AI Candidate Search & Analysis System for LinkedIn: Python + Playwright + OpenAI
A practical breakdown of an AI platform for automated sourcing, profile analysis, and candidate shortlist generation on LinkedIn.
TL;DR
We built an AI platform for automated candidate search on LinkedIn with intelligent filtering via OpenAI.
The system automatically searches for candidates, visits profiles, analyzes experience and skills, evaluates job fit, and generates a shortlist.
A FastAPI web interface lets you manage searches, view results, and work with the candidate database.
Key challenges: noisy profiles, inconsistent formats, LinkedIn rate limits, and bias risks in decision-making.
Core idea: not just "scrape LinkedIn" but build a full AI-assisted recruitment pipeline.
Why manual LinkedIn sourcing stopped working
The problem: modern recruitment is overloaded with manual actions β even a strong recruiter spends hours viewing profiles and sorting irrelevant candidates.
Manual sourcing
Dozens of LinkedIn tabs, hundreds of viewed profiles, and endless manual screening.
Wasted time
Hours spent on search, analysis, sorting, and re-checking candidates.
Lost talent
Hiring slows down, costs rise, and strong candidates go to competitors.
Hard segments
IT recruitment, mass hiring, senior engineers, executive search, and international hiring are especially labor-intensive.
Core idea: turn LinkedIn from a basic profile search into an intelligent candidate selection system with AI analysis.
Architecture Pipeline
Web UI (FastAPI)
β
LinkedIn Scraper Engine (Playwright)
β
LinkedIn Profile Extraction
β
OpenAI Candidate Analysis
β
Structured Candidate Database
This pipeline makes recruiting reproducible: each stage can be debugged and improved independently.
How the system works
The system follows a step-by-step recruitment pipeline:
LinkedIn authorization
On first launch, the user authenticates, cookies are saved locally, and 2FA is supported.
Candidate search
The user provides a query, selection criteria, and page count. The system opens LinkedIn Search and collects candidates.
Full profile analysis
The system visits each profile, extracting About, Experience, Skills, Education, headline, and career history.
AI evaluation via OpenAI
OpenAI evaluates job fit, seniority, relevance, potential risks, and match probability.
LinkedIn integration
LinkedIn integration is implemented through Playwright-based Chromium browser automation. The implementation handles:
Cookie auth
Cookies saved on first login, 2FA support, repeat authorization usually not required.
Search & collection
The system opens LinkedIn Search, collects candidate card data, removes duplicates, and saves results.
Deep extraction
Visits each profile to extract About, Experience, Skills, Education, headline, top skills, and career history.
Web UI management
FastAPI dashboard lets you launch scraping, manage searches, and view AI results in the browser.
Common source issues: incomplete profiles, uneven section completeness, and LinkedIn rate limits on requests.
Profile extraction and structured data
A key feature of the system is analyzing not just search cards but full candidate profile pages.
Challenges include uneven detail levels, partial section completeness, and varying profile depth.
Search cards vs full profiles
Search cards alone are not enough for precise candidate selection.
Production recruiting depends on full profile analysis: experience, skills, education, career dynamics, and role context.
Practical approach: first deep profile extraction, then AI scoring and result explanation for the recruiter.
AI-assisted Recruitment Pipeline
The system combines browser automation, structured extraction, and AI analysis:
Browser Automation
Playwright automatically opens LinkedIn, performs searches, and collects candidate profiles.
Structured Extraction
Profile data is normalized into a unified format for further analysis.
AI interpretation
OpenAI analyzes the profile and produces an explainable fit assessment.
Ready shortlist
Output is a ranked candidate list with AI scores, match reasons, and recommendations.
Scoring / Matching Engine
The key differentiator is AI evaluation via OpenAI, not simple keyword matching.
Profile data: - Experience: 5 years Python - Skills: FastAPI, PostgreSQL, React - Industry: B2B SaaS - Seniority: Senior OpenAI Evaluation: Vacancy match: 85% Seniority match: 90% Industry fit: 75% Risk factors: Low
AI output example: "Candidate fits: 5 years of Python, proven FastAPI and PostgreSQL experience, B2B SaaS background; gap: no Kubernetes data in production."
AI Layer (OpenAI)
OpenAI is used as an interpretation layer for candidate analysis:
1) vacancy fit and career track evaluation;
2) skill and term normalization across different formats;
3) seniority and experience relevance assessment;
4) concise candidate summary with risks and recommendations.
Anti-bias and limitations
AI recruiting carries bias risks related to gender, age, company background, and education path.
Risk mitigation practices: exclude sensitive attributes from scoring, keep reasoning transparent, enforce human-in-the-loop, and run regular recommendation audits.
Important: the system assists hiring decisions; it should not automatically hire people.
Docker / Production contour
Recommended production setup:
FastAPI web interface, Playwright engine for browser automation, OpenAI for AI analysis, structured database for result storage.
Service separation supports stability, batch-analysis scaling, and SLA control.
Common implementation mistakes
1) Using only search cards without deep profile analysis.
2) No normalization for skills and role naming across different profiles.
3) No explainability for recruiter decision flow.
4) Ignoring LinkedIn rate limits and not adding rate limiting.
5) No feedback loop from real hiring outcomes.
Tech stack
Backend
Python, asyncio, FastAPI, Uvicorn.
Browser Automation
Playwright, Chromium automation.
AI Layer
OpenAI API, semantic candidate evaluation.
Quality metrics
precision@k
Share of relevant candidates in top shortlist positions.
time-to-hire
How much the system reduces vacancy closing time.
match accuracy
How well final ranking correlates with expert recruiter/hiring-manager evaluation.
Recruiter acceptance rate
Percentage of AI recommendations accepted to the next hiring stage.
Usage examples
Launching candidate search
python linkedin_scraper.py "Python developer Berlin" \ --criteria "5+ years Python, FastAPI, PostgreSQL, B2B SaaS"
AI result fragment
{
"candidate": "John D.",
"profile_match": 85,
"skills_match": {
"matched": ["Python", "FastAPI", "PostgreSQL"],
"missing": ["Kubernetes"]
},
"seniority": "Senior",
"conclusion": "Strong candidate with relevant stack",
"risks": "No Kubernetes data in production"
}
FAQ
Is this just a scraper?
No. The system doesn't just collect profiles β it analyzes candidates and helps build a shortlist with AI scoring.
Does it use AI?
Yes. OpenAI analyzes experience, skills, tech stack, career track, and job fit.
Can it search by location?
Yes. Geo search and country-based sourcing are supported.
Is there a web interface?
Yes. A FastAPI dashboard for managing searches and viewing results is implemented.
Can AI analysis be disabled?
Yes, the system supports a --no-ai flag for collecting profiles without AI scoring.
Can scraping be sped up?
Yes, using the --no-profile-fetch flag for quick collection of search cards without visiting each profile.
Key Takeaways
1) This is an AI recruitment intelligence system, not just a scraper β a full pipeline from search to shortlist.
2) Python + Playwright + OpenAI enable scalable recruitment pipelines with AI analysis.
3) LLM analysis delivers higher-quality screening compared to keyword matching.
Who this is for
Recruitment agencies, IT recruiters, startup teams, and CTOs/founders who need fast, high-quality specialist search on LinkedIn without manually sifting through hundreds of profiles.
Contact via Telegram β