Mass resume parsing from hh.ru, SuperJob, Rabota.ru, and LinkedIn is a task we solve turnkey. The system processes thousands of new resumes daily, reducing manual recruiter work by 90%. Instead of copying data from 3–4 sources, you get a single, automatically updated candidate database with AI enrichment: grade (junior/middle/senior/lead), tech stack, total experience in years. Reducing costs by up to 80% saves over 500,000 RUB per year per every 1000 resumes. Using official APIs reduces the cost of collecting one resume to 0.1 RUB versus 10 RUB for manual copying. Below we cover the technical details: how to avoid blocks during parsing, normalize heterogeneous data schemas, and not drown in duplicates. All solutions comply with robots.txt and official APIs.
API vs Parsing: Strategy Choice
| Criterion | Official API | Parsing (HTML scraping) |
|---|---|---|
| Reliability | High, not blocked | Medium, requires anti-bypass |
| Speed | High (up to 1000 requests/min) | Low (≤5 requests/sec) |
| Data Completeness | Full structured info | Only visible, captchas possible |
| Legal Safety | Allowed by ToS | Gray areas, IP ban risk |
| Cost | Paid | Free but resource-heavy |
For Russia: hh.ru and SuperJob have official API for employers. We recommend starting with them. We use parsing only for Rabota.ru and LinkedIn, where APIs are absent or limited. When using API, the cost per resume is minimal, and reliability is 10 times higher than HTML scraping.
How to Reduce Parsing Block Risks?
For LinkedIn and Rabota.ru, we use Playwright with user-agent rotation and proxies. Once on a project with 500 resumes per day, we encountered a captcha on Rabota.ru—we had to integrate an image recognition service. After adaptation, parsing stability reached 98%.
Data Normalization: Key to a Unified Database
Each job site returns data in its own format. Without normalization, you cannot merge resumes into a single database. We convert all resumes to a unified schema using Pydantic:
class NormalizedResume(BaseModel): source: str # "hh.ru" | "superjob" | "rabota.ru" source_id: str # ID on the source full_name: str age: int | None city: str | None desired_position: str desired_salary: int | None currency: str experience: list[WorkExperience] education: list[Education] skills: list[str] # normalized skills languages: list[LanguageSkill] last_updated: datetime # AI enrichment seniority_level: str # junior/middle/senior/lead — AI estimate tech_stack: list[str] # tech stack — AI extracted experience_years: float # total experience Example of a normalized resume
{ "source": "hh.ru", "source_id": "123456", "full_name": "Ivan Ivanov", "age": 30, "city": "Moscow", "desired_position": "Python Developer", "desired_salary": 200000, "currency": "RUB", "experience": [ { "company": "LLC Technologies", "position": "Senior Python developer", "start_date": "2020-01", "end_date": "2023-06", "description": "Backend development on FastAPI" } ], "education": [ { "institution": "Moscow State University", "degree": "Bachelor", "field": "Applied Mathematics", "graduation_year": 2016 } ], "skills": ["Python", "FastAPI", "PostgreSQL"], "languages": [{"language": "English", "level": "B2"}], "last_updated": "2025-02-01T10:00:00", "seniority_level": "senior", "tech_stack": ["Python", "FastAPI", "PostgreSQL", "Docker"], "experience_years": 8.5 } What Does AI Enrichment of Resumes Provide?
The AI model (GPT-4o or LLaMA 3) determines the grade and technologies on the fly—it is 1.4 times more accurate than manual tagging. Recruiter time savings amount to up to 80% per resume.
Candidate Deduplication: Three-Level Method
One candidate often posts resumes on 2–3 sites. Our system detects duplicates using a three-level method:
| Method | Basis | Accuracy | Action on Match |
|---|---|---|---|
| Exact contact match | Phone/email (if public) | 100% | Automatic merge |
| Semantic similarity | Embeddings intfloat/multilingual-e5-large |
>0.85 | Suggest merge |
| Fuzzy matching | Name + city + current employer (Levenshtein distance) | >0.95 | Automatic merge |
At similarity > 0.85, we suggest merging; at > 0.95, we merge automatically. This eliminates up to 95% of duplicates without data loss.
Trigger-Based Resume Database Update
Resumes become outdated, so the system updates them based on triggers:
- Candidate updates resume on the source (webhook or periodic poll every hour).
- 30 days without changes—background reparsing.
- Candidate applies for a vacancy—priority update.
This approach ensures database freshness with a maximum delay of 1 hour.
Implementation Stages of Resume Parsing System
- Analysis: determine sources, data volumes, ATS requirements. Collect sample resumes for testing.
- Design: choose between API and parsing, design normalization schema, deduplication and enrichment pipeline.
- Implementation: write parsers (Scrapy/Playwright), connect AI model, set up deduplication and ATS integration.
- Testing: run on test data, check extraction accuracy, speed, and reliability.
- Deployment: deploy on servers (Docker, Kubernetes), set up monitoring (Grafana, Prometheus) and CI/CD.
What's Included in the Work
- Documentation: architecture description, data schemas, operation manual.
- Access: to backend (FastAPI), admin panel, Grafana metrics.
- Training: 2 sessions for your team (administration and rule configuration).
- Support: 2 weeks after launch + 6-month code warranty.
On request, we also add custom enrichment rules: for example, extracting certificates, projects, or soft skills via few-shot prompts for the LLM. We test on a sample of 100 resumes.
Get a consultation on your project today—we will evaluate your project in 1 day and propose the best solution. Order a turnkey system development and automate personnel recruitment.