AI Candidate Ranking System
We develop AI candidate ranking systems that solve the problem of multi-criteria selection: technical skills, relevant experience, cultural fit, career trajectory. Traditional methods like keyword matching achieve ~50% precision on real data. We use multi-vector scoring with separate components for each criterion and weighted aggregation, customizable per role. This not only improves accuracy but also provides an explanation for each score.
A typical recruiting scenario: 400 resumes for a position, an HR manager has time to read 30. The rest are filtered out by surface-level signals—job titles, university names, presence of keywords. This isn't selecting the best candidates—it's selecting the first ones that come along. Our AI ranking system gives a weighted score to each candidate relative to a specific vacancy, explains the score in terms understandable to HR, and doesn't miss strong profiles with non-traditional backgrounds. That's what true AI recruiting and recruitment automation is about.
Where the Naive Approach Breaks Down
The simplest approach—keyword matching or TF-IDF between job description and resume—yields precision of 0.45–0.55 on real data. A candidate with 10 years of experience in "software development" loses to someone who wrote the exact phrase from the JD. Semantics are lost.
The next step—embedding similarity via sentence-transformers—has a different problem: general-purpose models (all-MiniLM-L6-v2, text-embedding-3-small) perform poorly on HR-specific semantics. "Python" in a JD and "Python" in a data analyst's resume versus a backend developer's are different contexts, but the embedding doesn't distinguish. Semantic matching of resumes without context is ineffective.
Even trickier are implicit requirements. A JD requires "team management experience", a candidate writes "led a team of 5 analysts"—semantic match might work, but a fuzzy threshold on cosine similarity (<0.72) would cut it off. Our approach with cross-encoders overcomes this limitation.
How Multi-Dimensional Scoring Works
Not a single score, but a vector of evaluations across key dimensions:
- Hard skills match—semantic matching of technical skills. We use a bi-encoder for initial retrieval (FAISS, top-100), then a cross-encoder for precise scoring (ms-marco-MiniLM-L-12-v2 or fine-tuned on HR data from MS MARCO).
- Experience level—extract years of relevant experience from the resume via NER + regex. Compare with JD requirements.
- Career trajectory—analyze progression: growth in responsibility, relevance of previous positions. LLM call with structured output.
- Education & certifications—rule-based extraction + normalization of university names and certifications.
The final rank is a weighted sum with weights specific to the role. For junior developers, education carries more weight; for seniors, experience trajectory.
Fine-Tuning Models on Historical Data
If historical hiring data is available (who was hired, who passed interviews, who failed), it's gold for training. We build a ranking model: input (vacancy, resume), output—probability of successful hire. LambdaRank or ListNet on top of embedding features.
Caution with bias: if historical data contains bias (e.g., 90% of hires are male for technical positions), the model will reproduce it. A fairness audit before deployment is mandatory. We conduct fairness audits using the AIF360 library.
Explainable AI for HR
A score without explanation is a black box that HR ignores. For each candidate, we generate:
- Top-3 matches with job requirements (with quotes from resume)
- Top-2 gaps (what is required, what is missing)
- One recommendation for additional verification during interviews
This makes ranking a tool, not a replacement for judgment. Explainable AI is a key advantage of our system.
Practical Case: How We Cut Screening Time by 3x
Our client—an outsourcing IT company with 200+ open positions simultaneously. Existing process: manual review in an ATS (Huntflow). Funnel processing time: 8–12 days to first phone call.
We built the system: resume parsing (PDF/DOCX → structured JSON via LLM extraction + regex) → bi-encoder indexing in Qdrant → cross-encoder reranking of top-50 per vacancy → LLM-generated explanations → integration via Huntflow API, scores displayed directly in the candidate card.
Results after 3 months: average time to first call reduced from 9.4 to 3.1 days (3x faster compared to manual screening). HR managers reviewed top-15 instead of top-30 on average (fewer irrelevant ones). Offer rate for hires through the system: +22% compared to historical baseline. Average savings for the client: $38k–55k per year per 100 vacancies. Contact us to discuss a similar project.
Technical Stack
| Component | Tools |
|---|---|
| Resume Parsing | LlamaParse, Docling, custom LLM extraction |
| Embedding | text-embedding-3-large, E5-mistral-7b, jina-embeddings-v3 |
| Vector Store | Qdrant, pgvector |
| Reranker | cross-encoder ms-marco, Cohere Rerank |
| Ranking Model | LightGBM, LambdaRank (if historical data available) |
| Explanations | GPT-4o-mini, Claude Haiku (structured output) |
Comparison of ranking methods (precision@10 on our data):
| Method | Precision@10 | Processing Time (100 resumes) | Explainability |
|---|---|---|---|
| Keywords | 45% | 2 sec | Low |
| Embedding Similarity | 62% | 5 sec | Medium |
| Our System | 83% | 12 sec | High |
Our approach is 34% more accurate than embedding similarity and provides full explainability. This is an HR Tech and ML for HR solution.
What's Included
- Audit of current recruiting process and data quality
- Prototype development in 2 weeks: bi-encoder + scalar ranking
- Weight calibration with HR department
- Fairness audit and model debiasing
- Integration with ATS (Huntflow, Lever, Greenhouse, Workday)
- Documentation and team training
- 3-month post-launch support
Work Stages
- Data Audit—quality of resumes in the database, availability of historical hiring data, structure of JDs. This determines the approach.
- Prototype in 2 weeks—bi-encoder + simple scalar ranking, demo on 50 vacancies.
- Iterations with HR—weight calibration, validation on cases of "well-known" candidates. Without recruiter input, quality cannot be tuned.
- Fairness audit—mandatory before deployment, especially if historical data is used.
- ATS integration—via API for Huntflow, Lever, Greenhouse, or Workday. Or a standalone interface.
Timeline: MVP in 4–6 weeks, full system with fine-tuning and ATS integration in 3–4 months.
Technical Details on Fine-Tuning
For fine-tuning, we use LambdaRank with NDCG loss function. We train on pairs (vacancy, resume) with a successful hire label. We apply early stopping and dropout to prevent overfitting.Get a consultation on AI ranking implementation by contacting our team. We have 10+ years of experience in production and 40+ successful projects in NLP and recruiting. Request a demo to see the system in action.