Skip to content
All posts

Research

Scaling our job ranker to 800M parameters: 11% more interviews per 1,000 applications

Our new 800M-parameter ranker lifts NDCG@10 from 0.746 to 0.863. In 30 days of production, the same users got 11% more interviews per 1,000 applications.

Actify Team4 min read

parameters, up from 149M
800M
NDCG@10, up from 0.746
0.863
the training data
3×
interviews per 1,000 applications
+11%

The original ranker

Every Actify search runs in two stages. Vector search pulls candidates from our 20M+ job index, then a ranking model reorders them for you. The ranker is a cross-encoder: it reads your profile and each posting together, in one pass, and scores how well they fit.

The original model, which we open-sourced on GitHub, is a 149M-parameter ModernBERT cross-encoder trained on 5.7M LLM-graded profile–job pairs for two epochs. On held-out profiles it scored 0.746 NDCG@10, against 0.640 for plain retrieval order.

Scaling it up

The new ranker keeps the same architecture and training objective and scales everything else: 800M parameters, 3× the training data, and four more epochs.

The original and the scaled-up ranker. NDCG measured on held-out profiles never seen in training.
Spec800M rankerOriginal ranker
Parameters800M149M
ArchitectureSame cross-encoder, scaled upModernBERT cross-encoder
Training data3× the original5.7M graded profile–job pairs
Epochs6 (4 more)2
NDCG@100.8630.746
NDCG@full0.9120.794
The original and the scaled-up ranker. NDCG measured on held-out profiles never seen in training.

Offline: a 15.7% jump in NDCG@10

NDCG measures how close a ranked list is to the ideal order, weighting the top results most. A score of 1.0 means the best matches sit exactly at the top. The scaled model lifts NDCG@10 from 0.746 to 0.863, and NDCG over the full list from 0.794 to 0.912.

NDCG@10 on held-out profiles (higher is better)
  • Random order0.521
  • Retrieval order0.640
  • Original ranker0.746
  • 800M ranker0.863

Random and retrieval-order baselines are from the original ranker's evaluation.

Online: 30 days in production

Offline metrics only matter if they move real outcomes, so we measured one: interviews per 1,000 applications. We took every user who had been on Actify for at least two months and compared their last 30 days on the 800M ranker with their previous 30 days on the original.

The same users averaged 11% more interviews per 1,000 applications than in the 30 days before. The better ranking offline turned into better results for real people.

Why matching is the lever

An application only turns into an interview when the candidate fits the role. A stronger matching model sends each application to a job the candidate is genuinely suited for, so the same number of applications produces more interviews. The ceiling is an optimal matcher: every candidate in front of exactly the roles they fit best.

Beyond applications

Follow that logic far enough and the application itself stops making sense. Today candidates fire off hundreds of applications and recruiters wade through thousands to find a few good fits. It is slow and expensive on both sides.

A better model: candidates fill out a profile once, much like a LinkedIn profile, and say what they're looking for. Recruiters ask a matching engine for the best candidates for a role. Candidates get interviews for roles within their experience without applying, or even searching, and companies spend less time and money finding the right people. Stronger matching models like this one are how hiring gets there.

Frequently asked questions

How does Actify rank jobs?

In two stages. Vector search pulls candidate jobs from Actify's 20M+ job index, then a cross-encoder ranking model reads your profile and each posting together and scores how well they fit. The newest ranker has 800M parameters and scores 0.863 NDCG@10 on held-out profiles.

What is NDCG?

Normalized discounted cumulative gain measures how close a ranked list is to the ideal order, weighting the top positions most. 1.0 means the best matches sit exactly at the top. NDCG@10 scores the first ten results; NDCG@full scores the whole list.

Did the bigger model actually get users more interviews?

Yes. Over its first 30 days in production, users who had been on Actify for at least two months averaged 11% more interviews per 1,000 applications than in their previous 30 days on the original ranker. That is a before-and-after comparison, not a randomized test, and it doesn't account for changes users made to their profiles or resumes.

Let Actify apply for you

Download the app, upload your resume, and your first agent is applying in minutes. Start free.