Building State-of-the-Art Agents with Mercor | Applied Compute
CASE STUDY
Building State-of-the-Art Agents with Mercor
How Applied Compute's long-horizon RL stack turned Mercor's expert data into a top ranked agent at a fraction of the cost
Feb 24, 2026
We built APEX-Agents because we believed expert data could produce expert-level AI. The hardest part of building economically useful AI agents is knowing whether your data is teaching the right things. Applied Compute gave us that answer quickly and precisely with frontier training infrastructure.
Brendan Foody Founder and Co-CEO, Mercor
Mercor has solved one of the hardest problems in AI development: getting real experts to produce real training data at scale. Their platform recruits and deploys domain specialists from the world's top institutions, transforming hard-won professional judgment into labeled tasks that teach models how knowledge work actually gets done. Their APEX-Agents benchmark tests whether agents can perform professional tasks across investment banking, management consulting, and corporate law. In these industries, 480 tasks were created by experts with 10+ years of experience at top-tier firms.
Applied Compute's custom-trained model, Applied Compute: Small, ranks #1 on the APEX-Agents leaderboard in corporate law, and 4th overall, beating models like Opus 4.5 and GPT 5.2 at a fraction of the cost.
LawOverall
APEX-Agents: Corporate Lawyer
ModelRelease dateMean perf
Applied Compute: Small
54.8%
Gemini 3 Flash (High)
December 2025
52.4%
Opus 4.6 (High)
February 2026
50.2%
Gemini 3.1 Pro (High)
February 2026
49.4%
Gemini 3 Pro (High)
November 2025
48.7%
Opus 4.5 (High)
November 2025
47.1%
GPT 5.2 (High)
December 2025
44.3%
Grok 4
july 2025
41.0%
Kimi K2.5
January 2026
40.2%
GPT 5.2 Codex (High)
December 2025
38.2%
0%10%20%30%40%50%60%
Reaching state-of-the-art shows that domain-specific post-training, when paired with quality data and the right training infrastructure, can move a model from general competence to expert reliability, without collapsing general reasoning capabilities in the process. This research shows that current gaps that lead agents to fail in enterprise pilots can be addressed with a systematic approach.
Why professional work is hard to train on
A banker builds an LBO model from messy financial documents. A lawyer analyzes contract risk across hundreds of pages. These tasks are unstructured, and success is judged on outcomes rather than process. Reinforcement learning (RL) is a natural fit for professional domains where the standard for good work is already well-understood within the enterprise, even if the process for getting there varies.
But RL raises the stakes on data quality. Small flaws in task construction or reward design can dominate training dynamics entirely. When expert data is the binding constraint, not compute, every labeling decision matters more.
The dominant risk is investing scarce expert time in the wrong direction and only finding out thousands of examples later.
Michael HainesApplied AI Product Manager, Mercor
Initial derisking on GLM 4.6
Applied Compute first deployed its long-horizon RL stack on Mercor’s expert-labeled dev set of 874 tasks across 50 worlds. These self-contained environments simulated real enterprise contexts, including complicated spreadsheets, legal filings, and financial records. Training ran single-epoch with no SFT warmup, no filtering, and no modification to the tasks or rubrics. The base model was GLM 4.6, scoring 3.8% Pass@1 and 12.1% mean score at baseline.
What made the difference was the observability around the training run. Applied Compute provided full trajectory-level visibility into how the model attempted each task and made it possible to distinguish genuine learning from surface-level failures that aggregate metrics obscure.
Bertie VidgenAI Researcher, Mercor
For example, Mercor's initial grader imposed negative penalties for specific errors. Though a reasonable evaluation decision, during training this rewarded the model for doing nothing and aggregate scores plateaued. Applied Compute's platform flagged rising refusal rates through environment tracing and run observability tooling. Mercor opted to remove the negative criteria for this task, and without adding data, we were able to get past this local optima.
The post-trained model outperformed the baseline across every metric. Pass@1 (the share of problems solved correctly on the first attempt) and mean score (the average score across all attempts) nearly doubled. On corporate law, Pass@1 tripled, from 4.4% to 16.3%. Performance gains in law, banking, and consulting scaled nearly linearly with data volume. Rapid feedback from training runs allowed expert effort to be redirected toward concrete gaps instead of expanded uniformly, and indicated that adding more data would drive similar gains.
Example prompt with responses from the baseline and post-trained models
1
Prompt
User Request
We need to review Summit's historical distributions. Write me a memo analyzing whether Summit's shareholder distributions are in accordance with US tax code. Cite the exact tax code in each instance (short citations are acceptable). Reply to me right in here please.
2
Rubric
GLM 4.6
Criterion| Score
States that Summit's shareholder distributions are disproportionate| No States that Summit's disproportionate distributions violate US tax code| No States that a nonresident alien receiving shares violates tax code| Yes
Applied Compute
Criterion| Score
States that Summit's shareholder distributions are disproportionate| Yes States that Summit's disproportionate distributions violate US tax code| Yes States that a nonresident alien receiving shares violates tax code| Yes
3
Trajectories
State-of-the-Art performance at a fraction of the cost
Applied Compute upgraded to GLM 4.7, scaled the training dataset to 1,885 tasks, and increased training compute 10x through longer context lengths and more train steps.
The resulting post-trained model, Applied Compute: Small, achieved a Pass@1 score of 26.6% and a mean score of 54.8%, placing first on the APEX-Agents leaderboard in corporate law and fourth overall with a mean score of 40.1%. Applied Compute: Small navigated large documents efficiently rather than loop or abandon tasks. Average output tokens increased only modestly, so the model learned to reason efficiently.
Pass@1 Performance by Domain
Pass@1 Rate
Corporate Law
4.7%
26.6%+21.9 pp
Investment Banking
1.7%
20.9%+19.2 pp
Management Consulting
2.9%
21.6%+18.7 pp
30%
25%
20%
15%
10%
5%
0%
GLM-4.7 → GLM-4.7 + RL (1,885 tasks)
To verify the gains did not come at the cost of general capability, Applied Compute evaluated both models on HLE and GPQA alongside APEX-Agents. General reasoning performance was consistent: GPQA improved slightly (+0.4%) and HLE degraded by just 1.3%, though neither of these changes are statistically significant. What changed substantially was the ability to operate as an agent: finding files, reading them correctly, and reasoning over their contents to reach a defensible conclusion.
Intelligence Index: Mean Performance vs. Evaluation Cost
Applied Compute: Small
Google (Gemini)
Anthropic (Opus)
OpenAI (GPT)
xAI (Grok)
Moonshot (Kimi)
Zhipu (GLM)
MEAN PERFORMANCE
0%
10%
20%
30%
40%
50%
Applied Compute: Small
Gemini 3 Flash (High)
Gemini 3.1 Pro (High)
Gemini 3 Pro (High)
Opus 4.6 (High)
Opus 4.5 (High)
GPT 5.1 Codex (High)
GPT 5.2 Codex (High)
GPT 5.2 (High)
Grok 4
Kimi K2.5
GLM 4.7
$1
$2
$5
$10
$20
$30
OUTPUT COST ESTIMATES ($USD PER 1M TOKENS)OUTPUT COST ESTIMATES ($USD PER 1M TOKENS) / MEAN PERFORMANCE
What results suggest for enterprises
Fewer than 2,000 expert-labeled examples produced a model that outperforms every frontier model on a specific domain. This collaboration has a few direct implications.
First, data volume is not the primary constraint in domain-specific post-training. What determines whether learning occurs is data quality and the ability to measure what that data teaches at the trajectory level. This means that the proprietary workflows, documents, and judgment calls that accumulate inside any mature organization represent more usable training signals than most assume. The right infrastructure can turn these signals into reliably high-performing agents.
Second, the feedback loop between training and data decisions is where much of the leverage is. Each training run in this engagement functioned as a diagnostic: identifying which capabilities improved, which stalled, and where additional labeling would have the highest return. That kind of precision is what makes continued investment in expert data tractable.
Third, the gap between a general-purpose model and one that performs reliably on an organization's specific work is addressable at a scale that makes it practical and affordable. Law firms, banks, and consulting practices already possess the institutional knowledge required to close that gap. The work is in codifying it into training signals and building agents to learn from it continuously.
Applied Compute builds specific intelligence for enterprises: agents that perform reliably on your workflows, learn continuously from your data, and operate in your environment. If you're ready to turn institutional knowledge into performance, contact us to explore what's possible.