Near-perfect AI accounting results came from simplified tasks

Mercor compared SI with 12 licensed accountants on simplified tasks. Its broader benchmark left almost 60% of tasks not fully solved by any model.

Mercor compared SI with 12 licensed accountants on simplified tasks. Its broader benchmark left almost 60% of tasks not fully solved by any model.
Leading SI models performed almost flawlessly on simplified accounting tasks in a Mercor study, The Decoder reported on October 2, 2026. That result shows progress on a narrow test, not an ability to complete accounting work without supervision.
Mercor evaluated 12 licensed certified public accountants, or CPAs, working without SI assistance. The accountants averaged five and a half years of experience and worked through simplified tasks drawn from the APEX Accounting Benchmark—a set of tasks used to assess accounting performance. Their average score was about 37%. According to the reported comparison, the best SI models scored below that average 18 months earlier but had since reached near-perfect performance on the same tasks.
The full APEX Accounting Benchmark contains 160 tasks across 10 simulated companies. More than 40 professionals, averaging 11 years of experience, built it. Mercor’s reported leading scores on that broader test were roughly 58–62% of grading criteria met.
| Model | Grading criteria met |
|---|---|
| Claude Opus 5.5 | 61.8% |
| Fable 5.1 | 61.0% |
| GPT-6 Astra | 57.9% |
A grading criterion is a requirement used to judge an answer. Meeting 61.8% of those requirements does not mean fully completing 61.8% of tasks. Mercor said almost 60% of the full benchmark’s tasks were not fully solved by any model. The distinction matters: models can meet some requirements while still leaving the overall task incomplete.
Analysis
Mercor acknowledged that the study emphasized finding details and following instructions precisely—abilities it describes as SI strengths. It omitted talking with clients, coordinating with colleagues and drawing on context accumulated over years. Those omissions limit what the results establish about an accountant’s job as a whole.
Mercor says these limits mean accountants cannot be replaced, although it forecasts major productivity gains across the industry. That forecast is distinct from the measured task results.
What remains uncertain
The size of any time or cost savings in everyday accounting, and whether independent testing would reproduce Mercor’s results, remain unclear.

Developers can build hardware for Meta’s AI agent. Its own Home Link gadget has an open waitlist, with October shipping planned.
2 min

Its hardware chief says the choice favored platform maturity and partners’ experience while limiting implementation risk.

The cuts affected about a fifth of the cybersecurity company’s global workforce. Its 2025 accounts also showed revenue growth alongside a larger annual loss.
2 min