Applied World

Finance research · Accounting & spreadsheets

LedgerBench: The Accounting Benchmark AI Cannot Bluff Its Way Through

Evaluating spreadsheet work through source evidence, formulas, reconciliations, and the workbook an accountant actually receives.

Getting a total that balances is nice, but you can’t tell from a total where the data comes from or if a formula has been replaced by a number. You might not find an issue with a workbook until you need to update it next month.

Accounting benchmarks are elements of our job. As such, we’re meticulous, and a minor thing, like requesting an adjustment, can mean double checking calculations and reviewing a client’s work. When evaluating the work done, we have to inspect what’s been turned in.

Our focus with LedgerBench is the accounting work behind the figures a client receives. That includes finding source information, analysing data, performing calculations, and checking the results before a report goes out. A calculation can send you back to the PDF it came from. Sometimes you find that the figure belongs to another reporting period, and the work on the spreadsheet has to be checked again. We follow those steps and look closely at the file the agent returns. The reviewer needs to see where the numbers came from, follow the calculations, and use the workbook again next month.

The tasks are mainly data manipulation; searching and calculating. There’s a lot of validation and modeling. There is translation and reporting. Often there are a number of tasks inter-related and carried out one after the other. For instance, to reconcile a balance, you may need to locate the figures and confirm the reporting period.

FINCH takes you all the way to exporting the completed reports. This allows a reviewer to check the calculations and figures and other things that may impact the workflow, like the state of the report and if it’s ready to be presented to the next person.

What the benchmark actually asks

The researchers documented the workflows using various sources, including company emails and financial documents. Experts analyzed and edited these workflows, and the corresponding documentation, over 700 hours, the authors said.

Some ordinary office work involves interpreting and carrying out requests, as in office emails, and editing and interpreting work to instructors’ (sometimes confusing) instructions. Version histories may help determine the worksheet to be edited.

A SUM is simple to enter. However, depending on the context, determining the scope of the numbers to be summed may require reviewing supporting and sometimes lengthy notes. Using FINCH to evaluate a worksheet may provide an answer; however, it may take additional work to determine why, for instance, the answer may be wrong.

What the benchmark expects

The figures below are our estimates. We have not put these model versions through the same FINCH workflows. We place Claude Opus 5 at 62% and GPT-5.6 Sol at 59%, followed by Claude Sonnet 5, Gemini 3.5 Flash, and Grok 4.6. I would want to see them run with the same tools and review criteria before treating that order as a finding.

Projected automated workflow pass rates

Claude Opus 5~62%
GPT-5.6 Sol~59%
Claude Sonnet 5~50%
Gemini 3.5 Flash~47%
Grok 4.6~44%
Projected workflow pass rates, shown on a 0 to 100% scale. These models have not been scored here.

There are only three percentage points between the top two estimates. With numbers we have not measured, that gap tells me very little. A fair comparison would give each model the same business process, software, online services, and time to work.

I would spend as much time on the failed files as on the ranking. An agent could pull a figure from the wrong period, or leave a formula pointing to an old dataset while the chart still looks current. A reviewer might spot the out-of-range value without seeing why it happened. I want to know whether the error is visible in the workbook and how much work it takes to track down.

Projected pass rates by accounting task type

Claude Opus 5GPT-5.6 Sol
Data entry / import~56% / ~54%
Structuring / formatting~59% / ~57%
Web search~33% / ~32%
Cross-sheet/file retrieval~61% / ~59%
Calculation~64% / ~62%
Validation / review~62% / ~60%
Financial modeling~60% / ~57%
Summary / visualization~63% / ~61%
Projected rates by task type. In these estimates, data entry and web search are lowest for both models.

The accounting side

A hardcoded total is an easy mistake to miss. Suppose the agent sees a wrong answer, finds the correct figure, and types it straight into the cell. The report ties, and the screenshot looks fine. Next month the underlying figures change. That cell does not. Someone has to open the workbook and notice that the formula is gone.

Rebuilding the workbook can cause a different problem. The review team may have notes, links, or edits in the file they provided, and some of those can disappear in a new copy. In the original file, unrelated cells can change as well. Either way, the reviewer needs a clear record of what the agent was asked to do and what it actually touched.

FINCH looks at whether the work is complete, correct, restrained in its edits, and readable. For an accounting handover, I would check the business rules behind the figures and whether the validations still work. I would also look for missing notes or data that had nothing to do with the request. A report can balance while leaving those questions unanswered.

For this version of the tool, I would record the changes and the location of each figure in the edited workbook and maintain a log of the figures and why each figure was changed. Since work is often iterative, the review of formulas, links, and charts may not be completed until later and could be time consuming. It would also be beneficial to time how long it takes to do the edits, since it may take just as long to verify the edits.

An agent may be able to assist with locating data, creating and evaluating formulas, and summarizing work in a report. The accountant may evaluate the work and the report to ensure the work is accurate and the process is appropriate. I would evaluate the work to assess if the process is appropriate and if the final work is dependable.

Get in contact for internal use of benchmark.