We ran our clause extractor on 100 public contracts. Here is the score.

Legal AI vendors publish accuracy figures all the time. Very few publish the test behind them. We will not quote an accuracy number without a published method, so this post is the method, with the number attached.

What we tested

The Contract Understanding Atticus Dataset (CUAD) is 510 commercial contracts that lawyers annotated across 41 clause categories: governing law, termination, exclusivity, caps on liability and so on. The Atticus Project publishes it under a CC BY 4.0 licence, so anyone can rerun a test on it.

We took 100 of the 510 contracts, chosen by an even step through the sorted list so that a rerun scores exactly the same documents. Together they hold 2,716 clauses the lawyers marked. On each contract we ran LexCore's own clause extraction step, the one a review uses, and asked it for all 41 categories every time. It never saw the list of categories each contract actually contains, because handing a model the answer key's list inflates the score.

• Precision: 0.737. The share of clauses it flagged that matched a lawyer's annotation.
• Recall: 0.487. The share of lawyer-annotated clauses it found.
• F1: 0.586. The balance of the two.
• Average precision: 0.418. Precision with its answers ranked by its own confidence.
• Found word for word: 1,796 of 1,796. Every clause it extracted appears verbatim in the contract.
• Keyword baseline: F1 0.117 on the same 100 contracts.

The run used a Gemini model on Google Vertex AI, LexCore's default quality lane on the day. It made 972 model calls with none failed, took 45 minutes and cost $1.88 for all 100 contracts. A run the day before scored F1 0.585, so the result is stable between runs.

How to read it

• It is precise. About three in four clauses it flagged were clauses a lawyer had also marked, in the same category.
• It misses about half. A recall of 0.487 is the weakness, and it is the number to remember.
• It did not invent contract text. That is a separate question from whether it found the right clause.

Where it works, and where it fails

Strongest categories, by F1:

• Governing Law: 0.94
• Notice Period To Terminate Renewal: 0.89
• Document Name: 0.86
• Effective Date: 0.84
• Agreement Date: 0.80

Weakest categories, by F1:

• Volume Restriction: 0.10
• Covenant Not To Sue: 0.18
• Competitive Restriction Exception: 0.20
• Price Restrictions: 0.28
• Affiliate License-Licensor: 0.29

The pattern is plain. Clauses with a usual heading and usual wording score well. Rare restrictions and exceptions buried inside commercial terms score badly. Those are the clauses a reviewing lawyer should still read for.

What this number does not cover

• It tests one step. Clause extraction only. A full LexCore review adds specialist agents that challenge each finding, CitationGate, and the lawyer's own Human Gates. None of that is scored here.
• It is not Nigerian. CUAD's contracts are US commercial agreements filed with the SEC. We know of no public, lawyer-annotated set of Nigerian contracts to test on, and we will not invent one.
• It is one model. A different model, or a later version of this one, would move the numbers. We will rerun it on the same 100 contracts and publish the result, better or worse.

Why publish a score that says we miss half

Because a buyer should be able to check what a vendor says, and a lawyer should know where the tool is weak before relying on it. A recall of 0.487 is the reason LexCore keeps a lawyer in the loop, and the reason the Lawyer Analysis Metric records how much supervision each review received.