We asked eulaw.ai, GPT-5.6 and Claude Sonnet 5 the same legal questions and scored every answer against ground truth taken from the official sources themselves.
How many times as often eulaw.ai got the answer exactly right, on the same questions. Partial answers, refusals and errors all count as wrong.
Each line is indexed to eulaw.ai = 100. The hollow dot shows how far the bare model gets relative to eulaw.ai, and the shaded gap is what it falls short. The figure on the right is how many times as often eulaw.ai got it right. Claude Sonnet 5 is the model eulaw.ai writes its answers with, so the gap to Claude is what the retrieval pipeline adds.
We left out measures a model without a legal database cannot win by definition, such as acts passed after its training cutoff. Legal substance is a small lead and inside the noise; it stays because it is the fairest comparison in the set.
Asked which acts had amended a given law, GPT-5.6 named 59 acts that do not exist. Claude declined the question. On the long conversation Claude scored nothing, so no multiple can be given.
Every question is generated from a record that exists in the official sources, so “no data” is never the right answer and nothing is scored against a model’s memory.
The benchmark asks whether the user got an answer. Declining scores the same as a wrong answer.
All systems get identical question text. One extractor normalises every answer before scoring, so prose is not penalised against structured output. GPT-5.6 and Claude Sonnet 5 were tested through their APIs without a legal database, web search or retrieval.
Headline and measure-by-measure figures: run of 3 September 2026. The four fair measures: August 2026.