Legal AI is the rare software category where a good demo tells you almost nothing. Any modern model will summarize a contract impressively in a meeting. The question that matters is what happens on the two hundredth document, at 7pm, when a fee earner is under deadline pressure and the model is confidently wrong about a clause.
We build AI systems for accuracy-critical domains, including Magic Circle, an agent workspace for legal teams. This is the due-diligence framework we would hand a managing partner or head of knowledge who is being sold to — and the questions we expect to be asked ourselves.
What legal AI does reliably today — and what it does not
The distinction that matters is not "hard tasks versus easy tasks". It is tasks where the source of truth is in front of the model versus tasks that require the model to know something. The first category is largely solved. The second is where firms get hurt.
| Reliable today | Handle with strict controls | Not yet |
|---|---|---|
| Extracting terms, dates, parties, and defined values from a document set | First-draft clause generation from firm precedent | Unsupervised legal advice |
| Reviewing a contract against a written playbook and flagging deviations | Cross-document consistency checks at volume | Reasoning about unpublished or novel case law from memory |
| Summarizing with citations back to the source paragraph | Translation of legal text between languages | Anything where a wrong answer cannot be caught before it reaches a client |
| Search and retrieval across a matter archive | Chronology and issue-list construction | Quantifying legal risk as a number |
A vendor who blurs these columns in a sales conversation is telling you something useful about how they will behave during implementation.
Ten questions to ask a legal AI vendor
- 1. Does every answer cite its source, down to the paragraph? Not a document name — the specific passage, clickable, in the original file. If a fee earner cannot verify an output in five seconds, the tool creates review work rather than removing it.
- 2. Is the architecture retrieval-based, and can you explain what happens when retrieval finds nothing? The correct behaviour is to say so. A system that generates an answer anyway when the source set is silent is unsafe in this domain. Our write-up on RAG versus fine-tuning covers the trade-off.
- 3. Show me your evaluation set. A credible vendor has a fixed set of documents with known-correct answers, scores every release against it, and can show you the score. No evaluation set means no way to know whether the last update made things worse.
- 4. What is the measured error rate on work like mine, and what kind of errors are they? A missed clause and an invented clause are different risks. Ask for the breakdown, and ask what happened in the worst case they have seen.
- 5. Is our data used for training — by you or by any model provider in the chain? Get the answer in the contract, covering subprocessors. "We don't train on customer data" should be a clause, not a reassurance.
- 6. Where does our data live, and for how long? Region, retention period, deletion process, and whether documents leave your tenancy at inference time. For firms with client confidentiality undertakings or cross-border restrictions, this decides whether a pilot is even permissible.
- 7. What does the audit trail record? Which user asked what, which sources were retrieved, what was produced, and what the human changed. This is what lets you reconstruct a decision months later — and what a regulator or insurer will ask about.
- 8. How does human review work in the product, not in the policy? Review should be a designed step with a record, not an instruction in a training deck. Ask to see where the reviewer signs off and what happens to unreviewed output.
- 9. What happens to our work product if we leave? Export format for prompts, playbooks, extracted data, and audit logs. Firms invest real effort in configuring these systems; that configuration should be portable.
- 10. Who is accountable when it is wrong? Not to shift liability — that stays with the firm — but to see whether the vendor has thought seriously about failure. Vendors who have not been burned yet answer this question breezily.
The pilot is the diligence
References and security questionnaires filter out the obviously unsuitable. Only a pilot on your own documents tells you whether a tool earns its seat. A four-week structure that produces real evidence:
- Week 1 — build the ruler. Choose one recurring, high-volume task (NDA review against your playbook is the usual candidate). Assemble 30–50 real documents with answers you already trust. This set is the most valuable artifact of the whole pilot: it outlives the vendor.
- Week 2 — run blind. The tool processes the set; results are scored against your known answers by someone who did not configure the tool. Record misses and inventions separately.
- Week 3 — run live with two fee earners. Real matters, normal deadline pressure, with review steps enforced. Measure time-to-completed-review, not time-to-first-draft — the second number is easy to improve and does not pay for the licence.
- Week 4 — decide against pre-agreed thresholds. Write the accuracy and time-saving thresholds down in week 1, before anyone is invested. Then hold to them.
One warning from experience: pilots fail more often on workflow than on model quality. If the tool lives outside the system where the work already happens, usage decays after the novelty passes, however good the outputs are.
What we learned building Magic Circle
Building an agent workspace for a legal team taught us a few things that now shape every accuracy-critical build we take on — documented at more length in six lessons from building AI agents for a legal team:
The workflow is the product. The model is a component. Where the work starts, who reviews, what happens to an exception — that is what determines whether hours are actually saved.
Citations change adoption more than accuracy does. A slightly less capable system whose every claim can be verified in one click gets trusted and used. A more capable system that asks for faith does not.
Refusal is a feature. Teaching a system to say "the source set does not contain this" is harder than making it answer, and it is what makes the output safe to build a process on.
Frequently Asked Questions
Is it safe to put client documents into an AI tool?
It depends entirely on the contractual and technical arrangement, not on the model. The conditions to insist on: documents processed within a tenancy you control or under a no-training commitment covering subprocessors, a named data region, a stated retention and deletion policy, and an audit trail. Check your client engagement terms and any outside counsel guidelines as well — some prohibit third-party processing without specific consent.
Should we buy a legal AI product or build something custom?
Buy for tasks that look the same at every firm — document review against a standard playbook, e-discovery, research. Build when the workflow embodies something specific to your practice, when the system must integrate with a document management system in a non-standard way, or when data residency requirements rule out available products. Many firms end up with both, and that is a reasonable outcome.
How do we measure return on investment?
Measure time-to-completed-review on a defined task before and after, on comparable matters, then value the difference at the rate of the person whose time was freed — net of the review time the tool adds. Vendor-supplied percentages describe someone else's workflow.
What accuracy level is good enough?
There is no universal number, because it depends on whether a miss is caught downstream. Set the threshold per task in advance: for a first-pass extraction that a fee earner will verify anyway, a moderate error rate is tolerable; for anything reaching a client without a review step, the answer is that a review step is missing.
Will this replace junior lawyers?
The pattern we observe is a change in what juniors spend time on rather than a reduction in headcount — less mechanical extraction, more review, exception handling, and judgment. That shift does have training implications worth planning for deliberately, because some of the mechanical work was also how juniors learned.
If you are evaluating right now
We are happy to be tested against the ten questions above, and equally happy to tell you when an off-the-shelf product is the better answer for your use case. See our legal and professional services practice and our AI engineering work, or bring us a pilot to design.
