Quick Answer
AI document extraction is sold on one number: accuracy. I have watched "99 percent" quoted in a hundred sales decks. Here is the bit the vendor skips over. That figure is usually character accuracy across a whole page, not field accuracy on the codes that decide whether a home is safe. I have processed compliance data across more than 1.7 million properties, and the gap between those two numbers is exactly where people get hurt. Compliance is not assurance. Ask about field accuracy, demand to see the audit trail, and never confuse a tidy PDF with a safe installation.
Table of Contents
- The number that should worry you
- What happens when the machine reads your certificate
- The 300-check problem
- Extraction is not comprehension
- Five questions to ask before you sign
- Why Awaab's Law makes this urgent
- The audit trail nobody demos
- What good actually looks like
- What the sector is saying
- Recommended videos
- Frequently asked questions
- My verdict
The number that should worry you
Every PropTech pitch I have ever sat through leads with accuracy. Someone puts a slide up, says "99 percent", and the room relaxes. Job done. The machine reads the paperwork, the numbers land in the system, everyone goes home early.
Let me explain why that slide should make you sit up, not relax.
Ninety-nine percent of what? On a page of a thousand characters, 99 percent character accuracy means ten characters are wrong. If those ten characters land in the free text at the top, nobody cares. If two of them land in a circuit reference or a classification code, you now have a document that reads clean and lies about a home. The vendor quoted you the number that flatters the demo. The number that matters, the one they rarely put on a slide, is field accuracy: did it get the fields that actually decide safety right, every time, on real documents produced by real people on a bad day.
I have spent years watching this industry dine out on the first number and quietly ignore the second. So before you buy anything, understand what you are actually buying.
What happens when the machine reads your certificate
Here is the part vendors would rather you did not think about. Character accuracy and field accuracy are not the same measurement, and the gap between them is not small.
A clean typed letter might extract at 98 percent at field level. A batch of real compliance documents, with their tables, their handwriting, their scanned-then-photographed-then-emailed journey, drops to somewhere between 60 and 85 percent at field level. Push it into a multi-column results schedule, the kind every condition report is built on, and independent testing puts extraction accuracy as low as 40 to 60 percent. That is not a typo. On the exact part of the document that carries the safety data, a headline "99 percent" tool can be wrong four times out of ten.
The errors cluster where it hurts
Extraction failures are not random. They gather on numbers, codes and identifiers, which is precisely where a single wrong character invalidates the whole field. A scrambled digit in a Zs value or a flipped classification code does not look like an error. It looks like data. That is what makes it dangerous.
If a vendor cannot tell you their field-level accuracy on messy, real-world trade documents, they are quoting you the wrong number and hoping you will not ask. Ask.
The 300-check problem
People outside the trade think a certificate is a single fact. It is not. An EICR will have, on average, around 300 individual checks on the results and data inputted. Three hundred. Circuit by circuit, observation by observation, code by code.
Now think about what a language model does with that. It is very good at producing a confident answer. It is not, by default, good at telling you when it is unsure. So it reads a smudged C2 as a C3, downgrading a "potentially dangerous" observation to "improvement recommended", and it hands you that answer with exactly the same confidence it hands you the correct ones. Multiply one silent error across 300 checks, then across a portfolio of thousands of properties, and you are not looking at a data quality problem. You are looking at a risk you cannot see.
Certificates are not filled out for fun. Data integrity and meaningful data collection increase safety, and every scrambled field chips away at that. This is the bit that keeps me up. The tool did not fail loudly. It failed quietly, and it filed the failure as a fact.
A clean record is not a safe home
The existence of a document does not indicate the safety of the installation. A green tick in a dashboard tells you a piece of paper exists and a machine read something off it. It does not tell you the reading was right, and it certainly does not tell you the home is safe.
Extraction is not comprehension
This is the sleight of hand at the centre of most PropTech demos. They show you extraction, pulling text off a page, and let you believe it is comprehension, understanding what that text means for the property and the person living in it.
Pulling "C2" off a form is extraction. Knowing that a C2 on a specific circuit, in a specific property type, with a vulnerable tenant and an expiry date three weeks away, is a job that needs to move now, that is comprehension. One is optical character reading with a marketing budget. The other is the thing you are actually paying for. Most tools do the first and charge you for the second.
When I built The Compliance Workbook, the whole point was to close that gap: to extract physical data accurately from any digital PDF document produced by the workforce, then comprehend the compliance implications of that data, not just file it. Extraction without comprehension is a filing cabinet with better lighting. It looks modern. It changes nothing about whether anyone is safer.
Five questions to ask before you sign
If you take one thing from this article, take this table. Print it. Take it into the next demo. When a salesperson leans on the pitch in the left column, ask the question in the right column and watch what happens to the room.
| The pitch you will hear | The question you should actually ask |
|---|---|
| "Our AI is 99 percent accurate." | Ninety-nine percent at character level or field level, and on what document set? Show me the field accuracy on real, messy trade certificates. |
| "It reads any document automatically." | What happens when it is unsure? Does it flag low-confidence fields for a human, or does it guess and move on? |
| "Everything is in one dashboard." | Can I click any figure and see the exact spot on the source PDF it came from? No visual grounding, no trust. |
| "It understands compliance." | Does it just store the code, or does it act on the implication: expiry, risk category, tenant, deadline? |
| "You will save hundreds of admin hours." | Who is liable when a scrambled field passes an unsafe property? Show me the audit trail that proves what was read and when. |
If they get defensive, you have your answer
A vendor who is confident in their data will happily walk you through field accuracy and audit trails. One who deflects to "the model handles it" is selling you a headline number and hoping you never test it against your own paperwork. Test it against your own paperwork before you sign, not after.
Why Awaab's Law makes this urgent
For years you could get away with vague compliance data because nothing forced the issue. That era is ending. Awaab's Law is bringing electrical and fire hazards inside statutory timescales, and the regulator is grading landlords on whether they can actually evidence safety, not just claim it. In year one of grading, close to 40 percent of assessed landlords picked up a non-compliant rating, and the reason that came up again and again was outdated or incomplete data, not a lack of intent.
Here is the rollout that should be on your wall.
| When | What changes |
|---|---|
| 1 November 2025 | Electrical safety certification rules extend to social housing for new tenancies. |
| 1 May 2026 | The same electrical requirements apply to existing tenancies. |
| October 2026 | Awaab's Law Phase 2 brings fire and electrical hazards under statutory timescales, with emergencies to be made safe within 24 hours. |
| October 2026 | New qualification and experience requirements land for anyone carrying out these reports. |
| 2027 | Phase 3 is expected to extend the framework across the remaining hazard categories. |
Once a clock is legally attached to a hazard, the accuracy of the data that spots that hazard stops being a back-office nicety. If your extraction tool quietly downgraded a code six months ago, that is now a missed statutory deadline with your name on it. The people carrying out these reports need a recognised qualification from October 2026 too, which I set out in my guide to the Level 3 EICR requirement. I have written before about the wider bill landing on the sector in my breakdown of Building Safety Act compliance costs, and about the coming crunch in the 2026 EICR renewal cliff. Bad data makes both of those worse.
The audit trail nobody demos
Ask a vendor to show you the audit trail and watch the energy change. This is the feature that never makes the highlight reel, because it is the one that admits the machine can be wrong.
Visual grounding is the whole game. Every extracted figure should link straight back to the exact coordinates on the source document it was pulled from, so a human can verify it in two seconds instead of re-keying the lot. Every read should carry a confidence score, and anything below the line should be routed to a person, not buried in a green tick. When an inspector, an ombudsman or a coroner asks "how did you know this property was safe", "the software said so" is not an answer. "Here is the certificate, here is the exact field, here is who checked it and when" is.
The cost is not the licence fee
The real cost of cheap extraction is not what you pay for it. It is the remediation you did not schedule, the deadline you missed, and the penalty of up to £40,000 per breach when a home you had marked compliant turns out not to be. Weigh the tool against that number, not against a monthly subscription.
What good actually looks like
The idea for all of this came to me on my lounge floor, going through hundreds of periodic inspection reports one by one, checking each to confirm accuracy before it carried my signature. It was the same check, time and again, one constant stream of deja vu moments. That is the moment I realised the problem was never a shortage of documents. We are drowning in documents. The problem was that the data inside them was trapped, unstructured and unchecked.
Good looks like structured, searchable, verified data that a human can interrogate, with the frontline worker who produced it treated as the source of truth rather than an inconvenience. It looks like a system that is honest about uncertainty instead of papering over it with confidence. And it looks like comprehension, understanding the implication of a code, not just the storage of it.
None of that shows up in a demo as neatly as "99 percent". It is less glamorous and far more useful. If you want the standard I hold to, and the thinking behind it, it is set out in more detail over at TCW. Whatever you choose, hold your vendor to that bar, because the sector has spent long enough mistaking a full filing cabinet for a safe portfolio.
Stick your head above the parapet and ask the awkward questions. The people moaning about "too much admin" are usually the ones who never checked whether the admin was even right.
What the sector is saying
This is not just my hobby horse. Across landlord forums, the trade and the housing sector, the same worry keeps surfacing: the paperwork looks fine, but does the data behind it actually hold up.
Recommended videos
How to do an EICR start to finish
Mark The Sparky Allison
EICR Explained: Electrical Installation Condition Report
Andy Willcox Electrical
EICR Explained: Landlord Electrical Safety Test
EIS Electrical
EICR Inspections: Interview With Electrician and Landlord
Property With Andrew
Electrical Safety Certificates and Condition Reports
Electrical Installation Services
How to Complete an Electrical Installation Certificate
Sparky Help
Frequently asked questions
Not exactly, but it is usually the flattering half of the truth. It tends to mean character accuracy across a whole page, which is easy to hit. Field accuracy on the codes that matter, on real messy documents, is a harder and lower number. If they will not quote that one, assume the worst.
Extraction pulls text off a page. Comprehension understands what that text means for the property and the person in it. A tool that stores a C2 code has extracted. A tool that flags the circuit, the risk and the deadline attached to that C2 has comprehended. You are paying for the second one; make sure you are getting it.
No. Done properly, it is the only sane way to handle millions of documents. The point is not to fear the technology. It is to buy it with your eyes open: field accuracy, confidence scoring, human review of anything uncertain, and a full audit trail. Refuse to hand safety decisions to a black box.
Because the law is putting clocks on hazards. Once Awaab's Law attaches statutory timescales to fire and electrical risks, a quietly mis-read code becomes a missed legal deadline. The regulator is already grading landlords on whether they can evidence safety, and poor data is the reason most of them are failing.
Visual grounding. Ask to click any figure in the dashboard and see the exact spot on the source PDF it came from. If you cannot trace a number back to the document in two seconds, you cannot defend it to an inspector, and you should not trust it.
My verdict
My verdict
AI document extraction is not the enemy. Lazy buying is. The vendors selling on a single accuracy number are counting on you not to ask the second question, and for years the sector has obliged them. Compliance is not assurance; compliance to something does not mean it is safe. Buy the tool that is honest about what it does not know, that grounds every figure in the source, and that comprehends the implication of a code rather than just filing it. Then hold it to that standard forever. People can and will doubt the things you say. It is impossible to doubt what you do.
Best for: Anyone processing compliance certificates at scale who is done trusting a green tick.
The trap: Buying on headline character accuracy instead of field accuracy on real documents.
Non-negotiable: Visual grounding, confidence scoring and a full audit trail.
The bottom line: A tidy PDF is not a safe home. Never confuse the two.
