"A Human Reviewed It" — 3 Words That Protect Nobody When AI Decides Your Money

Somewhere right now, a bank is denying someone a loan, and a person is signing off on it. That signature feels like protection. Under Europe's new A.I. rules, it protects almost nobody — because if you can't explain what the machine did before the human looked at it, the human's approval is legally worth very little.

If you've ever been denied a refund, flagged as

If you've ever been denied a refund, flagged as high risk, or had a claim bounced back, an algorithm probably touched that decision first. And the sentence you'd get if you complained is the same three words every time. A human reviewed it. I understand why that sounds reassuring. A person looked at your file. Someone was accountable. But regulators have decided that's the wrong end of the process to be looking at. So why doesn't human review count for as much as we assume?

Start with the numbers, because they explain the whole problem. According to recent industry analysis, about fifty-seven percent of organizations are already deep into using A.I. — late-stage adoption, real decisions, real customers. But only around twenty-seven percent have a full governance framework in place. That's a thirty-point gap between companies using A.I. and companies who can explain their A.I. Enforcement under the E.U. A.I. Act starts in August of twenty twenty-six. That gap is the thing it's designed to close.

The Act works on a simple sequence that runs backwards from how most companies operate. For high-risk systems, the technical documentation has to exist before the system goes live. Not after a complaint. Not after a lawsuit. Before. Most organizations deploy first and scramble to write it all down when something breaks.

There's a comparison that makes this click. A home inspector can walk through a finished house and certify it meets code. But the architect's plans, the material specs, the engineering math — all of that had to exist before anyone poured concrete. If you can't produce the plans, no inspection proves the house was built right. The human reviewer is the inspector. The documentation is the proof of design. One of those can be faked by a signature. The other can't.

What actually has to be in those plans

So what actually has to be in those plans? The Act requires that training data be relevant, representative, and as far as possible, free of errors. Most companies have never audited their training data at all. Now it's mandatory and auditable. They also have to document accuracy metrics — and here's the part that matters to you personally. They have to quantify how accurate the system is for specific groups of people. Not one overall number. Broken out. Because a system that's ninety-nine percent accurate on average can still be badly wrong on the group you happen to belong to.

A confidence score isn't documentation, either. Saying the model was ninety-four percent sure proves nothing on its own. You have to show that score is appropriate for the decision you're making with it. Ninety-four percent is fine for sorting your photo library. It's not fine for denying someone medical coverage.

And the line into high-risk is thinner than people think. Most customer service chatbots sit in a lighter category with basic transparency rules. But the moment your tool starts judging creditworthiness on a big refund, or triaging an urgent insurance claim, you've crossed over into the strict tier. Nobody sends a memo when that happens. The use case moves, and the legal obligations move with it.

For anyone using facial comparison software in their work, this lands hard. A court won't accept "an expert built this tool." It'll ask whether you validated that tool on faces in lighting like your subject's, and whether you know when it fails. Professional tools make their documentation auditable. Consumer tools hide it. That difference is the whole ballgame.