Your Face Was Scanned Saturday. Nobody Asked If That Was Legal.

Here's something that should make you stop scrolling: a police force can point a facial recognition camera at tens of thousands of ordinary people — people who haven't done anything wrong, people who don't know they're being scanned — and the question of who set the rules might not get answered until after it's already happened.

That's not a hypothetical. In Western Australia, police launched a live facial recognition trial using a marked mobile van at public events and retail centers. The technology, supplied by NEC, scanned passing pedestrians and checked their faces against a watchlist of around 4,000 people with outstanding warrants. In one week of operation, more than 130,000 faces were processed. The Office of the Information Commissioner of Western Australia — the state's own privacy watchdog — was not invited to participate in any formal consultation before the trial launched.

Nobody's saying the technology itself is the whole problem. The real issue is quieter and weirder than that. It's about a review process most people have never heard of — one that should happen before the software runs, before the van parks outside a shopping center, before a single stranger's face gets compared to a database. When that process gets skipped, the consequences don't show up as an obvious error. They show up as an invisible gap: no one to ask "what happens to the innocent faces?" and no one required to answer.

The Pharmacy Analogy Nobody's Using (But Should Be)

Think about how a new drug gets approved. The pharmaceutical company might have stunning lab results — controlled conditions, ideal subjects, impressive numbers. But before that drug touches a single patient, an independent ethics board reviews the trial design. They don't just ask "does it work?" They ask: How will you handle adverse reactions? Who monitors the data? What happens if something goes wrong at week three? The board isn't there to slow things down for the fun of it. They're there because lab conditions and real-world conditions are two very different things, and someone has to be responsible for the gap.

A facial recognition trial is essentially the same setup — except in Western Australia, the ethics board never got called.

The privacy commissioner's role would have been to ask those exact questions before a single pedestrian got scanned: What images are collected and retained? How long do they stay in the system? Who reviews a possible match before officers act on it? And critically — what happens when the system gets it wrong? Without that review happening upfront, those questions don't disappear. They just go unanswered while the trial runs. This article is part of a series — start with Age Verification Api How It Works.

Why "90% Accurate" Is the Wrong Thing to Brag About

Here's where most people's instinct leads them astray — and honestly, it's an easy trap to fall into.

When a vendor announces their algorithm is 90% accurate, or posts top rankings from NIST (the National Institute of Standards and Technology — America's official measurement science agency, which runs the gold-standard facial recognition benchmarks), it sounds reassuring. Nine out of ten correct? That seems fine, especially if a human officer reviews the result anyway.

The problem is what those accuracy numbers are actually measuring. NIST testing happens under controlled conditions: frontal poses, consistent lighting, high-resolution images, minimal compression. The algorithm is essentially being tested on its best day, wearing its best outfit, in perfect light.

A pedestrian walking past a police van on a busy Saturday afternoon is not that.

Research from Carnegie Mellon's CyLab Biometrics Center has documented confidence score drops of 30–40% just from a change in camera angle — even for algorithms that perform brilliantly on frontal imagery. The algorithm doesn't stop and say "I'm not sure about this one." It still outputs a confidence score. It might say "87% match." But that 87% confidence rating on a side-angle, motion-blurred sidewalk photo may correspond to something closer to a coin flip in terms of actual accuracy. The officer sees a number. They trust the number. They can't see what conditions produced it.

That number deserves a second look. Not 10% worse. Not twice as bad. More than 100 times higher false positive error rates for the demographic groups the algorithm handles least accurately. Applied to 130,000 scanned faces in a single week, that's not a rounding error — that's a pattern. Specific communities end up generating far more false alerts, which means more unwanted police contact for people who did absolutely nothing wrong.

One reason this happens: as the Federation of American Scientists has documented, facial recognition datasets carry embedded biases — racial and gender biases baked into the training data — and darker skin tones may require different lighting conditions to capture facial structure clearly, leading to more recognition errors when lighting isn't controlled. A pedestrian in natural daylight doesn't come with adjustable studio lighting. Previously in this series: Your Face Is Not A Password And You Cant Reset It.

A pre-deployment privacy review would flag exactly this. It would require an independent audit of field accuracy across demographic groups before the system goes live. Without it, there's no baseline. You can't measure whether the system is performing fairly because you never established what "fairly" looks like before you started.

The Scale Problem: When Volume Breaks Everything

There's one more wrinkle that makes the governance gap even more urgent — and it's a math problem.

Accuracy benchmarks often measure what's called 1:1 verification: does this photo match that photo? But a live public trial runs 1:N — does this face match any of the thousands of targets on a watchlist? NIST's own benchmark program treats 1:N as a significantly harder test than 1:1. The more people you add to the comparison pool, the more chances the system has to find a wrong match.

Imagine an algorithm that's 99% accurate in a 1:1 comparison. Sounds excellent. Now run it against 4,000 watchlist targets, at 130,000 faces per week. Statistically, the false positive opportunities multiply with every additional comparison. The system isn't getting less accurate — but volume turns even tiny error rates into a steady stream of wrong alerts. And each wrong alert means a real person, who did nothing wrong, just had their face flagged to a nearby officer.

"The primary concern centers on the lack of explicit legislative frameworks governing how the biometric data is stored, processed, and eventually destroyed." — The Conversation, on the Western Australia facial recognition trial

This is the part that sounds like bureaucratic fine print but absolutely isn't: if the technology flags a match, the image is retained and an alert goes to nearby officers. Images of people not on the list are supposed to be deleted almost immediately. That sounds clean. But "almost immediately" needs a definition. Who verifies it happened? What's the audit trail? If innocent faces are accidentally retained longer than policy allows, who finds out?

These aren't hypothetical worries. They're exactly the questions a privacy impact assessment — a formal review done before deployment — is designed to answer with documented, enforceable answers. Not promises. Procedures. Up next: That Enter Your Birthday Box Is Dead Heres What Actually Che.

The 4 Questions That Should Come Before Any Trial

At CaraComp, we look at facial comparison as a methodology — not just a technology. Accuracy matters enormously. But accuracy without process is like an accurate scale in a courtroom with no chain of custody. The number means nothing if you can't defend how you got it.

The questions a privacy commissioner — or any responsible oversight body — should require answers to before a facial recognition trial touches real images aren't complicated. They're actually pretty intuitive once you see them laid out:

Australia currently has no dedicated legislation governing police use of AI surveillance or biometric data. The federal government's National AI Plan, published in December 2025, does not recommend introducing any. Which means right now, the answers to those four questions depend entirely on whether someone with authority thought to ask them before the van parked and the cameras started rolling.

So here's the question worth sitting with: if your face were scanned in a trial like this — walking past a van outside a shopping center on a Saturday — which of those four protections would matter most to you? Consent before scanning? Guaranteed deletion? A human who actually reviews the match? Or the ability to challenge the system if it got your face wrong?

Because right now, in at least one place that tried this, nobody officially asked that question before the cameras turned on. And "nobody asked yet" is a very different thing from "you're protected."