Facial Recognition Bias: What 905,117 Police Scans Hide

Western Australia Police scanned the faces of more than nine hundred thousand people in three months. They report that eight of those scans pointed at the wrong person. That works out to an error rate of less than one thousandth of one percent. It sounds like nothing. It's not nothing, and the number hides more than it shows.

If you've ever walked through a train station, a

If you've ever walked through a train station, a shopping district, or a stadium, your face may already have been scanned. You wouldn't have been asked, and you wouldn't have known. That's what happened here. Police ran live facial recognition across thirty-six locations. The cameras checked passing faces against a list of wanted people. Police say the trial led to seventy-nine arrests. They also say officers approached a hundred and fourteen registered sex offenders, and carried out eight welfare checks. The state's privacy watchdog wasn't consulted before it began. So the question running through this episode is simple. When police say the error rate is tiny, who is that number really describing?

Start with the eight people. Each one was a stranger walking through a public place. A computer told police that person matched someone wanted. Eight times, the computer was wrong. Police themselves say those errors were tied to darker skin tones, uneven lighting, and poor quality reference photos. The reference photo is the picture the system compares you against. If that picture is blurry or old, the match gets shakier.

Now look at how that tiny percentage was built. Lauren Perry, a researcher at the University of Technology Sydney, warns that the figure can give a false impression. The math divides eight errors by every face scanned. But nearly all of those faces were never a match for anyone. The crowd makes the error rate look small. It tells you little about how often an alert on your face would be right. For a police department, that's a reporting choice. For the rest of us, it means a reassuring number can be true and still leave out what matters.

The second problem is that errors don't land evenly. The National Institute of Standards and Technology, known as NIST, has tested these systems for years. Its researchers found that the gap between groups is much bigger for false positives than for false negatives. A false positive is when the system wrongly says you match. A false negative is when it misses someone it should have caught. Between demographic groups, false positive rates can differ by ten times, or even a hundred times.

Then there's the number that stopped me

Then there's the number that stopped me. As of March of this year, the top performing system in that testing produced three hundred fifty-eight times more false positives for West African women over sixty-five. The comparison group was Eastern European men between thirty-five and fifty. Three hundred fifty-eight times. That's the best system on the list, not the worst. An overall accuracy figure blends all of those groups together. Someone in the highest-risk group might face a very different reality than the average suggests.

For investigators who use face comparison tools, that's a practical test. Any vendor should be able to show how its tool performs across different groups, not just one overall score. For everyone else, it means a stranger's match can carry far more weight than it deserves. Imagine an officer walking up to you in public because a screen flashed your face.

Police do have a response, and it deserves a fair hearing. They say the system found people they were looking for, including registered sex offenders. That's a real public safety result. But critics point to what the results leave out. They don't say who the system missed. And they don't break down how it performed for different groups. Without those two pieces, nobody outside the program can judge whether it's fair.

The trial's success and its risk aren't opposites. Both can be true at once, and a single error rate can't show you either. The number that matters isn't how often the system is right overall. It's how often it's wrong for you.