Body-Only AI Isn't a Facial Recognition Workaround

Picture yourself on the witness stand. The defense attorney leans forward and asks you to explain exactly how you identified the suspect. You take a breath and say: "Medium build. Dark hoodie. Average height." The courtroom goes quiet — but not in the impressed way. In the oh no way.

This is the scenario nobody talks about when they celebrate the clever pivot from facial recognition to "body-only" AI systems. The workaround sounds pragmatic on paper: if we can't compare faces, we'll search by hoodie color, body silhouette, hair length, and accessories. Problem solved, right? Not even close. The gap between what these systems promise and what they can actually deliver — especially under legal pressure — is wide enough to drive a wrongful conviction through.

The Workaround That's Spreading Fast

This isn't hypothetical hand-wringing. MIT Technology Review recently reported on a tool called Track, built by video analytics company Veritone, which is already being used by roughly 400 customers — including state and local police departments and universities across the United States. As of last August, U.S. attorneys at the Department of Justice began using it for criminal investigations.

"The whole vision behind Track in the first place was 'if we're not allowed to track people's faces, how do we assist in trying to potentially identify criminals or malicious behavior or activity?'" — Ryan Steelberg, CEO of Veritone, MIT Technology Review

That's a refreshingly candid admission. The tool exists explicitly to route around restrictions on facial analysis. And on one level, that makes operational sense — there are genuinely situations where faces are obscured, turned away, or legally off-limits. But the critical error happens when investigators start treating these attribute-based results as equivalent to facial comparison. They are not. Not even in the same category. This article is part of a series — start with Airports Normalize Face Scans Investigators Eviden.

Why "Soft Biometrics" Are Scientifically Unreliable

Here's the core problem, and it has a name in computer vision research: intra-class variability. It means that the same individual can produce wildly different attribute readings depending on the day, the lighting, the camera angle, the season, or whether they stopped at a store on the way over. A red jacket becomes a maroon jacket under sodium-vapor streetlights. "Medium build" becomes "stocky" when captured from a low-angle camera. "Short hair" in January becomes "shoulder-length" by July.

Facial geometry, by contrast, doesn't do any of that. The Euclidean distances between your eye corners, the width of your nasal bridge, the spatial relationship between your jaw edges and cheekbones — these are structurally fixed. They don't change when you buy new clothes. They don't change much across decades. A face measured today and compared to a photo from ten years ago will still produce a meaningful, mathematically consistent similarity score. A hoodie worn yesterday might be in a donation bin today.

That number should stop you cold. In the computer science field of pedestrian re-identification — which is exactly the use case we're discussing — studies using benchmark datasets have found that accuracy drops below 50% when searches rely on clothing and body type alone, without any facial anchor. You'd genuinely do better flipping a coin in some scenarios. And that's under controlled research conditions, not the compressed timelines and degraded footage of a real investigation.

The "Silver Four-Door" Problem

There's an analogy that makes this click instantly. Searching for a specific suspect by hoodie color and medium build is like trying to identify a getaway car by saying "silver, four doors." Millions of vehicles match that description. You'd never find the right one. But a Vehicle Identification Number — a VIN — is unique, stable, and machine-verifiable. Facial geometry is the VIN. Everything else is just "silver, four doors."

The statistical severity of what researchers call the "lookalike problem" is genuinely alarming for soft-attribute searches. In crowded footage datasets, false positive rates spike dramatically when the search relies only on appearance attributes. The more people in the dataset, the worse it gets — because the more opportunities there are to find someone who happens to be wearing a similar jacket and is roughly the same height. This isn't a flaw that better cameras or faster processors can fix. It's a mathematical inevitability of searching by features that aren't unique. Previously in this series: Why Im Good With Faces Is Quietly Wrecking Investi.

The Courtroom Test That Quietly Exposes the Gap

Here's where the stakes become concrete. American courts evaluate scientific evidence under what's known as the Daubert standard — a framework requiring that forensic methodology be testable, peer-reviewed, have a known error rate, and be generally accepted within the relevant scientific community. Facial comparison performed by documented AI systems, with reproducible similarity scores and a clear mathematical methodology, can be walked through that framework step by step.

Attribute-based matches cannot — at least not with the same rigor. When a forensic scientist testifies that two face images produced a cosine similarity score of 0.94 using a validated model, that is metric evidence. It has a number. It can be challenged, replicated, and explained. When an investigator testifies that the suspect appeared to be "medium build, dark jacket, similar height," that is descriptive approximation. It has no reproducible measurement. Defense attorneys know exactly what to do with that distinction.

Look, nobody is arguing that attribute-based tracking has zero value. There are legitimate applications — tracking a known individual across a contiguous camera sequence within a single investigation, or narrowing a dataset before applying a more rigorous method. The problem isn't the tool; it's what happens when investigators treat a rough filter as a definitive identification. That's when cases quietly start collapsing.

Understanding how face comparison technology actually produces and documents similarity scores makes the contrast vivid. Facial analysis encodes a face as a high-dimensional vector — essentially a long string of numbers representing the spatial relationships between dozens of facial landmarks. Comparing two images means calculating the mathematical distance between two vectors. That distance is a number. It's the same number every time you run the comparison. It can be audited. Attribute matching produces a category label. "Dark jacket." That's it. That's the whole output.

What "Stable" Actually Means in Biometrics

It's worth being precise about why facial geometry sits in a different scientific category than clothing or body type, because "biometric" gets thrown around loosely. A true biometric is a measurable biological characteristic that is universal (everyone has it), unique (it differs between individuals), permanent (it doesn't change significantly over time), and collectible (it can be captured and measured reliably). Facial geometry hits all four. Body build hits maybe one, on a good day, and even then "collectible" is generous given how camera angle and clothing distort silhouette readings. Up next: Federal Biometrics Raising Bar Pi Face Evidence.

Research on super-recognizers — people with exceptional face-recognition abilities — provides a fascinating parallel here. Study Finds covered research from the University of New South Wales showing that super-recognizers don't just see more of a face — they instinctively sample the regions that carry the most identity information. What regions are those? The eyes, the nose bridge, the jaw geometry. The permanent structural features. Not the hair. Not the clothes. Even the human brain, at its most skilled, defaults to facial geometry for reliable identification. The AI systems doing this best are doing exactly the same thing — just faster and with a numerical output.

The clothes someone wore during a crime are temporary. The jacket gets ditched. The hair gets cut. The hat gets thrown out. Every single attribute an "appearance-only" system can search by is under voluntary human control and changes constantly. Meanwhile, the geometry of the face that wore those clothes? Unchanged. The real question for anyone building or relying on these systems isn't whether they're technically clever. It's this: when you're sitting across from a defense attorney asking you to justify your identification methodology, which would you rather have behind you — a documented similarity score derived from stable facial geometry, or a note that says "medium build, dark hoodie"?

One of those is evidence. The other is a description that matches half the people at any given bus stop.