Deepfakes Rebuild Faces From 128 Numbers — Why That Breaks Your Usual Evidence Gut-Check

Here's something that should stop you cold: a deepfake doesn't borrow your suspect's face. It rebuilds it — from scratch, pixel by pixel — using a compressed mathematical description of what that face fundamentally is. The original video might not even be in the room when the synthetic version gets generated. And that architectural fact — not poor lighting, not obvious glitching — is the real reason experienced investigators get fooled.

Most people's mental model of a deepfake is basically a very sophisticated Photoshop — someone carefully blending one face over another. That model is wrong in ways that matter enormously if you're evaluating video evidence. Understanding what's actually happening inside these systems isn't just interesting trivia. It's the difference between knowing what to measure and guessing at what looks right.

The Pipeline Nobody Tells You About

Modern deepfake systems — the kind that Vocal.media described as accessible enough for beginners to operate with minimal editing knowledge — run on a three-stage architecture that most users never see and most investigators never think about.

Stage one: encoding. The system ingests a face and compresses it down into a dramatically smaller representation — a vector of numbers that captures the essential geometry of that face. Not the pixels themselves. The concept of the face. Think of it as a highly efficient filing system: instead of storing every detail of a document, it stores a precise summary that contains enough information to reconstruct the document later. For faces, Alan Zucconi's technical breakdown of autoencoder architecture describes this compressed output as a "latent face" — a lower-dimensional representation that forces the network to identify structural similarities across all the faces it's ever trained on.

Stage two: latent space. This is where things get genuinely strange. The encoder doesn't just compress one face — it learns a kind of mathematical universe where every possible human face has a location. Researchers call this the latent space, and it's not metaphorical. Faces that look similar cluster together. Change one number in your vector, and the face subtly shifts. According to Metaphysic.ai's analysis of latent space manipulation, these latent codes directly represent facial features — which means you can modify specific attributes by moving through this mathematical space rather than painting over pixels.

Stage three: decoding. A separate network takes those numbers and reconstructs a face from them — generating brand-new pixels that the model believes should be there, based on everything it learned during training. This is not a copy. It's a generation. The decoded face never existed before this moment. This article is part of a series — start with Deepfakes Hit 8 Million Courts Still Cant Prove A Single One.

That 30-times surge isn't an abstract threat statistic. It means that if you're handling video evidence in fraud, identity theft, or credential verification cases today, the probability that you've already encountered synthetic facial media — without knowing it — is no longer negligible.

Why Your Eyes Are the Wrong Tool

Here's the misconception that gets investigators into trouble: "If I can't see obvious pixelation or glitching, the video is probably real — or at least real enough for comparison purposes."

The reason people get this wrong is completely understandable. For most of human history, detecting fakes meant looking harder. A forged signature, a doctored photograph, a poorly dubbed audio clip — these were caught by trained eyes noticing something off. That instinct is deeply wired. And early deepfakes rewarded it, because they did produce visible artifacts along hairlines, around ears, in the peripheral blur of a moving face.

Modern systems, increasingly available to non-specialists, are specifically optimized to eliminate those visual tells. The interface hides the complexity. The output looks clean. And human perception — which evolved to recognize faces, not to audit the biological plausibility of skin texture data — simply isn't equipped to catch what's missing.

What's missing is the biology. According to research published through the National Center for Biotechnology Information, deepfake content consistently fails to replicate two categories of subtle physiological signals: variations in skin tone caused by blood flow beneath the surface, and the natural temporal dynamics of behavioral cues like eye-blink frequency and micro-expressions. These signals are invisible in any single frame — they only appear across time. A human watching a video in real time can't perceive them. But they're measurable.

"Deepfake algorithms frequently fail to maintain the natural temporal dynamics of behavioral cues like eye-blinking frequency and lip synchronization." — National Center for Biotechnology Information (PMC), biological artifact detection research

There's also a structural problem that persists even in high-quality deepfakes. Academic survey research on arXiv notes that the semantic latent space learned by these systems is difficult to perfectly disentangle — meaning the encoding and decoding process leaves spatial and sequential manipulation traces in the output. The face looks right. The underlying geometry may not be. Previously in this series: Age Estimation Algorithm Lighting Accuracy Facial Analysis.

Generation vs. Comparison: Two Opposite Problems

This is the conceptual split that changes how you approach evidence review entirely.

A deepfake system is solving a generation problem: given a compressed mathematical description of a face, produce pixels that look plausible. It's optimizing for visual believability to a human observer. It succeeds when you can't tell the difference by looking.

Facial comparison — the kind used in disciplined forensic work — is solving a measurement problem. It isn't asking "does this face look right?" It's asking "do the geometric relationships between specific facial landmarks in this image match the known measurements for this identity?" Those are fundamentally different questions, and importantly, one of them is not fooled by visual plausibility.

Think of it this way. Imagine you have two hand-drawn maps of the same city. One was drawn by a talented artist who made it look exactly like a real map — the colors are right, the style is right, it passes a casual glance. The other was drawn by surveying actual coordinates. If you need to know whether two maps describe the same city, you don't admire the artwork. You measure the distances between specific landmarks and compare them to known coordinates. A beautiful forgery fails immediately under measurement. A genuine map from a bad artist passes.

That's the difference between eyeballing a deepfake and running a proper facial comparison. At CaraComp, this is the architectural principle that separates useful evidence from intuition dressed up as analysis: the system measures Euclidean distances between facial landmarks, not aesthetic similarity. A synthetic face might be photorealistic, but if the distance ratio between the inner canthi and the nasal bridge deviates from a subject's verified biometric record, the comparison flags it — regardless of how convincing it looks to a human reviewer.

What a Repeatable Method Actually Looks Like

The three-stage deepfake pipeline — encode, manipulate in latent space, decode — introduces detectable inconsistencies at each step, but only if you know what category of inconsistency to look for. Visual inspection catches stage-three artifacts when they're present. It catches almost nothing from stages one and two. Up next: Face Match Score Biometric Pipeline Explained.

Forensic facial comparison, done properly, works at a different layer. It's tracking landmark coordinates: the precise pixel locations of the inner and outer corners of each eye, the tip and base of the nose, the corners and peaks of the lips, the outer edges of the face. It's computing the ratios and distances between those points. It's comparing those measurements against a reference set with known provenance. According to detection research published on Preprints.org, temporal variation analysis — tracking how those landmarks move across frames, not just within a single frame — adds another layer that deepfake generation consistently struggles to replicate faithfully.

None of that is gut feeling. All of it is writable. And if your current process for evaluating video evidence isn't written down as a repeatable method — if "does this look right to me" is still somewhere in the chain — then you're relying on the one faculty that deepfakes were specifically built to defeat.

The real aha moment here isn't that deepfakes are sophisticated — it's that they're sophisticated in a direction that specifically exploits human visual perception. Every optimization in modern deepfake tools is aimed at fooling you, not at fooling a system that's measuring the distance between your left pupil and the center of your philtrum. Those are not the same problem. And the 30x surge in deepfake fraud cases means investigators who haven't internalized that distinction are increasingly working with a tool — their own visual judgment — that the adversary has already mapped and defeated.

When you review video evidence today, ask yourself one question: is any part of my current process something I could hand to a colleague and have them reproduce the same result? If the answer is no, you're not doing facial comparison. You're doing face recognition — the informal, biological kind — and that's exactly the system deepfakes were built to fool.