Your Family's Faces Are 128 Numbers — And Someone Else Has Them

Here's something that will change how you look at your photo app forever: it doesn't know your mom's face. It doesn't know anyone's face. What it actually does is convert every face in your library into 128 tiny measurements — eye distance, nose width, jawline curve, cheekbone angle — and then sorts those numbers into piles. The people in the "same pile" get grouped into an album. That's it. No names. No identity. Just math.

Once you understand that — once you see that "face recognition" is really just number-crunching — a surprisingly important privacy question comes into focus: whose computer is doing the math? When you use a cloud photo service, those 128 numbers get calculated on remote servers you don't control. When the same process runs locally on your own laptop or desktop, the numbers never leave your home network. Same algorithm. Completely different address.

So How Does a Face Become 128 Numbers?

The process has a name: face embedding (that's tech-speak for "turning a face into a list of measurements"). Here's what actually happens when you open your photo app and it groups Uncle Steve into his own album.

First, the app detects that there's a face in the photo at all — finding the rough rectangle where a face lives. Then it crops that face out and feeds it into a neural network (think of that as a very advanced pattern-recognizing machine that was trained on millions of photos). The network doesn't output a name. It outputs a list of 128 numbers. That list is called a facial embedding — basically your face's numerical fingerprint.

Those 128 numbers represent abstract measurements about the geometry of your face. Not measurements you'd recognize like "nose is 1.2 inches wide" — more like compressed, mathematical descriptions of how your face's features relate to each other. The specific model behind much of this technology, called FaceNet, was trained on 260 million images specifically to make one thing true: photos of the same person should produce 128-number lists that are close together, while photos of different people should produce lists that are far apart. According to Pluralsight, this process achieves up to 99.63% accuracy on standard face verification benchmarks. That's not "pretty good." That's extraordinary. This article is part of a series — start with Your Face Is Now Your Train Ticket And Nobody Asked You Firs.

The "How Far Apart?" Question

Now for the part that makes the whole thing click. Once every face in your library has been turned into its 128 numbers, the app needs to figure out which faces belong together. It does this by calculating distance between the number-lists — specifically something called Euclidean distance (basically: how far apart are these two sets of numbers if you plotted them in space?). Think of it like GPS coordinates, except instead of latitude and longitude you have 128 dimensions. Two faces that produce number-lists close together in that space? Probably the same person. Two faces far apart? Different people.

According to Tech Skill Guru, there's a specific threshold the algorithm uses to make the call. When the similarity score between two embeddings — their "closeness" — crosses a certain cutoff, the app groups them together. Set that cutoff too loose, and your sister and your cousin end up in the same album. Set it too tight, and you yourself get split into two albums because you wore glasses in half the photos.

Here's a useful analogy. Imagine a giant sorting machine that's learned to calculate the "distance" between two fingerprints — not by looking at swirls and loops the way a detective would, but by converting each fingerprint into a set of coordinates and measuring the gap between them. Smaller gap? Probably the same hand. Bigger gap? Different hand. The machine doesn't know it's a hand. It just measures. Face grouping is exactly that — except in 128 dimensions instead of two.

"Face recognition and search on local systems are close enough to cloud-based systems that users don't miss the feature set." — MakeUseOf

The Misconception That's Worth Correcting

Most people assume that if face grouping runs on their own computer instead of in the cloud, it must be slower, dumber, or less accurate. And honestly? That assumption makes total sense. Cloud companies have massive server farms. Your laptop does not. Of course the cloud version must be better, right?

Not really. The algorithm itself is mathematically identical whether it runs in a data center in Oregon or on the machine sitting on your desk. What changes is not the quality of the math — it's the speed of the first pass. A cloud service might process your 10,000 photos overnight because it has thousands of processors working at once. Your laptop might take a weekend. But once the embedding job is done? The grouping is just as accurate. Previously in this series: That Study You Just Read 66 Of Its Sources Dont Exist.

The reason this misconception sticks is that cloud photo apps feel faster and snappier day-to-day — because they're running on hardware specifically optimized for exactly this kind of work. But if you have a reasonably modern computer with a decent graphics card (GPU), MakeUseOf notes that embedding jobs which used to take days can finish overnight with GPU acceleration. The quality gap people imagine? It mostly doesn't exist. The speed gap? Temporary — it closes after the initial processing run.

So Why Does "Where" It Happens Actually Matter?

This is the part that matters for anyone who's ever felt a little uneasy about a photo service "knowing" what their family looks like.

When a cloud photo service processes your pictures, here's what happens: each face gets converted into its 128-number embedding on that company's servers. Those embeddings — your family's numerical face-fingerprints — are stored on infrastructure you don't control, under retention policies you probably never read, subject to data requests you'd never know about. The matching happens far from your home. You handed the math problem to someone else.

When local face grouping runs on your own machine? That entire process — the detection, the embedding, the distance calculation, the grouping — happens without a single number leaving your network. Nobody receives your family's face measurements. Nobody stores them. Nobody can hand them over. The algorithm is a guest that stays in your house, does its work, and doesn't call anyone.

At CaraComp, this is something we think about constantly: facial analysis technology is only as trustworthy as the boundaries around where the processing actually happens. The same algorithm that feels invasive in a cloud account feels genuinely private when it's running locally — not because the math changed, but because the address did. Up next: Ai Facial Recognition Doorbell Cameras Lawsuits Privacy.

The Question That Should Now Feel Different

Go back to your photo app — whichever one you use. Somewhere in its settings, there's a face grouping feature. It might be called "People & Pets" or "Face Groups" or something similar. It's on by default in most cloud photo services. And until about five minutes ago, you probably thought of it as a cute organizational feature, maybe slightly magical.

Now you know what's actually happening. Every face in every photo — your kids, your parents, your friends — gets converted into 128 numbers. Those numbers get compared to each other. The closest clusters get named albums. The question isn't whether that process is happening. It almost certainly is. The question is: on whose computer?

The beautiful irony here is that the thing that makes local face grouping feel "less powerful" — the fact that it runs on ordinary consumer hardware, quietly, without a data center — is exactly what makes it more private. The limitation is the feature. Your cousin's face measurements never leave the room. That's not a bug in the system. That's the whole point.