Your Selfie Gets Checked Once. It Could Train Their AI Forever.

Here's something that should stop you mid-scroll: a company can fully comply with the European Union's landmark AI privacy law — the biggest, most detailed AI rulebook ever written — and still never tell you that your uploaded photo is being used to teach their system for the next ten years.

That's not a loophole someone accidentally left open. It's a gap between two very different things that most people treat as the same: legal compliance and actual transparency. They are not the same thing. And once you see the difference, you'll never look at an "upload your photo" prompt the same way again.

The Question Nobody Asks at Signup

Think about the last time an app asked you to upload a photo of your face or your ID. You probably thought about one thing: what is this app checking? Is it verifying my age? Matching my face to my account? Comparing my selfie to my passport?

That's the right question. It's just not the only question. The better question — the one almost no one asks — is: after this check happens, where does my photo go?

Because "checking" and "learning from" are two completely different things. And a photo that gets used to train an AI system doesn't get used once. It gets baked into the model permanently. Every future user that system analyzes will be analyzed partly because of what it learned from your face.

Your photo may be analyzed once — or used to teach a system forever.

---

How a Photo Actually Becomes Part of an AI Model

Most people imagine AI as something like a very fast expert — you show it a face, it recognizes it, done. But that's the finished product. Building that expert takes a pipeline — a series of steps — and each step is a separate privacy event that most privacy policies don't address clearly. This article is part of a series — start with Your Rewards Points Just Became A Bribe For Your Face.

Here's what that pipeline actually looks like.

Step 1: Collection. The system pulls in large sets of images, often from public sources. According to technical documentation on facial recognition training pipelines, these raw datasets are frequently described as "loosely labeled" — meaning the images are tagged with a name or category, but nobody has carefully verified whether those tags are right. Early datasets were often built mostly from photos of celebrities and public figures, collected from the web at scale. The labels were guesses, essentially.

Step 2: Filtering and quality checks. A human or automated system goes through the raw data and removes images that are too blurry, incorrectly labeled, or don't contain a face at all. This sounds clean, but here's the kicker: this step often involves a second set of people — contractors, annotation workers, or a separate company entirely — reviewing images that were never meant to be reviewed by anyone.

Step 3: Labeling and annotation. Each surviving image gets tagged with structured information. For a facial recognition system, that might mean marking where the eyes, nose, and jawline sit, or whether this face matches another face in the dataset. This is the step where images get turned into data points — and where your face, if it's in the dataset, officially becomes part of the training record.

Step 4: Training and testing. The model actually learns from the labeled images, then gets tested on a separate set to see if it makes accurate comparisons. The model's behavior — how it identifies faces, how confidently it scores a match — is shaped by everything in steps one through three. Once it's trained, you can't "remove" a photo from what it learned any more than you can unlearn a word you've heard a thousand times.

That fine sounds enormous. And it is. But here's the thing about fines: they only happen when someone gets caught. And catching a training-data violation requires a regulator to actually inspect the dataset — something that is, in practice, extremely hard to do.

---

What the EU AI Act Actually Requires (and What It Skips)

The EU AI Act — which took effect in stages starting in 2024 — is genuinely ambitious. For powerful general-purpose AI systems (the kind that can do many things, not just one task), it requires companies to publish a summary of their training data. In July 2025, the European Commission went further and released a mandatory template that AI providers must use to disclose what data they trained on. Previously in this series: That Quick Selfie Verifying Your Id Its Three Secret Tests A.

"Companies have ample incentive to disclose as little information as possible, and opening datasets might redirect scrutiny toward those companies." — Analysis of EU AI Act Article 53 transparency requirements, TechPolicy.Press

A template doesn't guarantee meaningful information. It standardizes what companies choose to reveal. "Sufficiently detailed summary" — the actual phrase in the law — is interpreted by the company being asked to disclose. That's a bit like asking someone to write their own report card and then trusting the grade.

There's also a structural gap in what the law covers. As Open Future's analysis of Article 53 explains, the tension between disclosure requirements and trade secrecy claims means companies have legal cover to withhold the most specific details — including which exact datasets were used and whether user-uploaded images were retained for future model development.

The regulation draws a distinction between processing your data (which must be disclosed) and training on your data (which often falls outside mandatory disclosure). An app can analyze your face for an immediate check and remain entirely silent about whether that photo feeds the next version of its model.

---

The Misconception That Makes This Hard to See

Here's where most people — including a lot of founders building apps right now — get confused. They think: if I follow the EU AI Act's disclosure rules, I've handled the privacy problem. Fill out the template, publish the summary, done.

It's an easy mistake to make, honestly. The regulation exists, the template exists, the fines exist. It looks like a complete system. But legal compliance is a floor, not a ceiling. A company can tick every box and still leave users completely in the dark about the most important thing: will my photo ever become training data?

As WilmerHale's analysis of the mandatory disclosure template makes clear, the template creates obligations around what gets published — but verification is a separate, much harder problem. Regulators would need access to the actual dataset to confirm that disclosures are accurate. And individual users have no tools to check whether their specific photos were retained.

Think of it this way. Building a facial AI system is a bit like building a house. The EU AI Act says you have to publish a materials list. But if the builder writes "high-quality lumber" without saying which forest, which trees, or whether any of it was taken without permission — the list is technically complete and practically useless. And once the walls are up, nobody can look inside to verify. Up next: Digital Identity Verification Three Layer Process Explained.

The Question That Actually Protects You

At CaraComp, we work with facial recognition as a tool for verifying identity — and one thing we've learned is that the most important privacy questions aren't about what a system does in the moment. They're about what happens to your data afterward. Understanding the training pipeline is part of how we think about building responsibly.

So here's something practical you can take with you. Next time an app asks for your face or your photo ID, most people ask: what are you checking? That's good. But the better question is: will this photo be used to train anything later?

A trustworthy answer involves three things: where the data goes after the check, how long it's stored, and whether it can ever be used to improve or train future AI systems. If a company's privacy policy doesn't address that third question clearly, that silence is itself an answer.

The EU AI Act was written partly to protect people with what its own language calls a "legitimate interest" in knowing how AI systems were built — including anyone whose face, voice, or documents might have ended up in a training set. That's a meaningful idea. The gap is that "might have" is doing a lot of heavy lifting in that sentence, and the law doesn't yet give you reliable tools to find out whether you're in that category.

Here's the thing that sticks with me about all of this. The most powerful AI privacy protection isn't a regulation or a template. It's knowing the right question exists. Because the moment you start asking "will this train something?" instead of just "what does this check?" — you've already changed the conversation. And companies that can't answer that question clearly are telling you something important about how seriously they take the difference.