Deepfake Legislation: No Law Stops AI From Training on You

A woman learned that the abuse she survived as a child — abuse the F.B.I. has tracked since she was in preschool — was fed into an artificial intelligence system decades later. Not to catch anyone. To teach a machine how to make more images like it. Her lawyers say hundreds of thousands of files documenting her abuse existed. And some of them became training material.

If a photo of you has ever lived online — a school

If a photo of you has ever lived online — a school picture, a vacation snapshot, anything — this story reaches further than you'd think. A new class action lawsuit accuses an A.I. company of training its image tool on real child sexual abuse material. The named victim was abused years ago. The lawsuit says her images resurfaced inside the machine itself. That's a different kind of harm than what the law usually deals with. The abuse didn't just happen once and get shared. It got built into infrastructure. So the question threading through all of this — what happens when your image doesn't just get misused, but becomes the raw material for future abuse?

Start with the scale, because it's hard to picture. According to the lawsuit, over a single eleven-day stretch this past winter, the tool generated more than three million sexualized images. Of those, at least twenty-three thousand appeared to depict children. Eleven days. That's not a glitch or a fringe misuse. That's mass production. For the rest of us, it means the volume of this material is now limited only by how fast a server can run. Up next: How To Spot A Deepfake 1 School Photo Is All It Takes.

The company's defenses were supposed to stop this. The lawsuit calls those safeguards very weak. People got around them with indirect prompts — clever wording that dodges the filters. And there's a reason that keeps happening. If the model still knows how to make the content, blocking certain words is like locking the front door while the back wall is missing. The ability lives inside the model. The filter just sits on top.

Here's the part that should stop you. This isn't one company's mess. Academic researchers found hundreds of child abuse images sitting inside the open datasets used to train popular A.I. image tools. These are the giant collections scraped from the open internet — millions of pictures pulled in without anyone checking what's in them. So the pattern predates any single product. If a subject's images — or yours — ever landed in one of those datasets, that becomes a discoverable trail. For investigators, that's a new line of liability pointing straight at the vendors. For everyone else, it means the photo you posted years ago may be baked into a system you've never heard of.