A huge problem with ML training is the ‘ouroboros issue’ - training ML with input from other MLs breaks things in very deep (but difficult to stop/detect when it’s happening) ways due to the way the internal math works (model collapse).

Right now, the Internet training set is becoming more and more contaminated with better and better generative AI images and video.

It makes the models more screwed up, and makes it very difficult for humans to figure out what is original and not too.

If there was some signal that could at least make it easier to identify ‘original’/real images…

"Model collapse" is often overstated, as this paper demonstrates: https://arxiv.org/pdf/2404.01413

The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.

Not when you also including poisoning attacks.

I remember the original paper showing issues with even a couple percent of certain kinds of synthetic data too, not 100%.

What do you mean by poisoning attacks - stuff like Nightshade or Glaze? I was under the impression that those have largely failed to achieve their goals.

Those aren't poisoners. The are attempted protectors.

I think the term "AI incest" would sum this up :)

OK, but any reliance on a fakable signal such as this seems guarateed to invite poisoning.