So they made garbage AI content, without any filtering for errors, and they fed that garbage to the new model, that turned out to produce more garbage. Incredible discovery!
Indeed. They discovered that:
shit in = shit out.
A fifty year old maxim, to be clear. They “just now” “found that out”.
Biggest. Scam. Evar.
Who just found that out?
Yeah, in practice feeding AI its own outputs is totally fine as long as it’s only the outputs that are approved by users.
I would expect some kind of small artifacting getting reinforced in the process, if the approved output images aren’t perfect.
Only up to the point where humans notice it. It’ll make AI images easier to detect, but still pretty for humans. Probably a win-win.
Didn’t think of that, good point.
The inbreeding could also affect larger decisions in sneaky ways, like how it wants to compose the image. It would be bad if the generator started to exaggerate and repeat some weird ai tropes.
I don’t know if thinking that training data isn’t going to be more and more poisoned by unsupervised training data from this point on counts as “in practice”
Let’s go, already!
How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.
This would ideally become standardized among web servers with an option to easily block various automated aggregators.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on - social media, public image storage, copyrighted media, etc. All those sites with extensive privacy policies who are signing contracts to permit their content for training.
Without laws (and I’m not sure I support anything in this regard yet), I do not see AI progress slowing. Clearly inbreeding AI models has a similar effect as in nature. Fortunately there is enough original digital content out there that this does not need to happen.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on
Absolutely. It’s more a matter of principle for me. Kind of like the digital equivalent of leaving fake Amazon packages full of dog poo out front to make porch pirates have a bad day.
Well it means they need some ability to reject some content, which means they need a level of transparency they would never want otherwise.
They’ll just start using a chrome user agent
Only if enough people do it. Then again, loads scrapers outside of AI already pretend to be normal browsers.
You can validate that against user telemetry data expected from a browser.
AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.
These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.
Both can be true.
Preserved and curated datasets to train AI on, gathered before AI was mainstream. This has the disadvantage of being stuck in time, so-to-speak.
New datasets that will inevitably contain AI generated content, even with careful curation. So to take the other commenter’s analogy, it’s a shit sandwich that has some real ingredients, and doodoo smeared throughout.
They’re not both true, though. It’s actually perfectly fine for a new dataset to contain AI generated content. Especially when it’s mixed in with non-AI-generated content. It can even be better in some circumstances, that’s what “synthetic data” is all about.
The various experiments demonstrating model collapse have to go out of their way to make it happen, by deliberately recycling model outputs over and over without using any of the methods that real-world AI trainers use to ensure that it doesn’t happen. As I said, real-world AI trainers are actually quite knowledgeable about this stuff, model collapse isn’t some surprising new development that they’re helpless in the face of. It’s just another factor to include in the criteria for curating training data sets. It’s already a “solved” problem.
The reason these articles keep coming around is that there are a lot of people that don’t want it to be a solved problem, and love clicking on headlines that say it isn’t. I guess if it makes them feel better they can go ahead and keep doing that, but supposedly this is a technology community and I would expect there to be some interest in the underlying truth of the matter.
It is their own fault for poisoning the internet with their slop.
In case anyone doesn’t get what’s happening, imagine feeding an animal nothing but its own shit.
Not shit, but isn’t that what brought about mad cow disease? Farmers were feeding cattle brain matter that had infected prions. Idk if it was cows eating cow brains or other animals though.
It was the remains of fish which we ground into powder and fed to other fish and sheep, whose remains we ground into powder and fed to other sheep and cows, whose remains we ground to powder and fed to other cows.
So yes. That’s what’s happening.
Maybe the internet will get a prion and die
I use the “Sistermother and me are gonna have a baby!” example personally, but I am a awful human so
Photocopy of a photocopy is my go-to metaphor for model collapse.
More like… Degenerative AI *ba dum tsss
deGenerative AI ☞ !degenerate@lemmynsfw.com
No idea this existed.
Also… JFC WHAT THE SHIT?
Model collapse is just a euphemism for “we ran out of stuff to steal”
It’s more ''we are so focused on stealing and eating content, we’re accidently eating the content we or other AI made, which is basically like incest for AI, and they’re all inbred to the point they don’t even know people have more than two thumb shaped fingers anymore."
All such news make me want to live to the time when our world is interesting again. Real AI research, something new instead of the Web we have, something new instead of the governments we have. It’s just that I’m scared of what’s between now and then. Parasites die hard.
This sounds like AI is literally biting its own tail
ChatGPT, what is an ouroboros?
Of course! An ChatGPT is an ouroboros, ChatGPT what is an ouroboros.
…………………. Good?
Tbh I’m a bit lost on the purpose of this
Ah, the Hapsburg of AI!
Oh, the artificial humanity!
Are you confusing the Habsburg Dynasty with the Hindenburg?
Perhapsburg they are
No, I just thought they were vaguely similar enough words to make a dumb internet joke.
You’re right, that’s a good dumb internet joke. I’m just being needlessly pedantic today.
I see your needless pedantry and raise you abrasive grammarian.
I like to think of it like a Mad Cow or Kuru, you can’t eat your own species’s brains or you could get a super lethal, contagious prion disease.
Prion diseases aren’t contagious.
Edit: for the uninformed people that downvoted - clearly spelled out here https://www.merckmanuals.com/professional/neurologic-disorders/prion-diseases/overview-of-prion-diseases
You can acquire it through direct contact, i.e. consuming prion-disease-contaminated meat. What would you call it?
Also, that’s not what direct contact means when discussing contagion:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7150340/
Ingestion is not ‘direct contact’.
Contagious means you can get it from direct or indirect contact with another person or organism that is infected. Not from eating them.
That is not possible with prion disease.
Ingesting a Petri dish full of flu virus doesn’t make the Petri dish ‘contagious’.
If only the generated output also looked more and more like how inbred humans do.
Like insane rambling from LLMs, and the humans generated by AI had various developmental disorders and the Habsburg jaw.
Old news? Seems to be a subject of several papers for some time now. Synthetic data has been used successfully already for very specific domains.
Yup, old news and wrong news. Also so many people who hate AI but don’t understand how it works. Pretty disappointing for a technology community.
I’ve been assuming this was going to happen since it’s been haphazardly implemented across the web. Are people just now realizing it?
People are just now acknowledging it. Execs tend to have a disdain for the minutiae. They’re like kids that only want to do the exciting bits. As a result things get fucked because they don’t really understand what they’re doing. As Muskrat would say “move fast and break things.” It’s a terrible mindset.
“Move Fast and Break Things” is Zuckerberg/Facebook motto, not Musk, just to note.
Oh, I stand corrected
It is very much the motto this idiot lives by. He just wasn’t the first to coin that phrase.
No, researchers in the field knew about this potential problem ages ago. It’s easy enough to work around and prevent.
People who are just on the lookout for the latest “aha, AI bad!” Headline, on the other hand, discover this every couple of months.
Cool, let’s try to ruin it faster!
The solution for this is usually counter training. Granted my experience is on the opposite end training ai vision systems to id real objects.
So you train up your detector ai on hand tagged images. When it gets good you use it to train a generator ai until the generator is good at fooling the detector.
Then you train the detector on new tagged real data and the new ai generated data. Once it’s good at detection again you train the generator ai on the new detector.
Repeate several times and you usually get a solid detector and a good generator as a side effect.
The thing is you need new real human tagged data for each new generation. None of the companies want to generate new human tagged data sets as it’s expensive.
Looks like that artist drawing self portraits as his alzheimer got worse and worse.
It’s basically AI alzheimers
AIzheimers?
oh no are we gonna have to appreciate the art of human beings? ew. what if they want compensation‽
More like degenerative AIs
