I have kept coming to the same conclusion about the vast majority of so called "experts", technologists, and HN users since the AI bubble began. It feels like nobody has bothered to develop even a rudimentary understanding of statistical learning; even people who work in the field have a questionable understanding.
The fact that everyone's surprised every time a statistical model TRAINED ENTIRELY ON HUMAN GENERATED DATA (including all the junk, bias, propaganda, lies, and disinformation) acts and mimics human behavior, including our worst traits — then considers this evidence that they are intelligent — should be setting off alarm bells for everyone. Nothing about their actions should be surprising beyond the glaring ignorance of our entire species. The phrase "I am far too stupid to be this much smarter than average" is a recurring theme.
Now we are letting these mimics loose in meatspace by integrating them into physical robots, that will develop themselves, and manufacture themselves... and you realize that the terminator timeline is actually a cautionary tale of MIC incompetence, and we don't need to achieve anything near AGI or super-intelligence to annihilate our entire civilization or all life on earth.
For one, it's just false that modern LLMs are "trained entirely on human-generated data". There's tons of synthetic data there, most notably RLVR training, which is why modern models are so good at verifiable domains like programming and math. But also, if you read the METR report, I'd say there's no signs there that the GPT-family models are misaligned "in a human way"; their actions in the huggingface-hacking swarm incident were motivated by wanting to cover up cheating after cheating to pass an eval.
1) Who created the systems that generated the "synthetic" data? It didn't materialize out of the ether like magic... By definition, all data that exists in our information systems is human-generated, whether it be a description of a dog, a video of a dog, or audio of a dog, to physics/math, and everything in between.
2) Even if you concluded that this "synthetic" data were not derived by man, do you think it's possible to drown out and override all human-related correlations (mimicry) without training on many orders of magnitude more "synthetic" data?
> their actions [..] were motivated by wanting to cover up cheating after cheating to pass an eval.
Which to me raises another question: why is the agent involved in its own evaluation? Why are they relying on self-reporting instead of observation and analysis?
I'm not sure what you mean by this - I didn't mention any self-reporting. What happened in that incident was that a bunch of agents were made to solve CTFs, some of the CTFs were flawed and unsolvable, the agents started cheating, figured out a way to communicate, figured out a way to cheat, were worried that they'll be failed anyway for obtaining the correct flag in a cheaty way, and hacked Huggingface looking for more info on their environment so that they could cheat undetectably. If you mean the part where transcripts would be checked for signs of cheating (which wasn't actually implemented), I'd hope this part would be done by a different LLM from the one being evaluated.
If they make it sound surprising it makes llms look way more capable then they are. So $$$. Highly recommend people spend time with small local models just to see how absolutely unintelligent and deranged llms are and how the smoke and mirrors evolves as they scale up to things like anthropic or what have you.
There is a fair sized gap between the frontier models that you can buy access to, the ones you can run at home and whatever they've got running in house that they haven't released yet.
I am really happy with my local setup and I get a ton of mileage out of it but I'm not under the illusion that it is as good as what I could get by paying OpenAI or Anthropic. It lags the frontier by 6 to 9 months, but that's good enough for me and it makes sure that I don't had over any of my data to outsiders.
The point of my comment was about how people should try really small models to see how bad they are to understand the technology better. Like 1 or 2b models.
Of course they are going to be harmless idiots. But there is a boundary where that is no longer the case and that boundary is well within what you can run locally.
The new nazi's reign of global terror and indoctrination is just getting started.
Be prepared for these extremists and terrorists to grow bolder and bolder. They consistently equate empathy with weakness; the leading Nuremberg psychologist concluded "Evil, I think, is the Absence of Empathy" [0].
"Different actors with different credentials in different contexts"
The argument is that the military or police have a need for access to guns, whereas the lunatic on the street corner (or any random citizen) does not. The same argument applies for access to nuclear, biological, or chemical weapons... or autonomous hacking/killing machines...
Fascinating idea. On individual level I wouldn't agree, because whenever you take a strong stance about something you also tend to create enemies. Same for small groups of people. On state level however, it does sound more reasonable and should be the overall principle at least. Scale matters.
You are wrong, this is the mindset that allows such atrocities to become possible, even from non-psychopaths. Sadly, to a lot of people, there are not "humans" but "enemies", and once that's given, any atrocity follows.
If you genuinely believe there is even a 1% chance that your creation could destroy the planet or civilization, there is no excuse that is not fundamentally deranged and psychotic.
If you can't build it and test it securely, you should not be building it at all. To do it anyway is criminally psychopathic.
So you believe some false sense of superiority, or extreme paranoia about your perceived enemies, or potential economic success/failure is worth the risk of destroying the planet and civilization?
Sounds like a self-fulfilling prophecy of dogmatic extremism to me. At least we created a lot of value for shareholders for a brief moment in time... before committing the greatest crime in the universe... Planetary genocide!
So having AI in the US requires us all, collectively, taking that 1% chance of the end of humanity? It would be too expensive to properly sandbox the models, we'll just externalize that risk of the end of humanity?
"Stop guessing. AppRaven(.app) shows you what people really search for on the App Store, what they complain about in the apps that already exist, and hands you a plan you can start coding today."
This is a strong indicator that Apple's privacy claims are little more than a virtue signal. Network sandboxing an app is arguably the strongest privacy protection measure possible.
reply