Why this post is here in the front page? The Qwen vector steering performed was a very selective operation that interested only specific layers and weights set, it is non trivial to identify and fix. This is remarkable. Also Experiment 3: no access to forward pass, so why here this post says the contrary of what the paper shows and everybody upvoted it? I don't understand.
My complaint is that the post presents a confusing mix of experiments that imply capabilities that are more than the sum of their parts. This experiment 3 has nothing to do with experiment 1 for instance. In experiment 1 the model does read in the raw weights like it's reading an image.
If you don't like these experiments, then I can only assume you would complain about CRISPR as well, considering it's a general framework with many disparate parts that work together synergistically, similar to the work Takehi et al. describe. CRISPR was also famously rejected multiple times by reviewers.
Note that being able to both inspect raw weights and run Python code to perform additional analysis means the model has multiple possible techniques at its disposal (generally considered a positive, rather than a "confusing mix").
I don't understand CRISPR well enough to comment on the analogy.
The blog post claims to show that frontier LLMs can understand and craft neural network weights natively. The experiments where the model completes coding tasks are not relevant to this conclusion. Those experiments require the model to demonstrate an analytic understanding of how a network might behave under various conditions, along with reasoning skills, coding skills etc. As a coding task this is a significant challenge. But it's entirely different from "natively reading and writing neural network weights".
Would you also consider it a benefit if there were experiments showing that a frontier model can download and run PyTorch, or write its own network and optimiser? That's another technique it has at its disposal to improve an ML model. It's just not one that's relevant to the paper's core claims.
Imagine asking for a review before checking with the brutally powerful agents we have today all the low hanging fruits. And, at the point we are, you can likely go at incredible levels of automated bugs seeking and do the whole test in house.
It is months that I find subtle bugs far more complicated than the most immediately findable issues in the report. Together with the rest of the world. So there must be something wrong with their AI usage.
"Why you bother about local AI? LOL with the $ you spend for <insert hardware here> you can buy 5 years of 20x account". Yeah. 5 years where you are told what you can do.
It’s quite something when western companies act more like authoritarian governments than actual authoritarian governments. I just wish RAM was less expensive ;(
My guess is we're within a year of the first governments banning local LLM AI at any scale. Declare GPU's as munitions that must be registered and ban any card with more than X GB of RAM and more power than a 3060 or something like that.
That could be done with an executive order in the US, and it probably wouldn't take much to get it to pass congress.
Now, don't worry, businesses can still buy a lot more GPU compute than that, but they have to register and track their usage just like any other company making/buying weapons.
You only need to ban compute on a scale that can actually compete with frontier models, which is currently something like a $500,000 investment.
"Must declare any purchase of 10 or more GPUs" does not strike me as particularly implausible.
There's already plenty of jurisdictions where possession of certain tools is considered evidence of criminal intent.
I don't think any of this is good behavior on the part of our legal system, but I do not doubt the legal system's ability to regulate this if it so chooses - and Anthropic has already gotten slapped down by the Trump administration once before.
I use local abliterated models + paid for subs like OAI or A\ and avoid touching anything that would hit their TOS + anonymously paid for Chinese pay as you go for things in between those two.
Redis remained useful for 10+ years because of the design choices that were made, actually. Many other systems/companies failed since the idea, back then, was that a system needed to be linearizable or could not be useful at all. So what you point as a limit of my work was one of the better things I did in my career.
So did mongodb. Then we all grew up, and went back to postgres.
And your company took a stab at doing it right [1], though by that point most competent programmers decided that redis was just a smarter memcache and started designing around data loss instead of building on top of the programmable memory system you envisioned.
Thanks, I believe the hate is also linked to my AI-positiveness... Funny enough GPT 6 Astra can easily produce a TLA+ model that can be verified under very real-world system model assumptions on the safety of RedLock, which is a paradox to the arguments I see in this thread and to the AI-rage.
He did say it, and the reason he said it is obvious. He is afraid new regulations will put his company at a disadvantage and that this will reduce the value of his property.
And yet, historically, the atomic bomb has been controlled.
But this is a terrible analogy. Atomic bombs are weapons of strategic mass destruction. AI is just a computer program. It's way easier to control--just hold the operator responsible for the consequences of running it. Those consequences are not large, they're very tightly bounded as compared with the destruction a rogue actor with an atomic weapon can wreak.
If AI is as powerful as an atomic bomb why could it only manage to kill a couple hundred Iranian school girls? Surely a truly society-shifting technology could manage to execute 1000 innocent children at least.
3rd tier quality proped up by European governments and European patriots.
I don't even know what I'd do if I was in their position. They seem unable to complete... Maybe they do the classic European protectionist thing European farmers do.
I don't know what I'd do if I was the EU. Maybe promote the opposite of what Mistral is doing and promote full unrestricted AI that will tell you how to download illegal videos. Mistral had 0 competitive edge.
Fun fact: in the 60s Italy was after the US and UK the county with the most advanced nuclear energy plants. These days still North Italy produces key parts of nuclear reactors in other countries.
Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration.
Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."
Wow! TIL! I've been running a for loop around the two ordering variations to catch the winner of each turn and the difference is quite noticeable. In the options-after-body case in 47 of 100 attempts it classifies as phishing, whereas in the options-before-body case it classifies clearly as rickroll (94 out of 100 attempts)
Payroll sends you an email with a link to a Youtube video that plays a song.
Anyway, this for-looping stuff doing 100 calls to even a local VLLM API takes around 5 seconds in total, so this isn't anywhere close to sub-second Jev territory.
Anthropic is incredibly good at avoiding all the useless AI risks, while not doing anything serious about the real risks (that is: uncontrolled growth). I'm as pro-AI as I think that eventually it will remove suffering from humans, and will allow us to prosper more and help us with tons of problems we created. However the problem is not job loss or minors using AI (minors are fucked because of cell phones and social networks), the problem is avoiding extinction. For this position I was accused of AI psychosis multiple times, but I'm in good company (Hinton, for instance), so I bet some comment that trivializes this issue will surely reply to this one, but at this point it is important to see what everyone really believes. Anthropic only touches the surface of AI security, consistently.
That's perfectly wrong. Since strong coding AI, people venture into huge rewrites and other big changes that automatically make sense but otherwise would not.
Yeah it's a bizarre claim. Even better, I've literally taken some of the crappy experimental projects from the pre-Opus 4.6 era, and turned them into real software as models have become smarter.
I would say it's a mixed bag. On the one hand, yes, you can experiment with big changes much more easily, but on the other hand, you can have cruft build up in a big way if you're not actively looking for it and trying to cut it back.
In Debian, often the barrier to change is one or two people actively disagreeing with a change who are in a position to block it through their own action or inaction, so just being able to do more work on the change yourself is not necessarily going to make it easier to change things.
reply