Same old story. Criminals will find models with no limitations. Law abiding people will run up against the limitations when trying to defend themselves against criminals. Same reason that encryption back doors are a bad idea.
Isn't kernel development one of their banned topics in their public facing models? The ethics of this are just so inherently dirty. They can enjoy the benefits--the sum value of all that compiled knowledge--but not us. If open models don't succeed, the future is looking pretty bleak. Really. This normally feels like such a tin hat thing to say, but damn if it doesn't feel true right now.
That's because that's how it is, and not only that, they can use whatever un-used compute to do it. If they're at 30% capacity they can have 70% of max data center resources do all the things we can't do, with the things we can't and don't have (because of price fixing and circular economies).
Honestly it's probably just a matter of time before we all have a tiny little Rasp-Pi looking thing that powers AI locally, bring back self-hosting IMO after seeing the Office 365 stuff down.
You know damn well they have teams working on some crazy stuff behind the scenes. We're just here to fund them by buying tokens.
> Isn't kernel development one of their banned topics in their public facing models
I've been doing custom HIP kernels for a while now without any issue whatsoever. I guess it depends what you're trying to achieve? If Claude recognizes that your goal is password cracking or key breaking, pretty sure the session will be flagged, but I've been playing with custom HIP kernels to run open-source/open-weight models without any issues for a while now, even on the 5.5 series models.
> Isn't kernel development one of their banned topics in their public facing models?
I see various fable/opus-vibecoded "kernels" (and even benchmarks) of various flavors just searching for "fable kernel" or "fable kernel gpu".
The barrier to trying this sort of thing yourself is prob more the token consumption * API prices (and expert oversight?) required for a serious project than the security/CYA restrictions.
People constantly hit security guardrails doing kernel and inference engine development. Not too hard to work around them, but its very frustrating. Not to mention the publicly released Fable 5.2 intentionally nerfs its kernel optimization knowledge. Meanwhile, Mythos can write Anthropic engineers whatever kernels they'd like.
Honestly the rise of Chinese open models is the biggest stroke of 'dumb luck' (strategy, whatever you want to call it) that I would say we've ever had in the last 100 years or so. With how locked down the internet has become now, you'd never be able to source a corups of enough training data today.
My example case of not having an open model, is if I had to write half the software I do now, without the aid of agents. It would be beyond impossible to keep up with anyone.
Granted reading code and tests is boring and painful, but it's still much faster than having to write the code, and the tests, then read it all!
That’s why I really don’t care about Huang allegedly helping smuggle chips. Fuck US AI hegemony. I don’t want a handful of shitty American CEOs controlling the world. The future envisioned by Liang Wenfeng is so much more compelling.
Apples and oranges, really. SHA-256 is a fixed function with no memory. So it lends itself to being hardwired into pure logic. A CPU or GPU was absurdly wasteful for it. That's why ASICs saw like 100x gains over general purpose hardware. Most of the repeatable tasks in AI workloads, like matrix and vector math at reduced bit widths, have already been encoded into hardware in modern GPUs.
I called this in the comments of the post yesterday. Astra understands mechanistic interpretability literature and linear algebra and Python. So go figure what it would do with that knowledge.
Couldn't this just be a result of Astra training on massive amounts of circuit theory or mechanistic interpretability literature or other related topics? In other words, not black magic?
>Couldn't this just be a result of Astra training on massive amounts of circuit theory or mechanistic interpretability literature or other related topics?
Maybe, but it's more like Astra has been trained on enough practical linear algebra problems to be good enough at "eyeballing" how adjusting some weights will result in different outcomes (among other things).
To put it another way: a human can learn to do what Astra is doing here with enough time and assuming the weights are small enough to be practically comprehensible. I know I've gone through the practice of whiteboarding the training of simple neural networks on a whiteboard to understand how they work, so it's not an absurd concept in itself.
The scale of this, though, is certainly extreme and definitely not something that I would have thought any publically available model could handle at this point. It's worth noting, though, that there's virtually no way these frontier labs haven't been exploring this exact concept for a while now.
>In other words, not black magic?
It's just math. Extremely complex and impressive math, but we're still just talking about matrix multiplication under the hood, and none of this suggests anything otherwise.
There is obviously a mechanism to it. Astra isn't magic its an LLM. With perfect interrogation into its training data and weights you could reason about how this happened.
However, this doesn't mean its not magical when someone shows a next token predictor hand tune the weights in Qwen to make it generalize on math.
This is legit the stuff of science fiction if you presented it to mechanistic interpretability people a year or so ago.
Unfortunately, I'm a bit concerned that cool labor of love hacker projects like this will become much less common now that Anthropic (and possibly soon others) is moving to restrict this kind of usage. Oh well, so much for cool fun stuff.
Recently I did something similar (or even more labor-intensive). I recreated an existing game from scratch by manually rewriting its code without even thinking about LLM/"AI" tools usage. (see my submissions).
But I agree, that using an "AI" tool may be a serious temptation and thus way less new porting projects will be done with such care.
Most of the new game {decompiling, porting, modding} efforts use Codex anyway. And if they lose that they can switch to Chinese models that can get the same work done (because the results are easy to verify), just take more time to get it right.
The first project like this (of using AI to port old software from assembly) was interesting and novel. But every subsequent post has me wondering what new insight or lessons we have learned and if doing more of these is worth the energy burn.
I could write a well crafted reply, with a convincing argument, but why bother? You're just trolling. The whole anti AI coding thing is such a meme. I suppose you'd like to spend all the time it would take to reverse engineer a classic NES game by hand? Go familiarize yourself with 6502 assembly and everything else you'd need to know?
I would rather read a write up of someone doing just that. This exact thing has been done many times now and no one is really learning or demonstrating anything new. It is cool, insofar as AI tools are capable of reverse-engineering, mechanical translation, and automatic verification, but it is no longer _novel_ and borders on spam.
What if you've already played that game many other times during your 15 year long career? I mean the "game" of learning the ins and outs of a technical subtopic. And you know you could spend months ramping up on another thing just to see an idea take shape, but you only have limited resources in your life, a full time job, and so forth. And you'd still like to see an idea become a reality regardless of all that. So I guess that's not real work and it's cheating and it doesn't have value?
I mean, if you insist on doing everything by hand, go right ahead.
Afterthoughts: I really just don't get why people are so weird about this AI coding thing. What difference does it make to you if someone thinks they have more to learn from high-level planning work than from low-level coding work? A lot of the time, I get the vibe from people who talk this way that they are still early on in their career and think they have to prove their coding chops. I think that's why people like Linus Torvalds couldn't give a shit if people use AI or not, as long as it's not slop and suits its purpose. If you've already spent over a decade writing code and savoring the process, maybe you're just getting a little tired of it. And maybe agonizing over code perfection even drives you slightly insane sometimes. Who the f*** knows? In any case, it's none of your business. I don't know why I'm even writing all this. Because I know you'll just come back with the same old tired gatekeeping non-arguments.
It's not vapid. Half this web site at this poing is people saying "I-i made this" when AI made it for them. HALF. Nobody cares, it's old news. These stories are not relevant to tech, they're not thought-provoking nor is there anything to learn from them. It's about as cool as reading about some MIS directing a code sweatshop to make his or her latest web site.
reply