Hacker Newsnew | past | comments | ask | show | jobs | submit | nutanc's commentslogin

I don't think weight space interpretability is the only thing. I think it's the fact that Astra(and apparently Opus 5.5) can directly read the weights and find patterns and make changes. Writing code for interpretability is one thing. I think this approach shows models now understand weights directly without tools.


Not quite true. In earlier examples (like Pong vs Pacman) the models are quite small and a lot of the topological structure can be read from the input weights. Whereas, in latter tasks, they give the model a harness:

> We give Astra a... Python session, like a notebook, with the model’s weights loaded. Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512)

It's interesting that they can now understand weight patterns, but not much more interesting than LLMs being able to read/write directly in Base64.

However, I think this likely indicates that OpenAI and Anthropic are now including model weights directly in their training - rather than an emergent property as the blog suggests. I think this is the more concerning finding - especially if doing so is in a bid to find methods for RSI.


I think it is cool too, but with a caveat. For the large model (Qwen), their Astra used a Python environment, activations info, and iterative feedback to find an edit, it is not the same as looking at just pure weight numbers.

And in small MLPs they used, the I/O representations are kind of made to be more understandable because they explicitly encode objects and coordinates for games. If you set it up like that you can see some patterns yourself in the first/last layers without Astra. It is very different from interpreting distributed, entangled computations inside a large model. So I think the paper shows promising capabilities in weight-space analysis and editing, but not a general understanding of large models from their weights alone.

I'd love to see us reach that point, I am for one working in this area, but I don't think these experiments establish it.


Would this means remotion etc are not needed?


This is awesome. I am trying to build a full scale ASR system within 20-25MB. Now that we have Claude code to run experiments, I have started running some experiments. Promising results so far. First realization is that you can capture the nuances of speech in just 3300 embedding vectors(786d). This sequence can be decoded with a small CTC system to get text. Next experiments are on reducing the 768 dimension space into a 64D space. Thats also show some promising results. Hooking up my system so that the agent blogs the results everyday[1]. So my research "claw" setup does the experiments and posts results which I check in the morning and adjust the experiment direction as needed. Its not fully automated yet, but almost there.

[1] https://blog.trulm.com/posts/speech-as-independent-parts/


I think Google's Conformer paper is SOTA at the <30M model size, where I think they put an incredible amount of flops into a 10M param model to reach around 2% lsc clean (the whole model and RNN decoder were trained domain specific to librispeech here).

I think my small Talon models are next, around 3% lsc clean at ~28M (greedy CTC decoding, no external encoder, no LM, not trained in a domain specific way). I reached around 6.5% at 10M.

I've been working on some new baselines I want to release soon as public artifacts. This article is inspiring me to try pushing the param size down a bit. I suspect we can do large vocabulary end to end in the <5M range.


You can get very close to a working version by reverse engineering the javascript with Claude. I got a good version almost an year back. With Opus it might do a better job.


Most of the problems happen because we want to simulate human conversations. While thats a good goal to have, another approach is to let the user know clearly they are talking to a bot. You will be surprised at how accomodating users can be when they know they are talking to a bot and want their queries resolved.


Lets break this down. There is very little in newness in what Anthropic announced. Claude had skills for a long time. They have added one more layer of abstraction and called it plugins. This mainly comes with a set of integrations.

Thats the pitch.

But, what are Claude plugins?

Plugins=Commands+Skills+Integrations.

Commands are specific to Claude code. But commands and skills are nothing but prompts at their basest level.

So what is the main differentiator?

Integrations.

But what are you integrating with?

SaaS companies.

And what is the stock market doing?

Dumping SaaS stocks.

How do they think Claude cowork will work without the integrations. Without the system of records.

If anything, these SaaS products have become more important. If I was a trading guy, I would go to the github of claude plugins, see the default integrations and buy the stock of those companies.


Claude cowork and the SaaSpocalypse case makes no sense. What are Claude plugins? Plugins=Commands+Skills+Integrations. Commands and skills=prompts So differentiator? Integrations. Integrating with? SaaS companies. And what is the stock market doing? Dumping SaaS stocks.


2 years since this started. The effort is to protect the creators from the AI machine.

The AI machine is different from the printing press. We think we need protections more than just the copyright. Can we have a copyleft for data?

In this age of AI, how do we protect the creators?


Chat GPT has ads now. This is a dangerous domain we are entering and society is not ready.


Organizations adapting AI is the biggest problem that businesses are facing right now. Even in Ozonetel I face this problem day in and day out. The employees who really use AI to its full potential are minuscule. I can count on my fingertips. We need to overcome this in the right way or we will face the same problems we faced during industrial revolution.


All they (management) had to do was _not_ shove it down everyone's throats. Now everyone has a gag reflex to it.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: