Inspired by parameter golf and speedrun approaches I make the case for picking loss functions like a wallclock for LoRA on AI safety targets. The result when I tried it was a functional distillation of an Sparse AutoEncoder into a 5.3MB probe. I have a technical writeup below about it if anyone is interested.
paper looks nice! i think what they found was that they can recover the input sequence by trying all tokens from the vocab and finding a unique state. they do a forward pass to check each possible token at a given depth. i think this is since the model will encode the sequence in the mid flight token so this encoding is revealed to be unique by their paper. so one prompt of 'the cat sat on the mat' and 'the dog sat on the mat' can be recovered as distinct states via each token being encoded (unclear mechanism but it would be shocking if this wasn't the case) in the token (mid flight residual).
anyone who spends 10 years writing performance reviews and rebuilding a service they already rebuilt last year, all to collect RSUs, was never going to land us on mars
Well don't do it and instead of using an off the shelf library that is known to work while the rest of the development team isn't reinventing the wheel.
Doing it for fun and education is fine of course.
I am an AI interpretability researcher and have a new proposal for a way to measure the per token contribution of each head and neuron in LLMs.
I found that the normalisation that happens in every LLM is avoided by modern attribution methods despite it having a large impact on the model's computation.
Here is the full preprint paper and the code I used. https://github.com/patrickod32/landed_writes
Happy to some insight from any interested people and would like to know if other people here have been working on anything similar. This seems like a real gap in the research to me.
the average human certainly can not run that fast. treadmills in gyms usually cap at 20km/h and how many people on the street do you think could handle it? (very few of course)