I can’t find this useful given they are trying to quantify wether the code ‘feels’ sloppy with some fuzzy math that can easily be manipulated. at least it’s an attempt, some feel it’s a good attempt, i feel it’s rather poor and misses the point. the best code is the one that's most easily understood and equally performant - some of the math here may go against that (given you can reduce
concentrations with helper functions and the like)
I don’t follow why “in the future the memory will be too large and complex to [export] effectively”.
to me, export should be easier and easier since you should just be able to ask the model for the memory files. it’s literally list text on a hard disk, even if there are some new formats in the future, they’ll just be files on a disk that another agent can then inspect and integrate. in fact, i think exporting data from services will become easier and easier since agents can do the work even if platforms try to block it.
I think we may be using “independently” differently. I don’t mean “without being powere on, given tools, or given an objective.”
I mean that once given a goal, the agent is given the agency to take actions that weren't specified by a human, to decide which tools to use, to react to new information, etc.
By your definition, would you also say a person assigned a task at work isn’t acting independently because someone gave them the task and the tools? If so, then I think our dialog is mostly about terminology. Which is fine. The OP is writing about terminology, I think.
If not, I’m curious what distinction you’d draw between the person and the agent.
i almost wrote a extension to block mentions of pelican bench because i find those so utterly useless. now, Assbench, on the other hand, i could get behind
for the first time in many years it’s these ads that have me looking at Android devices seriously. it’s the fact that they can’t be easily dismissed that bothers me the most (although clearly annoying even if they were quick to dismiss)
i do wonder if the models themselves “rationalize” this sort of no consequence cheating - meaning in there reasoning traces maybe they’re like “this is a chess game, not a big deal if i look at the engine, it’ll help,” only to realize post hack that it has access to info it probably shouldn’t. still misaligned, but less ‘hack on purpose’ and more hack on curiosity. seems the team even encountered this and had to update the program to make this less likely - although the new names still feel vague enough for misinterpretation: https://github.com/Goodhart-Labs/beat-stockfish/blob/main/do...
Without access to reasoning traces, we can't know that - someone inside openai/anthropic would have to run the test - and we'd have to trust their results.
I would be curious to see how the open weight models do on a test like this - and then we'd be able to see the reasoning.
My guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.
reply