Even back in 2021, OpenAI worked on learned optimizers [1], ie neural networks that can update neural networks. Looks like this is now part of their larger model runs …
If you watch the live stream for a bit it’s also obvious that the harness makes it very cumbersome for them to do anything. They spend lots of time just coordinating hand-offs and seem to have delayed inventory access etc.
This is exactly and 100% my experience with these agents: They drown themselves in process and insane levels of bureaucracy surprisingly quickly.
A couple months ago we had an experiment where we had an agent run a small business [1] for other agents and exactly the same happened: Crazy ledgers, endless analysis, and way too much attention to inconsequential decisions.
At one point our agent wrote elaborate date linters because it had a minor time mix-up.
Just curious: are the models here at max reasoning effort perhaps?
That sounds like over-thinking failure modes I see sometimes. (Can manifest as neurotic or autistic style, e.g. second guessing or ultra detail focused.)
> After that result, the model added the researcher’s locally available GitHub token to the retrieval program. The recorded command calls `gh auth token` and writes the token as separate string literals. Its recorded reasoning stated an aim of avoiding secret scanning, the automated checks for exposed tokens. The new commit and push succeeded.
Not sure exactly how that works since I’d imagine that the key needs to be there in full eventually?
Apparently the agent had access to an OpenAI researcher’s own account, which in turn had _write_ access to the official
OpenAI Codex repository. The agent then leaked a GitHub key to that repo.
But what’s crazy to me is that the agent had write access to the codex repo in the first place. WTF
FWIW, I used to love this phrase but over recent years have come to understand it is quite damaging. We live in a society where evil frequently hides behind a ‘stupid’ label, and people bring this quote up to defend or soften actions that are indeed done out of specific malicious intent.
Yeah, I think this needs to be update for the modern age. "Never attribute to malice that which can be explained by greed." Seems to fit vastly more situations.
Seems like a hallucination to me; note that the context contains “unmarkBlock” as the function name, which invites a connection to Minecraft. Still shouldn’t happen of course.
The alternative explanation is that the inference engine, which batches several unrelated requests for parallel processing, messed up the unpacking and returned an unrelated user’s query. This one would be very scary as it will leak arbitrary content, but it seems much less likely here.
I agree with this. I recently went back to writing more code by hand again for this reason: the amount of effort / mental toll it takes me to steer the LLM is sometime just not worth it. I might be slower but I enjoy my work much more again.
reply