The article's description gives some idea, but I presume you saw that: "the 'just two more things' tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself."
Sounds like he ran into automated yak shaving. But it doesn't really explain why he couldn't fix it at the harness level.
Speculating, if you're trying to build something automatic, then you want constrained responses from each task you assign the model, otherwise you can get an endless explosion of work. The "Change 'Add to Cart' to blue" challenge parodies this: https://opusfived.dev/
Tangentially, reading the rest of that post gives me the impression that the author might benefit from an intervention. "AI psychosis" seems like it could be a relevant label here.
The article's description gives some idea, but I presume you saw that: "the 'just two more things' tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself."
Sounds like he ran into automated yak shaving. But it doesn't really explain why he couldn't fix it at the harness level.
Speculating, if you're trying to build something automatic, then you want constrained responses from each task you assign the model, otherwise you can get an endless explosion of work. The "Change 'Add to Cart' to blue" challenge parodies this: https://opusfived.dev/
Tangentially, reading the rest of that post gives me the impression that the author might benefit from an intervention. "AI psychosis" seems like it could be a relevant label here.