Hacker Newsnew | past | comments | ask | show | jobs | submit | codedrift_'s commentslogin

> automate a complex debugging process that would take me about 1-2 weeks of painstaking work before

This is the QoL improvements people should be talking about. Yes, I can build a custom harness to exercise this really obscure bug. Or the LLM does it while I go do the harder part of telling the PM why this is going to have a blast radius that requires another round of testing (which can also take a week or more).


I see it as more of a spectrum.

Like, maybe untyped Python is good enough for you with pylint or another linter that helps the code me more idiomatic. Or maybe you reach for ty because you want to have some type checking to avoid common coercion bugs.

And then there's this, for when ty simply wasn't enough and you've inherited some monster codebase where absolute rule over types is required. It's probably overkill for most codebases, but I can think of a few legacy codebases that would benefit from this...


> "Done does not mean DONE"

Ya know, respect. I don't feel like a lot of people who went the vibe coding route have had that moment.


Yeah there's so many "bad habits" you can stop via a gate. My go-to is static analysis tools in the tree-sitter family.

I am currently testing a "too simple" gate. Claude loves to turn everything into helper functions, and if the helper function isn't actually helping (too short, doesn't perform enough actions, etc) the commit gets bounced and Claude gets told why it's wrong and what to do about it.

This has generally kept the worst of things out of the codebase.


> For some reason I can “hold it right” and others cannot and for the life of me I cannot figure out what it is.

I don't think we talk about this as much as we should. I've been interviewing devs as part of [DAYJOB] and there's a lot of people who cannot get anything useful out of Claude. Forget features, we're just talking bugfixes.

I also don't know what's different about the behavior. My best guess is people are overprompting, adding a bunch of noise that interferes with what they're trying to achieve.


> skill behavior changes across different models

This has bitten me so badly SO many times. Every time there's a new model, it's time to re-evaluate the skills. It got so bad I put a lot of the skill behind a CLI.

Evals saved me, but I really wish skills had a `writtenFor` field in their frontmatter so an agent could warn you it was running a skill written for a different model.


Some of my current toolchain:

- I'm currently using Sentry's dotagents to share skills and pin versions

- I avoid a centralized "single brain" and instead have started a "knowledge" directory in each repo. I copy relevant learnings in so stuff stays isolated

- I use an OSS tool from my company to turn anything that could be enforced by code into something that can be enforced by code (keeps the markdown files leaner)

- My daily driver is cmux with a silly amount of sessions, so I'm constantly throwing away sessions and keeping context small


My biggest problem with that setup was to keep skills etc in sync across projects. That’s why I started using https://yarkr.com to distribute and synchronize them automatically.

> the more ignorant someone is, the faster they will produce

It's a lot easier to produce "correct" code when they don't have a reference point for what "incorrect" code looks like. The optimistic side of me believes we'll discover new ways to scale websites because of this knowledge gap. A lot of cool tech was built in the 2000s precisely because we didn't know how stuff scaled yet.


> To achieve this, we require that packages are built on a trusted CI/CD platform

Given what happened with NX [1], I'm hoping GitHub Actions disallows certain types of commands in their YAML. Otherwise we still have a straightforward way to attach provenance to malicious code. =\

1: https://x.com/adnanthekhan/status/1958722939534417989


How is this better than deterministic codemods?

Next 12 to 13 (with app router) is a struggle for even seasoned teams. Not looking forward to AI handing off a 162833292 line PR.


Great question! So our AI agents actually use a combination codemods plus generated AI results. We use static analysis and codemods as much as possible, but there are lots of situations where AI is the best tool. We have found that AI is particularly good at transforming EXISTING logic from one state to another, in a fairly predictable and consistent way, so long as the code transformations are individual functions or code files.

In combination with our RAG approach, you will find that if you run the same module multiple times, the generated results are incredibly similar with very little variation. Give it a go for yourself! You can try it for free on codebases up to 2MB, or use any of our example repos.


But I can’t ask the AI why it renamed all my files to mediocre 80s bands.


Hah yea. But, we use RAG to ensure that the choices made are really good. LLMs on their own can't be trusted, they are just great communicators. LLMs combined with reputable sources (like documentation and code examples) provide really great results! And these data sources actually can be queried, as they are shipped with each module.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: