Sometimes, but I think there are a few problems with this:
1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it.
2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look.
3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.
1: There are corner cases, no doubt, but the main source of information should remain near the code.
2 & 3: How to be sure that the agent will pick the "append only hint" from the memory? Especially when it starts growing and cannot be part of the context?
We should publish some stats. The fix rates on bugs are around 45% baseline, increasing over time as we learn from behavior within a given codebase. The merge rates for the codebase health work we do (e.g. deleting dead code) are very high, at least 90%.
Agreed that domain experts should be making these kinds of decisions, and the right way for an agent mistake to get caught is going to depend a lot on the cost and probability of the mistake. My only point here is that the agent side of the system needs to learn. The software production machine needs to improve over time.
I looked for an explanation of what the tool does on my behalf on your site but didn't see anything.
I guess I expected on the homepage or maybe "About" but I was looking for something related to whether you open PRs on my behalf given that OAuth prompt.
I think adding that or some explanation during onboarding about the permissions might help.
That's good to know, but I would still suggest an on-ramp that only uses GitHub for authentication (i.e. no permissions needed). To that end, it would be nice if I could also authenticate with other OAuth providers instead, like Google, etc.
Again, I understand that this would limit me to scanning public repos, but that would be fine.
Other auth providers for sure. We'll be adding shortly.
Using an alternate auth provider won't even prevent you from scanning non-public GitHub code. There's a GitHub OAuth App just for auth (which is what you're seeing here), and a separate GitHub App that you need to install either way to give Detail access to the right repos. We can swap out the former for Google/Okta/pw if you want to avoid this warning. GitHub Apps (the half that manages repo access) have a much finer grained permissions model.
Thanks for your reply. As I said, on your website there is no address, there is no legal entity name, there is no company registration number. You could sit in north korea for all I know.
Now I spotted in the last sentence of your "about us" that "We're based in SF". Oh and only now I see on the "terms" page has "15. Contact information
qqbot, Inc
3624 16th St
San Francisco, CA 94114
Email: support@detail.dev"
Why not put that address into the footer or add an imprint section to the website? It's such a quick win to establish trust. Also if guillermo rauch is an angel investor why mention him at the last sentence of the "about us" page and not in the middle of your landing page. Why did guillermo not post a testimonial that add to the landing page? Did he not like the product? Or did he not review the product?
PS: When I search for "qqbot" on kagi a lot of chinese-language results show up. Is the company affiliated with china?
Sorry for challenging you. I wish you good luck if your claims hold it is a worthwhile effort.
We've run it on a few firmware repos and gotten good results. A lot of firmware code tends to have really poor type-safety which means lots of low-hanging bugs.
We should be able to handle cross-compilation. Want to try it? Ping me in any direct channel (dan@detail.dev / @danlovesproofs) and we can keep an eye on your repo.
My immediate personal use case would be C# on a self-hosted Gitea instance.
Realistically, anything paid would need to be fully self-hostable, though. There's a bunch of Java codebases that I work on that would benefit from something like this, but they're all behind two or three layers of Citrix...
1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it.
2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look.
3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.