I have recenctly tried to move my codebase more to spec driven development, but I aborted it. For my use case it felt wrong. It just creates drift and more prose explaining what I need.
I’m not saying I’m right, but I want to understand. What’s the benefits of raw specs with AI, when AI can read the source of truth and explain to you?
The thesis is that you can come up with a tight spec that defines what correctness means for you. Then the AI writes lean proofs that your code follows this spec.
In practice, for real systems, this much moves source of truth, as you have pointed out.
However, there is a clear 'compactness' advantage for specs.
Specs can either define the operational (what it does) or the denotational semantics (what it is) for a system, while the code can only be operational. Some things are just easier and better defined in denotational semantics.
Who's paying for the tokens, and since all inputs and outputs are logged "for abuse and safety" what is stopping anthropic from identifying which account is running the daily tests once it gets popular, and A/Bing it into a different slot?
But if the intelligence has been calculated once it can then be utilized for the future. It might still make sense for cases in which you are looking for breakthroughs. I would assume the intelligence is gonna trickle down.
Very large codebase modernization/conversion to newer libraries for interop and compliance reasons. Fan out only works if you have enough to do without the agents stepping on each others work. It is amazing when you can do it. Conversions make it even easier because you have a baseline metric of tested functionality to meet. We did pull off mass fanout on an ambitious greenfield internal tool that went very well. It feels like my entire job now is figuring out how to parallelize as much work as possible for the agents.
I’ve written about this approach to and it doesn’t work the issue is that even if the information is presented perfectly you don’t know how many iterations it might take.
Some information is better presented in artifacts, but some is better presented in text, f.e. abstract concepts.
You will run in the same issue even if the handover is optimized. I’m still trying to solve it for me, but perfectionizing the handover is just a part of it.
I’ve even had an ochestrator sessions to prioritize by a mix of easiness to approve/priority.
I think it’s better to focus on doing the right base line tasks instead of continious duct-taping. Clear resonsibilities, built workflows and all the good patterns that keep the amount of tasks and complexity down. The worst thing to happen is that all the agents overcomplicate your software.
It’s an interesting problem. And i have following thoughts to it?
- Are all cases even decidable?
- Shouldn’t one false positive rank higher than a hundred rank negative.
-Shouldnt’t the AI just abstain from decisions to avoid false positives.
- Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?
- You should definitely clean up deterministically first.
I feel like if you really want to solve this problem, you would approach it differently. This seems like a wrong approach and a bad article on top?
> -Shouldnt’t the AI just abstain from decisions to avoid false positives.
Yes, and I think regulation should be quite strong in this regard.
> - Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?
For larger contexts I worry about agentic confabulation, but humans unquestionably look at the gestalt when doing matching. I don't want idiot clankers making enforced decisions because they have misunderstood one field.
> - You should definitely clean up deterministically first.
If you can't do field validation on entry then I'm not sure deterministic rules work better, they may be worse, although easier to reason about.
But surely there are some deterministic reasons to clean up.
I personally wouldn’t lean towards LLMs on identity resolution. I would like give it the dataset and then ask me questions about specs? I really struggle to find the LLM part reasonable here. I mean potentially some typos, or nicknames, but those could be potentially resolved deterministically also.
I would probably have reviewer running that suggests new rules that I could implement. To me this seems more like compiler work than LLM work.
its the upper bound to value. Net worth is calculated as market cap of your assets, but look at Elon Musk. Right now there’s not enough people to buy his net worth his liquidation potential is drastically lower than his net worth. Ergo it’s a lie. Some people understand that, most don’t. It’s abused and marketing in some cases.
the market cap is described in USD, but don’t get equal value of it and you definitely don’t get more.
If somebody would buy it for more, he would have already.
If somebody has a car and he gets 30_000 USD for it, that’s the value of it. If somebody wants a cart and he can get it for 30_000 USD that’s the price.
How do you plan to estimate value if you estimate it at the price somebody might pay for it in the future. It’s proofing too much.
I’m aware that buyouts happen at a premium, but I state that this is not a valid argument to conclude market cap => value. If that statement would be true. Every share would be worth infinite money, as theoretically a buyer could show up and bid for infinite money.
Nobody said market cap is > value. You seem to misinterpret basics. Buyouts happen at a premium because you can't buy all the shares at last trade or current offer or anything of that nature. You can't sell them all at last trade or current bid or anything of that nature either.
So, we are left with the last trade, or the mid point of the bid/offer, as the best proxies. It is what it is.
I’m not saying I’m right, but I want to understand. What’s the benefits of raw specs with AI, when AI can read the source of truth and explain to you?
reply