Usage does not correlate with valued output or ROI. This article smells like a bad attempt to say hey guys our customers see ROI without any actual evidence that they do.
Also, somewhat amusingly, the “founder” consistently being at the top of the use curve might just have something to do with everyone else also using it, but that more implies people are using it because the chief at the top wants them to use it… not because it’s actually useful. A pattern that’s typical of bad AI deployments.
> Usage does not correlate with valued output or ROI
This sounds possible but how can we know? It could just as well correlate. Effort of all kinds correlates with success even when it's not obvious or not... linear.
If that was true then there wouldn’t be a giant ROI crisis fueling calls of an AI bubble burst imminent. If the answer was “the ROI is clear” then folks would just say that.
The debate about ROI isn't about whether everyone is going to throw their hands up and stop using AI tooling - the debate is about where that value is going to be captured. It's pretty clear that using some degree AI assistance for programming is on the whole faster and better than writing code by hand for the majority of applications, and it's clear that the degree to which it's better than writing code by hand is increasing quickly.
I'm sure someone is going to reply to and say "actually it's not clear and the tech debt is about to explode everything real soon now" or something to that effect, but it's hard for me to explain this theoretically when it is so obvious in practice in my day to day.
ROI doesn’t mean “does it help.” Few question if it helps.
ROI is about how much its costs vs the value of the “help.” That’s where the present crisis is. Is the ROI there to pay for the trillions in commitments that have been bet on that ROI? That’s a clear no at this point hence the growing panic.
> Is the ROI there to pay for the trillions in commitments that have been bet on that ROI? That looks like a clear no at this point.
That's only a potential crisis for those who have made concrete investments.
On the use side, the 'cost' of AI spans more than two orders of magnitude. Looking at recent models (<6mo) with reasonable performance (intelligence index >= 45) on OpenRouter, the output cost ranges from $50/MTok (Fable) to $0.153/MTok (DeepSeek Flash 0731).
From the perspective of a user of LLM/agent assistance, there's very likely a range where the benefits outweigh the costs.
If the ROI for the model developers isn't there, then that just impairs the future trajectory of the field. Current models are just bits that aren't going anywhere, and as long as they can be served (in inference) above their marginal cost they will continue to be so-delivered.
On the value capture question, it's really not clear that the ability to write code at superhuman speed actually translates to an increase in the speed with which we can create new, better technology products that customers will pay more money for, vs. just enabling layoffs.
You could say the same thing about compilers versus assemblers, high-level languages versus low-level ones, and services and libraries versus monolithic programs.
All other things being equal, increasing the speed of some part of the development process will increase the overall pace of development. However, By Amdahl's law that increase will be sublinear, and that is why we should take "pull requests" as an imperfect metric.
We also don't get to pick the form that 'better technology products' take. While we'd probably like to keep cost(/effort) and complexity constant and increase robustness and performance, the market equilibrium might be 'worse is better' and reward whiz-bang features and lower effort.
Also, large companies are still experimenting on how to integrate LLMs into their workflows. Due to the fast cadence of releases, people forget that LLMs became robust (regardless of the capability level/parameter count/data size) enough to use semi-reliably in company-specific ways only 1 year ago. And bigger the org, the slower the process. I don't expect it to settle and get productive used across the majority of very large companies for another year atleast.
High prices are mostly a result of DC capacity. As more and more DCs get built out, prices will drop. At a unit level the inference business is extremely sound regardless.
> You could say the same thing about compilers versus assemblers, high-level languages versus low-level ones, and services and libraries versus monolithic programs.
Right, but we didn't invest a trillion dollars of capital into any of those things in the span of a couple years, thus forcing them to capture value and show a return on such a massive investment.
If you make an effort to use the new tools effectively, the gains are wild. Don't fall into a grumpy luddite trap, the train is leaving the station and you'll struggle to catch up if you don't learn and grow.
Not using AI for software development in 2027 will be like only knowing how to program via punchcard in 2010.
It's new. It's different. It's hard. It's your job. Learn how to use it effectively, embrace the new abundance mindset.
Children will jump on any train, that's why I'm in computers, and why there's an AI cheating crisis right now so severe that nobody is learning anything in schools.
> If you make an effort to use the new tools effectively, the gains are wild. Don't fall into a grumpy luddite trap, the train is leaving the station and you'll struggle to catch up if you don't learn and grow.
Correct, in fact you can tell most AI detractors aren't thinking about this from a _computer science_ point of view. You'll have many people claiming "why prompt an LLM when you can write it and commit it yourself in 45 minutes?" True, you could. But that's a purely _serial_ workload. LLMs are effectively data-independent and they can.. work in _parallel_. Meaning instead of working on a single task, you can juggle multiple simultaneously. So even if you're as fast or faster on a one-on-one basis, you _cannot physically_ be faster than 10 separate sessions running in parallel, working on wholly separate tasks. And that's actually where most 10x gains come from. See Amdahl's/Gustafson's laws.
His comment sucks but it is indeed a skill issue. Not the skill of "agentic coding" or whatever but the skill of having and communicating good ideas. A lot of engineers/people are bad at that, and are struggling to produce anything of value with AI coding.
Also, somewhat amusingly, the “founder” consistently being at the top of the use curve might just have something to do with everyone else also using it, but that more implies people are using it because the chief at the top wants them to use it… not because it’s actually useful. A pattern that’s typical of bad AI deployments.