I've observed that there are some companies and some folks that are very fanatical about a specific provider. "Only Claude" is very common. "Only GPT" less common, but still see it.
A lot of the non-technical and semi-technical folks are only comfortable using a specific client like the Claude desktop app. A lot of teams wrote skills and eval'd/benchmarked their skills specific to a model family and harness.
The "I can use any model, any harness" or "I only use a meta harness" (Orca, Paseo, Herdr, etc.) crew is relatively small IRL. Most people are still single-session, single platform user mode.
Most orgs have an enterprise contract with a single provider so all of their internal assets are built for the provider. It goes beyond skills, but also the telemetry, enterprise policies, and managed configuration.
For a company though, the idea that a few disgruntled employees will shift massive purchasing decisions is a bit of a stretch.
We have a developer on our team that keeps asking for a claude code enterprise license, we keep telling them "no, it's way too expensive, get used to using the copilot extension/cli".
For consumers, sure, but the bulk of the revenue opportunity is enterprise. Anthropic/OpenAI have a hard row to hoe IMO.
As an only-clauder, there's another elephant in the room. Anthropic is by no means perfect, but:
* using OpenAI supports the Altman/Thiel/Musk/Ellison/Karp/Trump billionaire Orwellian worldview and also ads
* using Grok supports a ketamine-addled open fascist sympathizer jizz-spewing 12-year-old-edginess memelord and his similarly broken worldview
* Google's models are pretty good but every single product they have ever had is ultimately either suborned or killed by the advertising division
* local models pretty much only half-exist because Jensen Huang wanted to cheat on his wife; future is unclear
One can argue for harnesses and so forth being lock-in, but the ultimate lock-in is morality and wanting to have a civilization at the end of all of this; I can't recommend either enough
China is literally banning Nvidia chips, over 600K Nvidia chips sitting collecting dust in Hong Kong warehouses due to China blocking their import. The notion that open weights half exist due to smuggling or Huang is a sick joke.
Are API testing UIs really relevant anymore? I haven't manually tested an API for almost two years, and my APIs are SO much better. This kind of GUI is going to die over the next 5 years.
No, not really. It's already unit and end-to-end tested.
But sometimes the tester needs to make a test where they create something via the official API to use it in other parts of the app. They just open Postman (soon to be replaced), select the collection, select the server, select POST-item and of the the races. Not everything needs to be AI.
We could vibe code something specific for our needs, but that would be more or less again just end up something like postman again? At least I currently don't have the vision to see a big change.
I did, and you really should take a step back and ask an LLM how it can automate what you're doing manually as described above. You'd be surprised what's possible now.
I'm thinking the same thing, I haven't have a need for a tool like Postman in a long time, prompting is enough at this point. I wonder what features others use from Postman that an agent can't do quickly.
Quickly eye-balling onprem responses, skipping over UIs and just eyeballing data in general is really useful. Curl can do it but with a couple headers it becomes messy, also RPCs, websockets, etc. It’s just convenient to have a single place where you can eyeball all comms.
This in fact makes me wonder if there is a good “operations center” not just “dashboard” but for all the dev things, websockets triggering, db stats, running queries etc. We have many semi-custom onprems, they do not behave properly as a fleet I cannot just act like they are cattle. These servers contain many products in many configurations. It’s all quite messy..
I find the CPU/disk space dashboards useless. Stuff like monit is useful but status of literal OS services is only part of the picture. There are many layers here and I sometimes wonder how others do this. Surely not all companies just have disposable fleets of “serverless” things?
You're not wrong - I bought a Bruno "Golden Edition" for $9 two years ago pre-release... and... I just don't use it. I'm not even sure why I keep it around, at 524.5 MB
That's an interesting point. I wonder if over the next X years consumers will free up ALOT of disk space by just using agents and not needing to download and install desktop software.
Full disclosure, I'm building a desktop app called DB Pro [1] which is a modern desktop database client.
We're still doing well, and selling a lot, but we're having to think about what the next 2/3/4/5 years look like...
Heyo! I'm Jeff, one of the co-founders here. Mira itself is for code review and does no e2e analysis. We actually automate your live web app or mobile app, so we're catching a whole class of issues that are just not possible with static analysis (is the button cut off in the UI? does your AI agent respond correctly? does the video render correctly? etc.)
But to your general point, this benchmark is showing we are finding 3-5X more bugs than the best OSS performance out there right now. We did also try it on tester army e2e and it underperformed Codex + Playwright MCP (this will make it in the benchmark soon).
The "better" comes from custom agents and massively parallelized infrastructure. I'm not sure how many folks want to or even can self-host fleets of GPUs, browsers and Mac minis. We're happy for folks who can but our current offering is for those who want to find the most bugs with the least amount of effort.
Have you used Copilot at a company?
I can literally change my model in 3 seconds.
The moat is as shallow as it gets. Investing in a single LLM provider at the moment would be a mistake.
reply