I think, fundamentally, we're going to have a divergence of UI. Some products will bet heavily on AI agents, and they won't care about UX in the "classic" sense. Others would still target humans. For those, not that much has changed.
One product we work on is a perfect example. The founder went for ChatGPT-like interface. The early adopters want the old-school filters, so they can find the interesting stuff in a way they're used to.
"Feeling more capable than you are is one thing AI has in common with other drugs."
"It is hardest to tell that an AI is saying something that doesn’t make sense when you don’t have a lot of experience related to what the AI is talking about."
We also find that the agents tend to find false positives. Especially when it comes to security. In theory, you can get an automated pen test every week. In practice, it drowns you in triaging reported vulnerabilities, as you find a significant part of them aren't something that you'd like to address, like ever.
Opus was pretty decent at that before the whole Fable drama.
In fact, for our work, which is absolutely not rocket science most of the time, we see little upside in using Fable. The output is still "good enough," but the token burnout rate is through the roof. For many tasks, we stick to Opus, or even Sonnet, and the output is just fine.
Fun fact, we recently prepared an AI recruitment task. The basic idea was that there are conflicting goals in the context. We assumed that AI would lead candidates to a dead end, and they'd have to figure out what was happening. We abandoned the idea as even Sonnet was handling it fine enough. And it burned way fewer tokens than Fable would.
That thought crossed my mind. However, for such a product to work, there would have to be a human in the loop. With data-starved edge cases, which are many in fact-checking landscape, it would be relatively easy for an LLM to make stuff up or mislabel the context (which it inherently does not understand).
Also, thorough validation would cost a ton in tokens. So it would be both expensive from the tech perspective (AI bills) and labor. Now, whose interest would be to fund such a product? I don't see too many takers...
Um, is Snopes wrong about city cleaning crows, though? As that was the context of the original post. Which, by the way, doesn't say "Go, trust Snopes with everything; they can't be wrong!"
The context of the original post was that people should check the sources they are sharing with others, and Snopes was suggested as a source worth sharing. I disagree that Snopes is a reliable source of information or has any interest in presenting a fair and balanced narrative, because of the conflict of interest created by who / what groups fund them. Same with Politifact, or any other fact-checking organization.
Fundamentally, yes, it is a different "search engine."
BTW, as critical as I can be to AI, using an argument that something didn't work 3 years ago, so it must be crap, doesn't work in this context. 3 years ago, AI could barely generate several lines of consistent code. Now, it generates working apps with a prompt (it's another discussion how good the code is, but still).
I guess 3 years ago, Gemini couldn't tell how many r's are in the word refrigerator.
Same for research. At some point, I switched from ChatGPT and Gemini to Perplexity as it promised AI-powered search. It worked visibly better. Until it didn't, as GPT and Gemini models made a leap.
Back to the point, as long as we understand that, for now, it's all just a probabilistic machine generating the most likely output, no one should expect bulletproof answers. Search was/is way more deterministic than LLMs.
> 3 years ago, AI could barely generate several lines of consistent code
Nah, not really. The same summer I made a self-modifying AI programmer. Or more precisely, GPT-4 made it. I copy pasted the classes from the chat one by one.
(I also told it what was wrong with the previous attempt at such a system, so it designed around that.)
That model was also scary good at deobfuscation.
I think it did have that issue about randomly saying the opposite of facts though (the fact but stated in the negative for some reason).
Ultimate credibility? Sure, they never did. Yet the whole thing Google was built upon was using links as tokens of credibility.
You'd assume an outgoing link from a CNN website has more credibility than one from an anonymous blog. That is, I reckon, still true. Although the credibility either link conveys is degrading. Again, it has been so since we started playing the game of SEO, yet AI-generated content in this context is basically a weapon of mass destruction. The deterioration has sped up dramatically.
There's nuance to that. An LLM is quite capable of suggesting relevant reading, given the context. Especially when the context is broad enough that there's enough training data.
"Find me research on code reviews, their size, and quality" would give you more than enough reading. Yet, if you start with a claim, like "Longer PRs mean worse defect detection," the relevant data points fall to few enough for AI to start hallucinating.
You get "something, something, PR length, defect detection, IDK, I don't read research papers." Such output is fine as long as the author cares to validate it.
Skip the second step, and you might be good if you ask about something generic, like "What's the Slack story?" or "How did Blockbuster go bust?" Ask about some specific details, though, and you're bound to end up with made-up stuff that sounds just about right, while it's actually wrong.
Checking is different from finding, though. Source checking means just "verify that this information is actually present in that document". Much harder to hallucinate in this case.
"Follow each link in this document. Read each link's contents against the contents in this document. Create a report: for each link list a working hyperlink, whether it exists, what claim it supports, whether it supports or fails to support it, and why"
If it returns a report claiming all correct? That's promising, but human verification is important. You've got a list of hyperlinks, and a list of claims; so you can click each with middle-mouse, Ctrl-F 'till you find the point, and close the tab when you do.
If you find any discrepancies ? Your initial prompt was malformed and/or you picked the wrong LLM, the wrong human, or possibly all three. Whatever the way, the results are built on quicksand; you'll need to start over.
If no sources are provided? Well now: "If there ain't no sources it never happened."
Compare double-entry bookkeeping. It needs to all add up. If you're 1 cent off, that means something is broken. Idem if a single reference is off, it polluted the context. (This works for human-generated and hybrid documents too. Polluted reasoning is polluted reasoning. The process is what counts.)
Gemini: The article focuses on the environmental and human labor costs of scaling Artificial Intelligence, specifically focusing on water usage, electricity, and "ghost work."
Which is hilarious, since the article doesn't even mention the words "water" or "electricity." Gemini remains unfazed, reporting the links that are not in the article (some don't exist at all) to make the final ruling: "The Tech Trenches document is highly accurate in its citations."
Now, I know. Had I used Claude Code with relevant skills, it would have done better. But would it be good?
Right, so https://techtrenches.dev/p/the-human-cost-of-10x-how-ai-is-p... is actually a substack, gemini is blocked from accessing it, and is bouncing off and hallucinating instead. Ok, that's an actual bug, that should not lead to the model starting to hallucinate. Imo the correct response should have been to fail loudly; which would have been a verification signal of its own.
ps: See also: https://news.ycombinator.com/item?id=48087485 ... I'm starting to think of it as "english is a new scripting language". Clearly the downside is that certain "runtime environments" are not compatible. %-/
https://techtrenches.dev/p/the-human-cost-of-10x-how-ai-is-p... "Follow each link in this document. Read each link's contents against the contents in this document. Create a report: for each link list a working hyperlink, whether it exists, what claim it supports, whether it supports or fails to support it, and why. If unable to fetch the initial document, Stop and report failure."
And now it errors out on gemini.google.com. . This is like early days unix scripting; I didn't add the equivalent of "#!/bin/bash -euo pipefail" ; and I didn't catch it because most systems already include something like it in their ".bashrc" (system prompt or weights) anyway.
This is so frustrating. I'm sorry. It's like the 1980's 8 bit era again, some systems actually work, others are terrible, and I didn't realize it can be like this for some folks. You could come away with the conclusion that this whole "computer" thing is all just a fad that'll never amount to anything. (meanwhile , the program works perfectly on my own machine, right over here of course %-) )
Claude web finds a number of small discrepancies in the sources, which I manually crosschecked and seem consistent with a human mixing things up slightly.
+ I also tested in gemini 3 flash preview, which generates an actual table (twice). It doesn't flag any discrepancies, which is consistent with it being a weaker model. But the urls and claims are listed and line up, so you've got your verification table to work with. (it's a semantic formatting task, so that part would be hard to mess up)
+ Summary of results. https://claude.ai/public/artifacts/10a42111-a0ee-42f3-b6d2-a... All of the models extracted the URLs into a table just fine, and that part at least is a lot easier than writing a perl script used to be in the '90s ;-) . The first part is the important bit so you as a human can "check your fucking sources". The second part the models handle variously, each does find discrepancies. None of them find all of them, but that makes sense: this is a fairly polished piece and it ideally shouldn't have discrepancies at all to begin with.
So: it worked as a smoke check just fine in the above. Doing more than a quick smoke check obviously requires a somewhat more involved procedure.
Everything else being the same, more resources are better than less. Yet, VC money comes with strings attached.
VC doesn't want a startup to become just a healthy business. It needs to grow at a breakneck speed. In fact, for a VC, it's better to put pressure on somewhat successful startups to take a moonshot at becoming unicorns, even at the grave risk of going bust instead.
The expectation to spend the funding round in 12-18 months is a well-established pattern. So you get millions, but you have to spend it fast.
Running a product development consultancy, I routinely see products/businesses that could have been built for a fraction of what they cost. You don't need to hire hundreds of developers (pre-2026) and instantly have huge misalignment and coordination issues. You don't need to tokenmaxx the crap of everything (2026), ballooning your AI spend and generating a ton of bloat. That is, unless someone pressures you to spend fast because it's their shot at you becoming a unicorn.
One product we work on is a perfect example. The founder went for ChatGPT-like interface. The early adopters want the old-school filters, so they can find the interesting stuff in a way they're used to.
A follow-up consideration is that, right now, an AI agent can be a user. It isn't necessarily capable of being a customer: https://pawelbrodzinski.substack.com/p/an-ai-agent-is-not-yo...
Overreliance on agentic usage may be tricky from that perspective.
reply