Stretching nothing, but encompassing a fairly standard payload. “The crisis of men is they don’t trust us, the authorities.” No need to pin down why. We’ve poisoned the well so the article is done.
This is logical, but it’s also a lemon market. You don’t know the author, editor and publisher did anything of the sort, unless you’re the sort of person who doesn’t need the book. But if you do see an LLMism, it’s a strong sign of cost-cutting.
If done right you couldn’t tell either way, and you probably wouldn’t care. The point of writing is to get a message out to the reader as efficiently as possible, don’t waste their time. The writer ensures that, AI is just a tool at that point. The problem with vibe writing with AI is that AI has no sense about wasting the reader’s time.
We should just be judging writing by its quality. There were always bad writers who wrote like LLMs or worse. It takes a lot of practice to write well.
Nothing has really changed: bad writers using AI will still be bad writers. Good writers using AI, with some effort put into learning how to use it, can benefit a lot from it in terms of (at least) writing good content faster.
Look, you're must not saying anything. "If done right ... you probably wouldn’t care". Yes, correct. It's done bad, so I don't like it, and the visible slack is a sign of slack. Would you like to push back in some way for some reason? What's the call to action here?
My pushback is that AI can be used for writing, and the results will depend on the person using it. It’s like a type writer doesn’t make anyone a great writer either, but it improve their legibility at least.
LLMisms are almost universally results of using a narrow set of mainstream offerings from Anthropic and Openai. Even Muse Spark has style much less "sloppy" than Claude et al, let alone Chinese LLMs. I mean yes, they have their own tics, but if you don know them you won't even notice.
But I do know them, and so do you, and they’re signs of cost cutting partly for the reason you mention. They’re the careless, basic “no-brainer” choice of models, plus a lack of editing, or a very light “edit” in which obvious, even famous tics are left in. (Many real editors would clean up writers’ tics.) So it’s possible care was put into it, but tics visibility is evidence against it and not evidence for it. If your text does pass and educated readers don’t see it, it’s a consequence of care and intent, which is what we’re really asking for.
Especially with the current rate of progress, you asking ChatGPT today probably means you'll get a way better answer than whatever the author got using a model from last year.
This, and the fact that GPT is writing for you personally. A text written to persuade a publisher, or to manipulate Google results, has the deck stacked hard against it vs. asking even a cheap AI that can size you up personally.
This goes back far, and I’ve even seen what appear to be shills for random B-list celebrities. Add SEO business as usual, political campaigns, international influence and the SnR is just radio static.
Their lack of self reference is a core problem that undergirds a lot of faults that do occur during inference, but their breadth + the agent harness successfully covers it well, so it requires a bit of poking to witness. The “hallucination” phenomenon is exactly this. They don’t know the scope of their own knowledge, and they just say stuff, so if you go out of band, it has a higher probability emitting claims that aren’t true. RAG (I don’t mean embedding indices, but any information ingest such as an agent harness executing a search) are somewhat effective in covering for it, enough to make them very useful! But when it does go wrong, it’s generally the same reasons. It has a certain nature and sometimes you run afoul of it.
From your comment, I noticed a sort of pattern that is often seen when discussing LLMs. That they are these amazing things that can run so fast they trip themselves in their attempts at achieving a task. So we resort to refining the models, creating guardrails, orchestrating harness, so as to alleviate the 'hallucination' problem.
In contrast to human intelligence, there is an underlying mechanism that propels intelligent behaviour. A person is no less intelligent just because they lose sight, sound or inner voice.
“trip themselves in their attempts at achieving a task”
I see this a lot in Claude Code. I assume it has to do with the training structure.
Example is “fallbacks”. Claude constantly sprinkles “fallbacks” in the code, even when I ask it not to. That is, write multiple candidate implementations into the same code with some kind of switch.
This is a problem because you only need one, and it would seem to have inflated the code for no reason. (The madness accelerates with code volume, so you must push back.) anyway, few people would do it this way.
But I thought, what could be the benefit?
If you’re being conditioned to pass evals one-shot with code that will be discarded and never read, it’s a great strategy. If you have more than one way to solve it, you can just put both. The behavior would easily be reinforced, if trained that way.
But in any case, again, a certain nature and certain conditioning.
I think we’ll learn to accept it as AGI but also that no intelligence is fully divorced from context, limits and conditioning.
They symptoms sound a lot like humans, so I don't see how it stems from their lack of self reference. Most people you need to keep them in areas they understand or they go to pieces. The lack of self reference just means every time the context clears they reset. They are systems in a permanent state of extreme amnesia.
It’s really not like that. If I ask you to tell me about a geographical place I just made up, you can trivially and generally instantly recognize you don’t recognize it. A child can do this. You wouldn’t be able to hold a job or generally get through life without this level of self awareness.
I had a great game as a kid where me and girls were playing for kisses on "words", i.e. "start the next word with the letter of the word I just said". So I invented a whole new vocabulary and then taught them that vocabulary. Apart from the fact that I got a lot of kisses from them they were so happy they learned something new they immediately went to share this with their (and my) parents. Obviously I got a slap but I just don't get how you can think "a kid can do this". You're either completely ignorant of different cultures (i.e. as an Eqstern European I still find it hard to remember Indian names, let alone tell if they are actually fucking with me) or intentionally simplifying the problem.
No, I’m referring to something else. Not the next word task. Being able to know whether you know something. Next time you encounter a child, ask them something they’re likely to be ignorant of, and see they’re able to say “I don’t know”.
Current frontier models are pretty good at telling you when they don't know something, or if you're asking about something nonexistent or nonsensical. Not always, but there has been massive improvement on this front lately.
If your last experience with frontier LLMs (read: not models that Google and ChatGPT are giving away for free, but models you have to pay for) was over a year ago, you may not realize that.
So can LLMs. I asked one about South Wollopop and it suggested that it'd never heard of it but maybe I meant South Wollo in Ethiopia. And that's just a local model with modest hardware and no internet access. First attempt.
I mean, if our position is that a hyper advanced statistical model is going to ultimately struggle with the concept of something being unlikely to be true then the statisticians may as well give up in despair. There is no theoretical obstacle here.
If you hate AI writing enough, this turns AI filters into a kind of humiliation ritual. AI will derank normal business writing for human readers, and uprank inflated, verbose, tic-heavy slop. So you have to put the heavy slop out with your name on it. Really perverse moment.
disagree that there is one kind of ranking and one kind of engine analyzing that ranking; sort of de-facto true that one company does run the ad world; strongly agree that this is a nightmare possibility and directly dystopian
“let the embedding be a latent space that forms naturally when training the network”
The embeddings are produced in concert with the network, to serve the network, and not created as a separate step.
It’s actually very cool
The look-up table is a matrix. Each row is an embedding and each row number is a token ID.
You get a differentiable transformation from token ID to token embedding using a “one hot vector” and a matrix multiplication
If you take the transpose of this matrix, you can convert an internal representation back to the same token form, but treat it as logits and give it to the sampler.
So token embeddings are produced on demand in service of the model, according to the model’s needs.
I found an example of this strategy in a paper as far back as 1980!
In the other reply I recommend the Bengio paper. But do bite the bullet and try it.
Embeddings are there to make continuous-space identities so you can run them through a differentiable model. Without this the tokens (no matter your granularity) are pure surrogate identities, and you can’t run a gradient through them. You also hit the curse of dimensionality hard because the model can’t perceive similarity. “Cat” and “kitten” for example are simply different atoms of text, but with embeddings, you can leverage what you learned about “cat” when you encounter “kitten”. Look at “A Neural Probabalistic Language Model” (Bengio, 2003).
You can actually rig up an embedding variant of a Markov chain with just a few tokens of context, and no position coding, transformers, attention, none of it, and only minutes of training time. As long as you have the embedding lookup table trainable it will do some neat stuff.
reply