I agree with the substance of this article, and disagree with the author's reaction to it.
The part I agree with:
It is true that watermarking can be done by "just" swapping one PRNG for another, and it is even true that with today's LLMs, it is possible that this will not degrade the output. But it has a cost, and as things improve, that cost will matter. You are intentionally reducing the degrees of freedom in the output, and using those bits of entropy for a purpose that does not improve the quality. If you maximize your tradeoff of bits for quality, those extra bits lower the ceiling of what's possible. It's a very simple information theoretic argument, and the only plausible argument against it (that we're using those bits so inefficiently now that the new PRNG is no worse than the old) only holds in the short term.
I also agree that having TOS that forbid removing the watermarking is deeply, deeply problematic. Hell, the whole essay is well-written and persuasive, and gives good reasons why this is a poor approach.
What I disagree with, and the reason for this comment, is the entitlement.
> The idea that anything other than my needs should factor into the generation of text for me is patently offensive.
This attitude is what is patently offensive for me. This is the argument that the world is beholden to my interests. It says that worrying about negative externalities is immoral. It's another form of certain people being above the law, shareholder profit maximization über alles, might makes right, we have to do it or someone else will, "we just help people connect", {code,a gun,roofie} is just a tool.
So I agree that the watermarking has a cost. But you can't leave out that it is an attempt to reduce negative externalities of AI. Whether it's a realistic or worthwhile attempt is a whole other debate (and Gruber does a good job of debating just that in the latter part of the essay), but saying that the user's needs are the only thing that should ever be considered is reprehensible.
> This is the argument that the world is beholden to my interests. It says that worrying about negative externalities is immoral.
I agree with you, in that the benefit of the commons (preventing damage AI is doing to the world etc) is better than one individual's 'right' to a perfect product.
But I don't think Gruber is saying quite this. He's worried about genuine semantic and intelligence loss, using the example of swapping two words:
> The semantic difference between banana and pineapple is just as noticeable to the human eye as the taste of the two are to the human tongue
Further, is this really a useful way to reduce the damage of AI? It only appears to work if someone already believes and then proactively checks if text is produced by AI. It doesn't really address the core issues.
> So I agree that the watermarking has a cost. But you can't leave out that it is an attempt to reduce negative externalities of AI. Whether it's a realistic or worthwhile attempt is a whole other debate (and Gruber does a good job of debating just that in the latter part of the essay), but saying that the user's needs are the only thing that should ever be considered is reprehensible.
Computers are tools that exist to serve. Creating some bizarro process where we are compromising the technology in service of it's owner to achieve some nebulous goal is gross.
Anthropic is crowing about this achievement because they are afraid of the dirt cheap AI models coming out of China and eventually other places impacting their valuation. Full stop. There's some vague notion of preventing harm without any backing, but a very real cost for startups to develop a compliant watermarked AI model.
I agree with you as long as things are at a small scale. But at a large scale, these things reshape society and what it means to be human. Social networks started out[1] as almost wholly good. The negative effects came from scale. Same with advertisement-funded websites and tons of other things that started out as being overall positive for the commons and ended up being highly negative. You can't just stick your fingers in your ears and ignore what is happening in reality.
> Anthropic is crowing about this achievement because they are afraid of the dirt cheap AI models coming out of China and eventually other places impacting their valuation. Full stop.
It's the "full stop" that I'm disagreeing with. Yes, you can make valid arguments about the motivations behind this, or the effectiveness of it, or whatever. You can even conclude that it's a net negative. That's my current leaning. But "computers are tools that exist to serve" is an excuse, a conscious decision to abandon responsibility. Hammers, nails, social media, biological weapons, and date rape drugs are tools that exist to serve.
[1] Ok, fine, at least one started out as a sleazy way to talk about people's physical attractiveness and behavior anonymously with repercussions, in a way where the victims would have no meaningful recourse. For the sake of my argument, pretend we're talking about Friendster instead of Facebook, please?
I love your point about social media. IMO, initially they didn't know what they had with social media - it took the right kind of uninhibited sociopath to monetize/weaponize it. That's why the MySpace guy is traveling and taking photos while Zuck is building island lairs.
This is where a functional government would step in. There's no magic in social media technology or LLMs. I'd think of them as a car. You can buy a Nissan Leaf of a Porsche 911. One is an ok basic car. The other is an (over) engineered experience in the form of a car. You cannot legally drive a Porsche to it's potential on the public highways... because we have laws that regulate driving and hold the operator accountable.
We're thinking about Claude Code or Gemini or whatever. The people running these companies are like "I want to exceed the power of John D. Rockefeller or Stalin." If we or the EU are going to regulate AI Labs, you need to grab them by the throat and they should be screaming about it. They seem to be very pleased with themselves. Barring real regulatory teeth, we would be opening the aperature to the Chinese companies to rationalize the valuations.
The part I agree with:
It is true that watermarking can be done by "just" swapping one PRNG for another, and it is even true that with today's LLMs, it is possible that this will not degrade the output. But it has a cost, and as things improve, that cost will matter. You are intentionally reducing the degrees of freedom in the output, and using those bits of entropy for a purpose that does not improve the quality. If you maximize your tradeoff of bits for quality, those extra bits lower the ceiling of what's possible. It's a very simple information theoretic argument, and the only plausible argument against it (that we're using those bits so inefficiently now that the new PRNG is no worse than the old) only holds in the short term.
I also agree that having TOS that forbid removing the watermarking is deeply, deeply problematic. Hell, the whole essay is well-written and persuasive, and gives good reasons why this is a poor approach.
What I disagree with, and the reason for this comment, is the entitlement.
> The idea that anything other than my needs should factor into the generation of text for me is patently offensive.
This attitude is what is patently offensive for me. This is the argument that the world is beholden to my interests. It says that worrying about negative externalities is immoral. It's another form of certain people being above the law, shareholder profit maximization über alles, might makes right, we have to do it or someone else will, "we just help people connect", {code,a gun,roofie} is just a tool.
So I agree that the watermarking has a cost. But you can't leave out that it is an attempt to reduce negative externalities of AI. Whether it's a realistic or worthwhile attempt is a whole other debate (and Gruber does a good job of debating just that in the latter part of the essay), but saying that the user's needs are the only thing that should ever be considered is reprehensible.