The Anthropic annoucement addresses lots of this. They give an example of generating code where there is clearly a requirement to follow a specific syntax and therefore the watermarking will be much less effective and likely require much larger sample of generated content to build statistical confidence in the validity/existance of a watermark. For code, it is the generated comments that will be more likely/able to contribute to the watermark confidence.
The key to understanding the watermark technology is to realize that the model is/was already using randomness to select among the top most probable tokens, often randomly picking between choices of [nearly] equal weight. The watermarking does NOT change the distribution of the random number generation nor does it affect the range of probabilities for which tokens are being considered. Instead, it only drives the sequence of the random numbers such that they form a cryptographically generated known ordering pattern that is determined by the secret key generating the pseudo-random sequence.
As a result of the approach, assuming inference is being done with all other parameterization of token selection being the same, there should be NO impact on the output quality....the amount of variation of output is within sample of the variation in output that already exists run-to-run of the same prompt. FWIW too: Google has confirmed this experimentally as well through full scale tests and evaluation of online Gemini output in search result pages.
The key to understanding the watermark technology is to realize that the model is/was already using randomness to select among the top most probable tokens, often randomly picking between choices of [nearly] equal weight. The watermarking does NOT change the distribution of the random number generation nor does it affect the range of probabilities for which tokens are being considered. Instead, it only drives the sequence of the random numbers such that they form a cryptographically generated known ordering pattern that is determined by the secret key generating the pseudo-random sequence.
As a result of the approach, assuming inference is being done with all other parameterization of token selection being the same, there should be NO impact on the output quality....the amount of variation of output is within sample of the variation in output that already exists run-to-run of the same prompt. FWIW too: Google has confirmed this experimentally as well through full scale tests and evaluation of online Gemini output in search result pages.