> To illustrate, in the special case that GPT had a bunch of possible tokens that it judged equally probable, you could simply choose whichever token maximized g [a cryptographic function]. The choice would look uniformly random to someone who didn’t know the key, but someone who did know the key could later sum g over all n-grams and see that it was anomalously large. The general case, where the token probabilities can all be different, is a little more technical, but the basic idea is similar.
> instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI.
Seems like - given enough text to encode information into - it would be possible for OA to uniquely identify the user account (and maybe even the specific request) that generated some content, even if the chat text itself isn’t stored.
Interesting argument in favor of local AI as a mechanism for privacy-preserving generated content. Though, I wonder if there’s a way to “bake” a hardware fingerprint into local models as well…
> To illustrate, in the special case that GPT had a bunch of possible tokens that it judged equally probable, you could simply choose whichever token maximized g [a cryptographic function]. The choice would look uniformly random to someone who didn’t know the key, but someone who did know the key could later sum g over all n-grams and see that it was anomalously large. The general case, where the token probabilities can all be different, is a little more technical, but the basic idea is similar.