My understanding, is that existing NCMEC policy includes non CSAM photos that were found alongside CSAM. I’m not talking about evil government attacks surreptitiously criminalizing your meme images in the US, I’m worried about boring dystopia where over time the false positive rate is much higher than anyone will admit and Apple employees are reviewing tens of millions of false positive photos at random from people’s iPhones a year.
And a more direct dystopia outside the US where they have already criminalized your memes. What stops China from saying we have our own database of images objectionable to the state that you must scan devices for?
This type of persistent, continuous, opaque scanning for "objectional material" makes prior discussion about whether law enforcement should have the ability to boot into a criminal's phone seem almost quaint by comparison.
I hope that there will be a transparency report for how many reports happened and how many were found to be false positives.
Regarding China, the answer is probably “nothing”, but the system as it exists is not amenable to what I imagine their goals would be: only monitors the photo library (not messages) and requires a certain threshold of matches. The question is, I think, what today prevents China from requiring devices to attempt to match a database of pictures with a mandated match algorithm and a mandated reporting server. I believe (admittedly without any knowledge of the matter) that the same standard would apply to both.
And a more direct dystopia outside the US where they have already criminalized your memes. What stops China from saying we have our own database of images objectionable to the state that you must scan devices for?