I love that it has a feature to prove images came from the real world:
> Users can now prove the authenticity of a photo taken on iPhone 18 Pro models with Apple Reference Image, powered by the new sensor in the Main camera that can sign every pixel it sees. When a photo is taken in the new Reference mode, the camera captures signed sensor data that Private Cloud Compute develops into an unalterable reference image.5 Reference images can be viewed in the Photos app alongside the main image, like a digital negative, to visually compare the two assets and determine if any edits were made. APIs are available in iOS, iPadOS, and macOS 27 for third-party apps to enable viewing of these reference images.
It would be great if they could use the LiDAR scanner to sign a depth map of the scene in front of the camera. It would show if you photographed a flat image.
The sensor already uses phase detection autofocus. You can create a depth map out of it. iPhone X used its dual cameras and that phase data for depth maps.
How? Is this for every image taken, or does one need to take a second photo? Or is it "in theory phase detection hardware would let you make a depth map"
The sensor has special pixels which can detect phase differences in incoming light. Camera uses these pixels (aka embedded detectors on the image sensor) to be able to focus where you want. This is same from phones to professional cameras.
(DSLR cameras have independent phase detection arrays. This is why the mirror has a small mirror behind. To illuminate that section).
Lenses are tuned. They know how distant the thing they are focusing on, also photos' EXIF generally carries the focus point information. Side note: Apple Aperture used to be able to show where you focused with that info. Still no app does this. I'm still mad that we don't have Aperture anymore. Anyway...
So, if you collect this phasing information alongside the photo which you're taking, you are capturing the depth map of the photo. Its resolution will be lower, but not lower enough to be useless.
For example, Sony (and most probably all other big camera manufacturers) cameras use this phasing data throughout sensor in real-time for following moving subjects to keep the focus on them by predicting where they are going.
So, there's no "in theory". The phasing data is the depth map. Otherwise your camera can't focus on anything. It's there to focus, and it's done by reading the phasing data and knowing where to go by how much.
All fast AF systems are PDAF. When light is too low, then it's CDAF, which does way slower, without using any phase/depth data.
Images that are meant to be 2d (E.g. a photo of something on paper) isn't going to look 3d. I guess this approach will just have to let anything flat through?
"I took a real photo of this image printed on paper" seems possible to falsify, yeah, but that doesn't seem too surprising to me, given that anyone with a printer can print literally any possible image.
How would that work? I thought the premise here is that you can fool the apple camera by taking a (very carefully aligned) picture of a still image (printed out).
A depth map from the apple camera (again, signed) would show that the entire image had the same distance from the camera.
The photos are cryptographically signed in the apple image pipeline so it's not as simple as just AI generating something. That said, I can't see how this is any different to all the other times we have embedded crypto keys in consumer hardware where eventually someone finds a way to extract the key and the whole thing is busted open.
I think there's a chain of trust. The sensor signs raws, and the private cloud takes signed raws, does minimal processing so they're at least coherent, and re signs that output (maybe even including the original signed raw as well in the image file).
How would the server know that the request is coming from a real iPhone?
This is a pretty standard application of trusted computing and can be done entirely on the iPhone. A server would only possibly be needed for anonymization (while retaining key revocation capabilities if a key does end up leaking), but there are serverless ways to do even that (TPMs have supported these for a while now).
It wouldn't, but you could validate that a particular picture was created at a particular time, and had not changed, for example, especially with metadata that you may not want to share but that establishes certain parameters like gps coordinates. A lock, rather than an end-to-end pixel signature, which shows what was contemporaneous rather than exact provenance. If an event happened on day 0000-00-00 00:00:00am, but your photo was taken at some other time, it casts doubt.
I think a big part of validation for things like these are just "could it have been modified since Z event happened", because Z was not something people paid attention to before.
That's just a timestamping service then, not a content provenance/authentication scheme. Timestamping has been a solved problem for years; certificate authorities offer this, or you could just throw a hash onto any sufficiently trusted blockchain.
Nothing prevents anyone from opportunistically pre-generating and timestamping millions of permutations of fake kompromat and then selectively revealing the one that turns out to be useful after the fact.
You could charge per attestation, but the economics of that don't look great; you could demand publication of the image itself before attestation, but that would obviously not fly for most use cases out of privacy concerns.
> Nothing prevents anyone from opportunistically pre-generating and timestamping millions of permutations of fake kompromat and then selectively revealing the one that turns out to be useful after the fact.
If you're doing server-side timestamping and someone is sending millions of items, I think you just ban them. Apple accounts are free but not inexpensive.
That's a good point, You might still be able to trick the cloud to sign your photos, but that's something they could patch in updates without losing control of the key. They could have the server only sign photos taken on the latest ios version.
And honestly you could have a similar antitampering oracle that was at least obscured, in terms of "we've detected tampering but won't tell you how or why", which is frustrating but I have to imagine that 99.9%+ of images are clean.
... don't make it so shallow then. Perfectly possible in a consumer-friendly 3D printer for a face or even human body if you have lots of time for the prints.
You’re well on the way to 1:1 replicas at this point. Next you’ll need to match the thermal signature and the exact weather in the sky for the time at that location.
An important distinction but I have taken pics of pics, and sometimes its hard to tell I didn't take it directly. So now, its hard to tell I didn't directly take the photo AND its got the stamp of approval.
The credibility comes from who took the photo. The next logical and easy step is for this metadata to flow to the user agent. I'll know that it was taken by a legit journalist photographer. And id not, I can have my user agent make it fuzzy or replaced with
a kitten photo.
I think this is where we are headed. This feature seems useless on it's own since someone will eventually find a way to extract the key from the iphone and sign any image. But if they could make it so every iphone uses it's own key and the photos show "Taken by xyz" and it's linked to their icloud or identity somehow. That way images shared around will still be able to be linked to a source that you can choose to trust and the viewer can know it wasn't modified from what that person shot.
Then we might move to an age where photos which were not signed by someone will be treated as fake.
You may have forgotten that the world is driven by screenshots, and so a screenshot of a picture with the `(X) Verified by Apple to be Joe Schmoe` tacked on will be just as good as real verification for a lot of people.
What we need is almost something like the classic "press C+A+D to log in" or the idea of "above the browser pane"... something an image cannot show you unless it's legit. Perhaps a personal thumbprint icon that is different for each viewer, so you know its your device telling you that something is real and not just some mock fake thing?
I don't think we'll treat them as fake but they would probably undergo more scrutiny - perhaps in a crowd-sourced manner that provides an overall score of probable legitimacy.
People have constructed video walls so high resolution they can film TV series in front of them and to an audience it's indistinguishable from a real set. It's extremely cost-effective for things like space fantasy that needs lots of exotic-looking backdrops, apparently.
It may not be in reach for you and I - but within reach of anyone with the budget to run a 'bot farm'
The high resolution video wall is quite simple. The hard part and why this has not been done earlier is that for filming, they had to synchronize the rendered background with the movement of the camera.
Movement is solved by tracking the camera's position and projecting the background from there.
Proof of captured image with hardware-based-attestation can be increased by adding extra information besides the RGB channels, like depth mapping (which a projected screen wouldn't be able to fake), and eventually light field recording (plenoptic imaging, e.g. Lytro).
also no, even in the high rez video walls its still possible to tell. maybe to an "audience" no but there are lots of fundamental flaws with video walls that make them not the same as filming it for real. all of those flaws show up in the final image and could be detected.
things like color rendering, sharpness, screen door/morie, motion blur, and yup even good ole depth queues in the lens system all show up as artifacts.
case in point, outside of a few specific niche cases like the mandalorian, that virtual production video wall thing is not actually being used all that much because of the amount of post shoot cleanup required to fix all those issues, it wasnt actually that much cheaper and its not really better either. especially when you factor in how hard it is to shoot that way.
> things like color rendering, sharpness, screen door/morie, motion blur, and yup even good ole depth queues in the lens system all show up as artifacts.
Pre-AI fake videos sidestep this sort of issue by lowering the video quality.
Add some motion blur, some camera shake, some poor lighting, the camera being slightly out of focus, and make the video 720p instead of 4k.
> I love that it has a feature to prove images came from the real world
Congrats to Apple achieving nation-wide tracking, embedding some kind of yellow dots[1], but this time this time this is based on strong cryptography and non-removable, targeting people who don't use AI under the slogan "we are fighting AI fakes", but people still "love the feature".
On the opposite side, this read to me like there was so much image processing going on in the background and off device that they felt like they needed to prove that there was a real image backing the creation, and that it was not all AI. It really begs the question of what the extent is of image processing.
The iphone image pipeline is mostly just playing with exposure and color, things that we widely regard as fair game in editing. Where the line was crossed was magic eraser and reframe where the iphone is straight up making objects in the image.
That could be a result of a failed exposure stack. Phone sensors and lenses suck for light gathering so they have to take multiple photos and merge them together to create one that's less noisy. Occasionally you get artifacts where the merge failed.
It's still distinct from actual AI generation imo.
This is correct (from someone who has worked on software to do this). Processing images for the 3D walkthroughs in real estate sometimes does this too.
Reporters who only capture pictures on an iPhone and not a "real" camera, and who also never need to edit the photo to crop off or obscure identifying information, innocent bystanders/victims, gory violence or nudity, etc...
The problem is real, but this solution seems to not really solve it in any practical sense.
Reporters who only capture pictures on an iPhone and not a "real" camera, and who also never need to edit the photo to crop off or obscure identifying information, innocent bystanders/victims, gory violence or nudity, etc...
So, 90% of photojournalism outside of the major cities, then.
You want to do all that, then still can prove the picture is authentic. The editing history show what had been changed compare to the previous version, from the color adjustment to blur areas... it's not about preventing editing, it's about proving the authenticity when needed.
> The editing history show what had been changed compare to the previous version, from the color adjustment to blur areas...
That's not a thing. Straight from Apple:
"When a photo is taken in the new Reference mode, the camera captures signed sensor data that Private Cloud Compute develops into an unalterable reference image. Reference images can be viewed in the Photos app alongside the main image"
So this only helps prove anything if you can share the original, unedited image. You can share it alongside an edited one, but you must also be comfortable sharing the unedited original if you want to prove you own an iPhone 18 Pro. Er, I mean prove it's "authentic"
Now I think C2PA had allowances for what you're talking about, but that also ended up resulting in it being easier to break.
You don't need to share the original image with everyone, just like how journalists don't share their audio recordings from interviews with everyone. You share with an editor. Or a court.
Yes, which is why this doesn't really do much. Readers still have to primarily rely on trust for whether or not they believe the news and who is reporting it.
Canon's original high end digital models had this as a feature, that your images could be digitally signed in-camera based on raw sensor data. Primarily for law enforcement. Obviously a far broarder use case, now.
> Users can now prove the authenticity of a photo taken on iPhone 18 Pro models with Apple Reference Image, powered by the new sensor in the Main camera that can sign every pixel it sees.
Will this also work with third-party camera apps like ProCam?
Not available in EU. The reason Siri AI and other iOS features are not available in the EU is the issue of Apple not opening up the same API for other apps. Therefore I think it might be the same case here, which would mean that third-party camera apps wouldn't be compatible.
I don’t know too much about image processing, but my intuition tells me that distinguishing a photo of a 3d scene from a photo of a photo of a 3d scene is a much easier engineering problem than distinguishing it from an AI-generated diffusion. So even just reducing the problem to the former is still a worthy accomplishment.
Agree with the weakness, but the benefit of provenance is still there. You can't tell if the picture that person took is of an AI image, but there's some value in saying "I took this" and being able to prove it.
I hope future social media presents this metadata so if someone claims a photo is from the BBC, it actually gets checked against the keys the BBC sign their photos with.
Ideally it’d be linked to public key on a DNS txt record or similar, and trustworthy clients (including social media apps) show the domain linked to the photo.
Finally. Android phones have been shipping with C2PA since last year, so it's great that the final holdout has capitulated, though with its own set of nonstandard tools.
Indeed, on top of Google's implementation being a bad joke, it sounds like the C2PA specifications in general are a massive fail so far. https://arxiv.org/html/2604.24890v1
On the other hand, ignoring standards altogether is the wrong way to go because the ecosystem after capture won't be there.
Nah, it’s an optional feature, so by using it you provide express consent. GDPR gives users more data ownership; GDPR does not stop you from choosing to do what you want with your data.
It’s likely DMA — Apple would be required to provide APIs for signing for 3rd party apps too.
It also sounds pretty useful for anyone involved in a lawsuit to be able to present photographic evidence again. I’m sure GenAI is already a big headache for forensics, and enforcing AI watermarks will clearly not be bulletproof.
Will it ever work until a TPM like module is directly integrated into camera sensor? Older attempts of Nikon and Canon got broken, same had happened with C2PA implementation for Android - if it gets anywhere close to CPU it is insecure.
Yes, presumably there will be a cat and mouse game until the entire system including all peripherals has been brought into the trusted computing base, either physically or logically (usually with secured/authenticated communication). This has been the case for all applications of trusted computing.
Security is never a binary property, however, and can usually be better expressed in terms of how expensive it would be to subvert a given mechanism. "This image is either authentic or would have cost at least $x to fake" is already much more useful than nothing at all even without $x trending to infinity.
Cynical take: A single photo to now consume up 50mb of cloud storage, adding to the demand for more data centers that supported the creation of this authenticity issue in the first place.
How will this work against rehosted images on all the platforms people actually share photos from? It’s a good feature but many popular services apply metadata stripping and basic image adjustment before resharing. That seems like the place where people would actually want to verify authenticity.
> Users can now prove the authenticity of a photo taken on iPhone 18 Pro models with Apple Reference Image, powered by the new sensor in the Main camera that can sign every pixel it sees. When a photo is taken in the new Reference mode, the camera captures signed sensor data that Private Cloud Compute develops into an unalterable reference image.5 Reference images can be viewed in the Photos app alongside the main image, like a digital negative, to visually compare the two assets and determine if any edits were made. APIs are available in iOS, iPadOS, and macOS 27 for third-party apps to enable viewing of these reference images.