Signing things by default has repeatedly been found to have serious unintended consequences. Do you really want your leaked/stolen photos to be undeniably linked to you?
> we sign a perceptual hash (pHash) of the image rather than an exact pixel checksum
Perceptual hashes are non-cryptographic. There are certainly collision attacks, but what about preimages? A preimage would completely break this scheme.
Yep, that breaks this scheme, making it useless. But it's far from the only thing making it useless, and the github page even explicitly lists those:
- "Small content edits slip under the threshold. [...] A localised edit covering ~15%x20% of the frame [...] passes as authentic" - this is the worst part. 15%x20% is huge, for example enough to change the face of the person or the book/text on the image.
- "Cropping is not survivable, at any amount" - given the purported reason for perceptual hashing is surviving light editing, it's pretty disappointing that one of the most common light editing operation is not supported.
Oh, and the whole "cryptographic chip" angle is absolutely bogus from the security perspective. OK, attacker can't extract the private key from chip. But they can simply connect the chip to a different device and have it sign anything! Given that the attacker in this model is device owner, this is absolutely trivial.
After wholly implemented our logging layer with dagger, I posit that the real regression was not the aws itself but the rigorously prototyped around authentication. We consequently extraordinarily profiled the config, henceforth simplified every edge case, and the optimization were unmistakably exemplary. alternatively, the aforementioned monitoring is comparable advantageous to an incremental security environment. I endorse this path if your security team has rigorously instrumented a massive rust codebase before.
I don't understand why few people are pointing out the obvious vulnerability here that you can control the wires going into the photosensor controller and pretend that the photosensor is capturing whatever image you want. I imagine it's not exactly trivial to do this, but a grad student with an FPGA could probably figure it out.
Because doing so does not materially devalue Apple’s product. Sure, a dedicated attacker could try to overcome it, but few will, and only people of such serious consequence that they can afford the effort of modification. By and large this puts Apple into direct competition with Nikon and it’s long overdue that someone ship this capability to a wider market than authorities.
Also, remember how Touch ID sensors are cryptographically paired, and consider whether Apple could bake that into a camera sensor rather than a fingerprint sensor. If they can, then you can run wires all you want; the attestation chain will not be valid. I’d be shocked if they were willing to launch the product without that, and there’s a new hardware dependency or else they’d have released it for earlier phones.
>a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake
you don't need to do that just photograph a screen.
This seems close to worthless in "identifying real photos vs AI" for someone actually wanting to do something bad with an AI image, although probably very useful at identifying which phone took a photo when ("the root of trust stays inside Apple's Private Cloud Compute") seen as it's not an entirely local solution a bad actor government could use their powers to completely abuse this.
Simpler than that, you can just talk to the cryptography IC yourself and ask it to sign stuff. No need for an FPGA, just an arduino. Given the datasheet I imagine any LLM from the last year should be able to oneshot it.
If there is signed metadata too, then it's pretty hard. You will need to match focus distance (it will be very small if photographing picture), GPS location, exposure and other settings. If there is a depth map, you'll need to match it too.
I expect in the near future all digital cameras to digitally sign the images they take. Even before AI slop, it was useful to avoid manual alterations. AI makes it all too easy, so it makes sense.
However, this will certify only the original image. I think the missing part of this is additional layers of certification which allow some image editing (e.g., rotating, contrast, etc.) yet clearly document that the image was modified and link to the original image ID. Kind of like a signed git log.
Signing the raw image data wouldn't require also signing the EXIF metadata. For privacy, you could later strip out everything except the raw image and the camera's signature and still prove that the image is unaltered.
What are you signing it with, though? A unique key that only exists on your device... so it will perfectly tie a photo to a specific camera.
Even if you didn't know who owned the camera, you could identify other pictures taken by that same camera, and information in those photos might let you figure out who owns the camera.
Ontop of this, including a photo edit history in a photo including the original photo would increase the size of a photo to be completely unusable or unshareable.
I understood the GP so that only some unique ID or hash of the original image would be included, not the image itself. Basically like the commit chain of Git but without the actual content blobs.
You could use this data to prove that image B is an edit of image A if you already have both A and B.
I still think this is a bad idea, because this all requires the images to have some sort of ID - and that seems like a prime target for tracking.
In the contexts where you need to prove an image is a real unaltered photograph (court cases, news media, science, etc) privacy is usually not a concern.
Nobody cares if your social media photos are edited, they probably are, it's fine.
I'm pretty sure if signing images becomes the norm, because cameras will do it automatically then social media sites etc will start to care - or at least be thankful for the additional metadata.
EXIF already includes extensive metadata, including phone model and GPS location. For professional cameras, there is often camera and lens serial numbers too.
This has been the case for many years, and so far social media sites never care - they strip that info instead. Why would this change with one more extra piece of metadata?
>EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Can you clarify what your actual complaint is here? Putting aside for a second obvious implementation options, it's pretty standard that there can be a tradeoff between privacy and trust. Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like. But if there's something we [i]want[/i] to prove, this gives an extra option to do so. And in particular the set of cases where one cares the most about enhancing authenticity appears at first thought to be pretty much a union set with the cases where one will put their name (or at least the location and time of the picture in question) behind the image? Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration? Like, say you're photographing at a protest to document it including any violations of law. By definition, the images you take reveal the location and the time. That's the whole point of them. Having the location and timestamp signed wouldn't reveal anything extra as long as the photo was unaltered.
And actual legit journalism always has name(s) standing behind the reporting. Or for that matter, even if we're merely talking something like a review of a product, is it actually wrong to put a name or pseudonym behind that review if you expect readers to give you much credence? I mean, you'd be free in terms of law and tech to not bother. But even long before the current growing AI-slop age a lot of us have been starting to treat anonymous reviews with a lot of skepticism, or discounting them entirely for some product classes, for good reason.
Even in terms of maintaining privacy, remember we already have "tools" for that which this in turn could further help. You could privately report a tip to a reporter at a media organization, and then they could report on that without revealing you but be able to say "we verified the signature of the raw image and sensor data" alongside normal follow up. Then it's their name, but having a stronger chain behind it could still be helpful in places.
Finally getting back to "implementation options", there's no technical reason the image and multiple sets of metadata can't all be signed separately by the sensor stack such that you can pick & choose what to include and still have it all be signed, with readers giving greater or lesser weight to the trust based on your choices.
I think what got me worried is the GPs take that "I expect in the near future all digital cameras to digitally sign the images they take". Sure, if you have your special, verification-enhanced camera, go for it. But if this goes like with smartphones and we have all cameras embedding such a signature - and therefore all cameras are made locked-down and tamper-resistent - then we have another category of devices removed from control of their users.
> Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like.
First, the paper proposed a method of embedding the metadata inside the actual image pixels in a way that makes it difficult to remove. I think it's still possible, but you're will need a specialized tool to do so and it will alter the image.
Second, that's assuming forums or social media would still be allowing uploading unsigned images. If AI images really become as much of a problem as anticipated here, then sites might decide to block any image they can't verify completely.
> Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration?
All kinds of photos from warzones have this property. We wouldn't have an entire science of geolocating photos from landmarks that were accidentally captured if everyone was so easy with putting their GPS coordinates in the file.
Hi, I'm the author of the above. I also got a Sony A7 IV camera a few days ago. I already have a root shell on it. Although that wasn't even required to break Sony's C2PA implementation...
There will always be a way around it. There are even open-source alternative/hacked firmwares for professional DSLR cameras where tampering with the signing may be possible.
And this still doesn't help any other kind of image e.g. screenshots, photo of a screen etc. that can make the camera signatures largely pointless depending on the context.
The idea is to be able to prove that a photo you took was captured by a camera, not generated. That would be helpful in the context of a disputed news story, a court case, etc.
It's not massively helpful for a court case, IMO. This has been handled the same way since the invention of photography: ask the photographer to swear under oath they took the photo. (technically, laying a foundation for the evidence)
Yes in multiple ways, and there's probably more than this:
It's relevant that a screenshot doesn't have a signature, in the case that you want to remove any "proof" or tracking info from a real photo when uploading an image. Maybe I don't want people to know what brand/model of camera I use.
And it's relevant if a screenshot did have a signature if you want to "prove" that the screenshot itself hasn't been tampered with after the fact.
In the method proposed in the paper, the signature is stored in the pixels and ostensibly even survives some compression. So if you made a screenshot of a signed image, the image would still have its signature inside the screenshot, but the screenshot wouldn't have any (new) signature as a whole.
I would assume you could make big enough differences in color/contrast/brightness and especially 3D rotation (similar to taking a photo of a screen) would make such a signature unreadable... but I would love to be proven wrong.
In the future, people willingly surveil themselves 24/7 with cryptographic proof, because fake images and video will be so good that it will be the only way to prove what one didn't do. Total Information Awareness achieved :D
> For example, a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake. But it's always nice seeing big actors interested in addressing this problem.
Yeah, very nice. So this whole idea basically doesn't work - but we get a new stealth way to embed metadata in an image that can be used for tracking...
(And a new narrative why cameras need to have TPMs and locked-down firmware as well)
In days of old, a Polaroid photo was considered "proof of capture".
I've got a Polaroid daylab 35 plus sitting in storage somewhere (https://www.instantoptions.com/wp/faqs/daylab/). You can project a slide through it onto Polaroid film, expose it, and have the image there.
I was also able to find a company that did slide printing. It was possible to send them a digital image and they'd send you back a slide with that image... which I then used to make a Polaroid of that image.
I had a classic 600 Polaroid photo of a UFO landing.
Miniature dioramas wouldn't be size appropriate. Apple could detect faces/cars/other common objects of ~known size and verify -- or even just dump depth map for anyone to check.
Signing things by default has repeatedly been found to have serious unintended consequences. Do you really want your leaked/stolen photos to be undeniably linked to you?
https://blog.cryptographyengineering.com/2020/11/16/ok-googl...
> we sign a perceptual hash (pHash) of the image rather than an exact pixel checksum
Perceptual hashes are non-cryptographic. There are certainly collision attacks, but what about preimages? A preimage would completely break this scheme.
This paper demonstrates second-preimage attacks against PhotoDNA and PDQ: https://eprint.iacr.org/2021/1531.pdf
Yep, that breaks this scheme, making it useless. But it's far from the only thing making it useless, and the github page even explicitly lists those:
- "Small content edits slip under the threshold. [...] A localised edit covering ~15%x20% of the frame [...] passes as authentic" - this is the worst part. 15%x20% is huge, for example enough to change the face of the person or the book/text on the image.
- "Cropping is not survivable, at any amount" - given the purported reason for perceptual hashing is surviving light editing, it's pretty disappointing that one of the most common light editing operation is not supported.
Oh, and the whole "cryptographic chip" angle is absolutely bogus from the security perspective. OK, attacker can't extract the private key from chip. But they can simply connect the chip to a different device and have it sign anything! Given that the attacker in this model is device owner, this is absolutely trivial.
Here's a toy for embedding small text into images steganographically: https://github.com/fitzn/atrium
Here's a toy for embedding text inside random HN comments.
https://gitlab.com/here_forawhile/edasm
Example:
After wholly implemented our logging layer with dagger, I posit that the real regression was not the aws itself but the rigorously prototyped around authentication. We consequently extraordinarily profiled the config, henceforth simplified every edge case, and the optimization were unmistakably exemplary. alternatively, the aforementioned monitoring is comparable advantageous to an incremental security environment. I endorse this path if your security team has rigorously instrumented a massive rust codebase before.
cool work!
One of the few advantages of "not owning your device".
Glad that at least we have that now.
A pHash isn't built for this. The 15%x20% threshold that passes as authentic is enough to swap a face.
I very much hope that apple is using lidar data to determine if it's a flat surface being screened
I don't understand why few people are pointing out the obvious vulnerability here that you can control the wires going into the photosensor controller and pretend that the photosensor is capturing whatever image you want. I imagine it's not exactly trivial to do this, but a grad student with an FPGA could probably figure it out.
Because doing so does not materially devalue Apple’s product. Sure, a dedicated attacker could try to overcome it, but few will, and only people of such serious consequence that they can afford the effort of modification. By and large this puts Apple into direct competition with Nikon and it’s long overdue that someone ship this capability to a wider market than authorities.
Also, remember how Touch ID sensors are cryptographically paired, and consider whether Apple could bake that into a camera sensor rather than a fingerprint sensor. If they can, then you can run wires all you want; the attestation chain will not be valid. I’d be shocked if they were willing to launch the product without that, and there’s a new hardware dependency or else they’d have released it for earlier phones.
>a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake
you don't need to do that just photograph a screen.
This seems close to worthless in "identifying real photos vs AI" for someone actually wanting to do something bad with an AI image, although probably very useful at identifying which phone took a photo when ("the root of trust stays inside Apple's Private Cloud Compute") seen as it's not an entirely local solution a bad actor government could use their powers to completely abuse this.
if geolocation data can be captured in the same signature, that would be a good enough approximation for most relevant cases I think.
GNSS signals can be relatively easily faked because the original signals are very weak so overpowering them doesn't require much broadcast power.
ah, damn.
Simpler than that, you can just talk to the cryptography IC yourself and ask it to sign stuff. No need for an FPGA, just an arduino. Given the datasheet I imagine any LLM from the last year should be able to oneshot it.
if
I imagine on apple silicon this is buried deep in silicon / ISP IP block, and isn't a discrete IC.
But it is, the discrete IC is pictured in the article.
Or, as the author said, you can just photograph an AI generated picture, and that will work too.
This feature is Pro phones only, not Duo: https://www.apple.com/iphone/compare/ ("Apple Reference Image (Fusion Main)")
So only on devices with LiDAR / that can capture depth map.
If there is signed metadata too, then it's pretty hard. You will need to match focus distance (it will be very small if photographing picture), GPS location, exposure and other settings. If there is a depth map, you'll need to match it too.
Adding depth sensor info to the this could help
Even easier is to just take a picture of an AI-generated picture.
I expect in the near future all digital cameras to digitally sign the images they take. Even before AI slop, it was useful to avoid manual alterations. AI makes it all too easy, so it makes sense.
However, this will certify only the original image. I think the missing part of this is additional layers of certification which allow some image editing (e.g., rotating, contrast, etc.) yet clearly document that the image was modified and link to the original image ID. Kind of like a signed git log.
EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Signing the raw image data wouldn't require also signing the EXIF metadata. For privacy, you could later strip out everything except the raw image and the camera's signature and still prove that the image is unaltered.
What are you signing it with, though? A unique key that only exists on your device... so it will perfectly tie a photo to a specific camera.
Even if you didn't know who owned the camera, you could identify other pictures taken by that same camera, and information in those photos might let you figure out who owns the camera.
I think he's saying the existence of a camera signature is the privacy issue maybe.
You can remove it all.
If authenticity later becomes an issue you can produce the original.
Ontop of this, including a photo edit history in a photo including the original photo would increase the size of a photo to be completely unusable or unshareable.
I understood the GP so that only some unique ID or hash of the original image would be included, not the image itself. Basically like the commit chain of Git but without the actual content blobs.
You could use this data to prove that image B is an edit of image A if you already have both A and B.
I still think this is a bad idea, because this all requires the images to have some sort of ID - and that seems like a prime target for tracking.
You can publish a fully stripped image (as people do now for exif), and retain the original
In the contexts where you need to prove an image is a real unaltered photograph (court cases, news media, science, etc) privacy is usually not a concern.
Nobody cares if your social media photos are edited, they probably are, it's fine.
I'm pretty sure if signing images becomes the norm, because cameras will do it automatically then social media sites etc will start to care - or at least be thankful for the additional metadata.
EXIF already includes extensive metadata, including phone model and GPS location. For professional cameras, there is often camera and lens serial numbers too.
This has been the case for many years, and so far social media sites never care - they strip that info instead. Why would this change with one more extra piece of metadata?
>EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Can you clarify what your actual complaint is here? Putting aside for a second obvious implementation options, it's pretty standard that there can be a tradeoff between privacy and trust. Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like. But if there's something we [i]want[/i] to prove, this gives an extra option to do so. And in particular the set of cases where one cares the most about enhancing authenticity appears at first thought to be pretty much a union set with the cases where one will put their name (or at least the location and time of the picture in question) behind the image? Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration? Like, say you're photographing at a protest to document it including any violations of law. By definition, the images you take reveal the location and the time. That's the whole point of them. Having the location and timestamp signed wouldn't reveal anything extra as long as the photo was unaltered.
And actual legit journalism always has name(s) standing behind the reporting. Or for that matter, even if we're merely talking something like a review of a product, is it actually wrong to put a name or pseudonym behind that review if you expect readers to give you much credence? I mean, you'd be free in terms of law and tech to not bother. But even long before the current growing AI-slop age a lot of us have been starting to treat anonymous reviews with a lot of skepticism, or discounting them entirely for some product classes, for good reason.
Even in terms of maintaining privacy, remember we already have "tools" for that which this in turn could further help. You could privately report a tip to a reporter at a media organization, and then they could report on that without revealing you but be able to say "we verified the signature of the raw image and sensor data" alongside normal follow up. Then it's their name, but having a stronger chain behind it could still be helpful in places.
Finally getting back to "implementation options", there's no technical reason the image and multiple sets of metadata can't all be signed separately by the sensor stack such that you can pick & choose what to include and still have it all be signed, with readers giving greater or lesser weight to the trust based on your choices.
I think what got me worried is the GPs take that "I expect in the near future all digital cameras to digitally sign the images they take". Sure, if you have your special, verification-enhanced camera, go for it. But if this goes like with smartphones and we have all cameras embedding such a signature - and therefore all cameras are made locked-down and tamper-resistent - then we have another category of devices removed from control of their users.
> Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like.
First, the paper proposed a method of embedding the metadata inside the actual image pixels in a way that makes it difficult to remove. I think it's still possible, but you're will need a specialized tool to do so and it will alter the image.
Second, that's assuming forums or social media would still be allowing uploading unsigned images. If AI images really become as much of a problem as anticipated here, then sites might decide to block any image they can't verify completely.
> Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration?
All kinds of photos from warzones have this property. We wouldn't have an entire science of geolocating photos from landmarks that were accidentally captured if everyone was so easy with putting their GPS coordinates in the file.
People may be interested in the work of the Coalition for Content Provenance and Authenticity (C2PA) https://c2pa.org/
https://www.da.vidbuchanan.co.uk/blog/android-c2pa.html
is a post about Google Pixel C2PA cameras experiencing contact with reality
Hi, I'm the author of the above. I also got a Sony A7 IV camera a few days ago. I already have a root shell on it. Although that wasn't even required to break Sony's C2PA implementation...
I'll write more about this in the future.
if such a method gains traction then so will the effort to bypass it. either by stealing private keys or just projecting light onto the sensor.
Even if it's flawed, adding any friction at all to the effort of deception is a positive step.
There will always be a way around it. There are even open-source alternative/hacked firmwares for professional DSLR cameras where tampering with the signing may be possible.
And this still doesn't help any other kind of image e.g. screenshots, photo of a screen etc. that can make the camera signatures largely pointless depending on the context.
The idea is to be able to prove that a photo you took was captured by a camera, not generated. That would be helpful in the context of a disputed news story, a court case, etc.
Would that ever be relevant for a screenshot?
It's not massively helpful for a court case, IMO. This has been handled the same way since the invention of photography: ask the photographer to swear under oath they took the photo. (technically, laying a foundation for the evidence)
Yes in multiple ways, and there's probably more than this:
It's relevant that a screenshot doesn't have a signature, in the case that you want to remove any "proof" or tracking info from a real photo when uploading an image. Maybe I don't want people to know what brand/model of camera I use.
And it's relevant if a screenshot did have a signature if you want to "prove" that the screenshot itself hasn't been tampered with after the fact.
In the method proposed in the paper, the signature is stored in the pixels and ostensibly even survives some compression. So if you made a screenshot of a signed image, the image would still have its signature inside the screenshot, but the screenshot wouldn't have any (new) signature as a whole.
That's how I understood it at least.
I would assume you could make big enough differences in color/contrast/brightness and especially 3D rotation (similar to taking a photo of a screen) would make such a signature unreadable... but I would love to be proven wrong.
Soon my phone can cryptographically prove the beauty filter lied at capture time.
In the future, people willingly surveil themselves 24/7 with cryptographic proof, because fake images and video will be so good that it will be the only way to prove what one didn't do. Total Information Awareness achieved :D
> For example, a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake. But it's always nice seeing big actors interested in addressing this problem.
Yeah, very nice. So this whole idea basically doesn't work - but we get a new stealth way to embed metadata in an image that can be used for tracking...
(And a new narrative why cameras need to have TPMs and locked-down firmware as well)
Photos of photos has always been a problem.
In days of old, a Polaroid photo was considered "proof of capture".
I've got a Polaroid daylab 35 plus sitting in storage somewhere (https://www.instantoptions.com/wp/faqs/daylab/). You can project a slide through it onto Polaroid film, expose it, and have the image there.
I was also able to find a company that did slide printing. It was possible to send them a digital image and they'd send you back a slide with that image... which I then used to make a Polaroid of that image.
I had a classic 600 Polaroid photo of a UFO landing.
Couldn't the camera encode information from the depth sensor and prevent this.
I will make miniature dioramas and photograph them.
At that point you've earned the fruits of your deception, just like the tricksters who spent time doing physical photo editing.
Miniature dioramas wouldn't be size appropriate. Apple could detect faces/cars/other common objects of ~known size and verify -- or even just dump depth map for anyone to check.
It seems to be what Apple is doing, this feature is only available on the 18 Pro's (which have depth sensor on the back), but not Duo.
That signature contains time and optionally gps coords, and taking photo of a screen is not simple. So overall does solve some problem
ps:most important: cam/lens settings also in the digital sig, what for a screen is different