TO FIGHT DEEPFAKES, RESEARCHERS BUILT A SMARTER CAMERA
But what if that tamper-resistant seal originated from the camera itself? The NYU team demonstrates that you could adapt the signal processors inside—whether it's a fancy DSLR or a regular smartphone camera—so they essentially place watermarks in each photo's code. The researchers propose training a neural network to power the photo development process that happens inside cameras, so as the sensors are interpreting the light hitting the lens and turning it into a high quality image, the neural network is also trained to mark the file with indelible indicators that can be checked later, if needed, by forensic analysts.
"People are still not thinking about security—you have to go close to the source where the image is captured," says Nasir Memon, one of the project researchers from NYU Tandon who specializes in multimedia security and forensics. "So what we’re doing in this work is we are creating an image which is forensics-friendly, which will allow better forensic analysis than a typical image. It's a proactive approach rather than just creating images for their visual quality and then hoping that forensics techniques work after the fact."
The main thing consumers expect from cameras is ever-improving image quality and fidelity. So one main focus of the project was showing that incorporating machine learning into the image signal processing that goes on inside of a camera doesn't visibly detract from photo quality as it paves the way for tamper-resistant elements. And adding these features within the image-generation hardware itself means that by the time files are being stored in the camera's SD card or other memory—where they're potentially at risk of manipulation—they are already imbued with their tamper-evident seals.
The researchers mainly insert their watermarks into certain color frequencies, so they will persist through typical post-processing—like compression or brightness adjustments—but show modification if the content of an image is altered. Overall, the forensic-friendly additions improved image manipulation detection accuracy from about 45 percent to more than 90 percent.
"As the research and industrial communities consider this technology, I do think they should be wary of potential risks posed by anti-forensic attacks and adversarial machine learning," says Matthew Stamm, an information forensics researcher at Drexel University. " ... But it’s feasible that an attacker might be able to create a deep learning network to remove these security artifacts, allow an image to be modified or falsified, then re-insert the security artifacts afterward."
For forensic watermarking to really make an impact on curbing deepfakes it would need to work on video also, something the researchers say they haven't broached yet, but that would be theoretically possible.
See the full story here: https://www.wired.com/story/detect-deepfakes-camera-watermark/