Taking your example of image analysis, companies in this space will build applications that - for instance - identify which images of the 10,000 on a bandit's computer are all the same kid. The software company would use sample images of clothed kids to demonstrate how it works, then only law enforcement would input confiscated CP images to do the analysis.
Perhaps a corollary to this: instead of trying to match children’s faces, the surveillance state can mine other parts of the image; Open Source intelligence. Language and sound cues in a video. Or, my personal favorite, detecting the age and manufacturing origin of floor tiles and wall trim from a photo.
Those could be a handy tool for law enforcement the image classification model could be trained to spot criminals?
I'm pretty sure there are mugshots from say a sex offender database, then the police just need to take those binoculars to a park… it would be like shooting fish and a barrel!
I'm against security theater, but this sounds like an information visualization problem: if the image contains enough signal that a theoretically optimally trained eye can detect the contraband, they should be able to use machine learning algorithms to detect suspicious items and make them show up with higher contrast.
In other words, the software should magnify anything out of the ordinary, like contraband. It's not foolproof of course, but would help.
Except they all get scanned, so the person who runs the scanner would, arguably, be able to detect this. I imagine the dugout laptop is exactly why we have mandatory scanning of laptops.
I also imagine the software can automatically detect things that looks like contraband, explosives, etc. If not then we'd be seeing a lot more terrorist success using these methods.
There must be a lot of pressure on the devs who make the software that run these things. Whatever computer vision processing it does needs to be pretty good or else.
If you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that a certain (innocuous) image is on his rival's phone. So he pays someone to generate a fake child-porn image with the same neuralhash, and ensure that it gets into the child porn DB. Then, apple reports the rival to the police, and they come and investigate him. OF course, they may notice that the image isn't the right one, but by that time they may have found other incriminating evidence.
Not sympathetic to a rival gangster? Ok lets find an innocent victim: not a rival criminal, but an innocent witness who our protag wants to intimidate. Gangster wants to intimidate the witness, but can't get at them, so cooks up a scheme to convince the witness that the police are in his pocket. Exactly as above, causing the police to investigate the witnesses phone.
Another one might be, a certain government wants to identify opposition groups using images associated with them . Apple is not keen to be associated with that, but the government can simply generate fake child-porn (remember, programmatically generated CP is just as illegal) for each image of interest.
There are practical applications too. For instance, getting around automated filters like copyright or porn detection. Making adversarial captchas that fool computers but not humans.
Why would this be of little risk? The good thing about software is that it is automatable. That's also the bad thing.
Create a malware (which due to some big company fuckups can be even embedded in a webpage these days). Capture frames indiscriminately. Add some image recognition algorithms (from OCR to machine learning, depending on what you want to do) to flag interesting hits.
Voila. Massive dragnet. Applications can range from simple blackmail (a-la Black Mirror) to industrial espionage.
I suspect there might be some pretty heavy use of computer vision as related to those images -- perhaps to pick out brand preference or otherwise personal/private information that could be rolled up/anonymized such that marketers would buy in aggregate.
You could have a software to simulate lab exercises.
But I'm curious how they'll solve the "mechanisms for identity checks". What could they do? If they wanna do real lab exercises and not software simulation, then both are similar problems. I can imagine either using a camera, like you said, or partnering with local schools around the world.
Both have many flaws, let's see what they come up with.
I'm sure it will be used for nefarious things too however,
Programs involving photography, yes the big dogs have their own solutions already but this lowers the bar for us hobbyists.
Photo affects (Blurring the background and not the face), sorting images by labels (search for trees and get your photos with trees), searching by people.
Potentially things traffic monitoring, foot traffic in a mall, security systems (record a photo of all customers in the case of theft), automating tracking objects in videos.
> Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is illegal in the user's jurisdiction)
I'm not an expert in AI so this might be totally off base but I feel like you would be able to use an "intersection" of sorts for this type of detection. You detect children and pornography, the children portion trains it for age recognition and the porn portion trains it to see sexual acts. Slap those two together and you've got CSAM detection.
I've always wondered about that. Surely possesing the 'training data' (basically child porn) is illegal. Only police can posses it for investigations etc. So how can you make something that can recongnise it?
reply