Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

Using Adversarial Machine Learning, Researchers Look to Foil Facial Recognition

For privacy-seeking users, good news: Computer scientists are finding more ways to thwart facial and image recognition. But there's also bad news: Gains will likely be short-lived.

Suspect identification using massive databases of facial images. Reputational attacks through deep fakes videos. Security access using the face as a biometric. Facial recognition is quickly becoming a disruptive technology with few limits imposed by privacy policy. 

Academic researchers, however, have found ways to — at least temporarily — cause problems for certain classes of facial-recognition algorithms, taking advantages of weaknesses in the training algorithm or the resultant recognition model. Last week, a team of computer-science researchers at the National University of Singapore (NUS) published a technique that locates the areas of an image where changes can best disrupt image-recognition algorithms, but where those changes are least noticeable to humans.

The technique is general in that it can be used to develop an attack against other machine-learning (ML) algorithms, but the researchers only developed a specific instance, says Mohan Kankanhalli, a professor in the NUS Department of Computer Science and co-author of a paper on the adversarial attack.

"Currently, we need to know the class [of algorithm] and can develop a solution for that," he says. "We are working on its generalization, to have one solution that works for every class, current and future. However, that is nontrivial and hence we anticipate it will take time."

The research raises the possibility of creating photos that people can easily perceive but that foils commonly used facial-recognition algorithms. Turned into a filter, for example, the technique could allow users to add imperceptible changes to photos to make them more difficult for ML algorithms to classify and foil the development of reverse image search engines.

Such methods, however, currently take advantage of the brittleness of training algorithms. As ML-focused companies develop more robust algorithms, privacy-seeking Internet users will have to decide whether to degrade photos to the point where the changes are noticeable to humans.

"It's a very difficult problem because pictures themselves have utility for us," says Joey Bose, a PhD student in computer science at McGill University, who published related adversarial ML research in 2018. "But the only surefire way to guarantee privacy is to remove content from the image, and if you remove content, it becomes less useful."

The developments come as several companies are using — some would say abusing — the plethora of image content readily available on the Internet. Social media sites make it easy to gather an enormous number of images on which to train neural networks, allowing companies to create sophisticated recognition models or turn people's photos into deep-fake videos. Clearview.ai, a company that sells its ability to find people's online presence from a provided photo, collected hundreds of millions of photos from social media sites to create its large reverse image search engine.

The NUS technique has to be tailored for the specific class of facial recognition, which means the researchers have to know the details of the facial-recognition system. Only then can they create a technique for confounding that approach.

While such techniques could help people protect their images and their privacy, most of the research is aimed at preventing such attacks in the future: Far from just attacking ML algorithms, the researchers are basically performing the role of a red team, checking the quality of the models, says McGill's Bose. 

"The research can inform policy and help companies know what they need to check off before these systems are put into the wild," he says. "Better models are more reliable."

And even if the researchers did find an approach that could foil the most common approaches to facial recognition, the surveillance system makers would likely be able to find ways to work around the technology, says the National University of Singapore's Kankanhalli.

Any success will be "temporary in the sense it is like a cat-and-mouse game," Kankanhalli explains. "There will be better techniques developed by the data harvesters, and in response there will be better privacy techniques to counter them. There probably will never be a final solution in such constantly evolving  problem areas."

Unlike encryption, which can provide privacy even against well-funded adversaries, ML attacks often only work until the model is retrained, he says. The best way to limit artificial intelligence is through policy, Kankanhalli says.

"When consumers frown upon companies who violate privacy, then there will be a backlash against them, which will change the behavior of such companies," he says. "We as consumers and users should therefore articulate what is acceptable and what is not. I strongly believe we need work on all these three fronts to tackle this data harvesting problem."

Related Content:

 

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/3/2020
Pen Testers Who Got Arrested Doing Their Jobs Tell All
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
New 'Nanodegree' Program Provides Hands-On Cybersecurity Training
Nicole Ferraro, Contributing Writer,  8/3/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-12777
PUBLISHED: 2020-08-10
A function in Combodo iTop contains a vulnerability of Broken Access Control, which allows unauthorized attacker to inject command and disclose system information.
CVE-2020-12778
PUBLISHED: 2020-08-10
Combodo iTop does not validate inputted parameters, attackers can inject malicious commands and launch XSS attack.
CVE-2020-12779
PUBLISHED: 2020-08-10
Combodo iTop contains a stored Cross-site Scripting vulnerability, which can be attacked by uploading file with malicious script.
CVE-2020-12780
PUBLISHED: 2020-08-10
A security misconfiguration exists in Combodo iTop, which can expose sensitive information.
CVE-2020-12781
PUBLISHED: 2020-08-10
Combodo iTop contains a cross-site request forgery (CSRF) vulnerability, attackers can execute specific commands via malicious site request forgery.