Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

Using Adversarial Machine Learning, Researchers Look to Foil Facial Recognition

For privacy-seeking users, good news: Computer scientists are finding more ways to thwart facial and image recognition. But there's also bad news: Gains will likely be short-lived.

Suspect identification using massive databases of facial images. Reputational attacks through deep fakes videos. Security access using the face as a biometric. Facial recognition is quickly becoming a disruptive technology with few limits imposed by privacy policy. 

Academic researchers, however, have found ways to — at least temporarily — cause problems for certain classes of facial-recognition algorithms, taking advantages of weaknesses in the training algorithm or the resultant recognition model. Last week, a team of computer-science researchers at the National University of Singapore (NUS) published a technique that locates the areas of an image where changes can best disrupt image-recognition algorithms, but where those changes are least noticeable to humans.

The technique is general in that it can be used to develop an attack against other machine-learning (ML) algorithms, but the researchers only developed a specific instance, says Mohan Kankanhalli, a professor in the NUS Department of Computer Science and co-author of a paper on the adversarial attack.

"Currently, we need to know the class [of algorithm] and can develop a solution for that," he says. "We are working on its generalization, to have one solution that works for every class, current and future. However, that is nontrivial and hence we anticipate it will take time."

The research raises the possibility of creating photos that people can easily perceive but that foils commonly used facial-recognition algorithms. Turned into a filter, for example, the technique could allow users to add imperceptible changes to photos to make them more difficult for ML algorithms to classify and foil the development of reverse image search engines.

Such methods, however, currently take advantage of the brittleness of training algorithms. As ML-focused companies develop more robust algorithms, privacy-seeking Internet users will have to decide whether to degrade photos to the point where the changes are noticeable to humans.

"It's a very difficult problem because pictures themselves have utility for us," says Joey Bose, a PhD student in computer science at McGill University, who published related adversarial ML research in 2018. "But the only surefire way to guarantee privacy is to remove content from the image, and if you remove content, it becomes less useful."

The developments come as several companies are using — some would say abusing — the plethora of image content readily available on the Internet. Social media sites make it easy to gather an enormous number of images on which to train neural networks, allowing companies to create sophisticated recognition models or turn people's photos into deep-fake videos. Clearview.ai, a company that sells its ability to find people's online presence from a provided photo, collected hundreds of millions of photos from social media sites to create its large reverse image search engine.

The NUS technique has to be tailored for the specific class of facial recognition, which means the researchers have to know the details of the facial-recognition system. Only then can they create a technique for confounding that approach.

While such techniques could help people protect their images and their privacy, most of the research is aimed at preventing such attacks in the future: Far from just attacking ML algorithms, the researchers are basically performing the role of a red team, checking the quality of the models, says McGill's Bose. 

"The research can inform policy and help companies know what they need to check off before these systems are put into the wild," he says. "Better models are more reliable."

And even if the researchers did find an approach that could foil the most common approaches to facial recognition, the surveillance system makers would likely be able to find ways to work around the technology, says the National University of Singapore's Kankanhalli.

Any success will be "temporary in the sense it is like a cat-and-mouse game," Kankanhalli explains. "There will be better techniques developed by the data harvesters, and in response there will be better privacy techniques to counter them. There probably will never be a final solution in such constantly evolving  problem areas."

Unlike encryption, which can provide privacy even against well-funded adversaries, ML attacks often only work until the model is retrained, he says. The best way to limit artificial intelligence is through policy, Kankanhalli says.

"When consumers frown upon companies who violate privacy, then there will be a backlash against them, which will change the behavior of such companies," he says. "We as consumers and users should therefore articulate what is acceptable and what is not. I strongly believe we need work on all these three fronts to tackle this data harvesting problem."

Related Content:

 

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/10/2020
Pen Testers Who Got Arrested Doing Their Jobs Tell All
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
Researcher Finds New Office Macro Attacks for MacOS
Curtis Franklin Jr., Senior Editor at Dark Reading,  8/7/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-9079
PUBLISHED: 2020-08-11
FusionSphere OpenStack 8.0.0 have a protection mechanism failure vulnerability. The product incorrectly uses a protection mechanism. An attacker has to find a way to exploit the vulnerability to conduct directed attacks against the affected product.
CVE-2020-16275
PUBLISHED: 2020-08-10
A cross-site scripting (XSS) vulnerability in the Credential Manager component in SAINT Security Suite 8.0 through 9.8.20 could allow arbitrary script to run in the context of a logged-in user when the user clicks on a specially crafted link.
CVE-2020-16276
PUBLISHED: 2020-08-10
An SQL injection vulnerability in the Assets component of SAINT Security Suite 8.0 through 9.8.20 allows a remote, authenticated attacker to gain unauthorized access to the database.
CVE-2020-16277
PUBLISHED: 2020-08-10
An SQL injection vulnerability in the Analytics component of SAINT Security Suite 8.0 through 9.8.20 allows a remote, authenticated attacker to gain unauthorized access to the database.
CVE-2020-16278
PUBLISHED: 2020-08-10
A cross-site scripting (XSS) vulnerability in the Permissions component in SAINT Security Suite 8.0 through 9.8.20 could allow arbitrary script to run in the context of a logged-in user when the user clicks on a specially crafted link.