Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

5/14/2020
04:35 PM
100%
0%

Project Aims to Unmask Disinformation Bots

BotSight, a machine learning research project, rates Twitter users based on the likelihood that there is a human behind the keyboard. Could such technology blunt the impact of disinformation campaigns?

Aiming to combat disinformation on social media, a research team published plug-ins for major Web browsers on Thursday that give users a score as to the likelihood that a Twitter handle is a bot or a human. 

Dubbed BotSight, the project is the brainchild of machine learning researchers at security firm NortonLifeLock, formerly a subsidiary of Symantec, which aims to help users determine which accounts are valid and which are not. The tool uses 4TB of data collected by the researchers over the past six months and looks at 20 features — from the randomness of the Twitter handle to the rate of follower acquisition — to classify the handle as a bot or human.

Through the project, the researchers hope to help people understand when they might encounter bots, says Daniel Kats, principal researcher with the NortonLifeLock Research Group (NRG).

"People have a conception that bots exist," he says. "But ... when you go on your timeline and you look around — people don't have a sense of where they are most likely to find bots. We want to give people that intuitive sense of where bots live and what they are."

The tool is the latest attempt to apply machine learning and artificial intelligence techniques to the thorny problem of the often-viral propagation of false information on social networks. While Facebook, Twitter, and other social networks have their own technology for flagging posts, tweets, and messages for further scrutiny, giving users their own ability to evaluate messages is important, says Chris Meserole, the deputy director of the Artificial Intelligence and Emerging Technology Initiative at the Brookings Institution in Washington, DC. 

"The more users of Twitter are aware that bot networks exist, the more aware they can be of whether the accounts they are interacting with are human or bot, the more they will understand how trustworthy that the information they encounter might be," he says. "It also provides a service by giving users tools to evaluate the claims of public figures who are retweeting misinformation from bot accounts."

The BotSight plug-ins insert a human or bot icon next to each Twitter handle, along with color indicating whether the algorithm believes the account to be human (green) or bot (red), as well a measure of confidence. (BotSight has classified this reporter — perhaps, not inaccurately — as only 88% human.)

A beta feature allows users to also see some of the criteria — currently, somewhat cryptic — for the machine learning model's determination. 

The technology uses 20 different attributes and metadata from tweets and user accounts to determine whether the user is more likely to be human or a bot. While the technology is focused on detecting bots that retweet and amplify certain messages and disinformation, the system can also detect certain types of human-operated disinformation networks, such as sock puppets, Kats says.

"You have these central accounts that collect a lot of followers — they are influencers in the network," he says. "And then you have a bigger network of amplifier accounts. The larger network, the amplification network, is what BotSight specifically targets. However, it is also capable of capturing some of the influencers accounts."

Bot networks on Twitter are responsible for amplifying the information of specific "influencer" accounts. Often, certain hashtags are included to push a certain political issue. During the current epidemic, Twitter has been really careful on finding and deleting bad information on symptoms and treatments for COVID-19, but the company has had less of a focused approach to bot activity on the platform in general.

"There is a background radiation of, oh, around 6% of bot activity in general on Twitter," Kats says. "But when you look at certain trending hashtags, it can go up to 20% when you look at some hot-button issues, but it really depends on the issue."

However, identifying Twitter bots is not the same as identifying the sources of disinformation, since bots typically only relay or amplify information. Bot Sentinel, a project that attempts to identify untrustworthy Twitter accounts and "trollbots" using a machine learning model built from some 5 million tweets. 

Such tools can help users better gauge what information online can be trusted, says the Brookings Institution's Meserole.

"Right now, everything we are doing is dancing around the margins and putting Band-Aids on things," he says. "The question is how do we insure the credibility of our political discourse. For that, these types of tools can be helpful."

NortonLifeLock researchers intend to add more features to the tool, but trying to categorize the content of the post based on machine learning analysis is not a currently a focus, Kats says.

"We don't want to block people from accessing certain content," he says. "We want to provide people with enough information to help them make wiser decisions for themselves."

Related Content:

 
 
 
 
Learn from industry experts in a setting that is conducive to interaction and conversation about how to prepare for that "really  bad day" in cybersecurity. Click for more information and to register
 
Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
News
Inside the Ransomware Campaigns Targeting Exchange Servers
Kelly Sheridan, Staff Editor, Dark Reading,  4/2/2021
Commentary
Beyond MITRE ATT&CK: The Case for a New Cyber Kill Chain
Rik Turner, Principal Analyst, Infrastructure Solutions, Omdia,  3/30/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2015-20001
PUBLISHED: 2021-04-11
In the standard library in Rust before 1.2.0, BinaryHeap is not panic-safe. The binary heap is left in an inconsistent state when the comparison of generic elements inside sift_up or sift_down_range panics. This bug leads to a drop of zeroed memory as an arbitrary type, which can result in a memory ...
CVE-2020-36317
PUBLISHED: 2021-04-11
In the standard library in Rust before 1.49.0, String::retain() function has a panic safety problem. It allows creation of a non-UTF-8 Rust string when the provided closure panics. This bug could result in a memory safety violation when other string APIs assume that UTF-8 encoding is used on the sam...
CVE-2020-36318
PUBLISHED: 2021-04-11
In the standard library in Rust before 1.49.0, VecDeque::make_contiguous has a bug that pops the same element more than once under certain condition. This bug could result in a use-after-free or double free.
CVE-2021-28875
PUBLISHED: 2021-04-11
In the standard library in Rust before 1.50.0, read_to_end() does not validate the return value from Read in an unsafe context. This bug could lead to a buffer overflow.
CVE-2021-28876
PUBLISHED: 2021-04-11
In the standard library in Rust before 1.52.0, the Zip implementation has a panic safety issue. It calls __iterator_get_unchecked() more than once for the same index when the underlying iterator panics (in certain conditions). This bug could lead to a memory safety violation due to an unmet safety r...