According to new research, specific words or phrases could help identify a level of perceived credibility of tweets on Twitter. This determination was based on a language model built by a team of researchers at Georgia Institute of Technology and suggests that social media may have a more considerable impact on a major event’s credibility than previously known.
“There have been many studies about social media credibility in recent years, but very little is known about what types of words or phrases create credibility perceptions during rapidly unfolding events,” said Tanushree Mitra, the lead researcher at Georgia Tech.
The study began by analyzing 66 million tweets associated with 1,400 real-world events. Mitra, and her team of researchers, scattered through a wealth of tweets pertaining to major events like the rise of the Ebola epidemic and backlash associated with the death of Eric Garner.
The tweets collected were then judged by participants on their credibility using a scale ranging from “certainly accurate” to “certainly inaccurate.” The results were later inputted into a model, which split them into various linguistic categories, which included positive and negative emotions, hedges and boosters, and anxiety, among other traits in the affective domain.
The system probed the data to determine if the tweets were credible or not. Based on the findings, the words matched the humans’ opinions about 68 percent of the time, considerably higher than the random baseline of 25 percent.
“Tweets with booster words, such as ‘undeniable,’ and positive emotion terms, such as ‘eager’ and ‘terrific,’ were viewed as highly credible,” the study found. “Words indicating positive sentiment but mocking the impracticality of the event, such as ‘ha,’ ‘grins’ or ‘joking,’ were perceived as less credible.”
Interestingly, posts with a higher number of retweets received lower credibility scores, while replies with longer word counts were deemed as more credible, the findings showed. “It could be that longer message lengths provide more information or reasoning, so they’re viewed as more trustworthy,” researchers suggested. “On the other hand, a higher number of retweets, which was scored lower on credibility, might represent an attempt to elicit collective reasoning during times of crisis or uncertainty.”
Although the system isn’t available to the public, researchers at Georgia Tech are looking at developing an app that perceives trustworthiness of an event as it unfolds on social media.
“When combined with other signals, such as event topics or structural information, our linguistic result could be an important building block of an automated system. Twitter is part of the problem with spreading untruthful news online. But it can also be part of the solution.”
The paper will be presented at the 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing.
