Editor’s Note: This study was originally published by VoxUkraine. It has been edited for style and clarity and republished with permission.
VoxUkraine analyzed over 755,000 tweets that are connected with the Russian “Internet Research Agency,” informally known as the ‘troll factory,” and found lots of evidence of mud-slinging. But how does Russian propaganda work?
In the middle of October social media Twitter published a dataset with more than 9 million tweets that are “believed to be connected to the Russian Internet Research Agency.” This move by the tech giant is perhaps the result of investigations into interference by the Russian special services in the 2016 U.S. presidential election.
The published data contains information about 3,667 accounts that are associated with the Internet Research Agency.
We have already written about the influence of Russian twitter-bots on the U.K.’s “Brexit” referendum on leaving the European Union: the co-founders of VoxUkraine Alexander Talavera and Yuriy Gorodnichenko were the first to prove that the Russians tried to affect the result of the Brexit referendum.
We set the goal of investigating whether any of the accounts that were associated with the Internet Research Agency made attempts to influence Twitter users on any Ukrainian issue or event. For this case, using keywords we found 755,000 tweets related to Ukraine. In this analysis, we have explored the topics of the tweets, defined the most popular events for coverage over the last eight years, and found accounts that were tweeting for almost 33 hours straight.
The dataset
In the dataset, 55 percent of the tweets were in Russian, 37 percent in English, and 8 percent in other languages. VoxUkraine decided to analyze how many tweets related to Ukraine directly. Using keywords, we filtered the 9 million tweets for those with messages related to Ukraine. The keywords were present in almost 755,000 tweets.
In order to filter messages, we searched the tweets on hot Ukrainian topics, personalities (e.g. Poroshenko and Yanukovych) and key narratives of Russian propaganda, in addition to different variations of the word “Ukraine”. A complete list of marker words follows:
South East, Donbas, Boeing, IMF, Bandera, banderovtsy, punisher, pro-Russian fighter, fascism, AFU, Kyiv, revolution, Maidan, Crimea, Donetsk, Luhansk, Russia-Ukraine, Poroshenko, Yanukovych, civil war, DPR, LPR, people’s republic, State Department, conflict in Ukraine, Ministry of Defense of Ukraine, MH17, natural gas transmission system, Tymoshenko, khokhly, and resettlers.
The search period was from October 2010 to May 2018.
What kind of accounts are these?
Over the more than eight years we found 774,957 tweets about Ukraine in the dataset of the “troll factory.” They were generated by 1,369 accounts (these accounts have since been deleted from the social network). Twitter anonymized accounts that have less than 5,000 followers, and that’s why we can see only 5 percent of the user’s names in the database. These names can be broken up into categories:
media pages: Vestnyk Moscvy, Vestnyk Peterburga) (24 regional “Vestnyks” in total), Federalnoe agentstvo novostey, and Novyny Kyeva
pages with names: Maksym Dementev), Maria Mozhayskaya
thematic pages: Ramzan Kadyrov – a parody page as mentioned in the description, Myusli Lavrova, and “Kholodnaya voyna 2.0.
The accounts with the most followers were Vestnyk Moscvy — 257,000, Vestnyk Peterburga — 149,000, and Maksym Dementev) — 134,000. The total number of followers of the 1,369 accounts is 3.4 million.
Most of the account owners wrote (that means, we don’t know their exact coordinates) that they were located in Moscow (26 percent) and St. Petersburg (16 percent), while 17 percent never mentioned their location.
Activity peaks
The first tweet from the sample is from January 2010, so we have the opportunity to analyze ”troll” activity before and after the Revolution of Dignity.
Before the start of the Revolution of Dignity in November 2013 and during the next three months, the Twitter trolls showed little interest in Ukraine: the accounts from our sample demonstrated a very low activity. But by the spring of 2014 all of the accounts were posting an average 10 tweets per day, and the maximum number of postings for this period was on the day of the parliament elections in Ukraine, on Oct. 28, 2012, when they averaged 90 posts per day.
A new high was reached on the next day after the “referendum” in Crimea on March 17, 2014, when the average was 116 tweets, and on March, 18 — the day of Russia’s annexation of the peninsula — it rose to 129 tweets.
The accounts connected with the Russian troll factory started “working” even more productively two days before the presidential elections in Ukraine — 263 tweets were published on May 23. Over the next six months the number of tweets was less than 115 per day.
The biggest number of tweets over the eight years was on July 18, 2014 — the day after the shooting down of Malaysia Airlines Flight MH17 over Russian-occupied eastern Ukraine. On this day the accounts together tweeted over 44,000 messages, and on the next day over 25,000.
An eloquent hashtag
One of the most effective methods for spreading a tweet and making it trend is to use a hashtag. In our selection, we found over 6,200 different hashtags. What are they about?
The most popular hashtags in tweets related to Ukraine were #ПровокацияКиев (22,300 references), #КиевСбилБоинг (22,100) and #КиевСкажиПравду (21,900). They appeared on the days just after the MH17 shootdown — on July 18-20. A total of 327 accounts participated in the campaign promoting these hashtags. Taking into consideration that these are depersonalized accounts, we tried to figure out their behavior in the period from July 18 until July 20, 2014.
Top performer accounts
The first thing that catches the eye is a number of tweets from one account. The most active account from the selection for the period from July 18 to July 19 (no posts on July 20) tweeted and retweeted 296 times in total. Oddly, there were many other such “overachievers.” If we make a “workaholics rating” for July 18-20, 2014, then 277 accounts are found to have made 196 or more posts in this period of time.
Why did pick the number 196? Because of a gap of 155 tweets between No. 277 and No. 278 in the rating. We will now continue analyzing these 277 accounts.
How could one person make 296 posts over two days? One would have to be, at least, a very hardy person. This account had posted tweets for 33 hours without any long breaks. On average, the account tweeted every 6 minutes and 42 seconds, and the largest break between tweets came only on July 19, from 11:31 until 14:50. As we can see on the visualization, over 200 accounts also had similarly “productive” work.
We can assume that several people were managing one account. One of the confirmations of this is the way of publishing tweets. Usually, users are tweeting directly from the Twitter website or from the mobile app. Such tweets were labeled as “Twitter Web Client” and “Twitter for Android/iOS” in the dataset.
The tweets also can be posted using a special app — a client service, such as Tweetdeck. This is a special service for posting and managing one or several accounts.
You can use already existing client services or create your own for working with several accounts. Twitter gives developers the opportunity to post, edit, delete and make other manipulations without visiting the website, only using API. Several services showed strange activity in July.
For example, the “twisofter” app on July 18-19 was used to post 19,300 and 11,200 tweets, or 43 percent and 40 percent respectively of the total number of posts on those days.
The “token app” app showed similar activity, being used to post 18,100 and 10,400 tweets on these days, also 43 percent and 40 percent respectively of the total number of tweets for these days. There are no mentions of these services in the search system. There is a possibility that these client services were created for a specific customer or a specific task alone, and weren’t promoted to a wide audience as a new general-use app.
Content
Some 55 percent of the tweets of the almost 755,000 analyzed were unique, meaning that they were not shared (retweeted) from other accounts.
Unfortunately, 98 percent of the tweets were generated by anonymized accounts, so we can’t figure out who shared the content. But we can analyze the pages and the content that was retweeted the most.
The most retweeted post belongs to the user with a nickname “_drBre” (now deleted). He’s an author of the first and the second (632 and 537 RT, respectively) top-shared tweets from our selection. The third place in the rate belongs to “Письма на фронт” (Pysma na front) (@frontlineletter).
In total, the most popular sources of shares among the 1,369 accounts were Russian mass media. The first place belongs to “РИА Новости” (RIA Novosti) (23,000 retweets), in the second place — “Федеральное агентство новостей” (Federalnoe agentstvo novostey) (16,000) and in the third place is “Russia Today” (14,000).
The individual posts of Russian media didn’t collect so many shares. For example, the leader in the total number of retweets, “РИА Новости” (RIA Novosti), had a maximum of 19 retweets for one post. “Федеральное агентство новостей” (Federalnoe agentstvo novostey) had 80 or four times more than the posts from “РИА Новости” (RIA Novosti). However, we are interested not only in the number of retweets, likes and content – the time of publishing is more confirmation that one person is managing several accounts.
A total of 18 out of 19 accounts that shared one piece of “РИА Новости” (RIA Novosti) content tweeted it simultaneously — at 16:28 on Sept. 28, 2015. “Федеральное агентство новостей” (Federalnoe agentstvo novostey) say similar activity: the most popular tweet (80 retweets) was published by 45 different accounts at 16:34 on July 17, 2014, and by 35 other accounts in 16:43 on the same day.
Summary
For a little over eight years, in the dataset of the “troll factory”, we found 774,957 tweets about Ukraine that were generated by 1,369 accounts. Before the Russian occupation of Crimea, twitter bots showed almost no activity: the maximum number of posts for the period from 2010 until 2013 were made on the day of the Verkhovna Rada elections in 2012 (90 tweets). The accounts in the selection began to tweet more actively at the end of 2014: before the presidential elections (May 23) 263 tweets were published. During the next six months the number of tweets was not less than 115 per day.
A real “twitter-storm” came on July 18, 2014 — the day after the shooting down of MH17. On this day the accounts tweeted over 44,000 messages, and on the next over 25,000. On these days 297 accounts, using the hashtags #ПровокацияКиев (22,300 references), #КиевСбилБоинг (22,100) and #КиевСкажиПравду (21,900), promoted claims that Ukraine was responsible for shooting down MH17.
Over 200 accounts were managed centrally. In addition to the fact that Twitter connects all the accounts from the dataset with Russia’s “Internet Research Agency,” we found two confirmations that several accounts belong to one owner. There is strange activity by two client services for working with Twitter since July. For example, the service called “twisofter” was used for tweeting only on July 16-19, 2014. The largest number of posts were made on July 18 and July 19 — 19,300 and 11,200 tweets (or 43% and 40% of the total amount of the corresponding day).
Another service called “token_app” showed similar activity. There are no mentions of these services in the search system. There is a possibility that these services were created only to achieve a specific task.
The second confirmation is the content. A total of 18 out of the 19 accounts that shared the most popular for “РИА Новости” (RIA Novosti) post over the eight years did it simultaneously — at 16:28 on Sept. 28, 2015.
“Федеральное агентство новостей” (Federalnoe agentstvo novostey) saw similar activity. Their most popular tweet (80 retweets) was published by 45 different accounts on 16:34 on July 17, 2014, and by 35 other accounts at 16:43 on the same day.
Oleksandr Nadelnyuk is a VoxUkraine analyst.
