(PONARS Policy Memo) The U.S. intelligence community’s January 6 report about Russian meddling in the 2016 U.S. presidential election highlighted the role of Russian media organizations in spreading fake news and amplifying leaked materials in an attempt to manipulate public opinion. While few Americans receive their news directly from Russian sources, it is hard to dispute that a major consequence is that U.S. journalists and policymakers now face the challenge of restoring public trust in the media. This would not be the first case of a society trying counter biased and false information in the press.
Ukraine has been engaged in full-fledged information warfare against Russian propaganda since 2014, when Russia annexed Crimea and vitalized rebellions in eastern Ukraine. Ukraine’s information war is not going very well. Rather than helping to establish “ground truths,” Ukraine’s response to Russian propaganda has actually made truth more elusive, particularly with respect to the conflict in the Donbas. It has potentially made the conflict more difficult to resolve.
Harvard political scientist Matthew Baum and I performed a parallel analysis of thousands of incident reports from Ukrainian, Russian, rebel, and third party sources. We investigated the extent to which different sources suggested different patterns of strategic interaction between warring sides, and advanced different conclusions about the causes, location, and timing of violence.
We found that information warfare profoundly affects inferences about armed conflict, particularly about which actors are most responsible for violence. According to Ukrainian sources, rebels are more likely than the government to use force, kill civilians, and violate ceasefires. According to Russian and rebel sources, the opposite is true. Both Ukrainian and rebel sources report more violence than do outside, third-party sources such as the OSCE.
Each perspective has its own implications for how different actors behave in war, the sustainability of ceasefire agreements, the need for sanctions or third-party intervention, and whether intervention should be neutral or one-sided.
Background on Ukraine’s Information War
Before the Euromaidan movement swept President Viktor Yanukovych from power in February 2014, the Russian media had a heavy presence in Ukraine, particularly in Crimea and in the south-east. In contrast to Western media portrayals of the Euromaidan as a largely peaceful protest movement confronting riot police and hired thugs, the mainstream Russian media devoted coverage to nationalist militants storming the Ukrainian parliament and hurling Molotov cocktails. Both images were in a narrow sense true, but neither represented the complete picture. The Russian perspective on events generated impressions on those rallying in Crimea and in the south-east, who then condemned the Euromaidan movement as a “Western-backed coup” and “fascist junta.”
Concerned about the Russian media’s potential for mobilization, Ukraine’s new authorities took a series of steps to counter it. In March 2014, before the first shots were fired in the east, Kyiv banned Russian federal broadcasters from Ukrainian television. Several months later, Kyiv banned some Russian films and television programs and placed travel bans on Russian journalists. In December 2014, Ukraine established the Ministry of Information Policy to protect Ukrainians from “unreliable information,” register media outlets, and define professional journalism standards. To spread government-approved content in social media, the Ministry launched an “Information Army” of patriotic volunteers.
Ukrainian authorities also exerted direct pressure on some information providers. In September 2014, Ukraine’s Security Service (SBU) raided the offices of the newspaper Vesti, accusing it of violating Ukraine’s territorial integrity through its coverage of the Donbas conflict. In February 2015, Ukrainian authorities arrested a blogger on charges of treason for posting a YouTube video criticizing the government’s military mobilization campaign. The same month, Ukraine’s Television and Radio Council accused popular television host Savik Shuster of violating the law on “incitement of hatred” after a Russian journalist criticized the government’s military operations on his show. Multiple similar incidents ensued.
In the rebel-held territories of the Donbas, separatists moved to create a similar closed information environment. After seizing the Donetsk regional administration building in April 2014, one of their next steps was to take control of the television towers in the region. Their aim was to take Ukrainian channels off the air and broadcast Russian programs. Later that year, the self-proclaimed Donetsk People’s Republic established an official News Agency (DAN), while multiple privately-owned pro-rebel outlets emerged to fill the regional media vacuum. Wary of journalists from outside Russia and the region, rebels detained several reporters on suspicions of espionage, including an American journalist with Vice News.
In 2014, across Ukraine (including rebel areas) there were: 7 documented killings of journalists, 286 physical assaults, 78 abductions, multiple physical attacks on offices, and cyber attacks on websites, according to Freedom House. Predictably, these developments raised concerns over freedom of speech, including that an informational firewall between dueling and contradictory media narratives would only deepen existing divisions.
A Post-Truth Armed Conflict
How has Ukraine’s information war affected public attitudes toward the conflict? Survey evidence suggests that very few Ukrainians outside of the Donbas see Russian state media as a reliable or truthful source, which may be evidence either of the success of Ukraine’s counter-propaganda efforts or the ineffectiveness of Russia’s. Residents of rebel-held areas appear to have a similarly skeptical view of Ukrainian media, particularly due to its unwillingness to report on civilians killed by pro-government troops—incidents which Kyiv routinely denies.
To take stock of reporting biases in the Ukrainian conflict, we collected data on 72,010 violent events, as reported by 17 information providers, between February 23, 2014, and May 2, 2016. Our sources included official newswires, television channels, Internet news sites, and blogs from Ukrainian, rebel, Russian, and external, third-party outlets. We also included the Russian-language edition of Wikipedia, and daily briefings from the OSCE Special Monitoring Mission to Ukraine. We used natural language processing and supervised machine learning to classify each event into a series of pre-defined categories, by event type, initiator, target, tactic, and casualties.
How do Russian and Ukrainian sources differ in their coverage of the Donbas conflict? To answer this question, we estimated the relative bias of each information provider in covering rebel versus government violence. We did so using methods developed by scholars of American electoral politics to estimate “house effects” (Jackman, 2005; Pickup and Johnston, 2008) of individual survey firms (for example, which pollsters have a “pro-Trump” bias and which have a “pro-Clinton” bias).
Figure 1 shows these estimates, with event reports published by the OSCE as the reference category (vertical line at zero). Positive values indicate that a source is more likely to cover rebel than government violence, and negative values indicate greater relative coverage of government violence. Where the margin of error covers zero, relative levels of coverage were similar to reports by the OSCE.
Figure 1. Ukrainian Sources Report on Rebel Violence, Pro-Russian Sources Report on Government Violence
Note: Dots are relative bias in reporting on rebel versus government violence. Lines are 95% confidence intervals.
The data reveal large systematic differences in the armed actors who receive coverage in Ukrainian, rebel, Russian, and international sources. Overall, Ukrainian information providers (blue circles) devote more news coverage to rebel violence and less to government operations than any other group of sources. Four out of the five sources that systematically “over-report” rebel attacks are Ukrainian: the military blog Information Resistance (Sprotyv), and the television channels 112, Espreso, and Channel 5 (the latter is owned by President Petro Poroshenko).[1]
Most sources that “over-report” government violence are based within Russia (red circles) or the self-proclaimed Peoples’ Republics of Donetsk and Luhansk (DNR, LNR) (orange circles). DNR-based media outlets NewsFront and Donetsk News Agency (DAN), in particular, have the most acute actor-specific bias in the data, reporting almost exclusively on violence by the Ukrainian government.
Russian sources have the same general direction of bias as rebel sources, but with somewhat lower magnitude. With a single exception—the independent, opposition-oriented Dozhd television channel, which is closer to the median Ukrainian source—Russian media report disproportionately on government violence. The only Ukrainian outlet with a comparable bias in the opposite direction is Interfax-Ukraine, a Russian-owned wire service. Between rebel and Ukrainian media, there is a much clearer separation—the “left-most” Ukrainian outlet is still to the right of the “right-most” rebel outlet.
A very different picture appears in third-party sources, like OSCE reports and Wikipedia. These sources are more “neutral,” in the sense that they are unlikely to attribute violence to any armed group at all. The language in these reports tends to be more passive and non-specific (“shelling was reported near village X”) than language in local media. For the OSCE, this finding is consistent with anecdotal reports that—because it must maintain working relations with all sides—the monitoring organization is cautious about attributing violence to specific initiators. For Wikipedia (green circles), this pattern may reflect the crowd-sourced nature of the data: users flag entries as biased, remove offending information, and eventually reach a “neutral” compromise.
Not only have Ukrainian and rebel media reported disproportionately on violence by the “other” side, they report mainly on indiscriminate violence (e.g., artillery shelling, rockets, heavy armor) by the “other” side. Ukrainian news coverage of rebel violence cites indiscriminate tactics 66 percent of the time, compared to 45 percent in rebel media. Coverage of government violence is a near-mirror image: 32 percent of the government violence reported by Ukrainian sources is indiscriminate, compared to 57 percent for rebel sources. Russian and international sources, again, fall somewhere in between.
Beyond simply making the opponent “look bad,” these biases have implications for conflict resolution. We looked specifically at coverage of ceasefire violations after the Minsk I and Minsk II agreements, and ran a series of simulations to see which actor is most likely to break the peace, according to each set of sources. Unsurprisingly, the greatest disparity here was between Ukrainian and rebel sources. Ukrainian sources predicted that rebels are more than twice as likely to unilaterally violate the ceasefire as are government troops. Rebel sources predicted an even stronger pattern in the opposite direction, with government troops almost ten times more likely to unilaterally escalate than the rebels. According to Russian and outside sources (OSCE, Wikipedia), however, ceasefire violations should be relatively rare overall, and both sides are about equally likely to violate.
These predictions have divergent implications for conflict resolution. In the case of outside sources like the OSCE, a news consumer or policymaker may conclude that sanctions or intervention are not necessary to reduce violence. Here, violence diminishes organically over time, and neither side appears likely to unilaterally escalate—a situation in which a negotiated settlement may become self-sustaining. Local sources yield very different lessons: here, transgressions appear to be more common, and a negotiated settlement less likely to hold. For violence to decline, enforcement efforts and sanctions should target whichever side is more prone to unilaterally escalate. According to Ukrainian sources, this intervention should seek to restrain rebels; according to rebel sources, it should target the government.
Conclusion
The relative direction and magnitude of actor-specific reporting biases in Ukraine represent the exact opposite of what would be needed to quickly resolve the conflict. The net effect is that domestic audiences (in government-controlled vs. rebel-controlled territories) may become less interested in striking a bargain with the opposing side, reasoning that an actor inclined to use unilateral violence is unlikely to stick to the terms of a negotiated agreement. Meanwhile, outside audiences (in Russia vs. the West) may develop contradictory perceptions of how intractable the conflict is likely to be, whether sanctions or third-party enforcement is necessary to stop it, and whether that response should be impartial or directed at one side.
Reversing these biases is, of course, easier said than done. Absent attributions of responsibility for violence, leaders and activists interested in conflict resolution will need to better inform journalists about the details of specific incidents. Where attribution exists, governments and NGOs will need to expand audiences’ access to multiple sources of information.
As the United States adapts to a more polarized and uncertain media landscape in 2017, the main lesson of Ukraine’s information war is that efforts to respond to propaganda through counter-propaganda are unlikely to bring us closer to the truth.
Yuri M. Zhukov is Assistant Professor of Political Science at the University of Michigan.
[PDF]
[1] The term “over-report” indicates that a source reports a higher share of rebel-to-government (or government-to-rebel) attacks than the OSCE.
