Since its full-scale invasion of Ukraine in February 2022, Russia has been subjected to wide-ranging and increasingly severe sanctions by the United States, the European Union, the United Kingdom, and other advanced industrial democracies. By many measures, Russia is now the most sanctioned country in the world. This followed the smaller-scale, coordinated sanctions imposed in July 2014 after Russia’s forcible annexation of Crimea that March. Assessments of the sanctions’ effectiveness have diverged widely, with analysts finding them to be everything from devastating to trivial. To provide clarity to this debate, those making public-facing claims about sanctions’ effectiveness should focus on three things: identifying the goals of sanctions, drawing on appropriate evidence, and avoiding confirmation bias.
Identify the Goals of Sanctions
What does it mean to says that sanctions “work”? Analyses should avoid claims stating or implying that inflicting economic pain alone indicates effectiveness. Causing economic pain through sanctions is a means of achieving political ends. This matters because when analysts treat the goals of sanctions as expansive, for example, they are less likely to find sanctions to be effective (and vice versa). Higher bars are harder to clear. Note also that governments’ avowed political goals may differ from analysts’ own yardsticks for effectiveness.
What goals might sanctions be intended to achieve? At the least ambitious end of the spectrum, the goal may simply be to display unity in condemning the target country’s actions in a symbolic expression of approbation short of military action. In this case, the actual economic pain caused is not relevant; effectiveness means successfully maintaining this display of unity rather than sparking a desired response from the target state. Ratcheting up, sanctions can be intended as deterrence, to convince a target state’s government that taking a particular action would not be worth the price. Pursuing this goal assumes that the target state’s government sees itself as economically vulnerable to sanctions and would be unwilling to pay that price. Next, sanctions can be intended to reduce the target state’s economic ability to achieve its military ends. Effectiveness here does not require forcing the target state’s government to change its own goals, but instead entails reducing its ability to meet them. Sanctions that target defense-related financing and production fall into this category.
More expansive goals might involve forcing the target government to change its own policies and goals through sanctions. Here, effectiveness would mean that the target government agrees to enter meaningful negotiations and/or makes significant policy concessions because of the pain caused by sanctions. Sanctions can aim to achieve such goals either through direct pressure on government decisionmakers or through indirect pressure on influential economic elites or the public that leads them to demand policy changes. At this level, it becomes more difficult to demonstrate cause and effect. At their most extreme, sanctions can even be intended to induce the fall of the target government. In this case, sanctions are not a prelude to negotiation; effectiveness would mean removing Putin from power or regime change.
Finally, outside the scale entirely are sanctions pursued with the vague goal of “punishing” the target state. This is a moralistic rather than strategic use of sanctions. As such, it is important to identify when this goal is in play, as well as to avoid invoking it in analyses of effectiveness. As sanctions expert Daniel Drezner argued in March 2022:
There are a lot of very good reasons to sanction Russia right now, but I am not entirely convinced that those reasons are informing the actual economic statecraft being announced. … The real reason, the one most consistent with all that anger and outrage, is that foreign policy leaders want to punish Russia for what it has done. … Some thought needs to be given about how the sanctions are supposed to work and the conditions under which they can be lifted. Those thoughts need to be codified and articulated to Russia and the rest of the world. … Otherwise, all this behavior is just an exercise in maximizing the economic pain of ordinary Russians without any conception of what that will achieve.
The analytical complexity does not end there. Sanctions can have multiple goals, different sanctioning countries can emphasize different goals, sanctions can have unexpected effects that undermine their goals, and goals can change as the situation evolves. Sanctions initially meant for deterrence can be repurposed if deterrence fails, for example, while initially expansive goals can be moderated over time (or vice versa). As ambitions temper and goals become more limited, sanctions may seem more effective. Public-facing analyses of the effectiveness of Russian sanctions should therefore make their standards of evaluation clear.
Consider the Appropriate Evidence
Scholars, journalists, activists, and politicians alike have consistently disagreed on the economic impact of the sanctions. Given the range, type, and quality of economic evidence available, as well as the pressure to predict rather than evaluate effectiveness, it can be especially tempting to make overly assertive judgements. This makes it vital not only to identify the political goal(s) against which sanctions’ effectiveness is measured, but also to draw on high-quality evidence about specific economic impacts that could plausibly advance those political goals—or, in the absence of high-quality evidence, to moderate effectiveness claims accordingly.
Analysts have not had an easy time of it, as short-term and longer-term economic indicators have often pointed in conflicting directions. The initial post-invasion sanctions in early 2022 were sweeping—and, for the Russian government, unexpectedly so. For example, Moscow did not anticipate that financial sanctions, including a freeze on the reserves of the Central Bank of Russia, would be imposed in a rapid, unified way by countries across Europe and North America. These coordinated government sanctions were complemented by “self-sanctions,” in which private Western companies began to shift financing and operations out of Russia. This led in the short term to a crash in the ruble’s exchange rate, an inflation spike, a temporary shut-down of the Russian stock market, capital flight, and predictions of a massive fall in Russian GDP.
According to many indicators, the Russian economy then bounced back from the initial hit, thanks in large part to skillful financial policy and planning by the Russian central bank and finance ministry, as well as to loopholes in the sanctions regime, particularly regarding oil and gas exports. Inflation and the exchange rate stabilized, available reserves were not excessively depleted, and until fairly recently Russia’s budget numbers remained solid. Russia has been running a (diminishing) current account surplus and its GDP year-on-year has not fallen significantly. The Russian government retains economic partners outside the West and, sometimes with their help, has proven adept at sanctions evasion. The more sanctions ratchet up, the more they result in collateral economic damage to sanctioning and third-party states as well.
That being said, the sanctioning states have actively been closing sanctions loopholes; EU countries have been diversifying away from dependence on Russian oil and gas; and following the introduction of the oil price cap, world oil prices fell and Russian oil traded at a discount. The Russian government has just reintroduced capital controls to ease downward pressure on the ruble, stifling Russia’s growth potential. The sanctions-driven collapse of imports has propped up the current account but has not sparked an import-substitution boom domestically. Russia’s better-than-expected GDP numbers reflect a wartime economy, with disproportionate spending on the military effort and on defense-related industry. Companies that rely on imported technology for their products, including defense-related companies, are having difficulty sourcing alternatives.
Russia has experienced a significant brain drain since the full-scale invasion, has become far more economically dependent on China, and has lost access to important channels of international finance and trade. Russian government statistics are also now less reliable than before, as basic data on capital flows, FDI, credit, and commodity exports are often kept hidden; some argue that this is done to hide the devastating impact of sanctions, while others believe it is to hide the extent of Russia’s successful sanctions evasion.
Systematic evidence of the various ways in which sanctions and self-sanctions are impacting the Russian economy is accumulating (see, for example, KSE Institute Sanctions Group, Bergmann et al. 2023, Hilgenstock et al. 2023, Chupilkin et al. 2023, Evenett and Pisani 2023, Simola 2023), but sanctions are a long-term tool and a moving target. By matching explicit political goals with evidence about the economic impacts that could plausibly advance them—and being honest about the quality of that evidence—analysts can provide more nuanced, less potentially biased accounts of sanctions’ effectiveness and suggestions for improvement. Analysts will still disagree, of course, but at least the terms of the disagreements will be clear.
Avoid Confirmation Bias
If analysts have predetermined arguments to make about sanctions’ effectiveness, they can find pieces of economic data to support them. Skeptics can focus on the rapid rebound, on macroeconomic stability, on loopholes, on the sanctions’ impacts outside Russia, and on Russia’s ability to evade sanctions and build non-Western partnerships, while advocates can point to the longer-term, diversifying, and accumulating economic pain and distortions within Russia itself. Self-consciously matching explicit goals to appropriate evidence can help to avoid this, but it cannot on its own protect our analyses from the broader pitfall of confirmation bias.
People naturally filter information through the lens of their prior beliefs and theories. This is not necessarily a problem in and of itself, but it can become so when these priors are not explicitly taken into consideration by both the producers and the consumers of political analyses. Although analytical priors can assist in framing and understanding complex events, they can also lead us to search for confirmatory evidence, to make overly confident predictions based on preliminary or superficial information about actual conditions on the ground, and to interpret sanctions’ goals (and thus effectiveness) in light of these priors. Moreover, analytical priors may not only spark a search for confirmatory evidence of whether sanctions are effective, but also lead to unquestioned or poorly supported assertions as to why sanctions are or are not effective. This, in turn, shapes the policy prescriptions that follow from such analyses.
Theoretically informed stances on the war and analytical priors regarding sanctions as a tool of statecraft can be two such potential sources of confirmation bias. In the absence of self-reflection, discussions about the effectiveness of sanctions can devolve into a means to advance broader, long-standing arguments about the international system. Is NATO expansion a security threat to Russia, even perhaps the root cause of the war? Can the U.S. negotiate successfully with its adversaries? Should analysts think of the world in terms of great powers and spheres of influence? Is U.S. leadership waxing or waning, and can the U.S. count on its European allies? Is Russia an existential threat to the international liberal order? Can and should sanctions ever be targeted? Does the overuse of sanctions undermine U.S. influence and lead to blowback? Whatever one’s answer to these questions, the challenge is to resist preemptively defining sanctions’ goals and cherry-picking evidence of the economic impact of sanctions primarily as a way to support them.
Confirmation bias may be a particular concern when analysts lack deep regional expertise and when analyses are prospective, attempting to predict rather than evaluate the effectiveness of sanctions policy. International relations scholars and think-tank analysts without Russian area expertise have dominated much of the public-facing discourse on sanctions, both in sheer number of interventions and in the speed with which they publish in response to events. This has given such analysts an important opportunity to shape the terms of the U.S. debate on sanctions. But it also means that they have typically intervened with less understanding of the situation on the ground, as they have tended to write earlier (when information is sketchiest) and without the store of region-specific knowledge that might help to fill in the gaps, increasing the danger of confirmation bias.
Key Takeaways
To sum up, public-facing policy analysis and prescriptions that rely in whole or in part on evaluating the effectiveness of sanctions should follow three guidelines.
First, be explicit about the goalposts. Claims of effectiveness require judgements not only about how and how much sanctions hurt the Russian economy, but also about the policy results of that pain. Saying that sanctions work (or not) should be a claim that their economic impact has succeeded (or not) in sending signals and compelling reactions that advance specific political goals.
Second, match the identified goals to the appropriate evidence. The empirical evidence on the economic impact of sanctions often points in contradictory directions and is of varying quality. Analysts have the daunting task of sifting through the noise as best they can and focusing on the economic indicators that most plausibly contribute to advancing the identified goals. In so doing, it is important to skeptically interrogate the data sources, focus where possible on trends over time rather than snapshots, and acknowledge when existing data is inadequate to support definitive claims.
Finally, be wary of confirmation bias. The combination of confounding empirical evidence and myriad potential policy goals can make analyses of sanctions’ effectiveness unusually susceptible to such bias. Moreover, the unprecedented nature of the current sanctions, both in terms of their scale and in terms of the economic size of the target state, makes it more challenging to extrapolate from prior experience.
Juliet Johnson is Professor in the Department of Political Science at McGill University, Quebec, Canada.
