global trends, values, and ethics

Chapter 4

Selection Bias

In the policy world, selection bias happens when all meaningful options are not given due consideration. A limited option set could preclude planning for a possible world not considered in that set. The chief importance of such biases is that they compromise planners’ abilities to steward policies that will safeguard our values.

For example, an option set that considers only multipolarity, demography, and climate change but not individual empowerment would preclude leaders from considering individual-level policies focused on civil society, behavioral economics, education, or psychology. Instead, such an option set would limit policymakers to focus on policies where the nation-state is at the center. For instance, influential hackers and networked individuals—embodied today, for example, by Julian Assange, Wikileaks, Anonymous, Bradley Manning, and Edward Snowden—may have the ability take advantage of such blind spots of states. However, arguably the most significant failing would be to neglect developing policies focused on liberty, justice, peace, and pluralism—values that even these actors, in their own ways, claim to aspire toward.

The great challenge for policy planners is how to properly conceptualize the way that we can predict—or whether we can predict—the future world. Drawing upon Isaiah Berlin’s seminal essay “The Hedgehog and the Fox,” psychologist Philip Tetlock frames two ways that forecasters can envision the world.[16] Hedgehogs know one big thing. They have one grand theory about the world, which extends to all matters. Their explanations about the world follow clock-like regular patterns and are parsimonious: simple, highly explanatory, and deterministic. Foxes on the other hand know many small things. They are skeptical about grand theories and are more willing to change their thinking based on circumstances and actual events. Their explanations are more cloud-like and complex and less ambitious and predictable. Just how good are expert hedgehogs and foxes able to accurately forecast the future? In a twenty-year study, Tetlock tested whether both hedgehog-like and fox-like experts from a number of fields (not unlike the ones who participated in the NIC study) were able to accurately predict the long-term future. He found that on the whole, their predictions performed only slightly better than chance, but less well than computer algorithms. In short, experts are not very good at predicting the future.

This does not mean that the NIC’s predictions are useless or that predictions cannot improve. Tetlock further found that foxes were better than hedgehogs at predicting the long-term future during the Cold War. Forecasters can make two types of mistakes: false positives or false negatives. False positives—committed more by hedgehogs—erroneously predict worlds that will not happen, e.g., the existence of weapons of mass destruction in Iraq. False negatives—committed more by foxes—fail to predict worlds that will happen, e.g., the start of the two world wars or the game-changing effects of the invention of nuclear weapons. One standout failing of the NIC study is that they do not assign to their predictions the probability of such errors. Tetlock’s primary takeaway message is not simply that we should think more like foxes because they are more often right. Instead, the goal of forecasting should be to better balance these two ways of thinking, finding the optimal forecasting frontier where the tradeoff between false positives and negatives can be no better, i.e., the denial of more false positives will not lead to more false negatives and the denial of more false negatives will not lead to more false positives.

In providing us with a spectrum of possible worlds rather than a dichotomy or single world, Walzer’s conceptualization helps us understand the range of true and false positive and negative worlds. Since this spectrum is neither nebulous nor unyieldingly deterministic, we can begin to arrive at this optimum frontier where the logics of the cloud and the clock converge, holding that the future is neither radically unpredictable nor completely predictable.

We can now better understand how our biases may lead to false positives or negatives—and take measures to mitigate them. Selection bias can originate from many sources. For example, incomplete information, limited budgets, limited time, or absentmindedness may lead to the exclusion of meaningful options. Here, however, I will focus on three kinds of biases: traditional, methodological, and temporal. The former type of bias arises from more conventional notions of the term. Idiosyncratic preferences of culture, ideology, or individual inclinations may lead a researcher or organization to favor one input over another. I define the second bias as those biases that arise from selective surveying, i.e., some important groups may have been excluded from a study or some groups’ opinions may be weighted more than others. Finally, temporal bias refers to the limited knowledge that steers policymakers into predicting certain trajectories at a given time. This risk points at the danger of making inferences about the future based on isolated incidences.

Traditional selection bias can come from more conventional understandings of the term. Personal or political preferences can lead one to believe in certain assumptions and to forego valid policy prescriptions. Perhaps the most glaring example of such a bias is that the NIC expected its publication to be read by a US audience rather than a European, Chinese, or Brazilian one. Is it a coincidence that a country that tends to value individual liberty over other values (e.g., distributive justice) compared to other nations also foresees a future where the individual is empowered to be freer than today? Would a Russian or Chinese government’s intelligence agency have predicted the same?

Further, methodological surveys of different groups may lead to different conclusions. For example, favoring the opinions from Google, Facebook, and Twitter may have led the NIC to have too Panglossian a view of the panacea of technological advancement and individual empowerment.[17] Thus, the potential for the malicious use of technology to worsen the world was not duly considered: the Luddite opinion was not heard.

A further consideration not considered was that new technology may have only a marginal effect on our values. Peter Thiel—a prominent Silicon Valley venture capitalist, co-founder of PayPal and Palantir, and the first outside investor of Facebook—in public discussions on vertical versus horizontal progress has surprisingly argued such.[18] Despite developments in today’s technologies, particularly in the realm of the Internet, we have not seen many fundamental innovations that have radically changed our lives and what we value. For example, drones may have reduced casualty numbers of those states (promoting the liberty of some) that deploy them and increased the number of casualties of innocent bystanders (denying the liberty of others), but no technology—not even the interconnectedness and uniting forces supposedly offered by Google, Facebook, and Twitter—have fundamentally changed human nature to prevent war from happening.

Another conclusion not considered might be when consultations of only government officials lead to a state-centric view of the future rather than one of a grassroots, international movement that may be foreseen by members of civil society. This bias might cause leaders to ignore the potential for individuals to affect their own futures with their own perceptions and volitions in a manner that states are either ill equipped to do on their own. For example, regardless of whether you despite or praise whistleblower Eric Snowden, he has irrefutably challenged the status quo for better or worse.

The NIC deserves much credit in its attempt to mitigate traditional and methodological biases. In particular, its many meetings with various national and international groups and its new considerations of so-called black swan events have led to the inclusion of many meaningful options. The NIC also commissioned reviews of its four previous studies and sought to address blind spots, biases, and strengths. The NIC compared notes with academics, think tanks, and governments across the globe including in the EU, China, India, Russia, and Africa. It does not boast about predetermined predictions, and its opening pages address not only its previous flaws but also international reactions to the report.

However, although the NIC drafters included as many options that they could think of, “Global Trends 2030” did not eliminate all biases. Perhaps the best way to appreciate how biases can shape futures is by comparing other possible future worlds not included in the US option set. A concrete way to do this is to look at others’ predictions of future worlds. How have other states and organizations envisioned the future?

[16] See Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2005).

[17] For a more detailed account of such optimism see, Schmidt and Cohen, The New Digital Age. For a critique of technological solutionism, see Evgeny Morozov, To Save Everything, Click Here (Philadelphia, PA: Perseus Book Groups, 2013).

[18] See the forthcoming book by Peter Theil and Blake Masters, Zero to One: Notes on Stratups or How to Build the Future (Crown Business, 2014) or Masters’s lecture notes on Thiel’s spring 2012 Stanford course, “CS183: Startup” at www.blakemasters.com/peter-thiels-cs183-startup .

Copyright © 2014 by Thong Nguyen

Creative Commons License
Of All Possible Future Worlds: Global Trends, Values, and Ethics by Thong Nguyen is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.