Black Swan Lessons - You Can't Graph the Future
Something about most UN and NGO security reports has always made me uneasy. Don’t get me wrong. It’s not that they aren’t thorough. A lot of work goes into fact checking and ensuring that what they say is ’correct’. It’s just that the typical security report is a comprehensive list of recent past incidents combined, if we are lucky, with their assessed causes. Incident statistics are then charted and 'trends' are identified. This always made me a little nervous.
To be fair I never really knew why it made me nervous until I read “The Black Swan”. Nicholas Taleb raises several points that help explain my unease.
The first is that more information is not necessarily better. Its very easy to get bogged down in detail that has no real relevance to the issue at hand.
The second factor is what Nicholas calls the Ludic Fallacy
. In brief this is the assumption that the unexpected can be predicted by extrapolating from statistics based on past observations. Nicholas argues that while this holds true for theoretical models based on games of chance it seldom holds true in the real world for the following reasons:∗
We don’t know what we don’t know. (See the Unknown Unknown
Very small (perhaps imperceptible) changes in the variables can have a huge impact in the outcome. This is commonly referred to as the Butterfly Effect
Theories based on experience are fundamentally flawed as events that have not occurred before (or are outside living memory) cannot be accounted for.
The Washington Post graphic below, which shows the frequency and lethality of suicide attacks since 1981, illustrates the problem. If we had examined the chart in 2000 would it have led us to predict 9/11(a classic Black Swan)? If we had re-examined it in 2003 would it have led us to predict the sudden increase in the frequency of attacks in 2007? What does 2007 tell us about 2008? Looking at the trend from 1981 to 1989 how many researchers would have concluded that suicide attacks were in decline and opined that such attacks were ineffective in accomplishing the attackers goals.