Don't Cop Out on Knightian Uncertainty
I apologize for the posting lull. I actually spotted an issue than I wanted to address a few weeks ago, but I’ve been pondering how to approach it. It’s pretty complicated and subtle. I even ran a couple of drafts by Rafe to refine my thinking. So please bear with me.
As I’ve mentioned before, I am a fan of Dave Zetland. When I saw him propagate what I think is a fundamentally false dichotomy in this post, I knew I had to take on the concept of Knightian uncertainty. It crops up rather often in discussions of forecasting complex systems and I think a lot of people use it as a cop out.
Uncertainty is all in your mind. You don’t know what will happen in the future. If you have an important decision to make, you need an implicit or explicit model that projects your current state of knowledge onto the space of potential future outcomes. To make the best possible decision, you need the best possible model.
Knight wanted us to distinguish between risk, which is quantifiable, and uncertainty, which is not. If you prefer the characterization of Donald Rumsfeld, risk consists of “known unknowns” and uncertainty consists of “unknown unknowns”. This taxonomy turns two continuous, intersecting spectra into a binary categorization.
There are some random events where we feel very confident about our estimation of their likelihood. There are other random events where we have very little confidence. These points define the confidence spectrum. Moreover, there are some events that we can describe very precisely in terms of the set of conditions that constitute them and the resulting outcomes. Others, we can hardly describe at all. These points define the precision spectrum. There’s obviously some correlation between confident likelihood estimation and precise event definition, but it’s far from perfect. Unsurprisingly, trying to cram all this subtlety into two pigeon holes causes some serious analytic problems.
The biggest problem is that proponents of the Knightian taxonomy say that you can use probability when talking about risk but not when talking about uncertainty. Where exactly is this bright line? If we’re talking about a 2 dimensional plane of confidence vs precision, drawing a line and saying that you can’t use probability on one side is hard to defend.
Now, the Knightians do have a point. As we get closer to the origin of the confidence vs precision plane, we enter a region where confidence and precision both become very low. If we’re looking at a decision with potentially tremendous consequences, being in this region should make us very nervous.
But that doesn’t mean we quit! “Knightian uncertainty” is not a semantic stopsign. We don’t just throw up our hands and stop analyzing. As I was writing this post, Arnold Kling pointed to a new essay by Nassim Taleb of The Black Swan fame. Funnily enough, Taleb has a chart very much like the confidence vs precision plane I propose. His lower right quadrant is similar to my origin region. Taleb says this area represents the limits of statistics and he’s right. But he still applies “probabilistic reasoning” to it. In fact, he has a highly technical statistical appendix where he does just that.
Before I saw Taleb’s essay, a draft of this post included a demonstration that for any probabilistic model M that ignored Knightian uncertainty, I could create a probabilistic model M’ that incorporated it. M’ wasn’t a “good” model mind you, I merely wanted an existence proof to illustrate that we could apply probability to Knightian uncertainty. The problem of course was that the new random variables in M’ all reside in the danger zones of Taleb’s and my respective taxonomies. But Taleb’s a pro and he’s done a far better job than I ever could of showing how to apply probabilistic reasoning to Knightian uncertainty. So I won’t inflict my full M vs M’ discussion on you.
The key take home point here is that you can in fact apply probability to Knightian uncertainty. Of course, you have to be careful. As Taleb wisely notes in the essay, you shouldn’t put much faith in precise estimates of their probability distributions. But this is actually good advice for all distributions, even well behaved ones.
Back when I was in graduate school, my concentration was in Decision Analysis, which included both theoretical underpinnings and real-world projects trying to construct probabilistic models. I dutifully got my first job applying this knowledge to electrical power grid planning. What I learned was that you should never rely on the final forecast having a lot of precision. Even if you’re dealing with well behaved variables. Because if you put a dozen or so well behaved variables together, the system still often becomes extremely sensitive.
However, “doing the math” leads to a much deeper qualitative understanding of the problem. You can identify structural deficiencies in your model and get a feel for how assumptions flow through to the final result. Most importantly, you identify which variables are the most important and which you can probably ignore. Often, the variation in a couple will swamp everything else.
For example, one of insights Taleb identifies is that you should be long in startup investments (properly diversified, of course). That’s because the distribution of Knightian outcomes is asymmetric. Your losses are bounded by your investment but your gains are unbounded. Moreover, other people probably underestimate the gains because we don’t have enough data points to have seen the upper bounds on success. There’s a bunch of somewhat complicated math here having to do with the tendency to underestimate the exponential parameter in a power law distribution, but most numerate folks can understand the gist and the gist is what counts. The potential of very early startups is systematically underestimated. Now, this isn’t just some empty speculation. I’m actually taking this insight to heart and trying to create a financial vehicle that takes advantage of this.
I’ll give you an example from another of my favorite topics, climate change. I would love for someone to try and apply this sort of analysis to climate change outcomes. We have a power law distribution on our expectations of climate sensitivity for both CO2 warming and aerosol cooling. We also have a power law distribution on our expectations of natural temperature variability. If someone really good at the math could build a model and run the calculations, there are some very interesting qualitative questions we might be able to answer.
First, is the anthropogenic affect on future temperatures roughly symmetric, i.e., could make things colder or warmer? Second, and more importantly, is the anthropogenic contribution to variability significant compared to natural variability? If it isn’t, we should budget more for adaptation than mitigation. If it is, the reverse. But to get these answers, we need to be able to manipulate symbols and make calculations. Probability is the only way I know to do this. So saying you can’t use probability to tackle Knightian uncertainty seems like a cop out to me. How else are we suppose to make big decisions that allocate societal resources and affect billions of people?
Possible Insight 
In doing research for an upcoming power point presentation I read your article and thought the following would be of interest to you.
“Best Fit for Best Practice Governance” http://www.sfomag.com/article.aspx?ID=1281&issueID=c posits that disclosure of the underlying economic environment randomness as either determinate or indeterminate is a precondition for effective capital market governance. Cash flow is the bright line that demarcates risk from uncertainty. The current “one-size-fits-all” legacy approach is obsolete and requires continual updating to accommodate capital market complexities. Trying to reconcile the informational discontinuities of determinate and indeterminate domains in a one-size-fits-all regulatory regime is analogous to having one motor vehicle code for the US and UK.
Stephen A. Boyko
Author of “We’re All Screwed: How Toxic Regulation Will Crush the Free Market System” and a series of five SFO articles on capital market governance.
http://w-apublishing.com/Shop/BookDetail.aspx?ID=D6575146-0B97-40A1-BFF7-1CD340424361
Book Review: Brenda Jubin, Ph.D Thursday, October 8, 2009
http://readingthemarkets.blogspot.com/2009/10/boyko-were-all-screwed.html#comments
Stephen A. Boyko
January 9, 2010 at 4:12 pm
Stephen, I cannot disagree with you more. You are creating an even more unstable system by “best fitting” capital requirements. what does historical cashflow have to do with unknown unknowns?!?! if you want REAL best practice, do this:
1. break up every systemically critical bank within 10 years
2. focus on gradually reducing leverage in the whole system. you can start with the 9:1 money creation fiat money scheme.
3. go after any incentive scheme that doesn’t have clawbacks until the date all the unkown unknowns of an executive’s decision are realize (aka maturity date)
but yeah, not surprising to see that the financials are pushing custom leverage. this way they can sell to the public how safe they are on x and y and then sell a bunch of leveraged paper on z, get the bonus, sell a little stock, and scream “unprecedented” when it all blows up.
Alex Golubev
January 10, 2010 at 9:06 pm
Your approach still appears to try and distinguish between risk and uncertainty which, as I originally expressed, I think is a false dichotomy. Moreover, your approach of trying to use current cash flow as a dividing line is doomed to failure. What matters is the expectation of future cash flows.
kevindick
January 10, 2010 at 7:50 pm
Messrs. Kevindick and Golub:
EF Comment: You are creating an even more unstable system by “best fitting” capital requirements.
Where was this ever stated? I argue for treating pricing and practice as reflexive propositions within the context of randomness.
EF Comment: what does historical cashflow have to do with unknown unknowns
True, I did not distinguish historical vs. future cash flows—but will discuss down the road as to regulatory practicalities. That is a valid consideration. My main objective is to change the balance sheet capital emphasis.
EF Comments:
1. break up every systemically critical bank within 10 years
2. focus on gradually reducing leverage in the whole system. you can start with the 9:1 money creation fiat money scheme.
It will be difficult to do this absent a change in regulation. Regulation fosters oligopolies not competitive enterprises. Track the correlation of capital market regulation as contained in the federal register with the “capital base” of the top 25 financial institutions.
EF Comment:. go after any incentive scheme that doesn’t have clawbacks until the date all the unknown unknowns of an executive’s decision are realize (aka maturity date)
Agreed
EF Comment: not surprising to see that the financials are pushing custom leverage. this way they can sell to the public how safe they are on x and y and then sell a bunch of leveraged paper on z, get the bonus, sell a little stock, and scream “unprecedented” when it all blows up
You make my point as to the need for pricing and practice being a reflexive proposition within the context of randomness
Stephen A. Boyko
January 11, 2010 at 8:47 am
I believe this requires three questions to be answered:
1. Does uncertainty (non-measurable randomness) exist in the capital market to a material degree?
2. If so, can it be governed by the legacy, one-size-fits-all deterministic communication metrics?
3. What is the bright line that separates risk from uncertainty?
For the record, I have argued that:
• Given that there is complexity in the capital markets, there is uncertainty.
• The capital market must undergo a Gaulian segmentation for predictable, probabilistic, and uncertain regimes for effective and efficient governance, and
• Transitional cash flow (historical and future) guidance demarcates risk from uncertain regimes.
Stephen A. Boyko
January 12, 2010 at 9:09 am
looking at past cash flow leads to a feedback loop between low volatility and higher leverage/credit. Our political and corporate decision makers are paid to take short term risks with small long term repercussions. I am unclear how you incorporate Minsky’s ideas. If you’re not addressing this, i’m afraid you’re just buying time.
Alex Golubev
January 12, 2010 at 10:09 am
Sorry, didn’t see your first comment… I would still like to hear whether you’re focusing on the Minsky cycle problem
Alex Golubev
January 12, 2010 at 10:11 am
Stephen. I will answer your three questions from my perspective.
(1) Depends on what you mean by non-measurable. If you mean non-frequentist, then yes. But you can always generate a Bayesian measurement of any uncertainty.
(2) In _Plight of the Fortune Tellers_, Rebonato makes a good argument that the answer is yes. He advocates “low-nines” VAR (say 95%), tempered by Bayesian judgement, and multiplied times a substantial safety factor. I found his argument fairly convincing. I agree with him that “high nines” VAR is silly.
(3) There is none. This is my problem with your whole approach. You never know for sure whether _any_ real world stochastic process is stationary. So no matter how high frequency your historical data, there is always an element of what you call uncertainty due to potential non-stationarity. Conversely, all “uncertainties” are measurable in a Bayesian sense, though your prior may be very diffuse. This is a fundamentally false dichotomy.
kevindick
January 12, 2010 at 1:39 pm
In the book “We’re All Screwed” four systemic crashes are analyzed through a descriptive version of Soros’ boom-bust model below. Below are excerpts from pages 29-31. If you desire more description with diagrams send your email address.
WAS excerpts
To analyze this phenomenon, a modified version of George Soros’ boom-bust model (SFO Magazine, April 2009) is used for the purpose of illustration. The model contains two segments, a Schumpeterian boom schematic and a Minsky bust schematic. The boom cycle integrates state-of-practice concepts with state-of-art applications. This changes the terms of competitive engagement and create investment opportunities for early-adapters.
trend analysis is a function of risk management. Increases in the trend’s risk are related to the trend’s biases.
Each schematic is comprised of four phases. The boom schematic consists of the introductory phase, acceleration phase, tipping point, and the manic phase.
The bust segment finds state-of-art technology being introduced. One of the most important decisions in economics is determining the demand for major capital goods relative to sources of funds from financial institutions and wealthy individuals. On both sides of the capital investment decision we have agents who must put a present value on various long-lived assets that are subject to large potential capital losses. To evaluate an investment project, the expected cash flows over the project’s lifetime must be estimated. To rationally compose a portfolio requires estimates of long-term financial asset prices over the planning horizon. The key question confronting the theory of agent choice is: What do the agents of choice know about the future and how do they come to know it?
To fund the development of next generation technology and capital goods, business plans’ financial pro formas employed long-term extrapolations based on short-term hyperbolic growth experience. This created far-from-equilibrium conditions that were unstable. This created the bust cycle’s Minsky moment. The bust schematic consists of the point of discontinuity, moment of truth, crash, and codification phases.
Bubbles, once broken, are not easily reassembled. The E*trade illustration of the trend’s inflection point represents subjective judgment as they changed their business model to spend a significant amount to get new customers. Other points that just as easily could have been chosen to illustrate a shift in sentiment are: the success tax of the Microsoft’s anti-trust suit, the Fed tightening causing a rise in interest rates as Greenspan wanted to curb “irrational exuberance,” and greater systemic risk from the political uncertainty attendant to the 2000 Presidential election. Whatever point was chosen, there was a point when investors should have become defensive because they recognized the impending Minsky moment.
Stephen A. Boyko
January 12, 2010 at 1:32 pm
Kevindick: Let’s dicuss #3 where we seem to be farther apart.
There is none. This is my problem with your whole approach. You never know for sure whether _any_ real world stochastic process is stationary. So no matter how high frequency your historical data, there is always an element of what you call uncertainty due to potential non-stationarity. Conversely, all “uncertainties” are measurable in a Bayesian sense, though your prior may be very diffuse. This is a fundamentally false dichotomy.
I follow your logic but disagree with your conclusion because your facts flow from a one-size-fits-all governance metric. I argue that you need separate regimes that would provide more correlated information and more efficient pricing as it would reduce noise traders.
See: 21. “Small is Beautiful”, The National Interest, No. 77 – Fall 2004.
http://www.findarticles.com/p/articles/mi_m2751/is_77/ai_n6353167/print
Presented at SEC Small Business Conference in 2004
Stephen A. Boyko
January 12, 2010 at 2:20 pm
Um, no. My facts don’t flow from any governance metric. They flow from a Bayesian philosophy of probability. What I said would be just as true if there were no governance at all.
As I have stated in this thread no less that three times now, I DON’T AGREE WITH YOUR DISTINCTION BETWEEN RISK AND UNCERTAINTY!
Until you internalize that statement, we can’t have a productive discussion.
kevindick
January 12, 2010 at 2:52 pm