Possible Insight

Archive for the ‘Markets’ Category

Moneyball for Tech Startups: Kevin’s Remix

with 4 comments

Several people have pointed me to Dan Frommer’s post on Moneyball for Tech Startups, noting that “Moneyball” is actually a pretty good summary of our approach to seed-stage investing at RSCM.  Steve Bennet, one of our advisors and investors, went so far as to kindly make this point publicly on his blog.

Regular readers already know that I’ve done a fair bit of Moneyball-type analysis using the available evidence for technology startups (see here, here, here, here, here, and here).  But I thought I’d take this opportunity to make the analogy explicit.

I’d like to start by pointing out two specific elements of Moneyball, one that relates directly to technology startups and one that relates only indirectly:

  • Don’t trust your gut feel, directly related.  There’s a quote in the movie where Beane says, “Your gut makes mistakes and makes them all the time.”  This is as true of tech startups as it is of baseball prospects.  In fact, there’s been a lot of research on gut feel (known in academic circles as “expert clinical judgement”).  I gave a fairly detailed account of the research in this post, but here’s the summary.  Expert judgement never beats a statistical model built on a substantial data set.  It rarely even beats a simple checklist, and then only in cases where the expert sees thousands of examples and gets feedback on most of the outcomes.  Even when it comes to evaluating people, gut feel just doesn’t work.  Unstructured interviews are the worst predictor of job performance.
  • Use a “player” rating algorithm, indirectly related.  In Moneyball, Beane advocates basing personnel decisions on statistical analyses of player performance.  Of course, the typical baseball player has hundreds to thousands of plate appearances, each recorded in minute detail.  A typical tech startup founder has 0-3 plate appearances, recorded at only the highest level.  Moreover, with startups, the top 10% of the startups account for about 80% of the all the returns.  I’m not a baseball stats guy, but I highly doubt the top 10% of players account for 80% of the offense in the Major Leagues.  So you’ve got much less data and much more variance with startups.  Any “player” rating system will therefore be much worse.

Despite the difficulty of constructing a founder rating algorithm, we can follow the general prescription of trying to find bargains.  Don’t invest in “pedigreed” founders, with startups in hot sectors, that have lots of “social proof”, located in the Bay Area.  Everyone wants to invest in those companies.  So, as we saw in Angel Gate, valuations in these deals go way up.  Instead, invest in a wide range of founders, in a wide range of sectors, before their startups have much social proof, across the entire US. Undoubtedly, these startups have a lower chance of succeeding. But the difference is more than made up for by lower valuations.  Therefore, achieving better returns is simply a matter of adequate diversification, as I’ve demonstrated before.

Now, to balance out the disadvantage in rating “players”, startup investors have an advantage over baseball managers.  The average return of pure seed stage angel deals is already plenty high, perhaps over 40% IRR in the US according to my calculation.  You don’t need to beat the market.  In fact, contrary to popular belief, you don’t even need to try and predict “homerun” startups.  I’ve shown you’d still crush top quartile VC returns even if you don’t get anything approaching a homerun.  Systematic base hits win the game.

But how do you pick seed stage startups?  Well, the good news from the research on gut feel is that experts are actually pretty good at identifying important variables and predicting whether they positively or negatively affect the outcome.  They just suck at combining lots of variables into an overall judgement.  So we went out and talked to angels and VCs.  Then, based on the the most commonly cited desirable characteristics, we built a simple checklist model for how to value seed-stage startups.

We’ve made the software that implements our model publicly available so anybody can try it out [Edit 3/16/2013: we took down the Web app in Jan 2013 because it wasn't getting enough hits anymore to justify maintaining it.  We continue to use the algorithm internally as a spreadsheet app].  We’ve calibrated it against a modest number of deals.  I’ll be the first to admit that this model is currently fairly crude.  But the great thing about an explicit model is that you can systematically measure results and refine it over time.  The even better thing about an explicit model is you can automate it, so you can construct a big enough portfolio.

That’s how we’re doing Moneyball for tech startups.

Written by Kevin

September 27, 2011 at 10:56 pm

The VC "Homerun" Myth

with 9 comments

In spreading the word about RSCM, I recently encountered a question that led to some interesting findings.  A VC from a respected firm, known for its innovative approach, brought up the issue of “homeruns”.  In his experience, every successful fund had at least one monster exit.  He was concerned that RSCM would never get into those deals and therefore, have trouble generating good returns.

My initial response was that we’ll get into those deals before they are monsters.  We don’t need the reputation of a name firm because the guys we want to fund don’t have any of the proof points name firms look for.  They’ll attract the big firms some time after they take our money.  Of course, this answer is open to debate.  Maybe there is some magical personal characteristics that allows the founders of Google, Facebook, and Groupon to get top-tier interest before having proof points.

So I went and looked at the data to answer the question, “What if we don’t get any homeruns at all?”  The answer was surprising.

I started with our formal backtest, which I produced using the general procedure described in a previous post.  It used the criteria of no follow-on and stage <= 2, as well as eliminating any company in a non-technology sector or capital-intensive one such as manufacturing and biotechnology.

Now, the AIPP data does not provide the valuation of the company at exit.  However, I figured that I could apply increasingly stringent criteria to weed out any homeruns:

  1. The payout to the investor was < $5M.
  2. The payout to the investor was < $2.5M
  3. The payout to the investor was < $2.5M AND the payout multiple was < 25X.

It’s hard to imagine an investment in any big winner that wouldn’t hit at least the third threshold.  In fact, even scenarios (1) and (2) are actually pretty unfair to us because they exclude outcomes where we invest $100K for 20% of a startup, get diluted to 5-10%, and then the company has a modest $50M exit.  That’s actually our target investment!  But I wanted to be as conservative as possible.

The base case was 42% IRR and a 3.7x payout multiple.  The results for the three scenarios are:

  1. 42% IRR, 2.7x multiple
  2. 36% IRR, 2.4x multiple
  3. 29% IRR, 2.1x multiple

Holy crap!  Even if you exclude anything that could be remotely considered a homerun, you’d still get a 29% IRR!

As you can see, the multiple goes down more quickly than the IRR. Large exits take longer than small exits so when you exclude the large exits, you get lower hold times, which helps maintain IRR.  But that also means you could turn around and reinvest your profits earlier.  So IRR is what you care about from an asset class perspective.

For comparison, the top-quartile VC funds currently have 10-year returns of less than 10% IRR, according to Cambridge AssociatesSo investing in an index of non-homerun startups is better than investing in the funds that are the best at picking homeruns. (Of course, VC returns could pick up if you believe that the IPO and large acquisition market is going to finally make a comeback after 10 years.)

I’ve got to admit that the clarity of these results surprised even me.  So in the words of Adam Savage and Jamie Hyneman, “I think we’ve got to call this myth BUSTED.”

(Excel files: basecase, scenario 1, scenario 2, scenario 3)

Written by Kevin

August 23, 2011 at 3:10 pm

Startups, Employment, and Growth

leave a comment »

In regards to my posts on how startups help drive both employment and growth, a couple of people have pointed me to this essay in BusinessWeek by Andy Grove.  He says that:

  1. We have a, “…misplaced faith in the power of startups to create U.S. jobs.”
  2. “The scaling process is no longer happening in the U.S. And as long as that’s the case, plowing capital into young companies that build their factories elsewhere will continue to yield a bad return in terms of American jobs.”
  3. “Scaling isn’t easy. The investments required are much higher than in the invention phase.”

His basic argument is more sophisticated than the typical “America no longer makes things” rhetoric. It boils down to network effect. If the US doesn’t manufacture technology-intensive products, we incur two penalties because we lack the corresponding network of skills: (a) in technology areas that exist today, we will not be able to innovate as effectively in the future and (b) we will lack the knowledge it takes to scale tomorrow’s technology areas altogether.  I don’t believe this argument for all the usual economic reasons, but let’s assume it’s true.  Would the US be in trouble?

Grove makes a lot of anecdotal observations and examines some manufacturing employment numbers.  However, I think it’s a mistake to generalize too much from any individual case or particular metric.  The economy is diverse enough to provide examples of almost any condition and we expect a priori to find specialization within sectors.  We should examine a variety of metrics to get the full picture of whether America is losing its ability to “scale up”.  Bear with me, Grove’s essay is four pages long so it will take me a while to fully address it.

Manufacturing Jobs

First, let’s look at the assertion that we aren’t creating jobs relevant to scaling up technologies.  The real question is: what kinds of jobs build these skills?  Does any old manufacturing job help?  I don’t think Foxconn-suicide inducing manufacturing jobs are what we really want.  Maybe what we want is “good” or “high-skill” manufacturing jobs?  Well take a look at this graph from Carpe Diem of real manufacturing output per worker:

That’s right, the average manufacturing worker in the US produces almost $280K worth of stuff per year.  More than 3x what his father or her mother produced 40 years ago. That should certainly support high quality jobs.  But is the quality of manufacturing jobs actually increasing?  Just check out these graphs of manufacturing employment by level of education from the Heritage Foundation:

This suggests to me that we’re replacing a bunch of low-skilled workers with fewer high-skilled workers.  But that’s good.  It means we’re creating jobs that require more knowledge (aka “human capital” that we can leverage).  Look at the 44% increase in manufacturing workers with advanced degrees!  Contrary to Grove, it seems like we’re accumulating a lot of high-powered know-how about how to scale up.

Manufacturing Capability

Now you might be thinking that we could still be losing ground if the productivity of our high-end manufacturing jobs isn’t enough to make up for job losses on the low-end.  In Grove’s terms, our critical mass of scaling-up ability might be eroding. Not the case.  Just consider the statistics on industrial production from the Federal Reserve.  This table shows that overall industrial production in the US has increased 68% in real terms over the 25 years from 1986 to 2010, which is 2.1% per year.

“Aha!” you say, “But Grove is talking about  the recent history of technology-related products.”  Then how about semiconductors and related equipment from 2001-2010?  That’s the sector Grove came from.  There, we have a 334% increase in output, or 16% per year . Conveniently, the Fed also has a category covering all of hi-tech manufacturing output (HITEK2): 163% increase over the same period, or 10% per year. Now the US economy only grew at 1.5% per year from 2001 to 2010 (actually 4Q00 to 3Q10 because the 4Q10 GDP numbers aren’t out).  So in the last ten years, our ability to manufacture high-technology products increased at almost 7x the rate of our overall economy.  We’re actually getting much better at scaling up new technologies!

I can think of one other potential objection. Lack of investment in future production. There is a remote possibility that, despite the terrific productivity and output growth in US high-tech manufacturing today, we won’t be able to maintain this strength in the future. I think the best measure of expected future capability is foreign direct investment (FDI). These are dollars that companies in one country invest directly in business ventures of another country. They do NOT include investments in financial instruments.  Because these dollars are coming from outside the country, they represent an objective assessment of which countries offer good opportunities.  So let’s compare net inflows of FDI for China and the US using World Bank data from 2009.  For China, we have $78B.  For the US, we have $135B. This isn’t terribly surprising given the relative sizes of the economies, but there certainly doesn’t seem to be any market wisdom that the US is going to lose lots of important capabilities in the future.

Will China outcompete the US in some hi-tech industries?  Absolutely.  But that’s just what we expect from the theory of comparative advantage.  They will specialize in the areas where they have advantages and we will specialize in other areas where we have advantages.  Both economies will benefit from this specialization.  An economist would be very surprised indeed if Grove couldn’t point to certain industries where China is “winning”.  However, the data clearly shows that China is not poised to dominate hi-tech manufacturing across the board.

Startup Job Creation

So we’ve addressed Grove’s concerns about the US losing its ability to “scale up”. Let’s move on to the issue of startups. Remember, he said that startups, “…will continue to yield a bad return in terms of American jobs.”  As I posted before, startups create a net of 3M jobs per year. Without startups, job growth would be negative. If Grove cares about jobs, he should care about startups. The data is clear.

The one plausible argument I’ve seen against this compelling data is that most of these jobs evaporate. It is true that many startups fail. The question is, what happens on average? Well, the Kauffman Foundation has recently done a study on that too, using Census Bureau Business Dynamic Statistics data.  They make a key point about what happens as a cohort of startups matures:

The upper line represents the number of jobs on average at all startups, relative to their year of birth. The way to interpret the graph is that a lot of startups fail, but the ones that succeed grow enough to support about 2/3 of the initial job creation over the long term; 2/3 appears to be the asymptote of the top line.  The number of firms continues declining, but job growth at survivors makes up the difference starting after about 15 years.  For example, a bunch of startups founded in the late 90s imploded. But Google keeps growing and hiring.  Same as in the mid 00s for Facebook.  Bottom line: of the 3M jobs created by startups each year, about 2M of them are “permanent” in some sense.  The other 1M get shifted to startups in later years.  So startups are in fact a reliable source of employment.

I’d like to make one last point, not about employment per se, but about capturing the economic gains from startups. If we generalize Grove’s point, we might be worried that the US develops innovations, but other countries capture the economic gains. To dispel this concern, we need only refer back to my post on the economic gains attributable to startups, using data across states in the US.  Recall that this study looked at differing rates of startup formation in states to conclude that a 5% increase in new firm births increases the GDP growth rate by 0.5 percentage points.

I would argue that it’s much more likely that a state next door could “siphon off” innovation gains from its neighbor than a distant country could siphon off innovation gains from the US: (a) the logistics make transactions more convenient, (b) there are no trade barriers between states, and (c) workers in New Mexico are a much closer substitute for workers in Texas than workers in China. But the study clearly shows that states are getting a good economic return from startups formed within their boundaries.  Now, I’m certain there are positive “spillover” effects to neighboring states.  But the states where the startups are located get a tremendous benefit even with the ease of trade among states.


I think it’s pretty clear that, even if you accept Grove’s logic, there’s no sign that the US is losing its ability to scale up.  However, I would be remiss if I didn’t point out my disagreement with the logic. I’ve seen no evidence of a need to be near manufacturing to be able to innovate.  In fact, every day I see evidence against it.

I live in Palo Alto. As far as I know, we don’t actually manufacture any technology products in significant quantities any more. Yet lots of people who live and work here make a great living focusing on technology innovation.  As Don Boudreaux is fond of pointing out on his blog and in letters to the mainstream media, there is no difference in the trade between Palo Alto and San Jose and the trade between Palo Alto and Shanghai.  In fact, I know lots of people in the technology industry who work on innovations here in the Bay Area and then fly to Singapore, Taipei, or Shanghai to work with people at the factories cranking out units.

Certainly, I acknowledge that a government can affect the ability of its citizens to compete in the global economy. But the best way to support its citizens is to reduce the barriers to creating new businesses and then enable those businesses to access markets, whether those markets are down the freeway or across the world.  One of the worst thing a government could do is fight a trade war, which is what Grove advocates in the third-to-last paragraph of his essay.

The ingenuity of American engineers and entrepreneurs is doing just fine, as my data shows.  We don’t need an industrial policy.

Written by Kevin

January 23, 2011 at 11:39 am

Yes, You Can Save the World with Startups

with 2 comments

Dave Lambert pointed me to this new Kauffman Foundation paper by Tim Kane about job creation in the US.  Then Will Ambrosini pointed to this Growthology post which reproduces the money diagram from page 5 :

Look carefully.  Then think about this statement about US job creation:

The only firms that create jobs on average are brand new ones.

So yes, you can save the world with startups.

Written by Kevin

July 9, 2010 at 1:27 pm

Simulating Angel Investment: Kevin’s Remix

with 15 comments

Jeff Miller has done a couple of nice posts on “A Simulation of Angel Investing” here and here. I think it’s terrific that Jeff actually asked the question and tried to answer it with simulation. However, his answer of 20 is way too low because of two key oversimplifications. Using a more sophisticated methodology, I’ll show that a better answer is 100 to 150.

Read the rest of this entry »

Written by Kevin

May 11, 2010 at 11:34 pm

Posted in Economics, Markets, Models

Saving the World with Startups

with 3 comments

On a recent business trip trying to drum up support for RSCM, someone asked Dave and me why such obviously talented guys were starting a fund instead of a company. I’ve been thinking about that question for the last week and have a much better answer than the one I gave.

I want to make the world a better place.  But it’s not clear precisely what interventions will lead to the best outcomes over the long term. I think I’m a really smart guy, but I’m quite sure I can’t evaluate all the potential interactions within a system as complex as the world society to figure out the optimal plan.

Luckily, I don’t have to be that smart. We just have to collectively be that smart. And economic markets are the best way I know to organize collective action. The more effectively we can all create value, the better off we’ll all be. Creating wealth won’t directly solve a lot of problems, but it enables the solution of an incredibly wide range of problems.

So here’s the math that leads to my conclusion that increasing the number of startups we can fund is the best thing I can do for the world. This study shows that a 5% improvement in startup creation leads to about a half a percentage point improvement in the economic growth rate. If we could increase the rate of startup creation by 10%, we could add a full percentage point to our economic growth rate.

From this dataset, I determined that the world GDP growth rate over the last 30 years has been about 4%.  So we could probably achieve a 5% growth rate by increasing startup formation by 10%.

This seemingly small shift has dramatic results over the long term. In 50 years, world GDP would be 60% (1.6x) greater.  In 100 years, GDP would be 160% (2.6x) greater.  I think a world in which everyone were 2.6x richer would be pretty sweet.  That’s a gift I want to give to my great-great grandchildren.

Seed-stage startups are the key because that’s where businesses are born. A larger pool of innovative seed stage companies will naturally attract a larger pool of investment in later stages.  About $10B every year goes to professional investments in seed-stage startups in the US. So if we can add $1B, that’s 10%.  Even better, if we develop a better process, this process can be copied all over the world.  If it’s a lot better, I bet we can do significantly exceed a 10% improvement.

That’s why I’m focusing my time on revolutionizing the process for funding seed-stage startups.

Written by Kevin

May 1, 2010 at 3:27 pm

Two Important Links

with one comment

If you do nothing else intellectual this Sunday, do these two things:

(1) Read Tyler Cowen’s NYTimes column on how the bestowing of political favors was at the heart of the financial crisis and how we’re about to make the same mistake with health care.

(2) Remember Norman Borlaug.  He is the scientist who led the “Green Revolution“.  In my opinion, he would be a strong candidate for the man who did the most good for the most people in the second half of the 20th Century.  And the mainstream media will not make nearly a big enough deal of his death at 95 compared to that of Ted Kennedy.

Written by Kevin

September 13, 2009 at 10:51 am

Is Money an Emergent Phenomenon?

with 15 comments

The two economists that have most informed my view of the current macroeconomy are Arnold Kling and Scott Sumner. In both cases, their models and explanations make sense to me.  They use solid reasoning and evidence; I don’t feel I’m getting a lot of hand waving. Unfortunately, at first glance, their views seem mutually exclusive.  Kling believes business cycles are the result of many planning errors by individual agents (for example, this recent post and this follow up).  Sumner believes business cycles are the result of contractionary monetary policy by the central bank (for example, this recent post and this one).

How can they both be right? I think they are operating at different levels. Yes, individual agents make their particular planning decisions.  In aggregate, these decisions drive monetary variables like interest rates, exchange rates, liquidity demand, etc.  However, these variables then feed back into the next round of planning decisions.  Moreover, at least some of these plans take into account the effect of the agent’s actions on the monetary variables.  So you get classic chaotic/complex behavior with temporarily stable attractors, perturbations, and establishing new regimes. There may even be aspects of synchronized chaos. I think the monetary variables are the key emergent phenomena here.  They are like “meta prices” that provide a shared signal across just about every modern economic endeavor.

Food for thought.  I’m going to keep this in mind when processing future articles on the economy and see if it helps my thinking.

Written by Kevin

August 11, 2009 at 2:46 pm

Specifying a Climate Bet

with 3 comments

As I mentioned in the comments on this post, I am currently in the process of negotiating a bet on Anthropogenic Global Warming (AGW) with another blogger. The challenges are interesting, so I thought I’d give you a peek inside the sausage factory.

Read the rest of this entry »

Written by Kevin

June 24, 2009 at 2:12 pm

Posted in Climate, Markets

Tagged with ,

You Can’t Pick Winners at the Seed Stage

with 9 comments

[EDITED 05/08/2009: see here] The majority of people I’ve talked to like the idea of revolutionizing angel funding. Among the skeptical minority, there are several common objections. Perhaps the weakest is that individual angels can pick winners at the seed stage.

Now, those who make this objection usually don’t state it that bluntly. They might say that investors need technical expertise to evaluate the feasibility of a technology, or industry expertise to evaluate the likelihood of demand materializing, or business expertise to evaluate the evaluate the plausibility of the revenue model. But whatever the detailed form of the assertion, it is predicated upon angels possessing specialized knowledge that allows them to reliably predict the future success of seed-stage companies in which they invest.

It should be no surprise to readers that I find this assertion hard to defend. Given the difficulty in principle of predicting the future state of a complex system given its initial state, one should produce very strong evidence to make such a claim and I haven’t seen any from proponents of angels’ abilities. Moreover, the general evidence of human’s ability to predict these sorts of outcomes makes it unlikely for a person to have a significant degree of forecasting skill in this area.

Read the rest of this entry »

Written by Kevin

April 27, 2009 at 9:59 am


Get every new post delivered to your Inbox.