Possible Insight

More Angel Investing Returns

with 15 comments

According to our Web statistics, my post on Angel Investing Returns was pretty popular, so I thought I’d dive a little deeper into the process of extracting information from this data set.  At the end of the last post, I hinted that there might be some value in, “…analyzing subsets of the AIPP data…”  Why would you want to do this?  To test hypotheses about angel investing.

Now, you must be careful here.  You should always construct your hypotheses before looking at the data.  Otherwise, it’s hard to know if this particular data is confirming your hypothesis or if you molded your hypothesis to fit this particular data.  You already have the challenge of assuming that past results will predict future results.  Don’t add to this burden by opening yourself to charges of “data mining”.

I can go ahead and play with this data all I want.  I already used it to “backtest” RSCM‘s investment strategy.  We developed it by reading research papers, analyzing other data sources, and running investment simulations.  When we found the AIPP download page, it was like Christmas: a chance to test our model against new data.  So I already took my shot.  But if you’re thinking about using the AIPP data in a serious way, you might want to stop reading unless you’ve written your hypotheses down already.  As they say, “Spoiler alert.”

But if you’re just curious, you might find my three example hypothesis tests interesting.  They’re all based loosely on questions that arose while doing research for RSCM.

Hypothesis 1: Follow On Investments Don’t Improve Returns

It’s an article of faith in the angel and VC community that you should “double down on your winners” by making follow on investments in companies that are doing well.  However, basic portfolio and game theory made me skeptical.  If early stage companies are riskier, they should have higher returns.  Investing in later stages just mixes higher returns with lower returns, reducing the average.  Now, some people think they have inside information that allows them to make better follow-on decisions and outperform the later stage average.  Of course, other investors know this too.  So if you follow on in some companies but not others, they will take it as a signal that the others are losers.  I don’t think an active angel investor could sustain much of an advantage for long.

But let’s see what the AIPP data says.  I took the Excel file from my last post and simply blanked out all the records with any follow on investment entries.  The resulting file with 330 records is here.  The IRR was 62%, the payout multiple was 3.2x, and the hold time was 3.4 years.  That’s a huge edge over 30% and 2.4x!

Now, let’s not get too excited here.  There’s a difference between deals where there was no follow on and deals where an investor was using a no-follow-on strategy.  We don’t know why an AIPP deal didn’t have any follow on.  It could be that the company was so successful it didn’t need more money.  Of course, the fact that this screen still yields 330 out of 452 records argues somewhat against a very specific sample bias, but there could easily be more subtle issues.

Given the magnitude of the difference, I do think we can safely say that the conventional wisdom doesn’t hold up.  You don’t need to do follow on. However, without data on investor strategies, there’s still some room for interpretation on whether a no-follow-on strategy actually improves returns.

Hypothesis 2: Small Investments Have Better Returns than Large Ones

Another common VC mantra is that you should “put a lot of money to work” in each investment.  To me, this strategy seems more like a way to reduce transaction costs than improve outcomes, which is fine, but the distinction is important.  Smaller investments probably occur earlier so they should be higher risk and thus higher return.  Also, if everyone is trying to get into the larger deals,  smaller investments may be less competitive and thus offer greater returns.

I chose $300K as the dividing line between small and large investments, primarily because that was our  original forecast of average investment for RSCM (BTW, we have revised this estimate downward based on recent trends in startup costs and valuations).  The Excel file with 399 records of “small” investments is here.  The IRR was 39% and the payout multiple was 4.0x.  Again, a huge edge over the entire sample!  Interestingly, less of an edge in IRR but more of an edge in multiple than the no-follow-on test.  But smaller investments may take longer to pay out if they are also earlier.  IRR really penalizes hold time.

Interesting side note.  When I backtested the RSCM strategy, I keyed on investment “stage” as the indicator of risky early investments.  Seeing as how this was the stated definition of “stage”, I thought I was safe.  Unfortunately, it turned out that almost 60% of the records had no entry for “stage”.  Also, many of the records that did have entries were  strange.  A set of 2002 “seed” investments in one software company for over $2.5M?  A 2003 “late growth” investment in a software company of only $50K?  My guess is that the definition wasn’t clear enough to investors filling out the survey.  But I had committed to my hypothesis already and went ahead with the backtest as specified.  Oh well, live and learn.

Hypothesis 3: Post-Crash Returns Are No Different than Pre-Crash Returns

As you probably remember, there was a bit of a bubble in technology startups that popped at the beginning of 2001. You might think this bubble would make angel investments from 2001 on worse.  However, my guess was that returns wouldn’t break that cleanly.  Sure, many 1998 and some 1999 investments might have done very well.  But other 1999 and most 2000 investments probably got caught in the crash.  Conversely, if you invested in 2001 and 2002 when everybody else was hunkered down, you could have picked up some real bargains.

The Excel file with 168 records of investments from 2001 and later is here.  23% IRR and 1.7x payout multiple.  Ouch!  Was I finally wrong?  Maybe.  Maybe not.  The first problem is that there are only 168 records.  The sample may be too small.  But I think the real issue is that  the dataset “cut off” many of the successful post-bubble investments because it ends in 2007.

To test this explanation, I examined the original AIPP data file.  I filtered it to include only investment records that had an investment date and where time didn’t run backwards.  That file is here.  It contains 304 records of investments before 2001 and 344 records of investments in 2001 or later.  My sample of exited investments contains 284 records from before 2001 and 168 records from 2001 or later.  So 93% of the earlier investments have corresponding exit records and 49% of the later ones do.  Note that the AIPP data includes bankruptcies as exits.

So I think we have an explanation.  About half of the later investments hadn’t run their course yet.  Because successes take longer than failures, this sample over-represents failures.  I wish I had thought of that before I ran the test!  But it would be disingenuous not to publish the results now.

Conclusion

So I think we’ve answered some interesting questions about angel investing.  More important, the process demonstrates why we need to collect much more data in this area.  According to the Center for Venture Research, there are about 50K angel investments per year in the US.  The AIPP data set has under 500 exited investments covering a decades long span.  We could do much more hypothesis testing, with several iterations of refinements, if we had a larger sample.

analyzing subsets of the AIPP dataneed
Advertisements

Written by Kevin

January 25, 2011 at 9:21 pm

15 Responses

Subscribe to comments with RSS.

  1. After reading this post, I went back and read “You Can’t Pick Winners at the Seed Stage”, which I agree with.

    But this raises the question “why should we expect that future angel investments will perform anything like past ones?”

    Paul

    January 26, 2011 at 8:58 pm

    • That’s a good question. But it’s the same one you have to ask for investments in _every_ asset class.

      If you’ve got money that you don’t want to spend now, you have to do something with it. Any possible choice could be wrong in retrospect. In prospect, the performance of any choice could suddenly shift from its previous path. So what do you do?

      A typical answer to this question is the “all asset” answer. You should construct a portfolio that includes all assets. Start with the proportions as they are in the economy as a whole and then adjust based on your risk tolerance.

      Now, as a Bayesian, I acknowledge that a person my have prior beliefs about the future potential of an asset class. But they should certainly take into account past performance.

      Kevin

      January 26, 2011 at 9:39 pm

      • No, for an investor with a fundamental valuation orientation, this is a very different situation compared to some other investment classes. My interpretation of “You Can’t Pick Winners …” is that there is a tremendous amount of Knightian uncertainty (yes, there’s a wikipedia entry) in angel investing. At the other end of the spectrum, you have instruments like a 1-year Treasury bill, which has almost zero uncertainty.

        The ‘all asset’ answer comes from orthodox finance, which is wrong. Taleb is more correct, that you should have a barbell allocation (some in ultra-safe, some in ultra-uncertain). But you will benefit further if you can evaluate relative prospects within an asset class.

        Paul

        January 26, 2011 at 10:02 pm

      • Ironically, I’ve actually got an old blog post on Knightian uncertainty that references Taleb:

        http://emergentfool.com/2008/09/19/dont-cop-out-on-knightian-uncertainty/

        So I don’t think you can say “Knightian” and make any difference in an argument about uncertainty. You still have to do the analysis.

        So let’s talk Taleb. You are right that there is _more_ fat tail uncertainty in angel than T-Bills. But the angel fat tail is is to the _right_ and the T-bills fat tail is to the _left_. According to Taleb, people underestimate the thickness of the tails so you should actually allocate more to angel than you think and less to T-bills than you think in his model (or short T-bills more).

        I don’t know why you think Taleb says you should evaluate relative prospects. His whole point is that you can’t. In fact, what you say about his barbell allocation is actually oversimplified. What Taleb does in his own trading (and yes I’ve talked to him about it), is try to go long on volatility, both positive and negative. So he’s actually in a lot of asset classes, he just takes volatile positions in them.

        So if you want to think about startups in Taleb terms, my recommendation is to look at the historical data and say, “Wow, this is really volatile.” Then go long on the most volatile segment of the market, which is the seed stage.

        Kevin

        January 26, 2011 at 11:00 pm

      • We should probably try to talk some time!

        Paul

        January 26, 2011 at 11:17 pm

      • I didn’t oversimplify what Taleb says about barbell allocation; I just emphasized what he got right: there isn’t much to gain from investing in moderate uncertainty.

        And I don’t think Taleb says you should evaluate relative prospects which is why I wrote “But [that is, contrary to Taleb] you will benefit further if you can evaluate relative prospects.”

        Oddly though, you report that Taleb does attempt to evaluate relative prospects: by using volatility as a surrogate.

        But relative prospects can definitely be measured, even within a highly uncertain asset class. As a trivial example, which investment would you rather make: $100,000 for 50% of a startup? Or $500,000 for 50% of that same startup? More examples of possible factors that may impact startup success appear on the RSCM site: risk share of the founders, preconceptions, constraints, drive, ….

        To the degree that you can find ways to assess relative prospects, you will make better investing decisions. Volatility is one method, but not the only one. Hopefully RSCM’s experiments will identify many new measures.

        Paul

        January 27, 2011 at 2:01 am

      • I think the issue has something to do with viewing radical innovation (i.e
        seed market) as an “asset class”. In some sense black swans are the
        anti-class. In other words, anytime you call something an asset you are
        excluding everything in the set of adjacent-possible, which is to say
        innovation. Investing in innovation is not about defining asset classes but
        rather commitment to certain processes.

        Rafe Furst

        January 27, 2011 at 3:58 am

      • I think we’re mostly in agreement, except for some very fine distinctions. Sorry I misunderstood what you were attributing to Taleb and what you weren’t.

        As far as I know, Taleb doesn’t evaluate the prospects of specific bets. He looks at the volatility of large classes of bets. I think this is significant, but I can see how someone else might not.

        I don’t want to discuss RSCM too much here (for legal reasons), but we actually aren’t evaluating the relative prospects within the class of bets we wish to make. We’re trying to filter out bets that aren’t in the class. Again, I think this is significant.

        Now, we do plan on collecting _a_lot_ of data and testing hypotheses to see if we can get some forecasting skill. But we’re going to be very careful. My prediction is that we won’t be able to come up with a fine-grained predictive algorithm, but rather identify broader sub-classes of investments with the best expected payoff. I’ll be happy if I’m proven wrong though.

        Anonymous

        January 27, 2011 at 3:58 am

  2. Why is it so common for early stage investors in companies to not invest in a company’s later stage rounds if the company is successful?…

    Actually, a lot of VCs do invest in follow rounds. It’s pretty common. But on average over the long term, later rounds return less than earlier rounds. This is basic portfolio theory: reward is proportional to risk. So you will dilute the return of yo…

    Quora

    May 5, 2011 at 8:07 pm

  3. […] Row”.  Then a bunch of the cell references will be broken.  [Update 1/27/11: I've done a follow up post on using this method to test various hypotheses.] Share this:TwitterFacebookLike this:LikeBe the […]

  4. […] of Moneyball-type analysis using the available evidence for technology startups (see here, here, here, here, here, and here).  But I thought I’d take this opportunity to make the analogy […]

  5. What is the best way to angel invest a million dollars?…

    Thanks to Jeremy above for referencing my blog post. As he says, the data shows pretty clearly that you need diversification. However, contra his final paragraph, the data also shows pretty clearly that you don’t need to do follow on. In the Kauffman …

    Quora

    March 22, 2012 at 3:18 am

  6. How do early stage investors avoid getting crammed down by VCs in later rounds?…

    I’m not sure this is the right question. At least, it depends on the answer to another question: Do early investors actually get crammed down often enough to warrant special precautions? The data I’ve seen for angels is that it doesn’t. If you look …

    Quora

    May 9, 2012 at 5:20 am

  7. […] capital-efficient technology companies (a sample I generated using the methodology described in this post), I found that the top 5% of outcomes accounted for 57% of the payout. That’s substantially […]

  8. […] post. In principle, you could create a filtered data file for your own strategy as described in this other post and then apply the procedure below to calculate the precise diversification tradeoff for your […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: