Survivorship Bias - Why Fund Databases Remember Only the Winners

By EC Assets Research Team, Performance Analytics · Published

Survivorship Bias: Survivorship bias is the error that arises when performance is measured only on the funds, stocks or strategies that still exist. Funds that close are mostly the ones that did badly, so an average over the survivors overstates what the whole group earned, by around 1 percent a year in long mutual fund samples.

What Survivorship Bias Is

Survivorship bias is the error of measuring a population by the members that are still around. In finance the population is usually a set of funds, stocks or strategies, and the members that disappear are not a random sample: funds close after losses, stocks are delisted after collapses, and strategies are abandoned once they stop working. An average taken over what remains is an average over the winners and over those that have not yet lost enough to leave.

The consequence is that a database can report positive alpha for a category in which nobody had any. Nothing in the surviving records is false. Every return was earned and reported correctly; it is the missing records that do the damage.

Brown, Goetzmann, Ibbotson and Ross (1992) formalised the problem for performance studies and showed that survivorship does more than shift averages: it can make performance appear persistent when it is not, which is the property that track-record analysis relies on most. Elton, Gruber and Blake (1996) measured it for mutual funds by following every fund that existed at the end of 1976, including those that later merged or closed.

How It Works

Let s be the share of funds still alive at the end of a window, with average return r_s, and let the closed funds have average return r_d up to the point at which they disappeared. The whole population earned the weighted average of the two, while a database of survivors reports r_s alone. The difference is

$$\text{bias} = r_s - \big[s,r_s + (1-s),r_d\big] = (1-s),(r_s - r_d)$$

Two quantities drive it: how many funds left, and how much worse they were than those that stayed. A category with low attrition, or one in which funds close for reasons unrelated to performance, carries little bias. A category in which poor results are the main reason for closing, as they are for active mutual funds and hedge funds, carries a lot.

The bias also depends on the length of the window. Sponsors rarely close a fund after one bad year; they wait for a run of them. Carhart, Carpenter, Lynch and Musto (2002) show that such a multi-year survival rule makes the bias grow with sample length, and find exactly that in US mutual fund data: 0.07 percent a year in one-year samples, rising to about 1 percent a year in samples longer than 15 years.

Hedge fund databases add a mirror-image distortion. Reporting is voluntary, so a fund typically joins after a good incubation period and brings that history with it. This backfill, or instant-history, bias adds good years at the start of records in the way survivorship removes bad years at the end. Fung and Hsieh (2000) set out both biases for hedge funds and proposed measuring the industry through funds of funds, whose own records carry less of them. Malkiel and Saha (2005) found backfilled returns about 5 percent a year higher than the returns funds reported after joining.

Worked Example

Take 1,000 managers with no skill. Each year a manager beats the benchmark by 5 percentage points or trails it by 5, with equal probability and independently across years and managers. Excess returns are added rather than compounded to keep the arithmetic exact. The sponsor closes a fund, and the database drops it, once its cumulative shortfall reaches 10 points.

After five years:

Group Funds Cumulative excess return Per year
Survivors in the database 625 +6.0 points +1.2%
Closed in year 2 250 -10.0 points -5.0%
Closed in year 4 125 -10.0 points -2.5%
All 1,000 managers 1,000 0.0 points 0.0%

Closures fall only in even years: after an odd number of years the cumulative excess is an odd multiple of 5 and cannot equal -10.

The database reports an average excess return of 1.2 percent a year and a survival rate of 62.5 percent, which reads like a modestly skilled category with ordinary attrition. The true figure is zero. The formula accounts for the gap exactly: (1 - 0.625) × (6 - (-10)) = 0.375 × 16 = 6 points of cumulative bias, which is precisely the survivors' apparent edge.

A reader of the database cannot see any of this. The 625 surviving records are genuine, and some of them are long runs of outperformance. Without the 375 closed funds there is no way to tell that the category's whole positive average is produced by the closure rule.

When It Applies (and Limitations)

Wherever exits depend on results. Fund databases, peer-group rankings, hedge fund indices built from reporting funds and backtests run on today's index members all qualify. A test of a stock-selection rule on the current constituents of an index never holds the companies that were removed after falling, which is why careful index construction and backtesting rely on point-in-time membership lists.

It distorts persistence in either direction. When funds differ in risk, volatile funds either do very well or leave, so their survivors look consistently strong and a survivor-only sample shows winners repeating even if every outcome is luck; this is the effect Brown and co-authors demonstrated. When funds are instead closed on multi-year records, Carhart and co-authors find that survivor conditioning weakens the evidence of persistence. Either way the sample answers a different question from the one asked.

Not every closure is a failure. Funds also disappear through mergers, the liquidation of small but successful vehicles, managers retiring on strong records, or funds that stop reporting after closing to new money. These exits push the bias the other way and are one reason estimates differ between studies.

The size is an estimate, not a constant. It depends on the window, the category, the database's rules for dropping funds and the period studied. The 1 percent figure for long mutual fund samples is an order of magnitude, not a correction to subtract mechanically.

Why It Matters for Institutional Investors

Category averages are upper bounds. A peer median built from live funds describes the managers that survived, not the opportunity set an allocator faced at the start of the period. Comparing a candidate with that median flatters the category and understates how often active choices fail.

The firm's whole record is the track record. A manager that has launched ten funds and closed six can present four excellent ones. In due diligence, asking for every product the firm has launched, merged or shut down, with dates, turns a highlight reel into a sample. It is one of the cheapest questions in manager selection and among the most informative.

Backtests need their dead. Systematic strategies are judged on simulated history before they have a real one. A backtest on a survivor-only universe is tilted towards whatever the survivors had in common, often size, quality or the absence of distress, and the live result tends to disappoint by roughly that amount.

References

  1. Brown, S. J., Goetzmann, W., Ibbotson, R. G., & Ross, S. A. (1992). Survivorship Bias in Performance Studies. Review of Financial Studies, 5(4), 553-580.
  2. Elton, E. J., Gruber, M. J., & Blake, C. R. (1996). Survivor Bias and Mutual Fund Performance. Review of Financial Studies, 9(4), 1097-1120.
  3. Carhart, M. M., Carpenter, J. N., Lynch, A. W., & Musto, D. K. (2002). Mutual Fund Survivorship. Review of Financial Studies, 15(5), 1439-1463.
  4. Fung, W., & Hsieh, D. A. (2000). Performance Characteristics of Hedge Funds and Commodity Funds: Natural versus Spurious Biases. Journal of Financial and Quantitative Analysis, 35(3), 291-307.
  5. Malkiel, B. G., & Saha, A. (2005). Hedge Funds: Risk and Return. Financial Analysts Journal, 61(6), 80-88.

Frequently asked questions

What is survivorship bias in simple terms?

Judging a group by its survivors. If funds that lose money close and vanish from the record, the funds left behind look better than the group ever was. The average of the living is not the average of everyone who started.

How large is survivorship bias in mutual fund data?

Carhart, Carpenter, Lynch and Musto (2002), using US fund data that includes dead funds, found an annual bias of 0.07 percent in one-year samples, rising to about 1 percent a year in samples longer than 15 years. The bias grows with the window because funds are closed on multi-year rather than single-year performance.

What is the difference between survivorship bias and backfill bias?

Survivorship bias removes the end of the losers' records, because funds stop reporting when they fail. Backfill, or instant-history, bias adds the beginning of the winners' records, because a fund that joins a database after a good incubation period brings those years with it. Both push database averages up, from opposite ends of the record.

Does survivorship bias affect backtests?

Yes, whenever a strategy is tested on today's index members or today's listed stocks. Companies that were delisted, bought out at distressed prices or went bankrupt are missing, so the test never holds the stocks that would have hurt it. A sound backtest uses point-in-time constituent lists, including the companies that later disappeared.

How can an investor correct for survivorship bias?

Use databases that keep defunct funds, compare a manager's flagship record with every fund the firm has launched and closed, and treat category averages built from live funds as an upper bound. In due diligence the question 'which products have you shut down?' is often more informative than the track record itself.

Stay informed

Market commentary, firm news and research from EC Assets - direct to your inbox.