The Popular Method for Benchmarking Closed-End Alternative Funds Against Public Indices Is, Well, Wrong

The PME method for comparing the performance of closed-end funds with that of publicly traded products was developed by Austin Long and Craig Nickels and presented in a 1996 paper. It is a genuinely elegant method and has become very popular for assessing closed-end fund performance. The method is based on the timing and size of the cash flows between the closed-end fund and the investor. In effect, it simulates the investment in the closed-end fund as follows: each capital call is invested in the public market index (such as the S&P 500, or an ETF on the index) on the same date and in the same amount, and each distribution is treated as a withdrawal from the market index on the same date. This makes it possible to calculate the internal rate of return (IRR) of the hypothetical investment in the public index and compare it precisely with the IRR of the private fund. In other words, we are comparing apples to apples.

Except that is not the case. In fact, with the PME method we are cutting oranges into the shape of apples, and then comparing those cut oranges to apples.

To simplify the discussion, let us look at a closed-end fund that made only a single investment. The fund called $50 on date A and returned $95 to the investor on date B. The comparison between investing in the closed-end fund (shown by the blue line) and investing in the ETF (the green line) can be seen in the following chart:

Well, the savvy investor who invested in the closed-end fund earned a 90% return over the period, while the ETF returned only 67% over the same period (between A and B). In other words, the fund outperformed the ETF.

This elegant method is, well, wrong.

Here is a conversation between two runners: Mr. Private and Ms. General:

Private: Hey, how about we measure who’s the better runner?

General: Great idea. Want to have a race?

Private: No, forget it. Let’s use the results we already have.

General: OK. But how do we do that? We’ve never taken part in the same run.

Private: Did you run yesterday?

General: Yes.

Private: Great, me too. Yesterday at around 14:15 I felt fantastic, a real surge of amazing energy, and the weather where I was was excellent too. So I went out for a sprint that started at 14:20. I ran for 16 seconds and covered exactly 120 meters.

General: OK.

Private: What were you doing at that time?

General: I was running a marathon barefoot with a 20-kilo pack on my back.

Private: No, I mean exactly at 14:20, for 16 seconds.

General: I was in the middle of the marathon; in those 16 seconds I covered 30 meters.

Private: Ah, so I’m 4 times faster than you.

General: OK, and what did you do for the rest of the day? While I was running a marathon?

Private: I didn’t run for the rest of the day. You can’t compare us during the time I’m not running.

General: OK. Makes sense, all things considered.

The investor’s investment decision regarding the fund was not to invest $50 on date A and redeem the money on date B, even though that is ultimately what the fund carried out. The investor decided something entirely different. He decided to commit $100 to a closed-end fund with a 4-year investment period and a further 6-year harvest period, with an option to extend by two years, and then by another two years. If one wants to compare this decision with a publicly traded product, the comparison dates and amounts must be set at the time of the decision, with all the uncertainty that this decision entails. And not in hindsight – after the fund’s cash flows are known, and all the additional uncertainty inherent in investing in a closed-end fund has already disappeared.

The “apples to apples” illusion arises because the PME method mixes two sets of uncertainty. The first is the uncertainty about the results of the public index and the results of the closed-end fund, both of which are indeed unknown at the time of the investment decision – so the “parity” between the products is preserved here. The second set is the uncertainty about the timing of capital calls, the amounts of capital calls, the timing of distributions, the share of the total commitment that is actually called, and the life of the fund, including the use of extensions. This second set of uncertainties does not exist at all when investing in an ETF: the investor can invest in the ETF whenever he wants and in whatever amount he wants, and withdraw the money on whatever date he wants (as long as the investment amount is small enough relative to the ETF’s daily trading volume).

Making a comparison between dates A and B on $50 is exactly as valid as making the comparison between points C and D on an investment of $1000. For the sake of argument, assume the investor actually invested $1000 in the ETF on date C and redeemed the entire investment on date D. Let us compare that decision with the investment in the closed-end fund over these dates (this is exactly the same fund and the same index described in the previous chart):

Had we invested $1000 in the ETF on date C, we would have received $1430 on date D, a 43% return (the green line).

What did the fund do between points C and D? Let us break performance into two periods. In the period from C to A, the fund did nothing (that is, it existed and the fund’s staff were presumably reviewing potential investments, but in terms of cash flow to the investor it did nothing). In the period from A to D, the fund declined to invest $950 (since, as noted, the fund called only $50 out of the investor’s original $100 commitment), and invested only $50. Then, from date A to D, the fund was invested, and on date D the fund is still invested and the money cannot be realized (it does not matter what the NAV was; we are looking at cash flow to the investor).

Which investment is preferable according to this comparison?

Do we prefer $1430 (including a $430 profit), or $50 invested in the fund with no ability to withdraw at present + $950 that was not invested?

One can argue about it. Does it even make sense to compare an advance commitment to invest $100 in the closed-end fund with a spontaneous decision to invest $1000 in an ETF? Except that this is exactly what investing in an ETF allows – flexibility in timing and amounts. And that is exactly what investing in a closed-end fund does not allow.

There are countless other comparisons that could be made, based on the investor’s original decision: to commit to an investment in the closed-end fund, for a long period and with a capped amount. Why, under the PME method, do we look back specifically at the exact schedule on which the fund made capital calls and distributions to the investor?

The ones who will really kill off the PME method are the managers of evergreen funds in the alternatives world. When people start comparing a manager’s closed-end funds with its evergreen fund using the PME method, the evergreen funds will show inferior performance. In other words, it is precisely the alternative managers – the very ones who raised the PME method to great heights – who are also likely to bring it crashing down.