OVER the years, big financial institutions have replaced wealthy individuals as the most important category of company owner. With the rise of private equity, they now have two main ways of acquiring their stakes. They can buy shares directly in public stockmarkets, or they can invest in a private-equity fund that will, on their behalf, buy shares in private firms, or in public companies that they then take private. But which model is best?

Institutional investors typically invest well under 10% of their money in private equity, and some still steer clear of it altogether. But to hear people in private equity talk, its superiority over public equities is now overwhelming. Private-equity firms offer the best solution to the biggest problem facing institutional owners: how to ensure that their companies' bosses act in shareholders' best interests. With their hands-on corporate governance—from close but unbureaucratic monitoring at board level to carefully designed incentives for the bosses—they reckon they can deliver a better performance and steer clear of scandals. Corporate governance in the public equity markets remains notoriously weak, despite the increase in activism in the past decade, and particularly since the collapse of Enron in late 2001.

Indeed, the private-equity folk think that recent attempts by politicians and regulators to improve public companies' corporate governance have further strengthened their hand. KKR's Henry Kravis has gone on record as saying that America's Sarbanes-Oxley legislation to tighten up public companies' corporate governance was a “positive development for shareholders. Today directors are taking their responsibilities to shareholders more seriously.” But, he continued, “they are also being more conservative and risk-averse,” and “to the extent that Sarbanes-Oxley causes public companies to be less competitive, there is an opportunity for the private-equity industry in taking these businesses private and putting some energy back into growing them.”

There is no doubt that Sarbanes-Oxley and other legislation and regulation on both sides of the Atlantic have raised the cost of running a public company. Smaller firms, in particular, may feel that an extra $1m a year in compliance costs, say, is enough to tip the cost-benefit analysis in favour of going private. Moreover, argues Larry Harris, who until recently was chief economist at the Securities and Exchange Commission, that is “not a bad thing”. After all, if you take a firm public, “you have a high responsibility to the people who are trusting you with their money.” The most expensive aspect of the Sarbanes-Oxley legislation, he says, is certifying financial results, which requires systems of accountability in public firms: “You need accountability if you are to have publicly distributed capital.”

Mr Harris does not think that there will be any macroeconomic cost if there is a shift away from public to private equity. “If you can't raise money for a business idea in the private domain, given how much capital there now is in private equity, the idea probably isn't any good.”

Yet it would be premature to celebrate the inevitable triumph of private equity. If most private-equity firms fail to improve their performance, investors may put their money elsewhere. And private-equity firms need healthy public markets as an exit route from their investments. Some leading private-equity bosses even hope to take their own firm public one day. They dislike private equity's long, secretive, inefficient fund-raising process, and the constraints imposed by each fund having a limited lifespan. They admit that the performance of publicly listed private-equity firms (such as Britain's 3i) has often been disappointing, but point to GE and Warren Buffett's Berkshire Hathaway, both of which have permanent capital and profit by doing lots of what are, in essence, private-equity deals.

And even if good exits are available, life is likely to remain tough for private equity as a whole, even as a few big firms and a larger number of niche operators do well. As Mr Kravis has said, many of the techniques that private-equity firms pioneered—paying managers with shares, concentrating on shareholder value, making good use of debt, maximising cashflow—are now standard practice in public firms, making it hard for private equity to outperform public equities. As they grow, private-equity firms may also suffer from diseconomies of scale, becoming more bureaucratic and risk-averse.

So institutional investors should look beyond the hype and question whether private equity really is a better bet than the alternatives. They are surely right to want to make private-equity firms more accountable, because they are not finding it easy to get general partners to accept responsibility when things go wrong. In 2002, the state of Connecticut's pension fund sued Forstmann Little for mismanaging its investment. The pension fund won, but was awarded no damages. It appealed, and Forstmann Little settled out of court for $15m in September this year. Yet this was a pyrrhic victory. In the words of one private-equity veteran, “The only funds that will take Connecticut's money in future will be those that can't raise money from anyone else.” That is quite a penalty in an industry where the best funds far outperform the rest and are able to pick their investors.

A similar punishment was meted out to some public pension funds, notably CalPERS, which last year published the results of the private-equity funds in which they invested. They were only trying to comply with the disclosure requirements of freedom-of-information legislation, but several funds—led by Sequoia, a top venture capitalist—said they would no longer accept money from public pension funds. Some states have since modified their laws to limit the need for such disclosure.

Limited partners are trying to exercise more clout by setting up organisations to represent their collective interests—though the people heading such groups are habitually poached by private-equity firms just as they start to find their voice. Some pension-fund managers responsible for investing in private equity hope to go on to much better-paid jobs in private-equity firms, so may be slow to criticise them for fear of marring their prospects.

Perhaps the growing fund-of-funds industry, led by firms such as HarbourVest and Parthenon, will eventually make a better job of holding general partners to account, not least by encouraging the development of a liquid secondary market in limited partnerships. So far, however, funds-of-funds have stood out mainly for their additional fees.

More accountability will require better information about the performance of private-equity portfolios. As things stand, valuations are a shambles, says Colin Blaydon of the Tuck Centre for Private Equity and Entrepreneurship. Valuation methodologies are not standardised and tend to be overly conservative, using historic cost rather than current market value; and values are rarely reduced when they should be, and are easily manipulated by general partners. But industry groups are slowly making progress with standardising valuation methodology. Herman Daems of the European Private Equity and Venture Capital Association (EVCA) points out that Europe is now well ahead of America on standardising valuations and reporting.

Government, too, is getting more involved in trying to make private equity more accountable and transparent, either through freedom-of-information legislation or via attempts by watchdogs such as the SEC to regulate the industry. The SEC has hinted that, having started to regulate hedge funds, it may turn to private equity next.

Even with the lobbying expertise of industry groups such as the EVCA, the British Venture Capital Association and America's National Venture Capital Association, it is hard to believe that private equity will be able to escape the growing pressure for institutions of every kind to be transparent. In many respects, greater transparency should improve the performance of private equity by making such firms more accountable to institutional investors—who, in turn, should then be more easily held to account for their investment choices by the public whose money they invest.

Yet transparency can go too far. Private-equity firms may be wrong to oppose attempts to make them disclose the performance of a fund as a whole at frequent intervals. But they are surely right to resist disclosure of the performance of each firm in a fund's portfolio. The one big advantage that private equity still has over public equities is that it can transform firms away from the public gaze and from the short-term pressures of the stockmarkets. Imposing full transparency on the kings of capitalism would destroy this advantage. As Walter Bagehot, a great early editor of The Economist, famously observed about the British monarchy, “Its mystery is its life. We must not let in daylight upon magic.”

Copyright © 2004 The Economist Newspaper and The Economist Group. All rights reserved.