CEO Acid Test

How good is a CEO? And how can you tell?

The available indicators of chief executive ability—compensation, tenure, media mentions, firm size, stock price, firm performance—are unscientific at best. But until now, they were all that investors, academics and boards of directors had to work with when attempting to assess the relative effectiveness of a firm’s top boss.

A new measure produces a vastly more precise picture of CEO worth. Outlined in a paper co-authored by Sarah McVay, an associate professor of accounting at the University of Washington Foster School of Business, the Manager Ability Score strips away variables that are specific to a firm or industry to reveal just how efficient a given CEO is, relative to industry peers, at transforming corporate resources into revenues.

“To put it simply,” says McVay, “we can isolate a CEO’s contribution to firm performance and show who generates the most sales based on the same inputs.”

An old conundrum

An accurate evaluation of a CEO’s ability has long eluded researchers as they attempt to understand the impacts of this head of the corporate hierarchy. The stakes are even higher for investors and especially directors, whose hires can mean millions, sometimes billions, to their companies—on either side of the balance sheet.

The problem is that no two businesses are exactly alike. So it’s a bit like trying to assess a college football quarterback against all others in every division of the NCAA. Sure, there are individual statistics and assumptions to be made about what it takes to lead a team. But how do you account for talent of teammates, strength of schedule, ability of opponents, and program size, ranking, and reputation? What about all the variables that support or suppress one player’s intrinsic ability?

Evaluating CEOs is an even bigger challenge. How do you stack the leaders of multinational conglomerates against those of old-line manufacturing firms, consumer products against high-tech ventures? It’s like comparing Apple (the electronics and software maker) and Orange (the European telecom).

“Until now,” McVay says, “it’s been really hard to partition the performance of the manager from the firm.”

Developing the measure

McVay and co-authors Peter Demerjian of Emory and Baruch Lev of NYU assessed the performance of more than 2,200 firms over 20 years. They used a process called Data Envelopment Analysis to establish the relative efficiency of each firm within its industry.

Then they began carving away key firm-specific characteristics—the effects of firm size and age, market share, positive free cash flow, multi-segment and international operations, industry and time. What emerged was a raw score, the unvarnished measure of a CEO’s efficiency. That is to say, his or her ability to convert firm resources—inventory, general and administrative expenses, fixed assets, operating leases, past R & D, intangible assets—into revenue.

“More able managers generate higher revenue for a given level of resources or, conversely, they minimize the resources used for a given level of revenue,” McVay says. “In other words, they maximize the efficiency of the resources used.”

Validity testing

The large number of firms and years in the study also allowed McVay and her co-authors to demonstrate the precision of their Management Ability Score and validate their individual CEO assessments in several different ways.

CEOs tend to turnover frequently. The study shows that the stock price reaction to the announcement of a change at the top is more positive the lower the departing CEO’s ability, and vice-versa.

And for those CEOs who switched firms within the sample, the data demonstrate that a top manager’s personal ability does make a significant economic impact—for good or ill—on his or her new firm.

“Good managers are good wherever they go, and they make their companies better,” McVay says. “Bad managers are bad wherever they go, and make their companies worse.”

A useful tool

McVay believes this more precise measure can open the door to research on the impact of CEOs on their firms and the economy. It also can be useful to boards of directors in their efforts to replace or compensate their companies’ CEOs.

She adds that the ability score can be adapted to measure the efficiency of chief financial and chief operating officers, as well.

“Assessing managers based on the efficiency with which they generate revenues, rather than by their pay or media mentions, is intuitively appealing,” she says. “And it’s more in line with the over-arching goal of profit-maximizing firms.”

“Quantifying Managerial Ability: A New Measure and Validity Tests” is published in the July 2012 issue of Management Science.