Process versus outcomes: Inequality indices as tests of fairness

Ravi Kanbur, Andy Snell 30 April 2017



The standard procedure for measuring income inequality in a society is to take the observed distribution of income and to calculate an inequality index from it. Such indices have also been interpreted as a measure of welfare loss entailed in departures from equality of outcomes for a society which is egalitarian. A classic treatment is that of the late Tony Atkinson, who asked the question: What fraction of national income would an egalitarian society be willing to give up in order to have equality? On this basis, he developed what we now call the Atkinson Index of Inequality.1

However, this procedure faces the well-known criticism that the observed distribution is nothing but the outcome of a process, and that it is the process which matters for normative assessment.2 In particular, it is the fairness of the underlying process which is held to be the appropriate normative standard, not whether the observed inequality of outcomes is high or low. But if we take the process versus outcomes criticism seriously, does this mean that we stop calculating inequality, since it no longer has normative validity in and of itself? The answer to this question is no. In our recent work, we argue that even within the process frame, overall indices of inequality still maintain their relevance, but now as statistical tests of fairness (Kanbur and Snell 2017).

An early proponent of the process versus outcome line of argument was Milton Friedman (1962), who brought in the consequences of risk taking for interpreting observed inequality:

“Another kind of inequality arising through the operation of the market is also required, in a somewhat more subtle sense, to produce equality of treatment. It can be illustrated most simply by a lottery. Consider a group of individuals who initially have equal endowments and who agree voluntarily to enter a lottery with very unequal prizes. The resultant inequality is surely required to permit the individuals in question to make most of their initial equality. Much of the inequality of income produced by payment in accordance with product reflects equalizing differences or the satisfaction of men’s taste for uncertainty.”

Friedman’s argument highlights simultaneously the issue of process versus outcomes, and the fact that even when the process implies ex ante equality (free lottery choice by identical individuals), the outcome may well show – misleadingly, in his view – inequality among individuals. Even though the process itself is fair, inherent randomness may show spurious inequality in outcomes.

Suppose we wish to evaluate the process in Friedman’s example. This would require an evaluation of whether the lotteries faced by the different individuals were indeed identical. If we could directly observe the lottery choices, that would be the end of the matter. But this is usually not the case. All we can observe are in fact the outcomes. The task is then to try and infer from these outcomes the nature of the process which generated them. It is clear that in order to do this we will have to provide a minimal structure to the class of processes. It is only within a given class of processes that we will be able to infer more specific properties of the process that gave rise to the outcomes we observe. In our work, we show that making these assumptions can provide considerable insight into the relationship between outcomes and process.

In particular, we motivate the use of two well-known and commonly used indices of inequality – the Theil Index and the Mean Log Deviation (MLD) – as tests of the null hypothesis of fairness versus the alternative of unfairness.3 We show that the likelihood ratio tests for fairness within two distinct income processes are proportional to these two well used indices respectively – that is to say, the tests are metrics of whether or not each individual faces the same income process. We then suggest that instead of presenting the indices as raw numbers, one could present the p-values that the raw numbers imply – a high Theil/MLD would imply a low p-value, which in turn would indicate that the probability that incomes were generated by a fair process was low.

We call the first stylised process we consider the ‘helicopter money drop’. Think of a helicopter dropping a fixed total amount of dollars onto a population. Each individual has a probability that he or she catches a dollar thrown from the helicopter. Fairness in this process is when all probabilities are equal. The likelihood ratio test for fairness in this frame is proportional to the Theil Index. We call the second stylised process the ‘helicopter money stop’. Suppose that within any year, each of our individuals receives the same amount of income each ‘hour’ and will continue to receive this hourly amount subject to a ‘stop’ which is dropped by the helicopter. Fairness is now characterised by the probability of ‘stopping’ being the same each hour for each individual regardless of how many hours they have survived the hazard. For this process, the likelihood ratio test for fairness is proportional to MLD.

The classical hypothesis testing approach takes fairness as a sharp null and asks whether the data support it. In a Bayesian framework, fairness and unfairness are ‘models’ and we ask which is more likely given the observed data. We show that if fairness and unfairness are treated as equally likely, then the two indices are also proportional to a measure of how the data modifies initial agnosticism in the direction of favouring fairness or unfairness as being the more likely model generating the observations.

Returning to Friedman’s example, it is obvious that in reality endowments are not equal and individuals face different lotteries. The final outcomes are thus the result of these inequalities across lotteries, as well as the inequality caused by the fact that even when a group of individuals faces the same lottery, and so are equal ex ante, there will be winners and losers ex post within that group. That portion of observed inequality which can be attributed to the initial ex ante differences across lotteries that individuals have access to, might be referred to as ‘inequality of opportunity’. This is related to John Roemer’s (1998) famous formulation of attributing variation in outcomes to variation in ‘circumstances’ (factors outside an individual’s control) and ‘effort’ (factors within an individual’s control).

We are then led to ask whether we can test for whether there are these ex ante differences (‘unfairness’) across groups defined by common circumstances (for example based on ethnicity and gender). We show that for the two processes, we can indeed use the Theil index and MLD to develop statistical tests for fairness across groups. Applying the methods to US data for groupings based on race, gender, health, location of birth state (North or South), and biological parents, we find that fairness is rejected. Similarly, in a Bayesian framework we find that starting with priors of fairness and unfairness being equally likely, the data move us in the direction of unfairness being more likely.

Thus, the answer to the process based critique of inequality measurement is not to stop calculating inequality indices. Rather, it is to interpret inequality indices differently, as test statistics to shed light on the fairness of the process.


Atkinson, A (1970), “On the Measurement of Inequality.” Journal of Economic Theory 2: 244-263.

Cowell, F A (2011), Measuring Inequality. Oxford University Press, Oxford.

Friedman, M (1962) Capitalism and Freedom. University of Chicago Press.

Dworkin, R (1981), “What is Equality? Part 2: Equality of Resources.” Philosophy & Public Affairs, 10(4), pages 283-345.

Kanbur, R and A Snell (2017), “Inequality Indices as Tests of Fairness.” CEPR Discussion Paper No. 11930.

Roemer, J (1998), Equality of Opportunity, Harvard University Press.

Roemer, J E and A Trannoy (2015), “Equality of Opportunity”, in A B Atkinson and F Bourguignon (eds), Handbook of Income Distribution, volume 2, pp. 217-300, Elsevier.


[1] See Atkinson (1970). For an overview and survey of standard methods, see Cowell (2011).

[2] Early critiques are by Sen (1979) and Dworkin (1981). A recent survey is by Roemer and Trannoy (2015).

[3] Note that MLD as sometimes also known as “Theil’s second index”.



Topics:  Frontiers of economic research Poverty and income inequality

Tags:  Inequality, measurement, US

T. H. Lee Professor of World Affairs, International Professor of Applied Economics and Management, Professor of Economics at Cornell University and CEPR Research Fellow

Professor of Economics, University of Edinburgh