A very poor means test?

Published on February 14, 2020 Updated on June 22, 2020
Dates

on the February 17, 2020

Focus on Research

A very poor means test?


Proxy Means Testing (PMT) – a popular method to target the poor – is subject to a lively debate among policy makers, civilian stakeholders and academics. In a recent article (Gazeaud, 2020), I provide empirical evidence on one largely ignored aspect of PMT targeting, namely its vulnerability to non-random measurement errors in survey-based consumption data.

Social programs such as cash and in-kind transfers have become an important tool for achieving poverty alleviation in developing countries: the number of developing countries with such programs doubled from 72 to 149 in the last two decades. But with an average spending of 1.6 per cent of GDP, coverage is far from universal, and policymakers often use targeting tools in order to concentrate the benefits on the poorest. However, identifying poor households is often complicated by a lack of verifiable records on earnings, especially in contexts where most households work in the informal sector or in traditional agriculture.

Against this background, Proxy Means Testing (PMT) has become an increasingly popular targeting method. In PMT, household income is predicted using a set of easily observable characteristics encompassing demographic characteristics (household size, number of children…), home attributes (floor type, roof type…), and household head’s features (education, occupation…). PMT has been implemented in large countries such as Indonesia, Pakistan, Mexico, and the Philippines, as well as in a number of smaller countries, ranging from Ecuador to Jamaica, and more recently to at least 20 African countries. PMT is subject to a lively debate among policy makers, civilian stakeholders and academics. The most debated issue is probably the claim that PMT is one of the best mechanisms, if not the best mechanism available for identifying households living in poverty. Del Ninno and Mills (2015, p. 20) argue that it “can accurately and cost-effectively target the chronic poor”. A recent World Bank report recommends the use of PMT to target beneficiaries of social benefits in Namibia because it “could provide better coverage at existing spending levels, providing a greater poverty and inequality impact” (Sulla, Zikhali, Schuler, & Jellema, 2017, p. 63). In contrast, critics often point to PMT’s high built-in errors, implementation issues and lack of transparency. For instance, Kidd and Wylde (2011, p. 2) argue that “PMT is inherently inaccurate, especially at low levels of coverage, and it relatively arbitrarily selects beneficiaries, while other methods (…) may be better at including intended beneficiaries”.

This debate has been fed by a surge of recent studies assessing the performances of PMT. In particular, Brown, Ravallion, and van de Walle (2018) provide a systematic assessment of PMT performances for nine countries in sub-Saharan Africa. They find that PMT reduces poverty only slightly more on average than a universal basic income, in which everyone gets the same transfer, whether they are rich, poor, or middle income. They conclude by suggesting that PMT is a “poor means test”. In a recent article, I investigate whether these already poor PMT performances may in fact be an overestimation of actual PMT performances (Gazeaud, 2020). An implicit assumption made by Brown, Ravallion, and van de Walle (2018) – and other studies assessing PMT performances – is that consumption data underlying PMT regressions are error-free or measured with random errors. However, this assumption has actually been challenged by recent literature. In particular, Gibson, Beegle, De Weerdt, and Friedman (2015) show that measurement errors in consumption have a mean-reverting negative correlation with true values. According to the typical textbook on the impact of measurement errors, this would lead to biased PMT estimates.
 

I provide empirical evidence on the vulnerability of PMT to non-random errors. I rely on a unique survey experiment that randomly assigned eight different designs of consumption module to more than 4,000 households in Tanzania. I compare the performances of PMT relying on gold standard consumption data with those of PMT using the more error-prone consumption data. I find that coefficients from PMT regressions are biased in the presence of non-random errors. This can be seen in the animation above, where I compare the error prone PMT distributions with the benchmark PMT distribution. Moreover, using the typical $1.25 poverty line, the incidence of targeting errors increase by a magnitude ranging from 10 to 34 per cent. More reassuringly, I find rather small and non-significant effects on targeting performances when poverty is defined in relative terms (such as with the typical 30 per cent threshold used in many development projects). This means that non-random errors in consumption have if anything, a limited impact on the ranking of households. Taken together, these results indicate that PMT performances are typically overestimated using absolute poverty lines, but remain largely unaffected using relative poverty lines.

References:

Brown, C., Ravallion, M., & van de Walle, D. (2018). “A poor means test? Econometric targeting in Africa”, Journal of Development Economics, 134, 109–124.

Del Ninno, C., & Mills, B. (2015). Safety nets in Africa: Effective mechanisms to reach the poor and most vulnerable, Washington, DC: World Bank and Agence française de développement Publication.

Jules Gazeaud (2020). “Proxy Means Testing Vulnerability to Measurement Errors?”, The Journal of Development Studies (forthcoming).

Gibson, J., Beegle, K., De Weerdt, J., & Friedman, J. (2015). “What does variation in survey design reveal about the nature of measurement errors in household consumption?”, Oxford Bulletin of Economics and Statistics, 77(3), 466–474.

Kidd, S., & Wylde, E. (2011). Targeting the poorest: An assessment of the Proxy Means Test methodology (Ausaid Research Paper), Canberra, Australia: Australian Agency for International Development.

Sulla, V., Zikhali, P., Schuler, P., & Jellema, J. (2017). Does fiscal policy benefit the poor and reduce inequality in Namibia? Washington, DC: World Bank Document.