A couple of weeks ago, France 24 ran a story featuring accusations by Belgian Professor Filip Reyntjens that the Government of Rwanda had manipulated its poverty statistics. The truth, to my relief*, is somewhat less exciting.

What seems to have actually happened, is that Rwanda quite reasonably decided to update the methodology for calculating what the poverty line should be, but then found that the new methodology led to an implausibly high poverty line, and so decided to (slightly arbitrarily) "adjust” the new methodology, resulting in the final poverty line being almost exactly what you would have expected it to be had you simply updated the original poverty line for inflation.

It took me a while to figure all this out, as the original criticism and rebuttal by NISR weren’t entirely clear, and it was only in Filip’s reaction to NISR’s rebuttal that I grasped his (mistaken) point (here’s also the Rwanda EICV4 Report and EICV3 Report).

How is poverty measured?

Rwanda has followed a fairly typical process – set a poverty line by first defining a minimum quantity of calories needed, second working out how much it would cost a poor person to buy that many calories, third increasing that amount by 40% to account for some basic minimum non-food spending needs. Then to get your poverty rate, just calculate how many people spend less than the poverty line.

What was the disagreement about?

Rwanda’s poverty line was set in 2001 based on how much it cost then to purchase a basket of goods that poor people bought back then. You need to keep your methodology consistent over time to allow for fair comparisons, but its also reasonable to think that the minimum consumption basket is likely to change over 15 years of rapid growth.

The government of Rwanda decided to keep the minimum assumed number of calories (2,500 per day, which is pretty high), but change step 2 – the way of working out how much it costs to buy these calories. In a normal survey year, this cost is simply updated for inflation (even if prices and consumption habits have changed in the meantime).

This year, Rwanda decided to make an update to the prices and consumption habits, but found something odd. Most poor people consume far fewer than the minimum number of calories – almost half.

So how do you construct a hypothetical " bare minimum" food consumption basket, that is twice as big as what people actually buy? Do you just double everything? Or do you assume that if people bought more food than they did, they might buy more of some items than others? This is where the big disagreement presumably came.

Rather than choosing to simply double everything, the Rwandan stats agency made a few arguably arbitrary choices about which items to increase and which to decrease, that has a big effect on the overall price of the basket, and therefore the overall poverty line, and therefore the poverty rate.

Why is Filip Reyntjens wrong?

Filip argues, correctly, that Rwanda’s assumptions about how to scale up consumption patterns to reach their minimum calorie basket, affects the overall line. In fact, their adjustments reduce the line by 19%. But his next step is wrong.

He argues that as this new methodology line has been reduced by 19%, we should also reduce the 2010/11 line by 19%, giving a substantially lower poverty rate in 2010/11, and therefore an increase in 2013/14. But he misses the intermediate step – the Rwandans didn’t just adjust the new food basket, they first also calculated a whole new food basket.

Yes, what should really happen is for the new methodology to applied retrospectively to all the old survey data to allow for truly comparable numbers, but the adjustment made to the new methodology leads you to a poverty line that is basically the same as it would have been with the old methodology anyway.

Implications for how we measure poverty?

One thing that this choice really highlights is the number of assumptions you sometimes need to make, and the fragility of the whole concept of poverty estimates.

Here's an example of another seemingly arbitrary choice of assumption with big consequences – the Indian stats agency used to measure poverty with surveys that asked people how much food they had bought in the last 30 days, longer than practice elsewhere which uses 7 day recall periods.

To their great credit the stats agency decided to run a randomised control trial to test these two methods against each other. The result was that moving from a 30 day to a 7 day recall period increased measured consumption massively – and reducing poverty by 175 million people – close to half of all those in poverty (from Angus Deaton’s 2014 LSE lecture (PDF), via Nic Spaull).

The bottom line: measurement is hard, and it is possible for reasonable people to disagree, without there necessarily being any nefarious trickery.

---

* Relief, because I have previously worked both for OPM as a staff member, on a project with OPM for the Rwandan Stats agency, and directly on a project for the Rwandan Ministry of Finance.

The story was first published on Lee Crawfurd's personal blog.