Using RCTs for Water Research

RCTs refer to Randomized Control Trials, a powerful statistical tool that has been used by the medical profession for quite some time. I am currently reading a book called "Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty" by Abhijit Banerjee and Esther Duflo. These MIT researchers (both development economists) are using RCTs in a lot of the research they do on poverty.

One example of an RCT experiment is giving computers to kids in school to see if it improves their overall learning and educational experience. The example provided is that 100 schools are randomly selected. 50 are given computers (treatment group) 50 are not (control). Banerjee asks "if we find that the schools where children did have access to a computer did in fact learn more, does this mean it's because of the computers that they learned more, or is it because the government gave the computers to the schools where the students were more enthusiastic and were more interested, and that's why the kids learned more?

Banerjee says that one can easily conflate reasons for why children are doing better now because they received computers. But what randomized control does is it solves that problem of inference. It basically says: with this school and that other school, all the names of the students are put into a hat and we draw out 50 of their names at random. And so the schools that did get the computers are chosen at random, and that gives you the advantage that you can compare the two groups. There's no difference between them; decide by lottery which will be in each group. 100 is a smaller sample size but nonetheless, the idea is randomization which helps minimize selection bias.

There is a lot more sophistication to these experiments which you read about here. As someone who is learning more about statistics everyday and starting to appreciate them for their ability to produce policy recommendations,  RCTs are certainly worth exploring beyond the medical field or international development. Indeed, water researchers can certainly use them. Suppose we wanted to use water meters as an intervention in a community to test whether meters help cut down on household water consumption. Meters are expensive to install and are politically and socially unpopular and some argue they don't even cut down on water consumption. Thus, doing an RCT to evaluate their effectiveness merits consideration. If our sample was 2000 community members (all with different socio-economic backgrounds), we could randomly select 1000 people for the trial (like pulling names out of a hat). The other 1000 would be homes that do not receive water meters so we can compare the two.

You would probably have to wait a few months for the meters to be fully installed, but this intervention (while not as interesting or significant as a drug experiment by the FDA or an experiment by Banerjee and Duflo), would still help decision-makers understand the effect of the meters on water consumption. The intervention might show us that water consumption has gone done considerably and because we have randomized the population (litterally by doing a coin toss) we can have a better idea of the effect of this intervention. Not that meters cause consumption to go down, but help give us more reliable information.

Every method of research has its limitations, but the more we can do to minimize biases, the more influential and cogent our recommendations will be for policymakers (whether it's water meters or computers in schools in developing countries).