Doctoral Dissertation Research in DRMS: Does participation in project decision-making affect how intended beneficiaries report project outcomes?


Malgosia Madajewicz
Associate Research Scientist in the Center for Climate Systems Research in the Earth Institute
Anna Tompsett
Assistant Professor, Stockholm University


Improving access to social services such as safe water and sanitation, education and health care depends on being able to learn what approaches work better than others, when and why. Household surveys of intended beneficiaries are an essential source of information about program effectiveness. Recent evidence, however, suggests that these survey data may be unreliable because program beneficiaries may change the way they report their behavior as a result of the program. For example, if a program expands access to safe drinking water, people living in areas covered by the program may become more likely to report that they are drinking safe water, even if they are not. Such reporting bias could distort estimates of the impact of social programs, or comparisons between different programs, resulting in misleading policy recommendations. This research measures the size and prevalence of reporting bias in the context of a recent experiment that investigates whether participation by intended beneficiaries in decisions made during program implementation improves program outcomes.
A recent randomized controlled trial in Bangladesh has provided the first experimental evidence that documents how beneficiary participation in decision making affects outcomes of a social program to provide access to safe drinking water. Preliminary results, based on reported changes in behavior, suggest that involving beneficiaries in decision making results in improved access to safe drinking water, but only when the decision-making structure incorporates protective measures to avoid the capture of program benefits by influential individuals or groups. Participation in decision making, however, may influence not only program outcomes, but also the way in which program beneficiaries report outcomes in household surveys.
The researchers directly observe the behavior and characteristics of users of the water sources installed, and compare these observations to water use reported in household surveys in order to determine the size and extent of reporting bias. In particular, the researchers measure how reporting bias varies depending on the degree of community participation in the decision-making process. Because the original project randomly assigned the decision-making structure, communities that implemented the program under different decision-making structures were comparable in terms of all other characteristics. This enables the researchers to measure the direct effect of community participation in program decision making on the size and extent of reporting bias.
The study will further understanding of the limitations of using self-reported survey data to measure the effectiveness of social programs. It will also make two further contributions: the first direct, experimental evidence on the connection between community participation in decision making and the likelihood of capture of program benefits by elite or influential community members. The research will provide experimental evidence on whether community participation in decision making improves project sustainability, as measured by maintenance of the installed water sources.