Mark Nichter and Prascilla Magrath (hereafter, M&N) have a nice new paper on pay-for-performance (P4P), stating that “understanding the processes by which P4P targets are reached demands a reorientation… towards an understanding of motivation as a component within a complex adaptive social system.” (Complex systems are so hot right now!)
the paper is worth reading in full. the authors also raise important points that i have wanted to address for some time and hopefully will do so in more depth in the future. these relate to process; motivation & context; and experiments.
1. a key question that is not always sufficiently asked or answered in health systems – or other – research is ‘how/why did X intervention not/work in Y setting?” this is an issue of both internal validity / credibility and external validity. internally, answering these questions means looking at the intervention from the perspective of multiple stakeholders to get a nuanced, full view of how or why something worked. externally, of course, the more the context and processes are understood, the easier it may be to predict when and where else the intervention may be successful.
answering these types of questions involve qualitative as well as quantitative work and looking at implementation process as well as impacts. hopefully i will have a write-up on studying implementation in the near future with @jonathan_payne [yes, Jon, I am putting it in writing as a commitment device]. the call for these types of work in the study of interventions – including but not limited to explicitly experimental approaches – include papers by Paluck, Mills, and Ssengooba, as well as the M&N paper discussed here.
i want to quickly make amends for the past times i have denigrated process indicators. it’s true that process indicators can become as meaningless as they are made into check boxes of routines (such as number of posters hung or meetings held). also, process indicators without impact indicators don’t get us as far as we need to go in saying whether something worked or should be done again. but, recording and evaluating process is necessary, if not sufficient, in saying how something worked and whether it could work again (and where).
2. M & N draw on Bordieu to provide a frame for studying the implementation and impacts of P4P – but the point is more widely applicable. M & N draw on the concept of ‘habits’ – attitudes, dispositions, actions – of individuals and social groups to help explain how interventions are received differently in different places. this has interesting parallels with Stein‘s work on ‘habits,’ for which he draws on the work of Velben; these parallels and lessons deserve more exploration than i give them here.
both roughly aim at the idea of shaping interventions to a given contextual reality, including political, economic, and administrative structures but also social processes and group, sub-group, and individual dispositions and ‘habits of thought.’
both also suggest the need to consider how an intervention’s positive & negative effects will ripple out in both time and space, as well as how positive effects can be sustained. Stein highlights that “relations among institutional constructs, habits and the transformation of behavior are at the core of development.” of course, M & N also point out that changing behavior is not everything. behavior relates to what actors ‘will do’ but what they ‘can do’ is may still be constrained by a lack of material, human, and time resources.
M & N also draw on Bourdieu’s multiple forms of capital to consider the ways that interventions may change dispositions and behavior; these include economic but also cultural, social, and symbolic capital. the importance of non-economic forms of capital in changing motivations and incentivizing behavior are increasingly recognized, including the many talks of Rory Sutherland and Ashraf’s recent paper on pro-social benefits. the way an intervention will change the distribution of all these types of capital should be considered in the design and the process captured over the course of implementation.
the ways that previous interventions have altered the distribution of these forms of capital is also an important consideration; tabula rasa non existunt.
3. finally, for quite some time (with a little e-input from Owen Barder – thanks!), i have been working up to saying all social interventions – explicitly experimental or not – are experiments. they change the context in ways that are not adequately considered. this includes the ethical ramifications of implementing an intervention and of stopping that intervention. at the very least, the presence of an experiment/intervention changes changes expectations. while i’ve been tiptoeing up to this point for months, M & N go right ahead and say, “short-term interventions can have long-term impacts on expectations.” an experiment or intervention might be billed as discrete, pilot, or otherwise, but if it is taking place outside a controlled laboratory setting, it is bringing about some change, from anchors and reference prices to much larger shifts in thinking on ‘how things work.’ experiments have social and political ramifications beyond the intended effects, at levels above the individuals directly involved.
to this end, M&N and others promote a ‘cyclical formative reformative research approach’ as a way of moving forwards with (health) systems research and i strongly back this idea as a way of experimenting and promoting development more generally. this sort of long-term research agenda does not always fit with the present structure of grants & funding but hopefully the latter will begin to change.
Heather E. Lanthorn 