Aside

john oliver on why context/setting matters

#lastweektonight, on mandatory minimums (video here, article with embedded video).

.

context is important. for instance, shouting the phrase, “i’m coming,” is fine when catching a bus but not ok when you’re already on the bus.”

tentative thoughts on ownership: work-in-progress

i am road-testing a few ideas from the conclusion of my thesis, in which i try to bring out two themes recurring throughout the analyses on adoption and implementation of the phase I pilot of the amfm in ghana, between 2010 and 2012. these themes are ownership and risk-taking. i have already written a bit about risk-taking here. below, i share some of my tentative ideas and questions on ownership (slightly edited from the thesis itself, including removing some citations of interviewees for now).

.

delighted for comments.

.

one undercurrent running throughout this thesis is the idea of ownership of: the definition of the problem and solution at hand, the process of adopting the amfm, and of the program itself and its implementation.

.

in chapter 4, i introduced stakeholder ideas of paths not taken, including how the program might have “develop[ed], not negate[d], local production capacity,” including through support to local manufacturers to upgrade to meet WHO prequalification and through work to bring local government and industry (rather than global industry) into closer partnership. both those ultimately receptive and resistant to the amfm acknowledged that all national stakeholders “would have preferred to have had quality, local drugs.” the very strength of the amfm design — high-level negotiations and subsidization — precluded local, structural changes.

.

in chapter 5, i highlighted that several key stakeholders refused to take — that is, to own — a stand on whether ghana should apply to the phase 1 pilot. moreover, the key, institutional decision-makers in the country coordinating mechanism for the global fund (ccm) vacillated on whether or not to send the application while a variety of circumstantial stakeholders felt they had stake in the decision and worked to influence the process. in chapter 6, i analyze how global ideas and actors played a role in ghana’s adoption of phase 1. in chapter 7, i describe the way the amfm coordination committee (amfm-cc) was set up which, in composition and process, differs from the ccm.

.

these points on alternatives not considered, on vacillation, on avoidance, and on outright resistance relate to conceptions of country ownership of development initiatives, as in the paris declaration. the absence of a national politics and aligned problem stream, in particular, neatly dovetails with the ideas of david booth that clarify what should be meant by country ownership (booth 2012). he proposes that it means an end to conditionality to “buy reform” and an end to channeling aid funding through “projects” as a way of by-passing country decision-making bodies, processes, and institutions (booth 2010)

.

the ccm represents an interesting example with which to examine country ownership. their explicit raison d’ětre is to foster ownership and they do indeed bring together representatives of government bureaucracy, business, and civil society, “representing the views and interests of grant recipient countries.” yet this structure allowed for vacillation within and strong views without. we must consider this and also juxtapose the make-up of the ccm versus the amfm-cc in terms of the stakeholders represented, the capacity and legitimacy to make relevant decisions, and a sense of ownership about the work ahead. having done this, it seems that, at a minimum, we must question whether the ccm composition when adopting phase 1 allowed for sufficient ownership. given the effort of ccm members to yield decision-making power to the minister of health, it seems that ccm members did not think so.

.

however, it is not fair to critique apparent limited ownership without raising three additional questions:

  • would ghana have tried out the amfm if political or bureaucratic actors had to take initial responsibility for the design?
  • did limited national ownership of the design and adoption decision allow national stakeholders to better, “energ[etically]” implement the initiative, maximizing credit-seeking after minimizing risk for blame during adoption (while recognizing that policy entrepreneurs and others still felt this risk keenly)?
  • how should we interpret ghana’s decision to continue with the global fund’s private sector co-payment mechanism?

these questions offer avenues for further analysis of the role of donors, the state, and the public.

.

indeed, ownership is not only an issue for capital-based elites; Fox (2015) recently highlighted that “the current aid architecture deprives both african governments and african publics of agency.” in chapter 7, I introduce views of the citizens and businesspeople at the street-level of implementation. about 20%, during in-depth interviews, spontaneously said they wanted to see the amfm continued — a view that seems to have had no way of entering any debates about the future of the amfm and is absent from the academic literature on this initiative.

.

though the minority, some respondents specifically voiced that they should have learned about the amfm through a government agency or professional association. two specifically raised their position as stakeholders. one, who heard from her supplier, said “i think it wasn’t fair because as major stakeholders, we should have been briefed before.” another, who heard first from the media, said “i felt this was wrong since we are a major stakeholder. we should have met as partners.” these concerns relate to relations between ghana and the global fund as well as between accra-based elites and tamale-based retailers.

.

the events of both adoption and implementation of the AMFm suggest that ownership is important (in no way a novel claim). note, though, that there may be certain amounts of freedom to innovate accorded by being just an implementer, rather than having clear ownership of a new idea, decision-making power over adopting and implementing that idea, and, accordingly, more risk if the idea does not pan out.

.

also, if we accept that ownership is indeed important, which seems a plausible lesson to draw from this thesis, we also learn that simply giving decision-making power to some national stakeholders is insufficient. the right national stakeholders and their existing decision-making structures need to be in play. we may glean something about relevant national stakeholders in this case through the composition of the amfm-cc and the committee characteristics raised as important (transparency, collaboration). but, given the views of some street-level implementers, ownership may require further consideration.

#rebunking

i attempted to start a very small trend on twitter tonight after reading a mcsweeny’s article on the topic of rebunking. for one, the article (here) is quite funny and deserves a read. but, for two, it also introduces the idea of rebunking, as in ‘that conspiracy theory was technically debunked but i rebunked it.’

.

this is a timely term given the run-up to the US presidential elections but it also relates to helpfully the idea of evidence-informed decision-making (not precisely as the author uses it but i think the general principle will generalize).

.

as author keanon patti notes:

one of the most well known conspiracy theories is that NASA never actually landed on the moon, and the apollo missions were actually filmed on earth, which is not on the moon. did we really land on the moon? maybe. i know i didn’t.

.

various forms of facts and evidence have been brought up to debunk this conspiracy theory, but as we all know, facts and evidence are just a set of quotation marks away from being “facts” and “evidence.”

.

two quick points:

  • if we want to produce evidence that will be convincing (un-rebunkable, as a term i don’t expect to take-off), it depends critically on understanding what key stakeholders need (and is possible) to be convinced of the evidence coming in. this means engaging stakeholders early on what they need to be convinced. helping facts not be “facts” requires early and on-going engagement about facts and “facts.”
  • there’s an important role for qualitative and descriptive data here. we can’t always get all stakeholders to the moon but we can better help them feel like they were there, rotating the flagpole so that Kubrick didn’t yell at them. i mean —
Aside

nice paragraph on local leadership

though i have read several of david booth‘s papers on country ownership, i appreciate craig valters pointing me (conversation here) to booth’s joint work with sue unsworth on doing development in ways that are politically smart and locally led. the whole paper is worth a read. this paragraph stood out:

the question of local leadership is not about the nationality of front-line actors; nor is it about donor agency staff not being involved in the process. it is about relationships in which aid money is not the primary motivator of what is done or a major influence on how it is done… the starting point is a genuine effort to seek out existing capacities, perceptions of problems and ideas about solutions, and to enter into some sort of relationship with leaders who are motivated to deploy these capabilities.

what i’d like to read now

the important, excellent thing about deaton’s work on aid, and it’s very incomplete overlap with development, is that it reminds me about all that ‘other stuff’ — arms trafficking, subsidies and fair trade, etc, that impinge profoundly on opportunities for economic and human development and capabilities around the globe.

.

the problem is, then, that i don’t know what to do. the bits about revamping aid to be more about unconditional technical support i get and can think about and can maybe even do something about. and the need to ground all research work in an awareness of history i get and can think about (it is not so often that i quote paul farmer but his call for us to consistently be historically deep and geographically broad in our work seems apt). this is something i’d like to do more about.

.

but i know very (embarrassingly) little about these other global flows and blockages that profoundly affect the people, processes, and outcomes that i study and care about. these are unlikely to be things to which i devote my career but i could certainly do with being more mindful of work that is going on in these areas and how the daily choices in make are bound up in these efforts. so, in my spare time, that is what i would like to read next. suggestions welcome.

Aside

Somali piracy as non-state governance

Source: Somali piracy as non-state governance

survey design and methods matter (and sometimes people in ghana aren’t truthful about loans)

a key part of data quality (and therefore research quality) is the design of questionnaires, interview schedules, and observation plans for generating primary data. it is somewhere between upsetting and galling, then, that these issues feature minimally in the training of social scientists planning to do (especially quantitative) empirical work. there are plenty of researchers who have devoted their careers to this topic (often with a high-income country focus), resulting in wonderful courses taught on this subject and wonderful books written. two of my favorite recommendations are the offering from groves et al and bernard tome covering both qualitative and quantitative methodologies.

.

on a basic shit in, shit out principle, it is very odd that so much empirical training goes into the nuances of analysis (important, yes) while so much on questionnaire design, measurement, and measurement error is left to be learned on-the-job, often under time pressure. thoughtful pre-testing is generally ignored and piloting a questionnaire and piloting an intervention often get muddled into a compressed whirlwind, followed by ad hoc tweaks rather than considered analysis and refinement (and, gasp, reporting the lessons learned in the research report).

.

blame the research cycle or funding timelines or publication word limits or the challenges of cross-disciplinary work or whatever but this happens far too often if we accept that bad data results in bad research. moreover, an unrooted and rushed approach to questionnaire and study design leads to inefficiently belated stock-taking and reinventing-the-wheel with regard to interviewer effects, construct validity, or thinking about how to limit respondent errors and misreporting. getting this right is a technical expertise that takes time, practice, and cross-disciplinary training.

.

it is my hope that a normative shift towards pre-analysis plans/commitment-to-analysis plans specifically linked with questionnaire items (and, please, a trend towards reporting the actual wording of the question that yielded the variables analyzed and the coding strategies used when turning open-ended answers into quantitative variables) will swing norms towards allotting more time and effort up-front to getting questionnaires and other data collection tools right. the best place to learn about new cool questionnaire techniques is probably in work by survey methodologists, psychometricians, and sector experts. to be slightly more pointed, at least at present, reading and citing only impact evaluations is probably not the best way to learn about new and best survey practices.

.

there are at least two reasons to think seriously about these issues. one is, of course, to get the most truthful answers possible, (perhaps) especially when we have an eye towards having a given study inform practice, policy, and funding in some way. the second is respect for the people we interview.

.

i’ll start with the latter point: we should be careful in curating our data collection instruments to place the least possible time (and emotional) burden on our respondents while getting the most useful truths out of them. again, the process of carefully constructing an analysis plan (and, for that matter, a theory of change and other dimensions of transparency) may better focus researchers on the variables that matter. there will be trade-offs between asking more questions (or using other data collection techniques like structured observations) to build in redundancies to triangulate on central variables while discarding items on which we expect respondents to be inaccurate or which are less likely to enter analysis. (a good rule of thumb is to ask yourself whether you could or would answer a question; if not, revise or toss.)

.

on the first point, about aiming for accurate measurements, david evans called our twittention to a nice example in a new paper on microcredit in ghana by erikson and lensink, published in jdeff. they triangulate on how recipients of microloans spend their loan by using both direct and indirect questions. this information is potentially important to the microfinance institutions but also relates to the policy broader debates (here and elsewhere) of how households in low- and middle-income settings make use of loans and un/conditional cash transfers. a completely non-paternalistic argument about loans and transfers says that how households decide to spend a transfer is entirely up to them and thus conditions and checks are unneeded and possibly unethical.

.

a different line of argument still holds that donors have (here, valid and generally vice-free) preferences about how loan and transfer recipients spend their money but that, for the most part, recipients behave in these ways of their own accord, so there’s no need to spend the time and effort to impose conditions. this line of argument depends on our having a fairly clear picture of how recipients are likely to spend their money — and this is what the present study calls into question. as the authors note, the basic theory of change linking a micro-loan to a sustained improvement in household or business welfare requires an increase in productive investments. therefore, microfinance organization would like loan recipients to invest in this way.

.

in this study, the authors directly ask respondents how they spent their loan. they also employ a randomized listing technique long used in asking respondents about sensitive behaviors like sexual practices and drug use (see, for example, meta-analysis of randomized response research: thirty-five years of validation for uses of the approach and cautions). in this case, the potentially ‘sensitive behavior’ relates to how microloan funds are spent, particularly if not on productive investments. the authors are able to triangulate the results between the two means of asking the same question. when asked directly, 0.1% of clients report using the loan primarily on household consumption rather than the purchase of productive assets. this jumps to 41% under the listing exercise.

.

to come back to the beginning, techniques like randomized responses do yield interesting, important, and useful results. they are also not new, even if relatively new in empirical social sciences focused on low- and middle-income countries. there are many other similar methodological details and strategies that would be beneficial to research on improving welfare in low- and middle-income countries. we should invest the time and cross-disciplinary effort in finding out — and training on — the best ways to ask questions and approach reality in ways that impose limited burdens on respondents.