i am now assisting with this particular ethics course for the first time. it was therefore no longer surprising  but still unsettling that on almost every midterm i graded last week, i wrote “a thesis sentence would be helpful.” i‘ll note that this is a masters-level course. i thought about reintroducing the (as noted by a friend, colossal ”In-and-Out 4-by-4 Animal Style”) sandwich as teaching tool but opted against it.

i find it upsetting that in a school of public health – covering topics for which communication skills are ostensibly quite important – there so little direct emphasis on improving writing and public speaking.

first, for better or for worse, at least in my department, TAs do the vast majority of the grading. we generally don’t have the time (nor, ahem, the commensurate pay) to comment on writing style and grammar as well as content.

second, the writing resources at school are limited to one man. he does a great deal of good work but we can hardly assign all students to go to him before a paper is due, as in undergrad the professor could mandate that a paper went to the writing center before it was turned submitted  (where you were forced to read your paper out loud, which was both terrifying and extremely helpful).

third, the above point is all the more upsetting given the school’s cultivation of an international student body. that there are no writing resource that ESL (or, likely EnL) students can access when working on a specific paper is fairly upsetting.

fourth, i am of the firm opinion that one really learns how to write or present by having to comment on or grade good and bad writing or presentations. however, most times when i try to insert a mandatory “read your paper out loud to another student” or “grade a fellow student’s paper” into a curriculum, it is shot down for one reason or another. yes, it’s a pain. yes, students might go easy on each other. nevertheless, i still think is a good idea and a necessary component of taking good writing seriously. rating other students’ presentations seems to go over slightly better – but only slightly.

Writing for ethics_HEL2013

on another track, there are always funny turns of phrase when working in english in india (and other places). one that always particularly tickled me was referring to “pressurizing” someone to do something, as opposed to “pressing” or “pressuring” someone into something, as i would say. it always seemed like an odd transformation of the word.

but now, thanks to miss marple,  i realize that ‘pressurizing’ was in fact a british construction, foisted at some point onto the colonies. i still think it sounds funny but at least i know where to place the blame for a goofy word.

in the meantime, i offer some of my take-aways from the conference – based on some great back-and-forths with some great folks (thanks!), below.

for me, the two most useful catchphrases were trying to get to “relevant rigor” (being relevantly rigorous and rigorously relevant) and to pay attention to both “glossy policy and dusty implementation.” lots of other turns-of-phrase and key terms were offered, not all of them – to my mind – terribly useful.

there was general agreement that evidence could be political in multiple dimensions. these included in:

• what questions are asked (and in skepticism of whose ideas they are directed), by whom, of whom, with whom in mind (who needs to be convinced), for whom – and why
• the way questions are asked and how evidence is collected
• how evidence is used and shared – by whom, where and why
• how impact is attributed – to interventions or to organizations (and whether this fuels competitiveness for funds and recognition)
• whether the originators of the idea (those who already ‘knew’ something was working in some way deemed insufficiently rigorous) or the folks who analyze evidence receive credit for the idea

questions and design. in terms of what evidence is collected and what questions are asked, a big part of the ‘push back’ relates to what questions are asked and whether they help goverments and organizations improve their practice. this requires getting input from many stakeholders on what questions are important to ask. in addition, it requires planning for how the evidence will be used, including what will be done if results are (a) null, (b) mixed, confused or inconclusive, and (c) negative. more generally, this requires recognizing that policy-makers aren’t making decisions about ‘average’ situations but rather decisions for specific situations. as such, impact evaluation and systematic reviews need to help them figure out what evidence applies to their situation. the sooner expectations are dispelled that an impact evaluation or a systematic review will provide a clear answer on the what should be done next, the better.

my sense, which was certainly not consensus, is that to be useful and to avoid being blocked by egos, impact questions need to shift away from “does X work?” to “does X work better than Y?” and/or “how an X be made to work better?” this also highlights the importance of monitoring and feedback of information into learning and decision-making (i.e.).

two more points on results for learning and decision-making. first, faced with the assertion that ‘impact evaluation doesn’t reveal *why* something works,’ it is unsatisfactory to say something along the lines of ‘we look for heterogenous treatment effects.’ it absolutely also requires asking front-line workers and program recipients why they think something is and is not working — not as the final word on the matter but as a very important source of information. second, as has been pointed about many places (e.g.), designing a good impact evaluation requires explication of a clear “Theory of Change” (still not my favorite term but apparently one that is here to stay). further, it is important to recognize that articulating a ToC (or LogFrame or use of any similar tool) should never be one person’s all-nighter for a funding proposal. rather, the tool is useful as a way of collectively building consensus around mission and why & how a certain idea is meant to work. as such, time and money need to allocated for a ToC to be developed.

collection. as for the actual collection of data, there was a reasonable amount of conversation about whether the method is extractive or empowering, though probably not enough on how to shift towards empowerment and the fact that extractive/empowering are not synonymous with quant/qual. an issue that received less attention than it should have was that data collection needs to align with an understanding of how long a program should take to work (and funding cycles should be realigned accordingly).

use. again, the conversation of the use of evidence was not as robust as i had hoped. however, it was pointed out early on (by duncan green) that organizations that have been comissioning systematic reviews in fact have no plan to use that evidence systematically. moreover, there was a reasonable amount of skepticism around whether such evidence would actually be used to make decisions to allocate resources to specific organizations or projects (for example, to kill or radically alter ineffective programs). rather, there is a sense that much impact evaluation is actually policy-based evidence-making, used to justify decisions already taken. alternatively, though, there was concern that the more such evidence was used to make specific funding decisions, the more organization would be incentivized to make ‘sausage‘ numbers that serve no one. thus, the learning, feedback and improving aspects of data need emphasis.

empowerment in the use of data (as opposed to its collection) was not as much a part of the conversation as i would have hoped, though certainly people raised issues of how monitoring and evaluation data were fed-back to and used by front-line workers, implementers, and ‘recipients.’  a few people stressed the importance of near-automated feedback mechanisms from monitoring data to generate ‘dashboards’ or other means of accessable data display, including alternatives to written reports.

a big concern on use of evidence was ownership and transparency of data (and results), including how this leads to the duplication/multiplication of data collection. surprisingly, with regards to transparency of data and analysis, no one mentioned the recent reinhart & rogoff mess, nor anything about mechanisms for improving data accessibility (e.g.)

one potential response is to improve our behavioral interventions to support adherence to treatment regimens (and prevention regimens) and integrate these methods more directly into medication trials. adherence and persistence with medication are global problems and we are just beginning to learn – with the help of health psychology and behavioral economics – how to tackle the challenge. efforts so far include high- and low-tech solutions, though not all the promises of the former, in terms of mhealth to facilitate behavior change and adherence, have yet been borne out.

another, not mutually exclusive, response would be to actually ask the women what they would like to see and use in the way of HIV prevention – a tool which should be empowering for them. the press seems full of comments like “the women are trying to tell us something!” why does it seem that, then, for a product made for them, they have to work so hard to tell us those things? why are we not hearing more sentences that start “the women told us…” that is, why, after such a big trial, am i not hearing anything about on-going qualitative and observational follow-up efforts to learn more about what exactly didn’t work about the methods offered to women?

there’s often a lot to learn from null results (that’s science, right?) but it doesn’t just come from brainstorming what went wrong. asking helps.

i don’t suggest that people are perfectly prescient about what they need or want. often, the innovations that we can’t live without now – smartphones, for example – weren’t a need or even desire that most people could have articulated 20 years ago. as such, directly asking people what it would take to get them to engage in X desirable behavior can’t determine the research agenda. but it should certainly be part of figuring it out.

the tweak is to separate out two categories of information that are important and may be better gleaned from qualitative inquiry and analysis, though more adaptive/iterative and process-focused quantitative data can also play a role. i think the general mindset on what constitutes (and who has) useful and useable information is as important as the difference between quant or qual analysis.

1. how the program actually brought about an effect. this point is the focus of ed’s post, as well as levy paluck’s nice paper on qualitative methods and field experiments, which distinguishes between causal effects and causal mechanisms. mechanisms can, to some degree, be pursued and examined through ever-proliferating treatment arms in RCTs… but observation and purposive, systematic conversation are very helpful. if i read ed carr’s piece correctly, he wants more (data collected with an eye towards) explanation in order to pursue (a) a deeper understanding – beyond ‘story time’ – of what moderates and mediates the (potentially causal) relationship between X and Y within the study context, to also help us gain (b) a deeper understanding of the external validity of the findings, which could inform adaptation and replication. both are important goals for studies with any intention of scaling.

2. how the program was experienced by a range of stakeholders and how it could have been done better. this part doesn’t feature in ed’s post or levy paluck’s piece but is important. sometimes i feel like when i talk about process, everyone breaks out in log-frame hives. take a deep breath. i don’t just mean process checklists and indicators. i mean recording how things went, deviations from the study design, and seeking feedback from study participants, study facilitators, study staff, and other study stakeholders. in the bombay experiment referenced above, the team had a process evaluation officer, who consistently surveyed staff and documented  meetings with the participants. these data allowed the researchers to know, among other things, that the participating urban women “balked” at collective action but were happy to share information one-on-one — a fairly useful finding for anyone else designing a program with similar goals or in a similar population. i think the researchers could have gone slightly further in asking participants about possible explanations for the similarity of outcomes in the treatment and comparison group — but the centrality of process evaluation is clear nevertheless. in a similar vein, campos et al draw lessons from experiments that didn’t happen and propose that researchers need to “work more on delivery and better incentivize project staff.” this, too, suggests a need to better collect (and use) information on delivery process and staff perceptions of projects, which means making time (and setting aside money) to solicit this information and finding ways to incorporate it into study findings. it also means taking program design as seriously as experimental design.

in sum, explanation matters. collect data that allows for better explanation of the mechanisms underlying causal effects as well as the process by which those mechanisms were put in place. in the meantime, everyone needs to do more work on figuring out how to present these types of data in a way that is easily accessible to and valued by a variety of researchers and practitioners.

on thursday, a controversial islamic cleric spoke out against the polio vaccination campaign, telling people that new cases of polio were caused by contaminated medicine.

more on targeting vaccination workers here.

i have written previously about the joys of getting and giving directions in lower-income countries – specifically for research and household follow-up, although the general taxi/auto/tuk-tuk stories of trying to reach any specific location purposefully are equally fun (in hindsight).

after reading my initial post, at least one friend reminded me that people at home (in the US) aren’t always so good at directions either, too familiar with a route to think about landmarks or to remember street names, and already too accustomed to google maps & similar being able to get the job done. the atlantic article from the title, about west virginia, re-emphasizes, for one, that a lack of street names, the use of landmarks, etc, is hardly only a poor-country phenomenon – rather, that “addresses have historically been an urban commodity” and one that probably belonged to highly literate urban areas with people who moved around the city a good deal.

formalizing addresses is more important than the inconvenience of trying to find a location or getting mail delivered. it is also essential for emergency services to find you and is presumably useful for tax collection and other basic services of the state.

which brings us to the second important part of the article: west virgina relied both on 911-services and a deal with verizon to get the mapping and road-naming word underway. knowing the power and visibility of mobile companies in many low- and middle-income countries, would this not be a reasonable way  to move the task forward? of all the potential projects for m-dev (e.g. and here, h/t tom paulson), it seems to me that mapping, paying taxes, and vital registration are some of the most promising and fundamental – as well as good public-private ventures. these would be fairly top-down and possibly foucauldian projects, and may be faulted for that, but i think we need more thinking about how the state can connect with its citizens.

finally, the atlantic article also points out the fun/difficulty of coming up with that many new street names. on absurd street names presumably combined by some random generator (although the linked article points to a single woman), i think my parent’s town has to take the cake.

drive-by truckers (highway 72):

“Don’t know why they even bother putting this highway on the map
Everybody that’s ever been on it knows exactly where they’re at.”

developing efforts have a greater chance of success when they stop treating cultural factors as a problem to be solved and try instead to harness them as a means to channel behavior in more positive ways.

over the past several years, a troubling thing would sometimes happen. i would go to a museum in europe (like the digs under geneva) or read a food history* and the author would mention ‘corn’ at a time that should have significantly pre-dated european contact with north america, and therefore, mention of maize in the ‘old world.’ i found this deeply confusing. sometimes i thought the author would go on to reveal that vikings or the basque had actually brought maize back from their early voyages and really rock my world.

but the authors were always silent and my brain would hurt.

however, mr. kurlansky tells me, “it was the 17th-century English who gave corned beef its name – corn being any kind of small bits, in this case, salt crystals.’

corn seems to just be a synonym for small things and sometimes used as a generic word for grain (e.g., which suggests it was the term for the most common grain in the region).

is this just my american english (or just my modern english) failing me?

*i really enjoy these commodity-specific histories - spice** and tea*** and all the rest. i know it is trendy but i hope it is a lasting one; it lets a history span geographic space and time frame without being too overwhelming.

**i first read spice around the time i was really digging into orientalism; nice pairing. not a very difficult one to figure out, either, but thought i would mention it!

*** i just found several additional tea histories while looking up that link. hooray! recommendations always welcome.