adapted writing guide, a few thoughts on writing and pedagogy

Posted on by hlanthorn

i just spent some time summarizing and adapting someone else’s adaptation of someone else’s guide to writing (specifically for ethics and philosophy but many of the points apply more generally). i have attached it in case it is of use!

i am now assisting with this particular ethics course for the first time. it was therefore no longer surprising  but still unsettling that on almost every midterm i graded last week, i wrote “a thesis sentence would be helpful.” i‘ll note that this is a masters-level course. i thought about reintroducing the (as noted by a friend, colossal ”In-and-Out 4-by-4 Animal Style”) sandwich as teaching tool but opted against it.

i find it upsetting that in a school of public health – covering topics for which communication skills are ostensibly quite important – there so little direct emphasis on improving writing and public speaking.

first, for better or for worse, at least in my department, TAs do the vast majority of the grading. we generally don’t have the time (nor, ahem, the commensurate pay) to comment on writing style and grammar as well as content.

second, the writing resources at school are limited to one man. he does a great deal of good work but we can hardly assign all students to go to him before a paper is due, as in undergrad the professor could mandate that a paper went to the writing center before it was turned submitted  (where you were forced to read your paper out loud, which was both terrifying and extremely helpful).

third, the above point is all the more upsetting given the school’s cultivation of an international student body. that there are no writing resource that ESL (or, likely EnL) students can access when working on a specific paper is fairly upsetting.

fourth, i am of the firm opinion that one really learns how to write or present by having to comment on or grade good and bad writing or presentations. however, most times when i try to insert a mandatory “read your paper out loud to another student” or “grade a fellow student’s paper” into a curriculum, it is shot down for one reason or another. yes, it’s a pain. yes, students might go easy on each other. nevertheless, i still think is a good idea and a necessary component of taking good writing seriously. rating other students’ presentations seems to go over slightly better – but only slightly.

Writing for ethics_HEL2013

miss marple & an apology to india

while i have been sick, i have watched/listened to (a fairly absurd amount of) british murder mysteries, including tommy & tuppance and miss marple (including the episode in which miss marple and tuppance team up!).

on another track, there are always funny turns of phrase when working in english in india (and other places). one that always particularly tickled me was referring to “pressurizing” someone to do something, as opposed to “pressing” or “pressuring” someone into something, as i would say. it always seemed like an odd transformation of the word.

but now, thanks to miss marple,  i realize that ‘pressurizing’ was in fact a british construction, foisted at some point onto the colonies. i still think it sounds funny but at least i know where to place the blame for a goofy word.

back (and forward) from ‘the big push forward’ – thoughts on why evidence is political and what to do about it

Posted on by hlanthorn

i spent the beginning of the week in brighton at the ‘big push forward‘ conference, on the politics of evidence (#evpolitics) which mixed the need for venting and catharsis (about the “results agenda” and “results-based management” and “impact evaluation”) with some productive conversation, though no immediate concreteness on how the evidence from the conference would itself be used.

in the meantime, i offer some of my take-aways from the conference – based on some great back-and-forths with some great folks (thanks!), below.

for me, the two most useful catchphrases were trying to get to “relevant rigor” (being relevantly rigorous and rigorously relevant) and to pay attention to both “glossy policy and dusty implementation.” lots of other turns-of-phrase and key terms were offered, not all of them – to my mind – terribly useful.

there was general agreement that evidence could be political in multiple dimensions. these included in:

  • what questions are asked (and in skepticism of whose ideas they are directed), by whom, of whom, with whom in mind (who needs to be convinced), for whom – and why
  • the way questions are asked and how evidence is collected
  • how evidence is used and shared – by whom, where and why
  • how impact is attributed – to interventions or to organizations (and whether this fuels competitiveness for funds and recognition)
  • whether the originators of the idea (those who already ‘knew’ something was working in some way deemed insufficiently rigorous) or the folks who analyze evidence receive credit for the idea

questions and design. in terms of what evidence is collected and what questions are asked, a big part of the ‘push back’ relates to what questions are asked and whether they help goverments and organizations improve their practice. this requires getting input from many stakeholders on what questions are important to ask. in addition, it requires planning for how the evidence will be used, including what will be done if results are (a) null, (b) mixed, confused or inconclusive, and (c) negative. more generally, this requires recognizing that policy-makers aren’t making decisions about ‘average’ situations but rather decisions for specific situations. as such, impact evaluation and systematic reviews need to help them figure out what evidence applies to their situation. the sooner expectations are dispelled that an impact evaluation or a systematic review will provide a clear answer on the what should be done next, the better.

my sense, which was certainly not consensus, is that to be useful and to avoid being blocked by egos, impact questions need to shift away from “does X work?” to “does X work better than Y?” and/or “how an X be made to work better?” this also highlights the importance of monitoring and feedback of information into learning and decision-making (i.e.).

two more points on results for learning and decision-making. first, faced with the assertion that ‘impact evaluation doesn’t reveal *why* something works,’ it is unsatisfactory to say something along the lines of ‘we look for heterogenous treatment effects.’ it absolutely also requires asking front-line workers and program recipients why they think something is and is not working — not as the final word on the matter but as a very important source of information. second, as has been pointed about many places (e.g.), designing a good impact evaluation requires explication of a clear “Theory of Change” (still not my favorite term but apparently one that is here to stay). further, it is important to recognize that articulating a ToC (or LogFrame or use of any similar tool) should never be one person’s all-nighter for a funding proposal. rather, the tool is useful as a way of collectively building consensus around mission and why & how a certain idea is meant to work. as such, time and money need to allocated for a ToC to be developed.

collection. as for the actual collection of data, there was a reasonable amount of conversation about whether the method is extractive or empowering, though probably not enough on how to shift towards empowerment and the fact that extractive/empowering are not synonymous with quant/qual. an issue that received less attention than it should have was that data collection needs to align with an understanding of how long a program should take to work (and funding cycles should be realigned accordingly).

use. again, the conversation of the use of evidence was not as robust as i had hoped. however, it was pointed out early on (by duncan green) that organizations that have been comissioning systematic reviews in fact have no plan to use that evidence systematically. moreover, there was a reasonable amount of skepticism around whether such evidence would actually be used to make decisions to allocate resources to specific organizations or projects (for example, to kill or radically alter ineffective programs). rather, there is a sense that much impact evaluation is actually policy-based evidence-making, used to justify decisions already taken. alternatively, though, there was concern that the more such evidence was used to make specific funding decisions, the more organization would be incentivized to make ‘sausage‘ numbers that serve no one. thus, the learning, feedback and improving aspects of data need emphasis.

empowerment in the use of data (as opposed to its collection) was not as much a part of the conversation as i would have hoped, though certainly people raised issues of how monitoring and evaluation data were fed-back to and used by front-line workers, implementers, and ‘recipients.’  a few people stressed the importance of near-automated feedback mechanisms from monitoring data to generate ‘dashboards’ or other means of accessable data display, including alternatives to written reports.

a big concern on use of evidence was ownership and transparency of data (and results), including how this leads to the duplication/multiplication of data collection. surprisingly, with regards to transparency of data and analysis, no one mentioned the recent reinhart & rogoff mess, nor anything about mechanisms for improving data accessibility (e.g.)

finally, there was a sense that data collected needs to be useful – that the pendulum has swung too far from a dearth of data about development programs and processes to an unused glut, such that the collection of evidence feels like ‘feeding the beast.’ again, this loops back to planning how data will be broadly used and useful before it is collected.

here’s an idea — if they are trying to tell you something, make it easy for them to do so.

Posted on by hlanthorn

there’s been a good deal of press around the unfortunately insignificant results of a major HIV prevention trial with products for women in south africa, uganda and zimbabwe. the results had little to do with efficacy of the products (a pill and a gel) but rather with the fact that most of the participating women did not use the treatments as recommended – or at all.

one potential response is to improve our behavioral interventions to support adherence to treatment regimens (and prevention regimens) and integrate these methods more directly into medication trials. adherence and persistence with medication are global problems and we are just beginning to learn – with the help of health psychology and behavioral economics – how to tackle the challenge. efforts so far include high- and low-tech solutions, though not all the promises of the former, in terms of mhealth to facilitate behavior change and adherence, have yet been borne out.

another, not mutually exclusive, response would be to actually ask the women what they would like to see and use in the way of HIV prevention – a tool which should be empowering for them. the press seems full of comments like “the women are trying to tell us something!” why does it seem that, then, for a product made for them, they have to work so hard to tell us those things? why are we not hearing more sentences that start “the women told us…” that is, why, after such a big trial, am i not hearing anything about on-going qualitative and observational follow-up efforts to learn more about what exactly didn’t work about the methods offered to women?

there’s often a lot to learn from null results (that’s science, right?) but it doesn’t just come from brainstorming what went wrong. asking helps.

i don’t suggest that people are perfectly prescient about what they need or want. often, the innovations that we can’t live without now – smartphones, for example – weren’t a need or even desire that most people could have articulated 20 years ago. as such, directly asking people what it would take to get them to engage in X desirable behavior can’t determine the research agenda. but it should certainly be part of figuring it out.

causes, explanations, & getting stuff done

Posted on by hlanthorn

@edwardcarr, i also have a confession, which is that i have a small crush on you right now for the post in which you make a confession about causality and try to disentangle causes, mechanisms, and information that can be used to understand, revamp, and scale programs. i‘d like to try to tweak the argument, especially in light of the recent excitement about the publication of null results from a bombay-based cluster-randomized trial on health and pregnancy, described here.

the tweak is to separate out two categories of information that are important and may be better gleaned from qualitative inquiry and analysis, though more adaptive/iterative and process-focused quantitative data can also play a role. i think the general mindset on what constitutes (and who has) useful and useable information is as important as the difference between quant or qual analysis.

1. how the program actually brought about an effect. this point is the focus of ed’s post, as well as levy paluck’s nice paper on qualitative methods and field experiments, which distinguishes between causal effects and causal mechanisms. mechanisms can, to some degree, be pursued and examined through ever-proliferating treatment arms in RCTs… but observation and purposive, systematic conversation are very helpful. if i read ed carr’s piece correctly, he wants more (data collected with an eye towards) explanation in order to pursue (a) a deeper understanding – beyond ‘story time’ – of what moderates and mediates the (potentially causal) relationship between X and Y within the study context, to also help us gain (b) a deeper understanding of the external validity of the findings, which could inform adaptation and replication. both are important goals for studies with any intention of scaling.

2. how the program was experienced by a range of stakeholders and how it could have been done better. this part doesn’t feature in ed’s post or levy paluck’s piece but is important. sometimes i feel like when i talk about process, everyone breaks out in log-frame hives. take a deep breath. i don’t just mean process checklists and indicators. i mean recording how things went, deviations from the study design, and seeking feedback from study participants, study facilitators, study staff, and other study stakeholders. in the bombay experiment referenced above, the team had a process evaluation officer, who consistently surveyed staff and documented  meetings with the participants. these data allowed the researchers to know, among other things, that the participating urban women “balked” at collective action but were happy to share information one-on-one — a fairly useful finding for anyone else designing a program with similar goals or in a similar population. i think the researchers could have gone slightly further in asking participants about possible explanations for the similarity of outcomes in the treatment and comparison group — but the centrality of process evaluation is clear nevertheless. in a similar vein, campos et al draw lessons from experiments that didn’t happen and propose that researchers need to “work more on delivery and better incentivize project staff.” this, too, suggests a need to better collect (and use) information on delivery process and staff perceptions of projects, which means making time (and setting aside money) to solicit this information and finding ways to incorporate it into study findings. it also means taking program design as seriously as experimental design.

in sum, explanation matters. collect data that allows for better explanation of the mechanisms underlying causal effects as well as the process by which those mechanisms were put in place. in the meantime, everyone needs to do more work on figuring out how to present these types of data in a way that is easily accessible to and valued by a variety of researchers and practitioners.

targeting vaccine teams: and now nigeria

Posted on by hlanthorn

this seems to be the first time workers were killed in nigeria, despite previous opposition to vaccines.

on thursday, a controversial islamic cleric spoke out against the polio vaccination campaign, telling people that new cases of polio were caused by contaminated medicine.

more on targeting vaccination workers here.