teaching qualitative analysis: an intro

teaching qualitative analysis is not easy for several reasons. first, an awful lot of material on doing qualitative research focuses on data collection. relatedly, then, a lot of academic papers that draw on qualitative data and analytic methods focus on data collection and organization. too often the use of an analytic software stands in for an explanation of how analysis was done.

.

second, lecturing on qualitative analysis is much like a powerpoint lecture on riding a bicycle. it sounds very easy (right foot down, then left foot down). it only gets hard when you try to do it.

.

nevertheless, a lecture must begin somewhere. i hope my notes, below, may prove useful to someone else.

.

despite my impulse to start with lincoln and guba’s paper, since this wasn’t an audience that spends all the their time reading academic papers or thinking about theory, i started with an example published qualitative piece. i found one that focused on a similar data source and level (interviews with high-level stakeholders as opposed to, say, a focus groups in a village or historical document review) as well as stated analytic approach (in this case, this paper by smit et al. was a good match).

.

the intuition was that — even though this was not an audience entirely used to reading research outputs — it would be helpful to get a handle on the type of research product toward which we wanted to build before getting lost in the nitty gritty of analysis. with a slightly different audience, i probably would have made the lincoln & guba piece mandatory to provide a touchstone for considering and critiquing the paper and then for storyboarding our own paper.

.

after reviewing a few key terms central to doing qualitative work (sources of qualitative data (talk, text, observations, images), positionality, inductive reasoning, deductive reasoning, codes and coding), we spent much of the first day discussing and critiquing the paper. 

.

first, individually and then in pairs, and then finally as a big group, we explored these questions:

• what are the goals set out by the researchers for this project and paper
• why did a qualitative approach make sense to answer these questions?
• what are the key conclusions the researchers draw from their analysis?
• what types of data do the researchers use to support their conclusions?
• what types of analyses do the researchers use to support their conclusions?
• what is convincing about the link between the researchers’ results and their conclusions? could anything have been done to make this more convincing?
• do the researchers achieve the goals they set out for themselves? why or why not? what could have been done differently?

.

then, as a larger group, we explored these additional questions (which are mostly notes to myself of topics to cover rather than and handout distributed to to participants; the first set of questions i did distribute):

• methods: data collection, organization, analysis
  • who were the data collectors? can we tell from the paper? how?
  • what was the positionality of the interviewers vis-à-vis the informants? what difference does this make?
  • what data collection strategy was used?
  • why do key informant interviews make sense as a data source given the research questions and goals?
  • were any other types of data used? how?
  • how were key informants chosen?
    • how many interviews were completed?
    • what does purposive sampling mean? snowball sampling?
    • how do the authors signal that the sample is representative of the relevant interests (i.e., what is thematic saturation or redundancy? what does this imply about the relationship between data collection, entry, and analysis in qualitative research?)? is this convincing? could it have been more convincing and if so, how?
  • how do the authors display their sample? is it helpful? what characteristics do they highlight and why? could it have been done better or differently?
  • what is a semi-structured interview guide? how does it differ from a completely structured or unstructured questionnaire or guide?
    • what types of questions did the researchers ask? how do we know? what else might we have liked to have known?
  • is there anything else we would have liked to have known about how the data were collected?
  • how were the data converted into transcripts? do the authors provide all the information we want on this?
  • what does it mean in this case that an inductive approach was used?
    • how did the researchers set about their induction? is this convincing?
      • what does it mean that key themes “emerged”?
    • what would have been different if the researchers had used a deductive approach? how would the analysis have changed? what would have been the trade-offs?
    • what did the authors actually do in analysis? do they provide us enough information to know?

.

• results
  • how do the authors reassure us that the information from different stakeholders is used and presented in a balanced way? could this have been done differently or better? if so, how?
  • figure 1 is the main display of the (descriptive) results.
    • did you look at it carefully when reading the paper or did you skip over it?
    • where did the figure come from? what do the bullet points in each box represent?
    • is this figure meant to be descriptive or analytic?
    • what is helpful about this display? what could have been done differently?
  • how are the results in this paper organized?
    • how does the presentation of results relate to the research questions?
    • how are quotations used to communicate the results? is this effective? convincing?
    • how were the quotes selected? are they meant to be representative or exceptional? how do you know?
    • were conflicting or diverging viewpoints represented? how do you know?
    • do you feel the researchers have drawn reasonable inferences from the data?
    • do the conclusions follow from the data?
    • do you feel that the researchers already had the conclusions in mind before they analyzed the data? does this affect the convincing-ness of the analysis?

.

• interpretations
  • what did the researchers do to make the present paper credible?
  • what did the researchers do to make the present paper balanced?
  • what did the researchers do to enhancing transparency?
  • is the paper ultimately convincing? why/not?