RCT, economics and qualitative research

Imagine how gratifying it is for me to wake up this morning and find this post by Edward Caar through a Twitter link:

What brings me to today’s post is the new piece on hunger in Foreign Policy by Abhijit Banerjee and Esther Duflo.  On one hand, this is great news – good to see development rising to the fore in an outlet like Foreign Policy.  I also largely agree with their conclusions – that the poverty trap/governance debate in development is oversimplified, that food security outcomes are not explicable through a single theory, etc.  On the other hand, from the perspective of a qualitative researcher looking at development, there is nothing new in this article.  Indeed, the implicit premise of the article is galling: When they argue that to address poverty, “In practical terms, that meant we’d have to start understanding how the poor really live their lives,” the implication is that nobody has been doing this.  But what of the tens of thousands of anthropologists, geographers and sociologists (as well as representatives of other cool, hybridized fields like new cultural historians and ethnoarchaeologists).  Hell, what of the Peace Corps?

Whether intentional or not, this article wipes the qualitative research slate clean, allowing the authors to present their work in a methodological and intellectual vacuum.  This is the first of my problems with this article – not so much with its findings, but with its appearance of method.  While I am sure that there is more to their research than presented in the article, the way their piece is structured, the case studies look like evidence/data for a new framing of food security.  They are not – they are illustrations of the larger conceptual points that Banerjee and Duflo are making.  I am sure that Banerjee and Duflo know this, but the reader does not – instead, most readers will think this represents some sort of qualitative research, or a mixed method approach that takes “hard numbers” and mixes it in with the loose suppositions that Banerjee and Duflo offer by way of explanation for the “surprising” outcomes they present.  But loose supposition is not qualitative research – at best, it is journalism. Bad journalism. My work, and the work of many, many colleagues, is based on rigorous methods of observation and analysis that produce validatable data on social phenomena.  The work that led to Delivering Development and many of my refereed publications took nearly two years of on-the-ground observation and interviewing, including follow-ups, focus groups and even the use of archaeology and remotely-sensed data on land use to cross-check and validate both my data and my analyses.

You really should read the whole thing.

As one who has a Masters degree in Development Studies but who chose to do a PhD in anthropology because I found that development research is all too often dealing with quantitave, “generalisable” data, and who has concluded said PhD, I find it really interesting that the RCT movement in economics seem to be taking credit, in the media and in policy circles, for what ethnographers – anthropologists, rural sociologists, historians, human geographers – have been saying all along. This is that things are a lot more complicated than people want to think, that it is extremely difficult to find a generalisable explanation, and that at the end of the day, what leads one to better understanding of issues is attention to personal stories, and an attempt to tease out how those stories are linked to larger structures, like local politics, regional politics, the economic structures, colonisation, culture etc. etc. One cannot arrive at this sort of understanding without spending time trying to understand the interaction between all these elements. Now that economists have discovered qualitative research it seems as if it were never there, as if there aren’t people who have been pointing to the importance of understanding nuances and personal stories.

I have deliberately refrained from commenting on RCT in economics because I wanted to read some of the texts, but since Edward Carr took thoughts out of my head I thought I would quote him and call attention to the fact that these kinds of studies have been going on for a long while. If economists are not aware of that (I think some of them are) it is their fault for not looking at other social sciences.

My copy of Karlan’s book is in the post to me, and I look forward to reading it. I doubt that I will learn anything new from it, but I feel it is important, as an economic anthropologist, to know what economists are doing. I wish economists would extend the same courtesy to other disciplines whose works often overlap with theirs.

And on RCT itself, check out the link that my brother, a medical doctor and researcher, sent me. It is a Lancet article titled “A philosopher’s view of the long road from RCTs to effectiveness”. Remember, RCT has been in medical and pharmacological research for a while.

PS, I promised a while ago to blog a list of must read economic anthropology books. I should get to it pretty soon.