Field Research Debated

On October 20, a debate entitled “Experimental Methods in the Social Sciences” brought together more than 200 students and faculty from departments in the social sciences to hear perspectives on field experimentation, a methodology gaining popularity with scholars in these disciplines. Faculty participating in the debate were Angus Deaton (Princeton), Donald Green (Yale), Ian Shapiro (Yale), Susan Stokes (Yale).

Dawn Teele, a first-year political science graduate student and the debate’s organizer, said that some funding organizations “want scientific evidence that their development programs are successful and that they believe experiments are the only way to understand cause and effect.”

Field experiments can be thought of as a combination of field research, where a scholar conducts research on phenomena of interest in the context in which they occur, and lab research, where a scholar performs experiments in a controlled laboratory setting. Through field experiments, scholars attempt to draw causal inferences about social phenomena by entering a real-world setting and observing responses to some intervention (experiment) in a randomly selected treatment population, in comparison with a randomly selected control population.

“…scholars must employ the very methods and the assumptions of traditional observational research that the experimental method was designed to avoid.”

Among the foremost proponents of the use of experimental methods in Political Science is Donald Green, A. Whitney Griswold Professor of Political Science at Yale University, and Director of Yale’s Institution for Social and Policy Studies, an interdisciplinary research center that emphasizes field experimentation. Professor Green argued that causal inferences can be confidently drawn only on the basis of evidence generated by experimental research. His assertion was challenged by Professors Angus Deaton, of Princeton’s Department of Economics, Susan Stokes, Chair of Yale’s Political Science department, and Ian Shapiro, Sterling Professor of Political Science and Henry R. Luce Director of the MacMillan Center at Yale.

Professor Deaton forwarded a multiple-pronged critique of the experimental method, noting, particularly, that in order to analyze the results of experiments, scholars must employ the very methods and the assumptions of traditional observational research that the experimental method was designed to avoid. Thus, experiments do not offer a means for social scientists to avoid the usual problems of causal inference, and as a result, should not be considered a “gold standard” in social scientific research.

Stokes offered a defense of traditional observational research, describing the ideology of the proponents of field experimentation as “radical skepticism,” and arguing that the principles of radical skepticism, if applied evenhandedly to field-experimental research, would raise similar problems with causal inferences that the skeptic identifies in traditional observational research. Finally, Shapiro proposed a systematic critique of research driven by methodological concerns, as opposed to research motivated by interest in persistent problems of politics and society. He argued that social scientists should ask big questions, and be open to using whatever methods are necessary to answer them satisfactorily. The experimental method, though sometimes very useful, simply cannot answer many of the perennial questions of social science.

Green responded convincingly to each of these important critiques, making it clear that the debate was only a starting point for further discussion about the uses and limitations of the experimental method in the social sciences.