Friday, March 30, 2012

Have conservatives lost faith in science? Or should the journal editor responsible for this paper resign?

Have conservatives lost faith in science? Many MSM outlets are reporting that. The LA Times, for example, writes:
As the Republican presidential race has shown, the conservatives who dominate the primaries are deeply skeptical of science — making Newt Gingrich, for one, regret he ever settled onto a couch with Nancy Pelosi to chat about global warming.
Is the story true? Instapundit reader Mary Ritenour researched the issue and reports:

I tracked back to the original paper (http://www.asanet.org/images/journals/docs/pdf/asr/Apr12ASRFeature.pdf) to see what the exact survey question was.

“The GSS asked respondents the following question: “I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them [the Scientific Community]?”(page 172)

The confidence in “people running these institutions” was being measured, not “Science” itself. Huge difference. HUGE!

After ClimateGate I, ClimateGate II, GlacierGate, and FakeGate, Ms. Ritenour is certainly correct: the difference is huge.

The misrepresentation of the paper's results started with the paper's author, Gordon Gauchat, a sociologist from the Univ. of N. Carolina at Chapel Hill. In the abstract of his paper, he misrepresents the meaning of his survey as "trust in science" rather than, as the survey asked, the "people running [the] institutions" :
This study explores time trends in public trust in science in the United States from 1974 to 2010. More precisely, I test Mooney's (2005) claim that conservatives in the United States have become increasingly distrustful of science. Using data from the 1974 to 2010 General Social Survey, I examine group differences in trust in science and group-specific change in these attitudes over time. Results show that group differences in trust in science are largely stable over the period, except for respondents identifying as conservative. Conservatives began the period with the highest trust in science, relative to liberals and moderates, and ended the period with the lowest. [Emph. added]
If one gets to the small print in footnote 2 at the end of the paper, the author does explain that he uses the word "science" to refer to something other than science:
For purposes of clarity and consistency, this study uses science to refer to a group of people, the organizations they belong to, and the professional boundary that central institutions in society agree is a source of credible expertise (Gieryn 1999). Terms like scientific establishment or organized science might be more appropriate, but these ideas are often simply referred to as "science."
In other words, the author was well aware that he was using the wrong word. Abstracts are supposed to be a brief, honest, and accurate summary of the paper. No one who depended on the abstract alone would be aware of his false terminology.

Worse still, the wording in his definition in footnote 2 is not an exact match for the wording used in the GSS survey which was the source of all his quantitative data. Thus, far from clarifying the issue, his footnote 2 just adds to the confusion.

The paper was published in the American Sociological Review, a peer-reviewed journal. Everyone expects the social sciences work to be sloppy but how did the reviewers not catch the conflict between the actual study and its abstract? Didn't they also notice the internal inconsistency in the study's definition of "science"? Shouldn't the reviewers have insisted that the abstract provide an accurate representation of the paper's main result? Where were the journal's editors in this process? Shouldn't the journal publish a retraction? Who was the editor responsible for this paper's faulty review process? Shouldn't he resign?

Will this paper about loss of trust in "science" cause people to lose trust in social science?

No comments:

Clicky Web Analytics