Wednesday, July 20, 2005

Fox pundit calls all science (and science-based policy) into question

Somehow I knew that Fox News would have a field day with this one -- especially in light of recent legisltative attacks on the science behind consensus theories on global warming via calls for the entire research and data records of three particular scientists to be handed over to some angry congressman with ties to the energy lobby -- and I just feel compelled to respond.

A Fox pundit led off her column today with the deceptive statement that "A review of medical studies published from 1990 to 2003 in three prestigious journals -- the New England Journal of Medicine, JAMA and Lancet -- has called the validity of approximately one-third of them into severe question." She then opined, "If a relatively 'hard' science (like medicine) has such difficulty with accuracy, then the results offered by the so-called 'soft' sciences (like sociology) should be approached with a high degree of skepticism," and that "Inaccurate studies become entrenched in laws that govern our daily lives. [...] In the best of circumstances, research is unreliable outside strictly defined limitations; even within those limits, research generally provides only an indication rather than a proof.

First of all, the original AP story she cites spins the review entirely differently: "Subsequent research contradicted results of seven studies -- 16 percent -- and reported weaker results for seven others, an additional 16 percent." The distinction is important: Fox leaves the impression that a single "review" found flaws in one-third of the the studies published in the three journals. But the reality is that it was a series of other studies subsequently published in those and other medical journals which, over time, modified the results of the original reports, and sometimes contradicted them outright.

Far from an indictment of academic research as "unreliable," this is, in fact, an example that the peer-review and research-duplication process for academic research works. The original studies in question were published because they met the standards for solid initial research results on novel ideas and questions. The subsequent studies built on these original reports, also worked their way through the peer review process, and were able to make claims of more general validity. It is likely that not only were these "scaled-up" studies, but that they were also studies with increased amounts of funding.

This is why it is unfathomable to me that the Fox pundit would use this moment to incite her readers to "Bring skepticism and common sense to all data you hear; withhold your tax dollars." She argues,

Consider the 'feminist' issues of rape or domestic violence. Studies that address these areas are often released in combination with policy recommendations. Indeed, they sometimes appear to be little more than a springboard from which advocates can launch a campaign for more law.

In turn, the laws that result often provide for more research. The Violence Against Women Act or VAWA -- now up for re-authorization before Congress -- is an example. VAWA includes provisions for more tax-funded research, for precisely the sort of research that created it in the first place.

And, so, a re-enforcing cycle is established: studies lead to laws that lead to similar tax-funded studies, which call for more law.

Leaving aside for the moment the unanswered question of why this pundit is attacking the Violence Against Women Act (and "feminists"), she seems to be arguing that (a) research by "advocates" is by definition tainted; (b) law should not in any way be based on research because research may be called into question; and (c) the federal government should not fund research with tax dollars because research may be used to formulate policy or law.

In answer to (a), the idea that research by persons with political interests is always tainted, I would argue that all persons have political interests (by definition) and that it is the set of structures and processes through which research flows which allows us to judge its truth value and/or utility. I expect and want researchers to be interested in the topics they are researching. I hope that people interested in curbing violence are the ones researching causes and patterns of violence, for example. But I want that research to be undertaken by someone trained in academic methods and ethics; I want the work funded by sources without political interests in reproducing their own position of power; I want that research to flow through a process of professional peer review and publication; and I want that research to then be called into question over and over again if necessary, by other persons from the same kind of training, with the same kind of interests in finding the truth, and working through the same funding-neutral, peer-review process.

This also answers (b), in that of course research should be called into question. That's the definition of rational argument, as opposed to, say, "faith" or "common sense" (which by definition cannot be called into question). I would rather have my government basing its laws and policies on rational argument which can, if conditions change or of new information is revealed, be called into question, rather than on someone's singular and unwavering idea of faith-based law and policy.

And finally, in terms of (c), I want my democratic, transparent, politically-accountable federal government funding research with tax dollars, because that is precisely how we may produce the kind of unbiased research results the Fox pundit claims to desire. If by contrast research is only funded by undemocratic, private, secretive, corporate sources, how likely is it that research which threatens to undermine the power position of these sources might be funded, distributed, and used?

1 comment:

Melanie said...

For an interesting read, go to this link of the 2005 Baylor University research annual and read the interview with Robert C. Roberts, Professor of Ethics at Baylor. This private university makes no bones about seeking out and hiring "information laborers" who are Christian, or about emphaszing research that centers on Christian topics or debates. While I was there, faculty complained publicly and bitterly that they lost good recruits to this practice, but the ones that were hired tended to be fully supportive of research that supported conservative and Christian beleifs. Two of my department's faculty told me they felt compelled to write articles or do research on "approved" topics to make tenure, even if they weren't interested, like creationism and geology or geography of religion. Although the emphasis is publicly on teaching, students felt constrained to not tick off the prof with un-Christian views. Baylor admin had strong ties to the Bush world, and as I left I had doubts as to the real academic freedom any of my teachers had, even with tenure. If one starts a research project with a bias, don't the results reflect that? Also, this research was published in peer-reviewed journals and quoted without mention as to the bias of the university. I suppose those information workers who didn't like the policy didn't accept offers, so to me that tends to concentrate a biased research pool in one place.