For years I listened as educators started their speeches with “research has shown,” or some phrase of similar meaning. I learned through my own educational research the narrow applicability of research findings, the limits inherent in studies’ designs and interpretations, and the fundamental importance of the assumptions underlying those designs. With those understandings, I came to hear something else in the “research has shown” introduction. Unless the speaker was intellectually naïve and broadly construing narrow findings, the presenter was usually saying, “This is what I believe, and you should, too.”
I began to think of this practice as application of an unwritten theorem which assigned the published study a truth value often well beyond its intended interpretation. I became an instant skeptic when presenters seemed to be applying this unwritten theorem by citing research only generally as justification for conclusions to be offered. Without a chance to review the studies cited, I remained skeptical of both the application of the studies’ findings and the derived conclusions.
Recently we’ve been working to introduce pellet-fired central heating to the US marketplace, particularly in the Northeast where oil dependency is extraordinary. Pellet central heating is commonplace in much of the world but is still relatively new in the US. Work in this business has led me to a couple of corollaries to this “theorem” of the published study as truth.
The more alarming of these two corollaries is “the first counts most” corollary. In several developing markets highly questionable studies have been released to the marketplace. As the first published pieces devoted to examination of the relationships between pellet heating and the environment or competitive energy sources, they have become the studies against which all other thinking is compared.
In one case in the Northeast, the commissioned study was intended to inform the policy-making of politicians through exploration of the use of biomass as an energy source for the production of electricity. The study has become a “truth” of sorts on issues ranging far from the production of electricity using biomass fuel to the use of wood products for heating buildings.
A Northwestern report of the comparative costs of heating using various energy sources has taken on unwarranted stature despite its deeply flawed analysis simply because it is the first such study published in the region. The study talks about “wood” heating and seems to conflate data associated with cordwood burning with data associated with pellet burning resulting in meaningless results.
A second corollary to the “published study as truth” theorem is the “misconstrual” corollary. This corollary is commonly applied both by the proponent of a particular point of view and by general consumers of studies. Widespread practice has those who have only read summaries of a study construing those studies in contexts well beyond those justified by study design or analysis. Whether these people are using their conclusions to support a point or just to formulate an opinion, they are assigning truth value based on poor application or lack of understanding of actual research findings.
In 1962 Thomas Kuhn told us that scientific change occurs by revolution, not evolution. He pointed out that most scientists spend their careers proving the existing point of view and that those who take a contrary position are often castigated for their efforts. When enough thoughtful researchers recognize that the newly proposed thinking has value beyond that of the old, a revolution occurs and thinking shifts and scientists begin careers proving the new model.
There is little different here. In the emerging biomass energy sector in the US marketplace, the first published “research” has come from those attempting to preserve the status quo for their own reasons, or to argue in favor of different energy sources. As more thoughtful members of the consuming public ignore these “truths” and move to biomass heating for its many values, a revolution in thinking will occur and the old first published truths will finally be recognized as passing efforts at preserving what currently is.
Dutch Dresser is the Managing Director of Maine Energy Systems in Bethel, Maine. He has a doctorate in science education.