My Science is Better than Your Science: Ideology, Objectivity and the Attack on Public Schools
Fred Barton

The fundamental idea of No Child Left Behind rests on the scientific approach to education. Curriculum is constructed applied and measured using the best available “science.” Of course science isn’t supposed to be biased, politically motivated, or driven by ideology.

So it was with great fanfare that the Bush administration rolled out the National Reading Panel report on the scientific approach to teaching reading. Critics of this report were branded as out of touch, lacking in scholarship, or worse, touchy feely. Science had spoken and the argument on how to best teach reading to children was closed.

Or was it? The use of science by the Bush administration has come under increasing fire. In February more than 60 leading scientists from many fields, including Nobel Laureates, National Medal of Science Recipients and former Federal Agency Directors, among others, issued a statement condemning the misuse of science by the Bush White House. The statement read in part: “Across a broad range of issues the administration has undermined the quality of the scientific advisory system and the morale of the government’s outstanding personnel…In case after case, scientific input to policymaking is being censored and distorted.”

Scientists around the country were so concerned with the misapplication of findings that an organization called Scientific Integrity in Policymaking conducted a study. Among their findings:

• There is a well-established pattern of suppression and distortion of scientific findings by high-ranking Bush administration political appointees across numerous federal agencies.
• There is strong documentation of a widening effort to manipulate government’s scientific advisory system to prevent the appearance of advice that might run counter to the administration’s political agenda.
• There is evidence that the administration often imposes restrictions on what government scientists can say or write about.
• There is significant evidence that the scope and scale of the manipulation, suppression and misrepresentation of science by the Bush administration is unprecedented.

Given what professional scientists have to say about the behavior of the Bush administration, it would be useful to examine the “science” behind educational policies they put forth. This may seem like a task for other scientists, but with a little study, teachers can become educated consumers of government policy supposedly based on “science.”

Conclusions are the endpoint of the scientific process and, if the process has been valid, those conclusions are rational, logical and free from bias. But just how much of a “conclusion” can be drawn from a scientific study? If a study of NASCAR races were conducted and it was found that most winning cars had Chevy engines, we could say there is a “correlation” between Chevy engines and winning on the circuit. But, if we were to say that driving a car with a Chevy engine “caused” drivers to win more often, that would be met with some skepticism because it vastly over simplifies the situation.

Yet that is precisely what is done in many studies of what goes on in a schoolroom. Most studies of educational activity are correlational in nature. Correlation simply means that there is a relationship between factors studied. What the nature of that relationship is may take further study. The “Nation At Risk” report of NAEP found a correlation between high test scores and a strong economy (at that time) in certain Asian nations. Soon however high test scores and good economy became inextricably linked in the popular press and politician’s speeches until it was taken as a given that one (high test scores) caused the other (strong economy). In logic this is called the fallacy of the faulty generalization and it is easily recognized because it over simplifies a complicated situation and usually takes out the human factor (the driver in the car, the teacher in the classroom).

Another way that studies can be used to paint inaccurate or biased pictures is in the publication of ranks and scores. Ranks are simply results compared to one another. Someone always has to be first; someone always has to be last, but the significance of that distance is obscured by the ranking procedure. In the NASCAR example, a driver will win the race and a driver will come in tenth. The difference between those two places may be a matter of mere seconds, yet one is first and one is tenth in the paper the next day. In education, it means the school ranked at the bottom in a particular district may not be practically different than the one ranked at the top, yet it is often made to look that way.
 
Scores, on the other hand give more information that is useful. If the NASCAR driver is consistently turning in lap times that are three seconds ahead of his nearest competitor, that gives us an indication of how well he is doing relative to the rest of the field. If another driver is improving his lap times by a half second a lap, that give us information that can be used to predict when he might catch up to the leader. Scoring students in like type groups (grade level, reading level, etc.) gives us information that can be used by teachers to adjust curriculum.
 
Perhaps the most misunderstood, and hence misused, statistical term is “average.” To understand average it is first necessary to know that there are three variations on the term that are used interchangeably by reporters and some researchers. They are mean, median and mode. Each is a type of average. Mean is what usually passes for average: the sum of all scores divided by the number of students who took the test. Extremely high or low scores can skew this number though and make it less representative. Median is the exact midpoint of all scores. Half the scores are above it and half below, and mode is the particular score that occurred the most in the test sample.
 
What this means is that “average” isn’t a number, it’s a range and depending on how it is computed that range can be significant. Researchers call this range the standard deviation. This is the famous “plus or minus 3% of national polls.” So when a politician receives a favorable rating from 50% of the public it really means he could have a rating of 47% or 53%--as best as the researches can tell from their instrument. In other words, it’s a guess. An educated guess, but a guess none-the-less.

The NASCAR driver will turn in an average lap speed during the time trials before the race. We know he went faster and slower than that during the trial and we know that particular number is meaningless once the race starts and he has to deal with traffic. So which “average” should we pay attention to? The answer is it depends on the type of performance we are seeking to measure.

The averages that are reported in studies of classroom activities and student’s performance also depend on several important contextual factors and should be clearly defined and viewed in the framework in which they occurred.

Finally, perhaps the most important research criterion to explore doesn’t have anything to do with percentiles, or quartiles, or standard deviations at all. It is simply, who is paying? Several conservative organizations like the Heritage Foundation and the Fordham Foundation have sponsored research that is clearly impacted by their political agenda. Studies have been done that show research into drugs sponsored by drug companies that does not show their drug in a favorable light doesn’t receive the same publication and dissemination as research that does.

Arguably the most egregious example of policy impacting research was the 1990 report on education conducted by Sandia Laboratories titled “Perspectives on Education in America.” The 176-page report found that “[T]here are many serious problems in American public education but there is no system wide crises.” Upon reading the report, Deputy Secretary of Education (and former Xerox CEO) David Kearns told the researchers “You bury this, or I’ll bury you.”

The researchers were forbidden to talk about their findings and the report itself wasn’t made public until 1993 because it was undergoing “peer review” although the peer review of one federal agency by another was previously unheard of. The simple fact that the study did not jive with the policy views of Reagan and Bush I administrations was enough to quash its publication.

Science in thrall to policy has a long, and not so illustrious history going back to The Bell Curve, eugenics all the way to Galileo. Caveat Emptor is a phrase that was coined to protect us in the marketplace. Today it is equally applicable in the realm of objective science.

If you are interested in learning more about science in the attack on public schools, Gerald W Bracey has written two excellent books on the subject. The War Against America’s Schools: Privatizing Schools, Commercializing Education (Allyn and Bacon, 2002) and Bail Me Out: Handling Difficult Data and Touch Questions About Public Schools (Corwin Press, Inc. 2000).