OAH 2009: Sam Wineburg Dares to Ask If the Teaching American History Program Is a Boondoggle

His audience knew in advance this would be no ordinary luncheon address for a convention of historians. They had to get no further than the title to know that much: "An Antidote for an Ailing Profession or a $836,000,000 Boondoggle: The Future of the TAH Program."

TAH--the Teaching American History program sponsored by the U.S. Department of Education--has been a fountain of beneficence at which historians have been drinking for years. And now at a history convention someone was dare raising the question that it might all have been a boondoggle? At the annual meeting of the Organization of American Historians (OAH)? That was like suggesting at a meeting of farmers who have been gorging themselves at the federal trough that government subsidies might be a bad idea. It wasn't exactly what you'd expect.

More surprising was the identity of the speaker. It was Sam Wineburg, an education professor who runs the Stanford History Education Group, co-sponsor of the National History Education Clearinghouse, a TAH funded program. No wonder Wineburg told us in advance he would be uncomfortable having his talk filmed by HNN. As both a TAH critic and collaborator he was like a person with split personality whose multiple identities are at war with one another--or at least in an uncomfortable state of coexistence. (Wineburg made it clear that he was speaking for himself and not for the Stanford History Education Group.)

It was, then, with rapt attention that people prepared to listen to his talk. It didn't disappoint. It was easily the most controversial lecture of the entire 2009 OAH convention--and one of the most riveting. Because of the importance of his talk and our keen desire to help readers evaluate the effectiveness of the TAH program (no easy task), HNN is providing a comprehensive blow-by-blow account. (Disclosure: George Mason University's Center for History and New Media is the other sponsor of the National History Education Clearinghouse.)

He began his talk by noting the unexpected windfall that was the TAH program. Back in 2000 when Sen. Robert Byrd (D-WVa.) slipped in an amendment to Title X providing for a $50 million appropriation expressly to promote the study of history (not social studies) no one in the profession had seen it coming. History teachers weren't much in favor on Capitol Hill. Not too long before a member of Congress had ridiculed a fifth grade curriculum funded by the federal government that supposedly promoted adultery and cannibalism. In the 1990s the effort to create national standards for the study of history had drawn fire from politicians from one end of the country to the other.

Naysayers expected another calamity, noted Wineburg. But instead of collapsing amidst controversy the TAH program quickly became an established institution. Within a few years the program's funding was more than doubled to about $120 million annually. Within a short time TAH succeeded in drawing together both academics and high school teachers. As one historian Wineburg quoted had remarked, "money has a way of thawing the most chilly relations." This line drew loud laughter from the crowd.

Thus far TAH has received an amount approaching a billion dollars: $838,172,000. As Wineburg said, "we're talking [real] money." TAH funds have been a boon to the history profession, providing steady jobs for both professors and their graduate students at institutions around the country, from Michigan State to Ohio State.

It's also been a boon to teachers, few of whom have much training in history. Wineburg cited a study from 1997 that found that 80 percent of middle and high school social studies teachers did not major or minor in history as undergraduates. Until TAH came along, the overwhelming majority had no opportunity to pick up skills in the teaching of history through professional development programs.

Wineburg believes the TAH grants should continue, but he told the crowd that he disagreed with those who believe the grants have made a difference. Seated in front of him was Arnita Jones, executive director of the American Historical Association, who recently wrote that they had. When Wineburg later took questions she was one of the first to take issue with him.

What does it mean to "make a difference"?

It was at this point in his talk that Wineburg began raising fundamental doubts about the program, which provides for teacher development. While he agreed with the program's stated goal--the wholesale transformation of the teaching of history--he said that the efforts being made are wholly inadequate to achieving it:

Imagine that physicians lacked the skill and knowledge to discharge their practice effectively. If we wanted to change doctor's knowledge would we circumvent the seven years of formal medical education and then allow our ill-trained doctors to go out and practice for a decade and then after they have been acculturated and socialized into practicing their craft despite huge gaps in their knowledge would we then create a program to bring them back to college campuses for two weeks in the summer to have courses in basic anatomy?

At this the audience broke into nervous laughter.

And how do we generally measure the effect of the TAH programs on teachers? By having them take multiple choice questions found in an AP history exam. Wineburg was incredulous about this. "In other words, we are paying millions of federal dollars per fiscal year to assure that school teachers possess the level of factual knowledge that we expect of bright seventeen year olds." This statement prompted loud murmuring (whether in dissent or support was unclear).

It made sense, said Wineburg, that teachers should have this level of knowledge before they walk into a classroom, not after. And yet in state after state teachers aren't required to take high-level history courses before earning their education degrees. Using Michigan as an example, he observed that a teacher could meet state teaching requirements after having taken just two 100 level history courses "in which no history book is read other than the textbook."

Let me suggest that one million dollars--the cost of one single TAH project--be spent bringing this scandalous state of affairs to light, lobbying Michigan legislators, working with them to draft a law that would put teeth into an undergraduate coursework necessary to get a license to teach history. Let me suggest that this simple change might do more for the future of teachers and students of American history in the great state of Michigan than the tens of millions that that state has received in TAH grants.

Impact of TAH

In case his listeners worried that his words might be used to undermine the program he was quick to reiterate his support for it. "Thanks to Senator Byrd TAH is what we have and we should all pause and write him a thank you note right now," he implored.

But has the program raised student knowledge and grasp of history? We know, he said, that TAH has helped make teachers and historians happy and as a by-product that's a good thing. But all taxpayers will care about is whether students are benefiting. Unfortunately, for this there simply isn't much hard evidence, he averred.

In fact, there's little in the way of hard evidence of the program's impact at all. Initially, the impact was measured by the people who staged the events. He cited one reputable study by SRI International (2005) that found that of the 174 TAH projects conducted between 2001 and 2002, "91 percent used self-reports to assess project outcomes." As "we have learned from research by Garrison Keillor conducted in Minnesota's Lake Woebegon district [laughter]," Wineburg wryly observed, "self-reports are the assessment of choice in places where all students and all teachers are above average." (More laughter.)

Measuring Success

In 2003 the Department of Education made what should have been a substantial change in the way the program's effectiveness was measured. From then on, officials decreed, programs should be evaluated on the basis of objective criteria, namely, by the rigorous testing of students. The scores of those whose teachers went through the program would be compared against those who hadn't. Unfortunately, Wineburg reported, the testing process itself was far from scientific, studies later found. The students tested were not randomly chosen. Sample sizes were too small. Control groups weren't used. And teachers who picked up tips in the TAH workshops passed them along to teachers who hadn't attended, contaminating the results of the control group. When one school official was confronted about the issue of contaminated results he reportedly said, "Sad to say I wish the project had had a bigger contamination impact!" Wineburg got a loud laugh citing that crack.

But as he went on it was obvious that Wineburg was deeply distressed by what has been happening. A glimmer of his anger came through in a sarcastic remark about the test questions students were given. One question asked students to determine "Which battle brought the American Revolution to an end." Wineburg derisively commented:

May I respectively suggest to you today that if our goal is to raise American students' achievement in U.S. history as defined by their ability to name Yorktown and not Saratoga as the decisive battle of the American Revolution, then Stanley Kaplan or Princeton Review can do a much more cost-effective job than a series of three to five year American history grants that cost taxpayers millions of dollars. The same goes for multiple-choice tests that we are administering to teachers.

What teachers need to be taught is not facts, he added, but how to put facts into "productive classroom use." When teachers were asked factual questions, SRI found in its 2005 study, they had nearly perfect scores. But when they were asked to do history--drawing up lesson plans relating cause and effect and the significance of events--scores dropped by half. As an example, SRI reported:

[One flawed lesson plan used by teachers] explored the meeting of Native Americans and Europeans in the New World, attempting to gauge the impact of their interaction on the native peoples. Although the lesson was also factually correct, no sense of time was apparent from the material presented, no historical context was provided, the explanation of cause and effect was vague and simplistic, and no attempt was made to consider the topic from multiple perspectives—in this case, those of Native Americans and European settlers. It was also unclear where in the New World this contact occurred, when it took place, and which Native Americans and Europeans were interacting.

For all the money that has been spent on TAH, said Wineburg, only one study evaluating the program's effectiveness has been sufficiently rigorous to "pass the review process of a scholarly journal that is qualified to evaluate inferential statistics." The fault is not with the evaluators.

Our evaluators are good and honest people. But it is difficult to go from AIDS education on Mondays, teen pregnancy on Tuesdays, highway safety on Wednesdays, and the nature of historical understanding on Thursdays.

So where are we?

Summing up, said Wineburg, the core problem we face is that we have no idea after spending millions of dollars on the TAH program what works. The problem is larger than the TAH program. We apparently don't really understand what works in virtually any educational programs, Wineburg asserted. A 2007 study commissioned by the Department of Education looked at studies that purported to examine over the past several decades "how teacher professional development affects student achievement." The study found that only 9 out of 1,343 studies met rigorous professional standards of evaluation (these dealt with math and reading in the lower grades).

However much TAH has spent on professional development, it's been but a "drop in the bucket." In 2005 a study estimated that the United States yearly spends $20 billion on professional development. The $119 million being spent this year on the TAH program is therefore a "tiny fracture of that $20 billion."

Wineburg's Advice

How to fix TAH? Wineburg was full of recommendations about the "direction the program should take." First, however, he had some general advice. Most importantly, he indicated educators need to adopt a sober attitude. While the newest and latest suggestions may sound great and commonsensical, educators should wait before wholeheartedly embracing them until they have been proven workable in scientific studies. As an example, he cited peer coaching. Great idea? Sure appears to be. But wait for results first before implementing it nationwide. A cautious note is that three solid studies seem to indicate that what really seems to work well is when teachers are given constant support by outside professionals.

He also advised historians "to stop with the anti-textbook rhetoric." "As long as TAH is tied to textbook standards," he stated, "teachers are not going to burn their textbooks. " Rather, professors in summer workshops should do some homework ahead of time and study the textbooks the teachers are using so they can offer helpful suggestions as to "how new historical knowledge can open up the textbook."

He had five concrete recommendations:

1. "Set aside 20 percent of TAH fiscal year funds for competitive grants ... to independent researchers ... to assess and evaluate projects in experimental and quasi-experimental ways." This is needed because one of the gravest threats to the integrity of the evaluation process is the cozy relationship that often grows up between teachers and evaluators, he said.

2. "For every $20 million in awards, [we should] set aside $1 million for new research and the development and testing of new measures to assess historical understanding and knowledge."

3. "We need to stop testing teachers with multiple choice items."

4. While communities love to invite marquee historians to do their summer workshops these are often not the right historians to be involved in TAH. "We need to engage those historians who are working on the scholarship of teaching and learning ... those people who are trying to create college classrooms where our students are thinking and working beyond the use of historical facts. These are the historians we must keenly engage in our projects so we can begin to articulate the problems between elementary and secondary and tertiary education."

5. "I dare anyone in this audience to dispute the following claim: We will not change history teaching by continuing to ignore how new teachers are trained. It’s that simple. We need innovative approaches for combining the strengths of university history departments and schools of education to create the kinds of courses and practice teaching assignments that put new teachers into the classroom already possessing deep knowledge and appropriate skill. We need new ways of thinking about alternative certification for history teachers and ways to deliver teacher training on-line. By ignoring how we socialize new teachers into the profession, we delude ourselves. More than any other issue, this one is the elephant in the TAH living room."

Follow these five recommendations, Wineburg said, and he is confident that after we have spent another billion dollars on TAH grants we will know through rigorous testing precisely what kind of teacher development programs work and help students arrive at the "knowledge, understanding and appreciation of history."

Related Links

HNN Hot Topics: OAH Annual Convention Highlights, 2009