His audience knew in advance this would be no ordinary luncheon address for a convention of historians. They had to get no further than the title to know that much: "An Antidote for an Ailing Profession or a $836,000,000 Boondoggle: The Future of the TAH Program."

TAH--the Teaching American History program sponsored by the U.S. Department of Education--has been a fountain of beneficence at which historians have been drinking for years. And now at a history convention someone was dare raising the question that it might all have been a boondoggle? At the annual meeting of the Organization of American Historians (OAH)? That was like suggesting at a meeting of farmers who have been gorging themselves at the federal trough that government subsidies might be a bad idea. It wasn't exactly what you'd expect.

More surprising was the identity of the speaker. It was Sam Wineburg, an education professor who runs the Stanford History Education Group, co-sponsor of the National History Education Clearinghouse, a TAH funded program. No wonder Wineburg told us in advance he would be uncomfortable having his talk filmed by HNN. As both a TAH critic and collaborator he was like a person with split personality whose multiple identities are at war with one another--or at least in an uncomfortable state of coexistence. (Wineburg made it clear that he was speaking for himself and not for the Stanford History Education Group.)

It was, then, with rapt attention that people prepared to listen to his talk. It didn't disappoint. It was easily the most controversial lecture of the entire 2009 OAH convention--and one of the most riveting. Because of the importance of his talk and our keen desire to help readers evaluate the effectiveness of the TAH program (no easy task), HNN is providing a comprehensive blow-by-blow account. (Disclosure: George Mason University's Center for History and New Media is the other sponsor of the National History Education Clearinghouse.)

He began his talk by noting the unexpected windfall that was the TAH program. Back in 2000 when Sen. Robert Byrd (D-WVa.) slipped in an amendment to Title X providing for a $50 million appropriation expressly to promote the study of history (not social studies) no one in the profession had seen it coming. History teachers weren't much in favor on Capitol Hill. Not too long before a member of Congress had ridiculed a fifth grade curriculum funded by the federal government that supposedly promoted adultery and cannibalism. In the 1990s the effort to create national standards for the study of history had drawn fire from politicians from one end of the country to the other.

Naysayers expected another calamity, noted Wineburg. But instead of collapsing amidst controversy the TAH program quickly became an established institution. Within a few years the program's funding was more than doubled to about $120 million annually. Within a short time TAH succeeded in drawing together both academics and high school teachers. As one historian Wineburg quoted had remarked, "money has a way of thawing the most chilly relations." This line drew loud laughter from the crowd.

Thus far TAH has received an amount approaching a billion dollars: $838,172,000. As Wineburg said, "we're talking [real] money." TAH funds have been a boon to the history profession, providing steady jobs for both professors and their graduate students at institutions around the country, from Michigan State to Ohio State.

It's also been a boon to teachers, few of whom have much training in history. Wineburg cited a study from 1997 that found that 80 percent of middle and high school social studies teachers did not major or minor in history as undergraduates. Until TAH came along, the overwhelming majority had no opportunity to pick up skills in the teaching of history through professional development programs.

Wineburg believes the TAH grants should continue, but he told the crowd that he disagreed with those who believe the grants have made a difference. Seated in front of him was Arnita Jones, executive director of the American Historical Association, who recently wrote that they had. When Wineburg later took questions she was one of the first to take issue with him.

What does it mean to "make a difference"?

It was at this point in his talk that Wineburg began raising fundamental doubts about the program, which provides for teacher development. While he agreed with the program's stated goal--the wholesale transformation of the teaching of history--he said that the efforts being made are wholly inadequate to achieving it:

Imagine that physicians lacked the skill and knowledge to discharge their practice effectively. If we wanted to change doctor's knowledge would we circumvent the seven years of formal medical education and then allow our ill-trained doctors to go out and practice for a decade and then after they have been acculturated and socialized into practicing their craft despite huge gaps in their knowledge would we then create a program to bring them back to college campuses for two weeks in the summer to have courses in basic anatomy?

At this the audience broke into nervous laughter.

And how do we generally measure the effect of the TAH programs on teachers? By having them take multiple choice questions found in an AP history exam. Wineburg was incredulous about this. "In other words, we are paying millions of federal dollars per fiscal year to assure that school teachers possess the level of factual knowledge that we expect of bright seventeen year olds." This statement prompted loud murmuring (whether in dissent or support was unclear).

It made sense, said Wineburg, that teachers should have this level of knowledge before they walk into a classroom, not after. And yet in state after state teachers aren't required to take high-level history courses before earning their education degrees. Using Michigan as an example, he observed that a teacher could meet state teaching requirements after having taken just two 100 level history courses "in which no history book is read other than the textbook."

Let me suggest that one million dollars--the cost of one single TAH project--be spent bringing this scandalous state of affairs to light, lobbying Michigan legislators, working with them to draft a law that would put teeth into an undergraduate coursework necessary to get a license to teach history. Let me suggest that this simple change might do more for the future of teachers and students of American history in the great state of Michigan than the tens of millions that that state has received in TAH grants.

Impact of TAH

In case his listeners worried that his words might be used to undermine the program he was quick to reiterate his support for it. "Thanks to Senator Byrd TAH is what we have and we should all pause and write him a thank you note right now," he implored.

But has the program raised student knowledge and grasp of history? We know, he said, that TAH has helped make teachers and historians happy and as a by-product that's a good thing. But all taxpayers will care about is whether students are benefiting. Unfortunately, for this there simply isn't much hard evidence, he averred.

In fact, there's little in the way of hard evidence of the program's impact at all. Initially, the impact was measured by the people who staged the events. He cited one reputable study by SRI International (2005) that found that of the 174 TAH projects conducted between 2001 and 2002, "91 percent used self-reports to assess project outcomes." As "we have learned from research by Garrison Keillor conducted in Minnesota's Lake Woebegon district [laughter]," Wineburg wryly observed, "self-reports are the assessment of choice in places where all students and all teachers are above average." (More laughter.)

Measuring Success

In 2003 the Department of Education made what should have been a substantial change in the way the program's effectiveness was measured. From then on, officials decreed, programs should be evaluated on the basis of objective criteria, namely, by the rigorous testing of students. The scores of those whose teachers went through the program would be compared against those who hadn't. Unfortunately, Wineburg reported, the testing process itself was far from scientific, studies later found. The students tested were not randomly chosen. Sample sizes were too small. Control groups weren't used. And teachers who picked up tips in the TAH workshops passed them along to teachers who hadn't attended, contaminating the results of the control group. When one school official was confronted about the issue of contaminated results he reportedly said, "Sad to say I wish the project had had a bigger contamination impact!" Wineburg got a loud laugh citing that crack.

But as he went on it was obvious that Wineburg was deeply distressed by what has been happening. A glimmer of his anger came through in a sarcastic remark about the test questions students were given. One question asked students to determine "Which battle brought the American Revolution to an end." Wineburg derisively commented:

May I respectively suggest to you today that if our goal is to raise American students' achievement in U.S. history as defined by their ability to name Yorktown and not Saratoga as the decisive battle of the American Revolution, then Stanley Kaplan or Princeton Review can do a much more cost-effective job than a series of three to five year American history grants that cost taxpayers millions of dollars. The same goes for multiple-choice tests that we are administering to teachers.

What teachers need to be taught is not facts, he added, but how to put facts into "productive classroom use." When teachers were asked factual questions, SRI found in its 2005 study, they had nearly perfect scores. But when they were asked to do history--drawing up lesson plans relating cause and effect and the significance of events--scores dropped by half. As an example, SRI reported:

[One flawed lesson plan used by teachers] explored the meeting of Native Americans and Europeans in the New World, attempting to gauge the impact of their interaction on the native peoples. Although the lesson was also factually correct, no sense of time was apparent from the material presented, no historical context was provided, the explanation of cause and effect was vague and simplistic, and no attempt was made to consider the topic from multiple perspectives—in this case, those of Native Americans and European settlers. It was also unclear where in the New World this contact occurred, when it took place, and which Native Americans and Europeans were interacting.

For all the money that has been spent on TAH, said Wineburg, only one study evaluating the program's effectiveness has been sufficiently rigorous to "pass the review process of a scholarly journal that is qualified to evaluate inferential statistics." The fault is not with the evaluators.

Our evaluators are good and honest people. But it is difficult to go from AIDS education on Mondays, teen pregnancy on Tuesdays, highway safety on Wednesdays, and the nature of historical understanding on Thursdays.

So where are we?

Summing up, said Wineburg, the core problem we face is that we have no idea after spending millions of dollars on the TAH program what works. The problem is larger than the TAH program. We apparently don't really understand what works in virtually any educational programs, Wineburg asserted. A 2007 study commissioned by the Department of Education looked at studies that purported to examine over the past several decades "how teacher professional development affects student achievement." The study found that only 9 out of 1,343 studies met rigorous professional standards of evaluation (these dealt with math and reading in the lower grades).

However much TAH has spent on professional development, it's been but a "drop in the bucket." In 2005 a study estimated that the United States yearly spends $20 billion on professional development. The $119 million being spent this year on the TAH program is therefore a "tiny fracture of that $20 billion."

Wineburg's Advice

How to fix TAH? Wineburg was full of recommendations about the "direction the program should take." First, however, he had some general advice. Most importantly, he indicated educators need to adopt a sober attitude. While the newest and latest suggestions may sound great and commonsensical, educators should wait before wholeheartedly embracing them until they have been proven workable in scientific studies. As an example, he cited peer coaching. Great idea? Sure appears to be. But wait for results first before implementing it nationwide. A cautious note is that three solid studies seem to indicate that what really seems to work well is when teachers are given constant support by outside professionals.

He also advised historians "to stop with the anti-textbook rhetoric." "As long as TAH is tied to textbook standards," he stated, "teachers are not going to burn their textbooks. " Rather, professors in summer workshops should do some homework ahead of time and study the textbooks the teachers are using so they can offer helpful suggestions as to "how new historical knowledge can open up the textbook."

He had five concrete recommendations:

1. "Set aside 20 percent of TAH fiscal year funds for competitive grants ... to independent researchers ... to assess and evaluate projects in experimental and quasi-experimental ways." This is needed because one of the gravest threats to the integrity of the evaluation process is the cozy relationship that often grows up between teachers and evaluators, he said.

2. "For every $20 million in awards, [we should] set aside $1 million for new research and the development and testing of new measures to assess historical understanding and knowledge."

3. "We need to stop testing teachers with multiple choice items."

4. While communities love to invite marquee historians to do their summer workshops these are often not the right historians to be involved in TAH. "We need to engage those historians who are working on the scholarship of teaching and learning ... those people who are trying to create college classrooms where our students are thinking and working beyond the use of historical facts. These are the historians we must keenly engage in our projects so we can begin to articulate the problems between elementary and secondary and tertiary education."

5. "I dare anyone in this audience to dispute the following claim: We will not change history teaching by continuing to ignore how new teachers are trained. It’s that simple. We need innovative approaches for combining the strengths of university history departments and schools of education to create the kinds of courses and practice teaching assignments that put new teachers into the classroom already possessing deep knowledge and appropriate skill. We need new ways of thinking about alternative certification for history teachers and ways to deliver teacher training on-line. By ignoring how we socialize new teachers into the profession, we delude ourselves. More than any other issue, this one is the elephant in the TAH living room."

Follow these five recommendations, Wineburg said, and he is confident that after we have spent another billion dollars on TAH grants we will know through rigorous testing precisely what kind of teacher development programs work and help students arrive at the "knowledge, understanding and appreciation of history."

Related Links

HNN Hot Topics: OAH Annual Convention Highlights, 2009

Elise Fillpot - 4/25/2009

Fact: My name is spelled "Fillpot"

Lawrence Brooks Hughes - 4/24/2009

You are getting dangerously close to reality in your final paragraph, I hope. When you check the views of Elise Philpot just above yours, you can see she favors sending out coercive utopians from Washington...

Those teachers who were given the AP tests with multiple choice answers are said to have had "nearly perfect scores" when it came to knowing the facts. (A link to that said they had "a firm grasp of historical facts.") This Doubting Thomas believes that at least some of them, and especially perhaps many others teaching history who did not come forward to take these tests, would have failed on the facts quite abysmally, and should be banished from the classroom. Whether anybody should be teaching theoretical and analytical history is debatable, but nobody should be doing so who cannot command the jumping off facts. Alas, nobody can uproot many thousands of poor history teachers from their existing jobs everywhere in this country today, and until someone comes to grips with that problem no improvements will be seen.

Dennis R. Lubeck - 4/23/2009

Let’s calm down and take a more careful look at Sam Wineburg’s important talk at OAH.
If Sam was a little provocative in making his points, the arguments should still set the agenda for future meetings. In my view, these meetings should include program directors, historians, teacher educators responsible for preparing future history teachers, program evaluators and program officers from the U.S. Department of Education.

Sam is arguing for continued support of TAH, not the culmination of these important opportunities for teachers to improve their content and pedagogical skills. He is not saying that TAH activities for teachers are worthless or insignificant, but he is claiming that we don’t have hard data to document the impact of the grants on student understanding of history. Let’s use what we have learned, increase expectations for teachers and students and move forward.

****This is the best way to insure future funding. This is the heart and soul of Sam’s talk.

Sam also argued for the participation of historians who are interested in the teaching and learning of history and willing to go beyond the formal presentations at TAH workshops, seminars and institutes. This is not an argument against having a well-known historian’s participation. However, keep in mind that the historian will be flying back to his or her university and, with a few exceptions, will be unavailable for future consultation with teachers. To the extent possible, let’s take advantage of academics at nearby universities, especially faculty who are interested in developing relationships with teachers.

And projects need to involve teacher educators responsible for preparing future history teachers, especially teacher educators willing to work with project directors on the teaching and learning of history. Let’s stop the battles between Arts and Science and Education. I have heard a few historians involved in leadership roles in TAH grants say that they want nothing to do with anybody associated with the School of Education at their university.

*****Clearly Sam believes that closing this division is the best way to insure continued funding and improve teacher and student learning of history.

I didn’t read that Sam was arguing for the end of program evaluation, including surveys of teachers that look at the satisfaction of teachers, evaluations of units of study, classroom visits, focus groups, staff implementation of the goals of the grants. However, he is arguing for the development of new instruments that raise the bar to focus on historical thinking skills (cause and effect, evaluation of evidence, time and place, etc).

In the previous context, Sam was certainly not presenting history as a social science. Sam has argued in many articles that history belongs to the humanities because of history’s unique way of knowing to enrich human experience. Sam would agree that the enrichment of human experience is not always measurable, especially in the short term. Nonetheless, we can still determine, as Laura Westhoff argues in “Lost in Translation,” that teachers don’t always use primary sources to teach students how to think historically. How do we develop instruments to document teaching practice and student understanding so that teachers improve their craft to increase student achievement in American history. This is one of the challenges Sam lays out for us.

Westhoff is project historian for the grant I direct and it was humbling for project staff to discover that our assumptions that teachers were translating historical content and teaching ideas learned at workshops, colloquiums, etc. into lessons that taught historical thinking skills to their students may have been mistaken in far too many cases. In some of these cases, we were told students were more interested in the lesson than in previous years, even if the lesson did not exactly teach what most of us think of as historical thinking. How do we provide more assistance to the teachers who are working hard to teach history as historians understand their craft? This is another challenge Sam lays out for us.

While I agree with Sam and the authors of several letters that teachers need to take more history classes, read more history, attend more history workshops and institutes, their preparation would even be better, as Sam argued in his talk, if professors read a secondary textbook or two and made use of these texts to show teachers how to use the text as one source among many to answer a particular historical question.

Changing requirements for history teachers may be desirable but unlikely to happen anytime soon. Therefore these grants are essential and are providing most of the professional development for American history teachers across the country.

What might help determine future policy is closer collaboration between the Deans of Arts and Sciences and Education. Imagine if the Deans decided that they were going to work together to improve the preparation of history teachers, as well as teachers of other disciplines. Professors are anxious to please their deans, so I am told. I am employed by an educational service agency and sit on the border between secondary schools and universities. This presents me with interesting lens into the lack of collaboration among the important players responsible for history teaching.

Let’s don’t get into an argument at this stage about where the money is coming from to support the development of instruments to assess historical thinking. The best way to reinforce the view that TAH grants are “boondoggles” is to get into a food fight over the importance of the many players in these grants, including academic historians, evaluators, methods teachers, the program staffs for the grants (including the author of this commentary), and something I have not mentioned, the many contractors who have provided important services to LEA’s, including three grants I have been privileged to direct. Above all, the most important players are the teachers and their students. This is another of Wineburg’s essential arguments.

In the final analysis, Congressional support for TAH grants will be determined by our collective capacity to persuade our funders (the American taxpayer) that these grants are about students, not providers and consultants.

Dennis Lubeck, Ph.D.
Director, Perspectives on Democracy (St. Louis)

Elise Fillpot - 4/23/2009

A few thoughts in response to Wineburg's recommendations...

1. "Set aside 20 percent of TAH fiscal year funds for competitive grants ... to independent researchers ... to assess and evaluate projects in experimental and quasi-experimental ways." This is needed because one of the gravest threats to the integrity of the evaluation process is the cozy relationship that often grows up between teachers and evaluators, he said.

Reaction: Set aside the 20%, but use it to establish an experimental design at the federal level. This would require several groups of uniform projects. All classrooms in each group of grants would be pooled for random assignment to treatment and control groups. The individual grant evaluators would provide formative assessment information as they seek to document the connections between professional development and classroom practice, so that student learning outcomes, if they are positively correlated with treatment, could be aligned with PD. This approach removes the local administrator from playing the bad guy in random assignment, and would begin to address problems with validity and generalizability that arise from experimental studies with inadequate participant numbers.
Such a plan would require the federal officers to identify a handful of projects that have provided evidence that they have at least positively impacted teachers' attitudes and changed classroom practice. These projects would become the template for other grants that would participate in the groups of experimental design evaluations. The replication of those project PD models and history topics could then be offered as a priority preference on the TAH rfp.
All of that said, I have to hedge with a caution against drawing hard and fast conclusions about any education outcomes based on one or two measures. Any classroom is, by nature, rife with human variables that can’t be adequately qualified, much less quantified. Which is why investigating what happens in classes using as many research methods as possible is essential to understand not just if something works or fails, but the intricacies of why.
As far as the dangers of a “cozy relationship between teachers and evaluators,” I could make the case that Wineburg’s list of recommendations might not need to include #3 if evaluators always worked closely with project administrators to design instruments aligned with project goals, i.e if developing in teachers an epistemological paradigm of history is a project goal, an evaluation team working closely with grant designers would never assess the success of that goal with a multiple choice test. And that is just one example of the importance of the importance of a solid working relationship between grant director and evaluator. If, on the other hand, the definition of “cozy” implies that some evaluators are skewing outcomes in order to secure ongoing financial security, that’s a different issue.

2. "For every $20 million in awards, [we should] set aside $1 million for new research and the development and testing of new measures to assess historical understanding and knowledge."
Reaction: Yes. And some of that $$ could be used to hire referees to maintain a degree of civility in the ongoing brawl over what historical understandings and knowledge should be assessed in the first place.
3. "We need to stop testing teachers with multiple choice items." Yes.
4. While communities love to invite marquee historians to do their summer workshops these are often not the right historians to be involved in TAH. "We need to engage those historians who are working on the scholarship of teaching and learning ... those people who are trying to create college classrooms where our students are thinking and working beyond the use of historical facts. These are the historians we must keenly engage in our projects so we can begin to articulate the problems between elementary and secondary and tertiary education."
5. "I dare anyone in this audience to dispute the following claim: We will not change history teaching by continuing to ignore how new teachers are trained. It’s that simple. We need innovative approaches for combining the strengths of university history departments and schools of education to create the kinds of courses and practice teaching assignments that put new teachers into the classroom already possessing deep knowledge and appropriate skill. We need new ways of thinking about alternative certification for history teachers and ways to deliver teacher training on-line. By ignoring how we socialize new teachers into the profession, we delude ourselves. More than any other issue, this one is the elephant in the TAH living room."
Reaction: Numbers 4-5 are closely related. When the history profession decides to share its skill set with all students of history instead of just the select few that end up in graduate programs, we’ll begin to make progress on both of these points; # 4 will become moot and #5 will begin to resolve via modeling. Come to think of it, when that day comes, I suspect we’ll see progress on every front in the struggle to truly engage students in history as an interpretive, evidence-based discipline.

Dana L. Carmichael - 4/23/2009

All of the comments posted are interesting, including the keynote's summary; but as a former TAHG Project Director in Minnesota and current TAHG evaluator, I believe one enormous problem went unaddressed:

Historians (whether local or national) providing "content" sessions rarely take the time to find out what teachers will be learning in the "pedagogy" sessions. This silo experience leaves teachers to integrate the content and pedagogy on their own, creating fragmented learning.

The best grants I've seen are the exception, where the historians model historical thinking by sharing HOW they analyze a primary source or HOW they considered multiple perspectives. Likewise the pedagogy specialists use what the historians bring to the workshops rather than sticking to an isolated teaching agenda.

These kinds of collaboration allow teachers to learn more and bring more in-depth historical learning to their students.

K Woestman - 4/22/2009

A clarification - no matter what value we as historians know it has, the historical method is not recognized by either the OMB or Congress as a was to effectively evaluate the spending of federal dollars. So, arguing here about history versus social science is a moot point here.

We simply have to have data to prove we are effective. Sam is trying to help us get there.

Echoing what Maris Vinovskis stated at the 1st TAH Symposium keynote - we as historians have to step up to the plate to protect our discipline. With the current age of accountability, we can no longer retreat to our corners with our best methods. We have to be part of the wider world that can provide data, data, data to the wider world. Sam is actually one of the few people who understands both worlds having been a history teacher himself prior to becoming a cognitive psychologist. He did a great deal to protect history from being lost in the social studies (re: his taking the NCSS standards head-on) and now he's trying to help us here.

Re: Context: There were many of these same discussions among the 75+ participants at the symposium as well as over the course of the past three years.

The bottom line, if we as historians don't participate in ways of choosing and providing effective data, others will do it for us to our own detriment.

Most importantly, we need the history scholars who normally do not have to deal with teacher education (both pre-service and in-service) to join us in the battle if anyone is ever going to listen to us. This is true at the state level where standards are determined and NCLB is "translated" as well as the national level.

Carl R. Weinberg - 4/22/2009

I recall that in your response, you said, somewhat sarcastically, something to the effect of, “what you call anecdotes, we historians call . . . primary sources.” I agree that there can be more than one way of approaching historical truth, but I hope we historians who do primarily “qualitative” research can agree that when we evaluate evidence, we should strive to base conclusions on the broadest sample of sources; and that reaching conclusions based on a single anecdotal document can be truly problematic. Surely, when we evaluate each others’ scholarly work, we reasonably want to know how representative the sources are and what selection biases may come into play.

For instance, we wouldn’t base a conclusion about antebellum slavery on a single WPA slave narrative. And even if we used hundreds, we would want to recognize that the subjects who narrated them are disproportionately from the upper south , disproportionately healthy and, as slaves, young (both by virtue of still being alive in the 1930s). That is to say, in a very broad sense, history must aim at being a scientific discipline. Otherwise, we are just telling stories. In that spirit, I think we should not be put off by a social scientific evaluation of TAH; and we should consider seriously what Sam Wineburg has to say.

Lendol Calder - 4/22/2009

Sure, social science methods have their limits. But on this basis to rule them out as a source of evidence is just (readers fill in the blank). Moreover, isn't there a difference between creating historical knowledge (one might have a dim eye toward social science methods here) and studying how people learn (here, one might see how social science could have more usefulness). Wineburg doesn't claim to be a historian. As a psychologist, he studies how people attain historical mindedness. To categorically rule out his kind of knowing reminds me of a saying: "Teaching is incredibly difficult, messy, indeterminate work. Teachers deal with this by erecting defenses which prevent them from confronting teaching."

Lendol Calder - 4/22/2009

I wasn't there to hear this exchange, but that doesn't sound like the Sam Wineburg I know--you've read his book, most of the chapters are based on methods ethnographers and humanistic historians would warm to if not claim as their own. Using your own method, what happens when you put what you heard Sam say in the talk in the context of his other written stuff?

Michael Green - 4/22/2009

Whenever someone mentions social science theory in connection with historical knowledge, I know that that person is either openly claiming not to be a historian or masquerading as one.

Larry Cebula - 4/21/2009

What Wineberg basically said is that rigorous social scientific method is the only way of knowing the truth, and that the TAH program was failing this measure.

I challenged him on this, pointing out that as historians we have a very different method of learning the truth. We investigate a range of primary sources, develop a thesis, compare it to the evidence, and make conclusions. Journalists also work this way. And evaluation can work this way--like journalism or ethnography.

In response he said he had social science theory that proved that social science theory is correct.

Bah.

Lendol Calder - 4/20/2009

Many at OAH were talking about this address, but until now I didn't have a good idea of what Sam said, so thanks to Rick Shenkman for the in-depth summary. I am one of the worst about ragging on textbooks so will take this part of the talk to heart when doing future work with teachers; Sam is right, most secondary teachers don't have the luxury of a college professor when it comes to selecting their texts. I would add a sixth idea for better uses of a small stream of TAH funds: fund a team of SoTL historians to take another crack at recommended standards for historical literacy. My colleagues and I are "lucky" to be in a state (IL) that holds future history teachers to standards but we have two questions about our state standards: 1) Who the heck came up with these items?!? and 2) How come we professionals do so terrible on the practice tests they let us see? I guess we just don't know our history.

Michael Green - 4/20/2009

I have been involved in running two TAH grants in Clark County, Nevada, and had the pleasure of participating in another in California. I do not dispute Professor Wineburg's recommendations, but I have a problem with his approach. He brings to mind the old joke about diplomacy that went something like this: a guy in the State Department says the solution to the Middle East is to get all Jews to convert or all Arabs to convert. He's asked how to do this. He says, "I'm just in charge of policy guidance."

Well, I wish history was a requirement everywhere in colleges. But it isn't. So the TAH problem helps teachers learn more than they otherwise might have. In essence, those of us who help teach teachers are providing them with information and approaches they might not get any other way. Do they use everything? I don't think all knowledge is there to be disseminated. Some of it might just be helpful for background.

Oscar Chamberlain - 4/20/2009

I missed Sam Wineburg's speech. It sounds stimulating, to say the least.

I've been directing a TAH grant the last few years. I think we have evaded some of the "problems" mentioned here. (Perhaps I am deluded by the Lake Wobegon syndrome--I hope not.) Still, his point about the problems in assessing the grant's impact ring true.

Our primary assessment instrument targeted the classes of the summer institute teachers. We've had control groups in our assessment. They do help, and our assessor has done a first rate job. So far, the results do show a statistically significant impact in a majority of the classrooms.

How important is that statistically significant impact? That's a harder question to answer.

I more and more believe that the a year-one test of classes (or of teachers for that matter)--no matter how good the assessment instrument--does not constitute a good measure of the long term impact. That impact does depend in large part on the historical skills imparted, on the willingness of the teachers to follow up by contacting the resources that we have introduced them to, and, in general, on their willingness to continue to try to think like historians within the press of their daily lives.

The grant has no mechanism for long term testing and evaluation. And before one blames the TAH people for this, the legislation makes no provision for this. The money spent on a grant must be spent in entirety within the grant period. Also Congress wants to see an immediate impact. Something separate would be needed to look five or then years down the road.

K Woestman - 4/19/2009

This is great coverage of Sam's keynote address at the 4th Annual TAH Symposium that is co-sponsored by the OAH and H-Net Humanities and Social Sciences Online through its H-TAH discussion list open to everyone interested in TAH grants [http://www.h-net.org/~tah]