This State Pays a Company $2.3 Million to Rank Teachers by an Algorithmic Formula?
It was only a matter of time. For years, we have classified our sons and daughters through standardized testing and now we are labeling our teachers by linking them to those very same results. Branding teachers with the composite results of high stakes testing is nothing new. Various states have used such measures to bludgeon the teachers and determine pay rates, bonuses, and job retention for more than a decade. Now, however, it is all the rage and spreading like an enormous stain; particularly so, since it has been sanctioned as a teacher evaluation tool by the Federal Government through their Race to the Top initiative. The specific weapon adopted for determining the very futures of our educators is called the value added model or VAM.
The value added model in education has been used by a few states in one way or another since at least 1990. It is a mathematical formula that claims to reveal just how much a particular teacher has added to or detracted from his/her students’ standardized test scores. Each teacher will be awarded a composite score and that score will place the teacher into one of five levels ranging from “least effective” to “most effective.”
In my state, Ohio, we will utilize the five level model, then combine the value added results in a 50/50 mix with administrative observations, thereby distilling it all into a four category range. The categories are: 1. Ineffective, 2. Developing, 3. Skilled, and 4. Accomplished.
Every teacher in my state, and the teachers in nearly all of the other states, will have their lives and their educational futures determined by the terrible arithmetic of the value added model either through the Common Core tests or through tests purchased from approved vendors. What exactly is the mathematical formula that will determine the fate of our teachers and our educational systems? Strangely enough, only the creators know the mysterious mix; and they refuse to reveal it.
The dominant corporation in the field of value added is SAS, a North Carolina company. Their Value Added Assessment and Research Manager is Dr. William Sanders who is also the primary designer of their model. While working at the University of Tennessee, his remarkable research into agricultural genetics and animal breeding inspired the very model now in use for teacher evaluation. The resultant SAS formula boasts a proprietary blend of numbers and probabilities. Since it is a closely guarded intellectual property, it becomes the classic enigma wrapped up in taxpayer dollars. As a result, we are urged to take its validity and usefulness as an article of faith. SAS and their ilk have, in fact, become the frackers of the educational world. They propose to drill into our educational foundations, inject their algorithmic chemicals into our students and instructors, and just like the frackers of the oil and gas world, demand that we trust them to magically get it right.
Strangely enough, Ohio is not worried about this situation. Indeed, no one at the Ohio Department of Education has embraced even the pretense of understanding the value added model it adopted. Quite to the contrary, they admitted to never having seen the complete model, let alone analyzing it. They have told us that it does not matter, for they do not need to understand it. In their own words, they have chosen to “rely upon the expertise of people who have been involved in the field.” Those are remarkable words and admissions and they are completely consistent with an educational bureaucracy sporting the backbone of an éclair.
In terms of dollars and cents, trust comes at a very high price. Ohio will pay SAS, Inc. an annual fee of 2.3 million dollars to calculate value added scores. I found very similar fees in the other states making use of their proprietary expertise.
Should we be afraid of this mystical undertaking? Of course not, instead, we should be terrified. Not only are we stumbling into the dark, unseen room and facing all the horror that implies, but the research into the effectiveness of the model shows it to be as educationally decrepit as the high stakes testing upon which it is based.
Research on VAM and its efficacy in teacher evaluation is plentiful; and it does not bode well for anyone. Dr. Linda Darling-Hammond, initially a supporter of the value added concept, and Dr. Edward Haertel headed up a 2012 study on VAM at Stanford University. They were stunned by the flaws they uncovered. They found wild swings in year to year teacher ratings, dramatic shifts in ratings based solely upon the students assigned to a given teacher, and no control over the life forces outside the classroom that impact a student’s life and hence his or her test scores. Their conclusion: “value added scores should not be used in high stakes evaluations of teachers.” That statement mirrored the findings of the National Research Council, which dovetail with the research done by the American Statistical Association. In a 2014 report, ASA found that “teachers account for about 1% to 14% of the variability in test scores” and that “the majority of variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum and unmeasured influences.” Dr. Stephen J Caldas, a statistician with vast experience in the value added model, gets directly to the point. He deems VAM as “psychometrically indefensible.” Then goes on to say, “A grave injustice is being foisted from the top down on educators who are caught up in the most recent crush of reform initiatives.”
The lack of controls over socioeconomics and life events are among the most disturbing aspects of VAM. Life hardly exists in a vacuum and the impact of life events on students are never left outside the classroom door on the day of a high stakes test. I doubt that even Francis Galton could concoct an algorithm that could control for all of life’s factors and therefore make any test a true measure of an instructor’s contribution. Teachers have a tremendous impact on the lives of their students, but the value added is impossible to measure through statistical analysis. What formula, for example, can we create to measure inspiration and how will we then assign a score for it to a particular instructor? Even more to the point, when it boils down to a single or even to multiple high stakes tests, the individual teacher’s impact becomes, at best, a chip in a much wider and complex mosaic.
The challenges of socioeconomic conditions and other life events seem lost upon Dr. William Sanders, the man behind the SAS VAM. In an interview with the Raleigh Public Record in 2011 he completely downplayed life events as a significant factor and defended his system as an excellent evaluative tool. Others are not so sure. Stanford University’s Dr. Linda Darling-Hammond stated: “Value added ratings cannot disentangle the many home, school, and student factors that influence learning gains. These matter more than the individual teacher in explaining changes in scores.” Dr. Stephen J. Caldas put it more bluntly: “The error in these models can be huge, which invalidates their use for making accurate individual-level predictions.” He then declared that “using value added models to rate teachers and principals is folly.” Oh ye of little faith.
I left teaching and I am so glad I left when I did. I went through nearly twenty years of proficiency testing. That rubbish was bad enough, for it forced me to lower my standards in order to please the data driven crowd. Had I been forced to dive down any further, I would have had to reach up to touch bottom. But, throughout all that, at least my job and my value were never linked to those high stakes frauds. Had I stayed on, my strategy would have changed even more radically in order to secure my job and establish my value.
People who have everything to lose, generally speaking lose everything, whether it is, in this case, their careers or their sense of honor. We saw horrible cheating scandals develop in state after state where teacher jobs and proficiency testing were linked. To save their jobs or obtain bonuses, many teachers and administrators sacrificed their honor. Many are facing prison time for their deeds. The stakes now are just as critical and much more widespread.
I could never compromise my honor and break the law in order to establish my data driven value as an educator. But there is another way, a completely insidious and legal way, for me to save myself and obtain the accolades. If I was back in the classroom, I know exactly how I would approach the value added canard. Since everything would be on the line, nothing would be left to chance.
Back in the day, I taught history and government. As of now, Common Core standards have not yet reached those fields. Instead, I would be subjected to approved vendor testing and I would have to make the horrifying adjustments required. When I was teaching, I created everything for my classes---learning packets, homework assignments, slide shows, power points, projects, and tests. In addition, I brought in rare artifacts from the periods of history under study. On my unit tests, I eschewed the multiple guess style. My tests were always fill in, short answer essay, and essay. My students had to master the material, apply the material, and analyze it. All of the things I used to do would now have to be discarded. Times have changed for the far worse and my classes would have to follow that odious trend. Given the penalties, it could be no other way.
I would teach my students every trick imaginable for gaming a multiple guess test. Should there be any essay style questions, I would teach them formulaic writing---the kind that pleases the cursory glance analysis standardized test graders utilize. Above all, I would drill the information into them day in and day out until they could spit the answers back to me with the staccato bursts of a machine gun. They would live only in a world of high stakes testing. At night, in the deep recesses of their sleep, they would dream of bubble sheets. My classroom would make a Skinnerian Box look like anarchy.
As to my fellow teachers, they would no longer be my colleagues. They would be my competitors. I would share nothing with them and I would accept nothing from them; for now it is survival of the fittest. It would all pay off on the day of the test. When the results are revealed and the evaluations done, I would emerge as a FOUR---an educational god with all the rights and privileges pertaining thereto. Hand me my job security and my bonus money. As to my competitors, I would wave goodbye to them as they trudge off to the unemployment lines. The Social Darwinists would be so proud.
Everything done in the above scenario to my colleagues, and most importantly, to my students was entirely legal. Indeed, it is demanded by our value added widget world. Did I cheat my students? Absolutely. Did I violate every value that led me into education? Of course. But, I saved myself; or at least I saved whatever was left of me. In the meantime, I wonder just how long I would be able to sustain this academic charade until conscience would force me to quit. We are creating a system not of value but one that turns our children into robots and our teachers into puppet masters with value added accelerating the pace.
How then should we evaluate teachers? The answer lies within the schools themselves and costs us no additional funding. We need a strict, objective system of observational evaluation. I never feared administrative presence for we had clearly established and well understood guidelines. Second, we should have a well ordered peer review system; one which allows master educators to not only evaluate but also to assist their colleagues. Last, teachers need to keep copies of their students’ work as demonstrations of the challenges offered and the growth attained. For many years, with their permission, I kept copies of my students’ projects and, even to this day, I still go back through them----box load after box load. I am constantly amazed at the brilliance they demonstrated on their position papers, historical fiction stories, and their magazine projects. By utilizing this three tiered approach, we can weed out the bad, preserve the great and good, and help those who can be helped.
But, let us not stop there; I believe the teachers should review the administrators as well. Too often I watched sublime goons going through the motions of leadership with no one permitted to challenge them. No more should such people be allowed to haunt our school buildings.
As to evaluating our students; set the standards high and then unleash the teachers to create, instruct, and evaluate. We have known for decades that teacher issued grades through the school years are better predictors of college success, for example, than any standardized tests; including the ACT and SAT.
Mark Twain supposedly said, “Sometimes I wonder whether the world is being run by smart people who are just putting us on or by imbeciles who really mean it.” Whether the value added advocates are smart people or imbeciles is unknown to me. What is known to me is that value added has no value. Through it and through standardized testing we have become the architects of an educational system of breathtaking mediocrity. One more thing is abundantly clear; no student and no teacher should ever accept a ride from the “Value Added Valkyries.”