Norming the grades
I'm home from England (thanks, Ralph, for the link), where I spent some time talking with my brother (a lecturer in English Lit at Exeter) about the differences between American and English universities.
One difference that struck me as remarkable was the lack of autonomy my brother and his colleagues have over grading. In order to make sure that there is a uniform (or near uniform) standard of grading, my brother and his colleagues read samples of each other's students' papers. The objective is to ensure that all professors are giving similar marks for the same sort of work; in other words, it is expected that there will be widespread agreement about what constitutes a "first" or a "second" (in this country, an A or a B are the rough equivalents.)
When I was a teaching assistant at UCLA in the early 90s, I had one professor who asked her TAs to swap graded exams in the hopes of achieving a normative standard of what constituted an A. I can remember this leading to some real arguments! Most of the disagreements were over slight differences (such as a B- or a C+); in a few instances, however, we had TAs who disagreed as to the merit of a given paper by well over a whole grade's difference.
In my eleven years of community college teaching, I've never reviewed a colleague's grading, or had my own examined. I admitI would bristle if I were asked to have my grading supervised! I'm curious to know if any of my American colleagues, particularly those who have been teaching for a while, "norm" their grades with other faculty in their same discipline. If so, is this voluntary, or, as at Exeter, mandated by the institution? Does this sound like a useful practice?
One difference that struck me as remarkable was the lack of autonomy my brother and his colleagues have over grading. In order to make sure that there is a uniform (or near uniform) standard of grading, my brother and his colleagues read samples of each other's students' papers. The objective is to ensure that all professors are giving similar marks for the same sort of work; in other words, it is expected that there will be widespread agreement about what constitutes a "first" or a "second" (in this country, an A or a B are the rough equivalents.)
When I was a teaching assistant at UCLA in the early 90s, I had one professor who asked her TAs to swap graded exams in the hopes of achieving a normative standard of what constituted an A. I can remember this leading to some real arguments! Most of the disagreements were over slight differences (such as a B- or a C+); in a few instances, however, we had TAs who disagreed as to the merit of a given paper by well over a whole grade's difference.
In my eleven years of community college teaching, I've never reviewed a colleague's grading, or had my own examined. I admitI would bristle if I were asked to have my grading supervised! I'm curious to know if any of my American colleagues, particularly those who have been teaching for a while, "norm" their grades with other faculty in their same discipline. If so, is this voluntary, or, as at Exeter, mandated by the institution? Does this sound like a useful practice?
Tooting my own horn....
I did norming exercises with fellow graduate students both as TAs and in writing/teaching seminars, and always found the exercises useful. Our perspectives were usually pretty close, as long as our disciplines were pretty close....
But I think it's nearly meaningless to have a quantitative grading system without a clear shared understanding of what those grades mean.
Re: Tooting my own horn....
How about externals?
CP
www.wicper.org
Re: How about externals?
Re: Student Perspective
Student Perspective
Two examples:
1.
I had a guy (Mr. X) who was quite clearly and infamously a ludicrously high marker in my first year, and seemed to give almost everybody 'First' class marks for their essays. A lot of people ended up being 'marked down',* which, while initally obviously quite disappointing to them, did mean that
a) there was no risk of them getting 90 and some other student flippantly going "well, he's an easy marker"
b) students themselves didn't feel like they were cheating a grade.
To have your grade from Mr. X upheld by the second marker was a really nice sign of confidence. Now, admittedly, if this guy didn't have such a reputation that kind of effect wouldn't occur, and you have to wonder how effective the system is if people like that still give out ludicrously high marks. But my main point is that, as a student, I feel my grade is earned when it's been second-marked.
2.
Also, I don't know how it is in the US, but it's a fundamental part of degree and course guidelines at UK universities that every grade has a fairly specific description, so it's difficult to imagine grades being too far from standard. For example, with my own course, if you haven't argued a strong, well-read but original argument which shows awareness of course objectives in your essay, then theoretically you shouldn't get a First. And there's a bit of paper somewhere that says so. Which also gives you a bit more confidence that your grade 'means something'. Not that I'm suggesting US grades mean nothing to students, just trying to give a sense of how UK students feel about it, at least at times when the system appears to be working.
*Sorry for the repeated quotation marks, I was trying to give a sense of just how common these phrases are.
Re: Student Perspective
I'm somewhat ambivalent on the issue of grade inflation. I teach a lot of upper-division electives that tend to be somewhat self-selective, and worry about grade policies that would, in effect, punish students for taking more challenging classes.
Re: Student Perspective
Norming as Faculty
However, I've also been working with an interdisciplinary assessment group (one of those "do we assess on the college's stated outcomes?" exercises), and what I've found is that, even across the disciplines, we all have very close notions of what an A essay looks like as compared to a B essay. This becomes even more true if we use rubrics for our grading. To be fair, I think that working in such groups also helps to norm the grades, because we all have to articulate what we are looking for to our peers and explain why we might look for some things in our disciplines that others don't consider as important.
In the last case, in terms of disciplinary norms, it is entirely informal. If a student questions a grade, I simply hand the paper over to an appropriate faculty member for a second look. Moreover, I think many of us -- especially those of us who make up half of a department -- tend to norm a lot via hallway conversations, listservs, and blogs, but in a very informal way.
As I understand the AP grading process, all of the grades are normed. It would be interesting to see how those grades compare to grades given by college faculty who don't norm.
TAs and norming