First, congratulations on your terrific initiative! You clearly have the right components in place. You are looking at student work and you are committed to collaboration. Moreover, you have created a safe way for teachers to share student work because you are making the submissions anonymous so that neither the identity of the student nor the teacher is disclosed.

You asked for some ways of adding structure to this process, so let me share what I do in these circumstances.

First, I hand out a single piece of student work—remember that all of the samples started in life as “proficient“—and ask each teacher to evaluate it ALONE, using the scoring rubric that was used for the assessment. I then track the percentage of faculty members who rated that piece of work Exemplary, Proficient, Progressing, or Not Meeting Standards. I note the percentage of agreement—typically fairly low.

Second, I give teachers the opportunity to collaboratively score the assessment in groups of two or three, and make any revisions they wish. I note the percentage of agreement again—typically a little bit better.

Third, we have a group meeting to revise the scoring rubric. The theme of this meetings is “the enemy is not each other; the enemy is ambiguity.” Where there is a disagreement it is not the fault of the teacher, but the fault of an ambiguous scoring rubric. We use a collaborative process to revise and improve the scoring rubric, making it more specific. Then we score the same piece of work AGAIN. The third time, the level of agreement is always higher than the first two times.

Finally, we look at this process from the student's ponit of view. What would they think of the first level of scoring? Pretty unfair, right? As we worked together, collaborated, and refined the rubric, our fairness improved. This process is not about some state mandate, but about our fundamental shared value of FAIRNESS.

In most schools, this takes ALL YEAR LONG - it's not a one-short staff development meeting. In some districts, they open EVERY faculty meeting, including cabinet meetings at the central office, with collaborative scoring of student work. They learn that the faster they reach 80% consensus, the faster they move on to other agenda items.