Whether you use SPSS or Microsoft Excel or just a hand-written data set, here are the keys to bear in mind:

First, be sure to make a “same student to same student” comparison. In other words, it is of some value to compare last year’s third grade students to this year’s third grade students. Perhaps the differences in scores will be due to the differences in curriculum. But perhaps the differences in scores will due to the fact that they are different children. It is therefore far more persuasive to have pre- and post- test data that will – for the very same group of children – the gains that they make with treatment A (the previous curriculum) and then the gains that they make with treatment B (the new curriculum). You would expect both data sets to show gains – after all, the kids were in school. But your hypothesis is that the gains would be of a different magnitude.

Second, if you are examining effects across many different classrooms, then it is imperative that we acknowledge that there is more going on here than the “presence or absence” or a new curriculum. In every curriculum or teaching reform, the fundamental issue is DEGREE OF IMPLEMENTATAION. I have seen new reading curricula implemented, for example, with some classrooms devoting 90 minutes each day, others devoting 120 minutes each day, and others devoting 180 minutes each day to the “same” curriculum. In other cases, I have seen many schools claiming to have the same new curriculum, but the actual use and implementation varied widely. When researchers tracked the actual degree of use of the new curriculum, they found that low levels of implementation were worse than no change at all, and only the highest levels of implementation were effective. This will help you avoid the “brand name fallacy” in which vendors attempt to claim that the brand name of the curriculum is the salient variable in student achievement, when you and I know that teaching, leadership, and daily implementation are far more important.

Thanks very much for a very thoughtful note. I agree that MAP is not a great reflection of the success of your students. However, I would not exclude MAP from the equation. One common problem in gifted education is that teachers assume "they already know that" and fail to give students the grade-level instruction that they need in addition to enrichment opportunities. That's one reason that gifted kids drop out in high school - they have never experienced failure, and then all of a sudden a high school teacher expects them to have some basic skills that they were never taught because teachers in earlier grades assumed that they had map reading, number operations, and essay construction down pat.

The real question is, as you suggest, what additional measures beyond MAP can we use? First, I think we need to stick with same student to same student comparisons, not this year's group to last year's group. There implies, therefore, some sort of pre- and post- test information during the same academic year.

Second, we'll need a variety of instruments, including traditional ones. As David Perkins reminds us, students who are gifted in one area may not be gifted in others, and therefore the student with superior mathematical ability may nevertheless use traditional literacy tests, or vice versa.

Third, we'll need tests that are sensitive to exceptionally fact progress. That's why test of a single grade level do not work with gift students, because even if we give a 5th grade test to a 4th grade student, we have essentially created a "ceiling effect" of one year of progress above grade level. More nuanced tests, such as those that use Item Response Theory to give progressively more challenging questions each time a student answers correctly, are better suited for this task. But at the end of the day, you and your colleagues (and students) will complain of test fatigue if we just test the kids all the time.

Fourth, to address the test fatigue issue and also challenge students appropriately, I would favor have them create their own assessments. First, I'd have an honest talk about times when they felt that a test did not allow them to show all that they know. When a 5th grader can do algebra, scoring 100% on a multiplication and fractions test is not very satisfying. So let them think about different types of assessment --- closed end, open-end, performance, etc. - and have them construct an assessment each month that, if they were the teacher, would allow their students to demonstrate all that they have learned. The worst case is that some kids will sand-bag, using this as an opportunity to get out of work by dumbing down the task. I think that risk is minimal, particularly when other students are going to create some wonderfully creative and challenging tasks in a variety of different assessment formats.