For the latest Oklahoma Test and Item Specification Documents, visit http://sde.ok.gov/office-assessments
The long-awaited Item Specifications have finally arrived! You can easily access each grade level document by visiting http://sde.ok.gov/office-assessments.
Edit January 17, 2017: As a follow-up to this post, read The OKMath Framework Released.
The challenge now is deciding what to do with these documents.
I see a number of districts across the state utilizing state testing materials, like Item Specifications, to construct locally administered assessments that “mirror” that state assessment. The thinking is that teachers can give mini-versions of the state assessment during the year and have a pretty good idea of how their students will perform on the OCCT.
From my perspective, this practice sounds better than it actually is. Jim Pellegrino confirmed my thinking on this matter a bit earlier this week when he discussed the different validation methods used for state assessments and local assessments. For much more on that matter, check out his article here.
Let me tackle three reasons this mirrored testing model isn’t what it’s cut out to be. Stick with me though. I’m putting my ideas out there to be considered, not to make anyone feel bad. Help me improve my thinking and let’s work together to keep improving mathematics instruction in Oklahoma!
1. Sorry, but your test just won’t be as good.
Let’s face it. Each item developed on the statewide assessment (like them or not) is developed by item writers who have made their careers out of building items that align to the intent of the standards and are constrained by the content limits. Each item goes through numerous review committees verifying alignment, redesigning to reduce bias, and ensuring that the item is actually appropriate for all students. After countless reviews, the items are then put through a remarkably rigorous process to ensure the items are reliable. This part of the process can give you a pretty extreme headache if you aren’t careful! They measure all sorts of variables, exploring variation in performance across demographics and ensuring the item is actually measuring what it says it will. It doesn’t even show up on the final, “live” assessment if it isn’t really doing what it’s supposed to.
Meanwhile, items developed locally go through just a few reviews (at best) and then land on a quiz or benchmark test; these tests are used to make instructional decisions about remediation, funding, and teacher quality. Thinking about smaller districts that don’t benefit from a full-time district-level math curriculum coordinator, the likelihood these items are free from bias and well-aligned to the intent of the standards is even lower.
Bottom line – If you’re trying to build a mini-state assessment based on the Item Specs, it probably doesn’t match up.
2. If we’re honest, your test won’t be as good as it could be.
The classroom is not hindered by the time and funding issues that constrain the state assessment. For state assessments, we have to design a test that covers all of the standards. Because of that, we are stuck with a huge number of items that don’t really allow us to dig into what kids know and can do in a way that provides actionable feedback to the teacher. I think we all know multiple choice just doesn’t give us a sense of what kids actually know and what misconceptions they are having.
In the classroom, instructional assessments can be such a powerful part of the teacher’s tool belt. Items that seem dull and are best completed by test-taking strategies become rich, worthwhile tasks if they are left as open response items in the classroom. You can do so much more if you aren’t limited by the same constraints we are in the state assessment.
Let’s tackle an example just to make my point.
In 7th grade, pretty much the only thing that matters is proportional reasoning. With that in mind, let’s look at the released item for Standard 7.A.1. Here’s the Standard and related Objectives for context:
So, what does the state assessment look like? The example provided is a DOK 1 task, so it’s pretty basic…
Right, this has a pretty straight forward structure… does the student recognize the form of y=kx or not? If so, A is obviously the answer. But, what could this look like in a classroom? Well, I actually have a quiz I designed using the Desmos Activity Builder. I think you’ll like it, but I also want feedback.
[scbutton link=”https://student.desmos.com?prepopulateCode=gf4dr” target=”blank” variation=”steel_blue” size=”small” align=”center”]Take the Quiz Here[/scbutton]
Here are the 7 different problems I made. I think you’ll get the idea pretty fast that they are fundamentally different than what you’ll find on the state test. You’ll get a better sense of the tasks by clicking the button above and trying them out.
Bottom line – You can really do so much without the constraints of the Item Specs… why hold yourself back?
3. The test-prep culture narrows curriculum and is boring.
Understandably, teachers want to ensure their students have covered all of the tested materials. Because the pressure on teachers is so intense, this often means that students don’t interact with mathematics that doesn’t explicitly show up on a multiple choice high-stakes state assessment.
The vision for mathematics cast by the new Oklahoma Academic Standards for Mathematics make it clear that all students should have excellent mathematics experiences that pique their curiosity and engage them in becoming problem solvers, not just problem doers. Mathematics is much more than test-prep and we all deserve the opportunity to explore and enjoy mathematics rather than constantly fear the state assessment.
Bottom line – the standards are meant to guide a patient, purposeful progression of the learner’s experience across grades. If we only teach to the test, we might miss out on the rich opportunities to interact with the mathematics entirely.
Now what?
We have to work together to find a way to both ensure success on the state assessments, but also make sure our students are engaged in appreciating and deeply understanding mathematics. Isn’t that what it is all about?!
We must focus our attention on using worthwhile instructional tasks so we are focused on the standards above all else.
A few reflection questions:
- What does the testing culture make you focus on that you wouldn’t otherwise?
- How does your local item development and validation process match up?
- Does the test-prep culture narrow your curriculum and bore you? Are you the exception? If so, I want to hear from you!
Featured Comments from the #OKMath Facebook Group
I think limits are necessary, item specs not so much… limits are necessary with or without a state test. I can’t go back to every students previous teacher and ask what they taught, and I can’t predict who they will have next year to ask them what they need to know to be successful in their next class.
Challenge your kids, teach them to think, explore, question. The test will take care of itself. What if………..you “sacrificed” a year and didn’t teach to the test, used pbl, hands-on learning, it would be interesting to see how your students would perform. Just a thought.
I’ve been wanting guidance & perspective of how deep to go, guidelines for projects that would at minimum be to the real world ideals that are referenced in the standards. I prefer to bridge them to the next level when I can. I enjoy letting them see how it ties to other areas, cross curricular, or how they might use it as an adult someday soon.
I have so many mixed feelings. I don’t want to have to teach to the test, and I keep hearing that I don’t have to continue doing this. However, my school is still being judged by testing. Am I wrong? I use the item specs for clarity of what I am to teach. I am a very visual person and actual examples of standards are very pleasing to me. I am very frustrated about this tenth grade test. I feel a lot of pressure not only to teach Geometry but also halfway reteach Algebra I. Last year I was responsible for my students knowing Geometry, but this year, I am responsible for my students knowing Geometry and reminding them of all areas of Algebra 1. And I know. There is a lot of Algebra I in Geometry. It’s just that I feel like my job got harder instead of easier. I just want to have fun teaching without possible doom hanging over my head. Rant over. I’m sure things will look brighter tomorrow morning. I will continue to do my best preparing my students for what may lay ahead of them.
It is a relief that we are moving into the direction of having professional discretionary judgement to just teach math the best and most interesting way we can without having to worry so much about the standardized tests. I’ve always felt that if we have the freedom to teach from a set of topics within a particular topic area (like algebra 2 or geometry), without worrying about covering every single topic, then we can instill an enjoyment of learning math that will enable students to pick up the math skills they need when they need it.
Thank you for the reminder that when we teach to a test, we only touch a fraction of student capabilities…and we are bored/boring…and we perpetuate the negative math culture that has kept many a bright mind from the love of math that we wish for for our students. I always enjoy your ideas on math instruction. Thank you so much for sharing your thoughts!
