ARCHIVE: Oklahoma Math 2014 Assessment Update

During this time of year, teachers are working their hardest to summarize three-quarters of a year worth of learning to ensure students are ready for the Oklahoma Core Curriculum Test (OCCT) for grades 3-8 and the End of Instruction (EOI) tests for Algebra 1, Geometry, and Algebra 2. This process can be fun and exciting a little stressful, but it is even more so when information is hard to come by and understand. So, with a little luck, this post will piece together the information teachers, schools, and parents most often request from me.

Perspective on Assessments

Because assessment is such a hot topic, I want to provide a little perspective. The math assessments for grades 3-8 and Algebra 1 are all required by the United States Department of Education. For Geometry and Algebra 2, Oklahoma included these assessments as part of the Achieving Classroom Excellence (ACE) Act (read more about ACE here). Regardless of what standards we have in Oklahoma, we are still required to give these assessments. Indeed, regardless of what standards we have in Oklahoma, reform efforts such as TLE and A-F are still happening. And perhaps most importantly, regardless of what standards we have in Oklahoma, teachers, schools, and districts are still in charge of determining their curriculum, instructional approach, and their local assessments (including benchmarks). Hopefully, my effort to distinguish between standards, curriculum, and assessments is clear.

Taking the Test

Calculator Policy

The calculator policy for this year’s operational assessments is a little different than we expected due to variations required to ensure that embedded items that are being piloted for next year are valid. We all know this is frustrating, but great care has been taken to ensure that the changes to the calculator policy does not impact operational items.

So, the following table shows the difference for the Operational Test, which is consistent with the information provided by the Office of Assessment and CTB McGraw-Hill in the Test Administration Manual for the Math EOIs. Note: Details about the third section of “Item Tryouts” are provided later in this post.

2014 Operational Assessment Calculator Policy

Unfortunately, these changes are not consistent with the Calculator Policy Memo (March 19, 2014) that Jennifer Lamb, Sonya Fitzgerald, and I worked so diligently to send out. However, the changes are very much necessary, albeit frustrating, as we are having to carefully field test these items to ensure we are ready for the new OAS-aligned assessments for next year.

Please know that my recommendation would be to ensure that students know that what matters is they are using the calculator they feel comfortable with. If they have no experience with Graphing Calculators, I would advise that you explicitly discuss with them that using the embedded Graphing Calculator would not be a good idea.

Reference Sheets

Supposing you have navigated the changes in the Calculator Policy, the next thing to understand is the additional Reference Sheet that has been provided this year.

The Operational Items in this year’s assessments will continue to have necessary formulas embedded within the item. 

Again, because the embedded field test items must be validated for next year, the Reference Sheets are provided for students throughout the entire test on the math EOIs. For 5-8, Reference Sheets are not provided on the Operational OCCT, but will be provided on the Item Tryout section, which is discussed further in the next section.

Download the Reference Sheets Here:

• 5th Grade Reference Sheet
• 6th Grade Reference Sheet
• 7th Grade Reference Sheet
• 8th Grade Reference Sheet
• High School Reference Sheet

Item Tryouts

As you may have heard, we are having to do a separate “Item Tryout” section for all grades this year to be able to validate enough items to be ready for the new OAS-aligned assessments for next year. Your District Testing Coordinator should know if your school site was selected to pilot the math portion or not. There are all sorts of things to know about the Item Tryouts, but I am proud to say that the Calculator Policy for the “Item Tryouts” as defined in the Calculator Policy Memo (March 19, 2014) is actually accurate!

For more information about the “Item Tryouts,” here’s a message the Office of Assessment sent out in March.

Information for Oklahoma Item Tryouts Spring 2014
This spring, students will be participating in Oklahoma Item Tryouts in Grades 3-8 Math, Reading, and Writing and EOI Algebra I, Algebra II, Geometry, English II, and English III in preparation for the full implementation of the new standards for College and Career Readiness in 2014-2015.

The Item Tryouts should be administered upon completion of the operational tests. Districts will have two additional weeks to test students with the Grades 3-8 window closing on Wednesday, May 21, and EOI window closing on Friday, May 23, 2014. The one exception to the Item Tryout Window is for Grade 7 Geography. Schools selected for this sample should administer this subject during the regular operational window. Grades 3-5 will be testing with paper/pencil books. Grades 6-8 and EOI will test online. Test Administrator Manuals will include instructions for the Item Tryouts.
Suggested times for the tryouts are as follows:

• Grades 3-8 Math: 45-60 minutes (approximately 25 items)
• Grades 3-8 Reading: 45-60 minutes (one single passage and one paired passage with approximately 26 items)
• Grades 3-8 Writing: 60-90 minutes (Students will read either a single or paired passage and respond to one writing prompt.)
• Grade 7 Geography: 30-45 minutes (32 items)
• EOI Algebra I, Algebra II, and Geometry: 45-60 minutes (approximately 12 items which includes technology enhanced and constructed response items)
• EOI English II and English III: 60-90 minutes (Students will read a paired passage, answer two multiple-choice questions, and respond to one writing prompt.)

Calculator Policies & Reference Sheets do not apply to this year’s operational test – only applies to this year’s Item Tryout and next year’s operational test. The Oklahoma College and Career Readiness Assessment (OCCRA) Calculator Policy (which will begin on the 2015 assessment) can be found at http://ok.gov/sde/sites/ok.gov.sde/files/documents/files/OCCRA_Calculator_Policy.pdf

• Math Grades 3-5: Calculators are only allowed as an approved accommodation for students on an IEP or 504 Plan, and only a basic four-function calculator with square root.
• Math Grades 6-8: Item Tryouts will have two sections; one section will be a calculator allowed section and the other will be a non-calculator section. Your students will only pilot one section, so the Sampling Plan has designated whether your students will or will not be allowed to use calculators.
• Math Grades 6-7: Students are allowed to use a basic four-function calculator with square root.
• Math Grade 8: Students are allowed to use a Scientific Calculator meeting general requirements.

Students in Grades 5-8 will be provided a Reference Sheet. These can be found on the State Department’s website and a link for each is provided below. Measured Progress will send copies with the testing materials for the students to use at the time of testing.

• 5th Grade Reference Sheet
• 6th Grade Reference Sheet
• 7th Grade Reference Sheet
• 8th Grade Reference Sheet
• High School Reference Sheet

Accommodation Guidance for Item Tryouts for students on an IEP or Section 504 Plan
Assessment accommodations for Item Tryouts for students with an IEP or Section 504 Plan may be those accommodation(s) documented on the IEP or Section 504 plan and normally employed during classroom instruction, EXCEPT for the following:

• No read aloud for the reading or writing tests
• No paper accommodated forms (e.g., Large Print, Braille)
• 3-5 testing will be a paper test only
• 6-8 testing will be online only
• No chunking or multiple sessions are permissible as the item tryouts are between 1-26 items

Advanced Middle School Math Students

The OSDE submitted a waiver to the USDE to not require advanced middle school math students to double test. The announcement was made on January 14, 2014. The Office of Assessment provided clarification on this policy in an April 4, 2014 “OSDE Testing Update” memo. The relevant excerpt is provided here:

Students who have been uploaded into a middle school math test but are taking a Math EOI may either be marked with the Not Tested Code of “Other” to remove their test record from being processed, or they may remain in the session without a Not Testing Code indicated. For the latter, the Accountability Office will run a match between the two records and suppress the middle school math record as long as the State Testing IDs match between the two test records.

It is important to note that this policy has implications for school’s A-F Report Card, whereas middle school students who are in high school courses will no longer benefit the school by providing two scores. For example, a school with 100 students that offers Algebra 1 to 20 of the eighth graders would have received the following score on their Student Performance section. For this example, we’ll assume 70 of the eighth graders would be Proficient or Advanced and all advanced eighth graders would be Proficient or Advanced on both the eighth grade and Algebra 1 exam. We can assume the 6th grades and 7th grades have a similar Proficient/Advanced rate of 70 per 100.

The current A-F score would be as follows:

Here is how the same students would score if all things were the same except Algebra 1 students were not taking and passing the eighth grade assessment as well:

In this case, the hypothetical school scrapes by with the same overall letter grade, but has lost 2 percentage points. You can imagine if the Algebra 1 class was slightly larger, the school  letter grade might have slipped to a D, having seen no difference in student learning. Of course, we are working on a strategy for communicating this issue, but the key concern from my perspective is reducing the burden on the student. Obviously, please make your voice heard on this issue. If you see a solution for ensuring schools and districts are making the best decision for students, please make your voice heard.

Passing the Test

Grades 3-8

For Grades 3-5, these tests are taken the old-fashioned way, using pencil and paper; for Grades 6-8, the tests are online. Assessments at these grades are post-equated, it means they are a little slower to grade. This type of analysis allows the various test forms to be more standardized and improves the accuracy of the measurement (Tong, Wu, & Xu 2008). Unfortunately, this means we don’t know the cut score ahead of time. In essence, the goal of post-equating is to replicate the pre-equated data gathered during field testing. This is a little bit in the weeds, but the point is that we just don’t know the cut score.

Even still, I know there are some folks who carefully track the past cut scores of various tests to have a little better expectation on this year’s test. I’m all for anticipating the cut score, but beware how you communicate this to students and parents. If you’re telling them they passed before the cut score is finalized and communicated, you could be setting the student up for a serious emotional swing.

Thanks to Katie Lamiell for posting the following table on the #OKMath Facebook Group.

Algebra 1, Geometry, and Algebra 2

In contrast to the 3-8 Assessments, the EOIs are pre-equated, allowing for significantly more timely results, which is especially important since students are graduating, etc. In fact, you can see the cut score tables in the CTB McGraw-Hill memo from April 3, 2014 right now!

As with all forms of assessment, there are limitations and it is to your advantage to inform yourself on the pros and cons of both types of “equating.” For pre-equating, the biggest risk occurs when field tests occur separately (Tong, Wu, Xu, 2008, p. 8). Thankfully, I expect all field test items to again be embedded next year as they have been in the past.

Demonstrated Mastery Guide

// ** Added 4/24/2014 ** //

In May of 2013, the Oklahoma Legislature passed Senate Bill 559, which provides exemption from certain End of Instruction (EOI) exams by demonstration of mastery over the subject through the means of an alternate exam. This is a huge change from the way things have worked in the past. You can read the 2014 Demonstrated Mastery Guide here. 

Here’s the Alternate Cut Score Table from page 7 of the Demonstrated Mastery Guide:

I hope this post has been helpful. I’m only human, so if you see a mistake, please don’t hesitate to kindly correct it in the comments section.