Building assessments for the future

Last week I worked with science educators from around the state to review state assessment items that the MDOE will potentially include in future MEA Science assessments at grades 5, 8 and 11.  These groups of teachers, Item Review Committees (IRC), represent one step in a two-year development process that assessment items move through before the MDOE can include items in the State science assessment.

The teacher voice in IRCs is invaluable.  The teachers who participate help the MDOE and our assessment vendor, Measured Progress, ensure that items:

  • are accurate and, in the case of multiple choice items, have only one correct answer;
  • are clearly and appropriately worded for the grade level;
  • are aligned to the performance indicators and descriptors of the Maine Learning Results;
  • avoid bias and provide access; and
  • are correctly coded for Depth of Knowledge*.

I have been thinking a LOT about what future science assessments aligned to the NGSS might look like and I eagerly await the results of the National Academies report that will address this topic.  The National Academies expect to have this report completed in 2013.

Assessments aligned to standards that combine practices, crosscutting concepts and core ideas will certainly have to look different from current assessments in some ways. For example,  I am excited by what I see as the need for more computer enhanced simulations in future assessments.  The recent NAEP Science Assessment gives us a window into what this might possibly look like and the important opportunity these simulations can provide for understanding student thinking. I also see the need for more constructed response items that probe at student ability to make  and defend claims and connect their arguments to core ideas in science.

I am grateful that Maine has continued to include constructed response items in our MEA Science Assessment. Many states use a multiple choice only format.  Many states also do not release assessments items to the public.  Maine currently releases 50% of the common items annually.  Budget constraints forced MDOE to cut back the percentage of released items from 100% to a 50% several years ago.

Maine certainly doesn’t have the resources as a small, single, rural state to develop rich science simulation assessments we would like.  However, the MDOE and Measured Progress are collaborating, within our current assessment design, to enhance constructed response items to make them more forward thinking.  We already incorporate graphs, tables, and representations into MEA questions and we will be field testing items that ask students to make and defend claims and use models to predict and describe science ideas related to the Physical Setting and Living Environment. These efforts will help us to better understand the opportunities and challenges of science assessments of the future.

We fully expect that in the future states will band together in consortia for Science Assessment.  Working together, it is my hope that state will be able to pool resources to develop assessments that reflect the vision for teaching and learning (and assessment) we see in A Framework for K-12 Education.

How fast will the change take place?  As I described above, assessment development takes at least two years so… if the Legislature adopts the NGSS in June of 2013 in theory it would take minimally two years to develop items aligned to NGSS standards.  However there are many complications. For example, there may be potentially many states collaborating this could take time. In addition, states will be trying to blend this work with the roll out CCSS Assessments in 2014-2015, another complicating factor.

What does these mean?  It means that we have more reasons to understand the Framework. How much reading in the Framework have you completed this summer?

*Depth of Knowledge is a coding for the complexity of thinking that a student must perform to respond to the item.  This is very different from the difficulty of the item.  Item difficulty is determined by the number of students who get the item correct.