Linda Suskie

  A Common Sense Approach to Assessment & Accreditation

Blog

A landmark study questions the validity of rubric design

Posted on July 30, 2014 at 6:45 AM

A study reported in the latest issue of Educational Researcher promises to rock the world of rubrics.

 

While I’ve always advocated for flexibility in designing rubrics, one particular format is widely regarded as the gold standard for rubric design. It’s what I call a descriptive rubric and what others call an analytic rubric. It’s a table or matrix, with the rubric criteria listed in a column on the left and student performance levels listed across the top. Then each box in the table has a short description of student performance for that criterion at that level. Probably the best known examples of descriptive or analytic rubrics are the VALUE rubrics published by the Association of American Colleges & Universities.

 

The study’s authors challenge this format, specifically the idea that every criterion in a rubric must have the same number of performance levels. A team used observed, qualitative differences in student writing to reconsider a writing rubric. The new rubric had, for example, seven performance levels for vocabulary but just three for paragraphing. Breaking away from the table/matrix rubric format helped raters make more independent judgments on each criterion…and thereby arrive at more valid judgments of student writing performance.

 

I’ve always been uncomfortable forcing faculty to come up with a set number of student performance levels, whether 3, 4, 5, or more. This study confirms my uneasiness.

 

The citation for the study is:

 

Humphry, S. M., & Heldsinger, S. A. (2014, June/July). Common structural design features of rubrics may represent a threat to validity. Educational Researcher, 43(5), 253-263. DOI: 10.3102/0013189X14542154

 

Categories: Ideas

Post a Comment

Oops!

Oops, you forgot something.

Oops!

The words you entered did not match the given text. Please try again.

0 Comments