|Posted on March 23, 2015 at 7:50 AM|
One of the biggest barriers to understanding and using assessment results is figuring out whether or not the results are good enough by setting appropriate benchmarks or standards. Most faculty and administrators don’t have any idea how to do this and end up pulling numbers out of a hat!
Here are seven steps to setting good-quality benchmarks or standards for rubrics:
1. Know how the assessment results will be used: who will use them and what decisions the results will inform. If the purpose is to fund things that need improvement (something many accreditors want to see), you’ll want to set a relatively high bar so you identify all potential areas for improvement. If the purpose is to maintain the status quo, you’ll want to set a relatively low bar so your students appear to be successful.
2. Clarify the potential harm of setting the bar too high or too low. If the bar is set too high, you may identify too many problems and spread yourselves too thin trying to address them all. If the bar is set too low, you increase the risk of graduating incompetent students.
3. Bring in external information to inform your discussions. Disciplinary and professional standards, employers and alumni, peer programs and colleges, faculty teaching more advanced courses—any of these help you develop justifiable benchmarks.
4. Have a clear rubric, with clear descriptions of performance in every box. The fuzzier your rubric, the harder it is to set meaningful benchmarks.
5. Look at the assignment that the rubric is evaluating, as well as samples of student work, to inform your thinking. Students’ organization and grammar will likely be weaker on an essay exam question completed in a short amount of class time than on a research paper subject to multiple revisions.
6. For each rubric criterion, identify the performance level that represents a minimally competent student: one whose performance would not embarrass you. Setting standards or benchmarks is inherently a value judgment, so a group of people should do this—the more the merrier—by voting and going with the majority. Not all rubric criteria will have the same benchmark: basic or essential competencies like grammar may have a higher benchmark than “aspirational competencies” like creative thinking.
7. Ground your benchmarks with data…but after that first vote. If you have assessment results in hand, or results from peer colleges or programs, share them and let everyone discuss and perhaps vote again. If your students’ performance is far below your benchmark, think about setting short-term targets to move your students toward that long-term benchmark.
If you’d like to learn more, I’ll be talking about these seven steps at the Higher Learning Commission’s Annual Conference. My session, “How Good is Good Enough? Principles for Setting Benchmarks,” is on Sunday, March 29, at 1:00 p.m.