Linda Suskie

  A Common Sense Approach to Assessment & Accreditation

Blog

Why do (some) faculty hate assessment?

Posted on February 28, 2018 at 10:25 AM

Two recent op-ed pieces in the Chronicle of Higher Education and the New York Times –and the hundreds of online comments regarding them—make clear that, 25 years into the assessment movement, a lot of faculty really hate assessment.


It’s tempting for assessment people to spring into a defensive posture and dismiss what these people are saying. (They’re misinformed! The world has changed!) But if that’s our response, aren’t we modeling the fractures deeply dividing the US today, with people existing in their own echo chambers and talking past each other rather than really listening and trying to find common ground on which to build? And shouldn’t we be practicing what we preach, using systematic evidence to inform what we say and do?


So I took a deeper dive into those comments. I did a content analysis of the articles and many of the comments that followed. (The New York Times article had over 500 comments—too many for me to handle—so I looked only at NYT comments with at least 12 recommendations.)


If you’re not familiar with content analysis, it’s looking through text to identify the frequency of ideas or themes. For example, I counted how many comments mentioned that assessment is expensive. I do content analysis by listing all the comments as bullets in a Word document, then cutting and pasting the bulleted comments to group similar comments together under headings. I then cut and paste the groups so the most frequently mentioned themes are at the top of the document. There is qualitative analysis software that can help if you don’t want to do this manually.


A caveat: Comments don’t always fall into neat, discrete categories; judgement is needed to decide where to place some. I did this analysis quickly, and it’s entirely possible that, if you’d done this instead of me, you might have come up with somewhat different results. But assessment is not rigorous research; we just need information good enough to help inform our thinking, and I think my analysis is fine for the purpose of figuring out how we might deal with this.


Why take the time to do a content analysis instead of just reading through the comments? Because, when we process a list of comments, there’s a good chance we won’t identify the most frequently mentioned ideas accurately. As I was doing my content analysis, I was struck by how many faculty complained that assessment is (I’m being snarky here) either a vast right-wing conspiracy or a vast left-wing conspiracy, simply because I’d never heard that before. It turned out, however, that there were other themes that emerged far more frequently. This is a good lesson for faculty who think they don’t need to formally assess because they “know” what their students are struggling with. Maybe they do…but maybe not.


So what did I find? As I’d expected, there are many reasons why faculty may hate assessment. I found that most of their complaints fall into just four broad categories:


It’s a waste of an enormous amount of time and resources that could be better spent on teaching. Almost 40% of the comments fell into this category. Some examples:

  • We faculty are angry over the time and dollars wasted.
  • The assessment craze is not only of little value, but it saps the meager resources of time and money available for classroom instruction.
  • Faced with outrage over the high cost of higher education, universities responded by encouraging expensive administrative bloat.
  • It is not that the faculty are not trying, but the data and methods in general use are very poor at measuring learning.
  • Our “assessment expert” told us to just put down as a goal the % of students we wanted to rate us as very good or good on a self-report survey. Which we all know is junk.


I and what I think is important is not valued or respected. Over 30% of the comments fell into this category. Some examples:

  • Assessment of student learning outcomes is an add-on activity that says your standard examination and grading scheme isn’t enough so you need to do a second layer of grading in a particular numerical format.
  • The fundamental, flawed premise of most of modern education is that teaching is a science.
  • Bureaucratic jargon subtly shapes the expectations of students and teachers alike.
  • When the effort to reduce learning to a list of job-ready skills goes too far, it misses the point of a university education.
  • Learning outcomes have disempowered faculty.
  • The only learning outcomes I value: students complete their formal education with a desire to learn more
  • Assessment reflects a misguided belief that learning is quantifiable.


External and economic forces are behind this. About 15% of comments fell into this category, including those right-wing/left-wing conspiracy comments. Some examples:

  • There’s a whole industry out there that’s invested in outcomes assessment.
  • The assessment boom coincided with the decision of state legislatures to reduce spending on public universities.
  • Educational institutions have been forced to operate out of a business model.
  • It is the rise of adjuncts and online classes that has led to the assessment push.


I’m unfairly held responsible for student learning. About 10% of comments fell into this category. Some examples:

  • Students, not faculty, are responsible for student learning.
  • It is much more profitable to skim money from institutions of higher learning than fixing the underlying causes of the poverty and lack of focus that harm students.
  • The root cause is lack of a solid foundation built in K-12.


Two things struck me about these four broad categories. The first one was that they don’t quite align with what I’ve heard as I’ve worked with literally thousands of faculty at hundreds of colleges over the last two decades. Yes, I’ve heard plenty about assessment being useless, and I’ve written about faculty feeling devalued and disrespected by assessment, but I’d never heard the external-forces or blame-game reasons before. And I’ve heard plenty about other reasons that weren’t mentioned in these comments, especially finding time to work on assessment, not understanding how to assess (or how to teach), and moving from a culture of silos to one of collaboration. I think the reason for the disconnect between what I’ve heard and what was expressed here is that these comments reflect the angriest faculty, not all faculty. But their anger is legitimate and something we should all work to address.


[UPDATED 2/28/2018 4:36 PM EST] So what should we do? First, we clearly need better information on faculty experiences and views regarding assessment so we can understand which issues are most pervasive and address them. The Surveys of Assessment Culture developed by Matt Fuller at Sam Houston State University is an important start.


In the meanwhile, the good news is the comments in and accompanying these two pieces all represent solvable problems. (No, we can’t solve all of society’s ills, but we can help faculty deal with them.) I’ll share some ideas in upcoming blog posts. If you don’t want to wait, you’ll find plenty of practical suggestions in the new 3rd edition of my book Assessing Student Learning: A Common Sense Guide.

Categories: Assessment culture, State of assessment

Post a Comment

Oops!

Oops, you forgot something.

Oops!

The words you entered did not match the given text. Please try again.

14 Comments

Reply Clieecus
10:07 PM on December 3, 2018 
, ?
!!! +79169295582 89169295582 . ! !
, , ? .
, . .
Reply Clieecus
4:18 AM on November 15, 2018 
So you! want something extremely new? Look at this page. Only here the choice of women for every desire and completely free! They are hardcore slaves, they will and want perform everything you order !
https://rebrand.ly/governy#K52
Reply annerReyes
1:52 AM on September 6, 2018 
good
Reply Popular Primary
9:03 AM on July 30, 2018 
Reply Popular Primary
9:01 AM on July 30, 2018 
Formal assessment take so much time!


However, Popular Primary has a great recommendation to speed that up at: top 5 teacher apps according to Popular Primary.
Reply nike air max homme
12:15 AM on July 1, 2018 
"But it was good for Pep, too, to see how Aguero responded to the challenge."
nike air max homme http://nikesport.soldes365.fr/Nike-Air-Max-2016-Hommes-Chaussures
-806771-020-Nike-Air-Max-Homme-T1032.html
Reply Xiaomei Song
12:42 PM on March 1, 2018 
Thanks for writing and promoting the post! And it is a good idea to explain the qualitative data analysis procedure and inductive approach. Sometimes anecdotal observations and systematic, scientific conclusions are just one step away, though there are many hours' differences in terms of workload.

Similarly, we found the theme of "who's to blame" when we surveyed instructors. They blame "high schools", "admitting too many students who are not college ready", and "lower-level non-major teachers." Overall, I feel there might be four types of faculty: those who are not really passionate about teaching and do not intend to improve, those who blame other people, those who want to improve and don't have time to learn, and those who take actions. Shared responsibility is the key.
Reply Catherine Wehlburg
7:34 PM on February 28, 2018 
Thanks, Linda, for this post -- I think that we do need to seriously look at the issues that we're hearing from faculty (and others). And, we've been hearing these things for years -- we need to incorporate these into how we "do" assessment and how we talk about it with our institutions and our accreditors.
Reply ★ Owner
4:40 PM on February 28, 2018 
I've updated my blog post to provide info on the Survey of Assessment Culture, with a link. Again, apologies to Matt and everyone else for my brain freeze. For what it's worth, I was writing this on vacation while looking out over the Gulf of Mexico. Clearly my brain is on vacation in more ways than one!
Reply Lynne
1:03 PM on February 28, 2018 
HI Paul, your link is broken. Maybe the site got overwhelmed with traffic? I would appreciate it if you could please send me a copy of your survey - if you do not mind if I use it for inspiration for one I have been considering developing here at UCCS.

Lynne, Director of Assessment, University of Colorado Colorado Springs
Reply ★ Owner
11:54 AM on February 28, 2018 
Duh... I'm actually on the advisory board for Matt Fuller's faculty survey at Sam Houston State U, and I've even advised him on survey questions. I had a complete brain freeze! Matt, my apologies!
Reply Paul Henley
11:18 AM on February 28, 2018 
Nice post, Linda. FWIW, the SHSU gang does an assessment survey that may provide some insight. Here's a link to that one. http://www.shsu.edu/research/survey-of-assessment-culture/faculty
.html

It certainly changed things at SFA.

Best,
pth
Reply Gina Polychronopoulos
11:11 AM on February 28, 2018 
For the second theme: "I and what I think is important is not valued or respected." That is a tough battle, and it takes a long time to shift the "culture" of assessment on campus. Faculty might feel reluctant to trust assessment professionals because of historical "assessment, for assessment's sake." However, what has helped me is to be patient and remind them that by working closely with faculty members, the assessment process may become more meaningful.

Thank you for sharing this!
Reply Jill Millard
11:10 AM on February 28, 2018 
Thank you for the informative blog post. I find the categories very interesting. In response to your comment on faculty surveys regarding assessment, Matthew Fuller, Associate Professor and Director, Higher Education Leadership Doctoral Program
Sam Houston State University, has been doing faculty surveys of assessment culture for a few years now. The 2018 Survey has just been made available.