The responses to the October’s Question, “GLOBAL RATING SCALES: USELESS AS A TOOL FOR IMPROVING PERFORMANCE” were insightful and honest as well as practical. If you have not already read them, I encourage you to go to the OCTOBER Archives and read each one. The following are just a few “pearls” from the list.
There was a consensus that “global ratings” alone do little to assist our learners in improving practice. Some quotes:
- “The student requires a specific notion of what should be improved and therefore numeric value will be useless if there are no specific comments that will provide further feedback and insight”.
- “I agree that a global rating scale number such as "7" alone provides little information to a student about performance. Descriptive feedback is definitely better at elucidating areas of strength and weakness. This type of feedback allows the student to internalize and re-assess their performance with opportunity to adapt to expectations and/or standards.”
Others responded to GRS reliability and the use of data to compare students or residents. Some notes and quotes:
- “Just like with Likert scales, things are given a numerical assessment and each number is exactly the same distance from the other, yet what these numbers mean is not uniform in distance. For example, the distance between what someone would rate a 5 and an 8 on these scales is much smaller than the difference between 1 and 4. In fact I don't even know if 1 exists! This renders the scale even more arbitrary than what is just personal point of view of what an 8 means, etc. This has direct implications on using the numbers in any way - for example averaging and comparing.”
- “…. I think that the information from these GRFs becomes more meaningful when there are a number of raters. In this case, if a student consistently is scroing much lower than his or her peers, (across several stations) then we can intervene with that student and provide targeted input, correction, remediation -- whatever you want to call it. Yet, the students say that they learn the most from the narrative comments that the SP's put on each section of the form, noting what the student did well and what could be improved. I don't think that it's really a question of GRFs vs. narative feedback; we need both as they meet different aims
There are good questions about the value and psychometric properties of Global Rating Scales. A nice short article that addresses the inherent problems with GRS is listed below.
Farrell, S E. (2005) Evaluation of student performance: clinical and professional performance. Academic Emergency Medicine. 12(4): 302e6.
Posts
