I had an experience the other day that was just so incredibly cool that I have to share it. I am the Director of Faculty Development and so my "students", for the most part, are the faculty. I was sitting at an early meeting, half asleep, when I heard one of the faculty raise the issue of "instructional alignment" and the need to align learning objectives, instruction and assessment process for an old course that had received sub-standard reviews. Now my ears always perk up when I hear any of the faculty using educational terminology, but this case was classic. Not only had he used the terms correctly, he was descibing in great detail how and why this was so important. And... in a moment of pure egocentrism I thought... "he learned that from me'!
Well this has certainly happened before, but this was a special case. This particular individual had let me know early in our conversations that he thought "all this education mumbo-jumbo" was just that. He had attended two or three faculty development sessions and seemed to be changing his opinion; then stopped coming. This meeting was the first time I had seen him for a few months, so hearing him describe "instructional alignment" to his peers was one of those "peak moments".
Now, I'm sure my clinical colleagues have those moments all the time as they teach residents, students and patients. You get the chance to see daily the intellectual growth of your learners and the progress that patients make based on your careful management. In my field, however, those moments don't come everyday. I'll bet that if you devote your time to faculty development, you are smiling right now and thinking about a moment like this that you have had. It is great isn't it
Wednesday, June 16, 2010
Wednesday, June 2, 2010
A Case for High-stakes Summative Examinations?
The role of examinations like the USMLE Step 1 has been called into question due to:
Medical Education recently published a very interesting study [pdf; abstract] regarding year 4 medical students in Germany learning EKG interpretation.
Summary of the study:
In addition to a test before and after the EKG course, each cohort had an unannounced test 8 weeks after the EKG course.
Thus the study was designed to look at the impact of 2 types of learning (traditional lecture vs. small group peer learning) and the impact of a high stakes summative examination on the learning.
They found that in the Cohort 2 where there was only a formative assessment at the end of the course (performance did not count towards grades) the small group peer learning group did better at the immediate post test and also retained the information better at the 8 week surprise test.
On the other hand in Cohort 1, the high stakes summative test seemed to eliminate all the differences between the 2 groups.
The question:
What is it about the summative test that improved learning and retention? Did it force students to spend more time in self-study preparing for the test or did it force them to concentrate better at the lectures, taking notes etc?
Reading this study made me think about how this might apply to even higher-stakes summative tests like the USMLE Step 1. Scores from this one test can determine a student's career.
One student who had spent the first 2 years in small group PBL sessions, told me that as he was preparing for the USMLE Step 1 at the end of year 2, everything seemed to fall in place. We discussed this further and it seemed that having to read all the different subjects in a concentrated time span helped him to develop mental connections between various pieces of information in his memory thus improving his understanding.
So what do you think? What is the role of high stakes summative tests in medical education? Are they good, bad, a waste of time, a distraction, or something that we should have more of?
- Tests only medical knowledge - not the other competencies that go towards making a "good physician"
- Forces students to memorize content just to regurgitate it at exam time
- Most physicians may not need to use the content memorized for this test in later life.
- Biomedical science information is growing so fast that there is no way to cover all this content within the span of an average medical school curriculum
Medical Education recently published a very interesting study [pdf; abstract] regarding year 4 medical students in Germany learning EKG interpretation.
Summary of the study:
- Cohort 1 - winter 2008-09 - Summative assessment at end of EKG course - randomized to:
- Traditional lectures
- Small group peer teaching
- Cohort 2 - summer 2009 - formative assessment at end of EKG course - randomized to:
- Traditional lectures
- Small group peer teaching
In addition to a test before and after the EKG course, each cohort had an unannounced test 8 weeks after the EKG course.
Thus the study was designed to look at the impact of 2 types of learning (traditional lecture vs. small group peer learning) and the impact of a high stakes summative examination on the learning.
They found that in the Cohort 2 where there was only a formative assessment at the end of the course (performance did not count towards grades) the small group peer learning group did better at the immediate post test and also retained the information better at the 8 week surprise test.
On the other hand in Cohort 1, the high stakes summative test seemed to eliminate all the differences between the 2 groups.
The question:
What is it about the summative test that improved learning and retention? Did it force students to spend more time in self-study preparing for the test or did it force them to concentrate better at the lectures, taking notes etc?
Reading this study made me think about how this might apply to even higher-stakes summative tests like the USMLE Step 1. Scores from this one test can determine a student's career.
One student who had spent the first 2 years in small group PBL sessions, told me that as he was preparing for the USMLE Step 1 at the end of year 2, everything seemed to fall in place. We discussed this further and it seemed that having to read all the different subjects in a concentrated time span helped him to develop mental connections between various pieces of information in his memory thus improving his understanding.
So what do you think? What is the role of high stakes summative tests in medical education? Are they good, bad, a waste of time, a distraction, or something that we should have more of?
Subscribe to:
Posts (Atom)
Posts
