- Tests only medical knowledge - not the other competencies that go towards making a "good physician"
- Forces students to memorize content just to regurgitate it at exam time
- Most physicians may not need to use the content memorized for this test in later life.
- Biomedical science information is growing so fast that there is no way to cover all this content within the span of an average medical school curriculum
Medical Education recently published a very interesting study [pdf; abstract] regarding year 4 medical students in Germany learning EKG interpretation.
Summary of the study:
- Cohort 1 - winter 2008-09 - Summative assessment at end of EKG course - randomized to:
- Traditional lectures
- Small group peer teaching
- Cohort 2 - summer 2009 - formative assessment at end of EKG course - randomized to:
- Traditional lectures
- Small group peer teaching
In addition to a test before and after the EKG course, each cohort had an unannounced test 8 weeks after the EKG course.
Thus the study was designed to look at the impact of 2 types of learning (traditional lecture vs. small group peer learning) and the impact of a high stakes summative examination on the learning.
They found that in the Cohort 2 where there was only a formative assessment at the end of the course (performance did not count towards grades) the small group peer learning group did better at the immediate post test and also retained the information better at the 8 week surprise test.
On the other hand in Cohort 1, the high stakes summative test seemed to eliminate all the differences between the 2 groups.
The question:
What is it about the summative test that improved learning and retention? Did it force students to spend more time in self-study preparing for the test or did it force them to concentrate better at the lectures, taking notes etc?
Reading this study made me think about how this might apply to even higher-stakes summative tests like the USMLE Step 1. Scores from this one test can determine a student's career.
One student who had spent the first 2 years in small group PBL sessions, told me that as he was preparing for the USMLE Step 1 at the end of year 2, everything seemed to fall in place. We discussed this further and it seemed that having to read all the different subjects in a concentrated time span helped him to develop mental connections between various pieces of information in his memory thus improving his understanding.
So what do you think? What is the role of high stakes summative tests in medical education? Are they good, bad, a waste of time, a distraction, or something that we should have more of?
Posts
7 comments:
Kind of a catch 22. Keeping the USMLE as a high-stakes examination may force students to read and thus develop a deeper understanding of the science.
But we should train program directors to ignore the USMLE results when reviewing applications for residency programs - which would mean it would not remain a high-stakes examination
Since "time on task" is one of the best predictors of success, time spent studying for an exam may increase time on task and hence lead to successful test taking. Now we have to be clear about how we want our students spending their time. I remember my doctoral "major exams". I needed to prepare to write for two days on questions supplied by my teachers as a means to demonstrate that I could integrate what I'd learned in my program and could "think systematically" about unique problems. Studying for that "high stakes test" was very beneficial (although I thought it was barbaric at the time). Medical student, like doctoral students need to be motivated to think across topics. High stakes examinations can provide that motivation.
As a program director, the USMLE has only one real use - it helps you identify who can study for a high stakes exam and who can't. The problem is that most subspecialties use high stakes summitive exams for board certification, so if you get someone with high USMLEs you know they will pass thier boards and you can spend more time focusing your curriculum on the aspects of clinical practice that are really important. If you recruit residents with low USMLE scores you will need to focus more of your curriculum on passing the boards - and the residents will actually have to figure out how to "practice medicine" after they graduate.
Bottom line - our education system in the US is broken - we run people from one high stakes exam to another SAT - MCAT - USMLE 1,2,3 - Boards --- and then we wonder why our physicians never "grow up" and become life long learners. We continue to cling to these high stakes, nationally standardized exams because no one "fails" out of school any more and diplomas are given for "attendance." If our education system actually gave up on being politically correct, and a dipolma actually meant you met a standard, we could move beyone these exams.
Program directors are willing to ignore USMLE scores if a candidate is from a competitive medical school with a reputation for excellence- but these are few and far between.
As a program director, the USMLE has only one real use - it helps you identify who can study for a high stakes exam and who can't. The problem is that most subspecialties use high stakes summitive exams for board certification, so if you get someone with high USMLEs you know they will pass thier boards and you can spend more time focusing your curriculum on the aspects of clinical practice that are really important. If you recruit residents with low USMLE scores you will need to focus more of your curriculum on passing the boards - and the residents will actually have to figure out how to "practice medicine" after they graduate.
Bottom line - our education system in the US is broken - we run people from one high stakes exam to another SAT - MCAT - USMLE 1,2,3 - Boards --- and then we wonder why our physicians never "grow up" and become life long learners. We continue to cling to these high stakes, nationally standardized exams because no one "fails" out of school any more and diplomas are given for "attendance." If our education system actually gave up on being politically correct, and a dipolma actually meant you met a standard, we could move beyone these exams.
Program directors are willing to ignore USMLE scores if a candidate is from a competitive medical school with a reputation for excellence- but these are few and far between.
The most interesting aspect of this study appears to be that the high stakes exam somehow brought the lecture group up to the level of the small group, and aided with retention. It is almost as if studying for the high stakes exam acted as a substitute for small group work, presumably the process component of small group work. However, 8 weeks of retention is not that long - an assessment a year out might see the lecture group return to baseline (or both groups).
The Services For Better Education, read more free http://goo.gl/fHE2
The Services For Better Education in Indonesia, read more free http://goo.gl/fHE2
Post a Comment