- "Another feature about written assessments that the class talked about was the fact that narrative assessment is a permanent record, hence the difficulty and reluctance with providing modifying feedback and why most written feedback is focused on strengths, even if the assessment is formative".
- "The faculty mentioned the need to balance the narrative assessment with face-to-face dialogue with trainees especially with regards to areas for improvement".
Monday, April 1, 2013
Using Narrative Assessment Methods to Give Feedback
I spoke with Dr. Pien and she mentioned a couple areas of narrative feedback about which there was much discussion. See below for possible stimuli for discussion:
Subscribe to:
Post Comments (Atom)
Posts
14 comments:
Kathy Baker
I believe that anyone in an educational related roll, mentor, or administrator wants to see others succeed and to learn and grow. Narrative written feedback I feel is very beneficial and informative for the individual receiving it, as it is areas to strive to work for and to continue to excel in. With narrative feedback, it can be difficult and very time consuming when you are dealing with someone who might not be at par with others or the expectation that they should already have. I feel verbal feedback is essential and can be applied quickly, but making sure there is some sort of documentation is vital. A lot of times it can be thought of that something that is not written and documented never occurred. Feedback can be tough, but it is important for growth and advancement of others. The truth is tough to bear at times, but wouldn't you feel better knowing earlier on that you are making mistakes or you need to refocus on the big picture, as opposed to waiting until graduation or certification and find out you are unable to complete it due to your behavior or experience? I think Lily highlighted some important narrative feedback methods that people use and showed some decent examples and some not so decent. How many times do we find ourselves using the not so decent methods of evaluation?
There are challenges and limitations to written assessment that likely contribute to the general reluctance of teachers to engage. 1) It takes time. In the current medical climate, time is a limited commodity. Filling out evaluations is not a billable activity. Finding time to complete a carefully worded and sufficiently detailed written feedback form is a challenge. 2) It is difficult to always put into words what you want to say. The written evaluation does not as easily afford the learner an opportunity to explain her or his behavior. Since the feedback often gets to the learner after a significant time interval after the observed behaviors, the learner may not even remember the event being discussed in the evaluation. It is hard to gauge whether the learner understands what is being said and whether the feedback is being clearly communicated in a way that is understood by the learner. As teachers, we may be clear what we want to say but whether we actually accomplish this or not is a different story. 3) There is also a fear of medicolegal or other negative fallout from submitting such evaluations. Written feedback is a permanent record. We’ve all heard horror stories, which fortunately are rare, about medicolegal action or punitive actions taken by the learner in response to negative feedback. No one wants to deal with a resident who starts crying or gets angry in response to feedback. A few years ago, a well known and generally beloved pathologist, who was a residency program director, was killed by a disgruntled resident over an evaluation. It causes one to think twice about giving negative feedback on paper.
Higher education is an educational level that follows a completion of a school providing a secondary education, such as a high school, secondary school, or gymnasium. Tertiary education is normally taken to include undergraduate and postgraduate education, as well as vocational education and training. Colleges, universities, and institutes of technology are the main institutions that provide tertiary education (sometimes known collectively as tertiary institutions).to know more Visit Here.
David Wheeler
It is essential that we create a narrative assessment that provides the student with an opportunity for growth, where it is needed, and an affirmation of skill or knowledge where it exists. The narrative assessment must be an authentic observation of the students’ performance or more specifically, discrete slices or dimensions of that performance. I think it is helpful to view a narrative assessment in terms of "coaching" the learner. In the initial phase we are examining in a step-by-step basis their performance of an essential task or skill-set as they grow in both knowledge and ability. We then step back and form a narrative based on the multidimensional performance of knowledge and skill. As an educator one must develop competency both as a formative and a summative creator of narrative assessment. In the formative assessments we look at raw performance and help the learner improve. In the summative aspect of our assessment you must help our learner think critically when making decisions and judgments. A well crafted narrative assessment should obviously be based on our direct experiences and observations with the students. We should strive to individualize the narrative and focus on skill/behavior and offer suggestions for growth. The best narrative would be crafted over time and illustrate the "story” of the learner's growth in a chronological fashion.
Maged Argalious
In the article by Smither and Walker on narrative comments, the authors found that unfavorable feedback may not necessarily lead to improvement in behavior. They also cited prior work that suggests that a large amount of negative feedback could cause recipients to abondon or lower their goals or direct their attention elsewhere. While these findings may be discouraging, the paper by Dudek et al offers a roadmap on how to write meaningful comments. The authors proposed interventions aimed at assisting supervisors at writing better comments and provided a roadmap to help accomplish this goal (table 1).
Some of these important features include focusing on strengths as well as weaknesses ( with specific examples for both), basing comments on behaviors rather than character, providing suggestions for improvement and writing the comments in a clear and supportive tone. Once we establish a pool of educators that are willing and capable of providing meaningful and constructive feedback, learners will hopefully perceive these comments as an opportunity for improvement rather than rejecting them altogether.
I
I agree with the direction in which Maged and David are headed. It is the responsibility of the educator to provide sound, written feedback to his/her students. The feedback should be timely and the two articles suggest guidelines for writing the most effective feedback. First, the narrative feedback should justify any quantitative ratings, and therefore reinforce the quantitative ratings. Second, the narrative feedback should specify behaviors and provide examples, and then suggest methods to improve behaviors, if necessary. Finally, good feedback will provide an appropriate balance of positive and negative feedback without overwhelming the student with negative comments. I liked the fact that the paper by Smither and Walker pointed out that too much negative feedback leads learners to abandon goals and avoid activities with which they are less comfortable, and this is the antithesis of what we as educators want to promote. Our job as educators is to enhance learning, and to assist in this we need to be courageous in providing fair and honest feedback, whether it be formative or summative. Further, we need to teach and model to our colleagues in education the best way to provide useful feedback to students. We can improve feedback self efficacy in both ourselves and others with effort, and this will ultimately help our students. BB
The overall purpose of Narrative Assessments is to help an individual improve performance. As teachers or supervisors we all want to help our students be successful. In order to help individuals do their best, both written and verbal feedback is necessary. An assessment at the end of a program can provide an individual with information about their overall performance. However, I feel that as we go along in our day to day activities, it is important to regularly give verbal feedback on performance to individuals, both positive comments and suggestions for improvement. I feel educators need to be observing, documenting and communicating to their students throughout the whole class, semester or year, whatever the time period may be. If there is ongoing communication between the teacher and student when written summative assessments are completed, the information or content provided should not be as much of a surprise to the student. In addition if observations and documentation are ongoing, it will assist in creating a more accurate and useful narrative assessment. I thought the slides on useful narrative assessments are helpful to keep in mind when writing individual narratives. The exercises in class illustrated how challenging it can be to provide concrete examples in a narrative assessment but also made me realize how important it is to provide in-depth, quality information in the assessments as students can use these as a reference and a learning tool for future improvement. As teachers and educators it is our responsibility to guide the students with accurate, honest and specific assessments.
Rachel Steines
Although I could not attend the last session due to kidney stones, and subsequently high volumes of painkillers, I did read up on the topics. From the beginning of the masters program, we have discussed in detail the importance of appropriate feedback for our students. Although the article suggested that negative feedback was sometimes not beneficial, it is often times necessary; However, this feedback should be useful and highlight beneficial, logical improvements with a respectfully direct reinforcement. I do believe this feedback would be most beneficial if exhibited in both a written and verbal format. Ongoing feedback through face to face communication will lead to a better prepared student, and this student in the end will have less vices to critique, leaving the written feedback to be encompass the core of the students abilities. I believe having an applicable balance between written and verbal feedback may be the key to appropriately preparing a student for the next stage in his or her career.
Completing written narratives is not only important for the student but also for the educator. It helps us organize our thoughts about students in a concise and usable manner. I think direct observation and interaction with the student is a vital component to student assessment and not taking others observations/input as factual. Well defined performance competencies is essential. Being descriptive, using examples and making sure you are discussing behaviors not personality are key points I will use. My concern is with the push for metrics in health care. Hopefully the narrative format will continue to give us a venue to evaluate and help improve our students. I too found the Smither article interesting in that unfavorable comments in large doses can result in discouragement and smaller doses of unfavorable comments results in improvement.I think many qualifiers exist in the feedback aspect of the narrative assessment and we discussed this previously in our giving feedback course.
I agree with Rich's comments regarding the time required to provide provide quality written assessments. I appreciated the discussion and class activities around providing specific feedback that reinforces the positive aspects and provides guidance on elements where improvements should occur. So many assessments that I've seen compliment the personality of the individual instead of calling out specific actions or behaviors that we want to see continue. As Rachel mentioned, providing guidance around negative feedback is important. Keeping the individual in mind as the audience receiving the feedback is helpful in driving improvements in performance. The suggestion of phrasing the written summary to the person instead of writing their name is a novel way of preparing these statements. I concur with Brian that this is the responsibility of educators to be courageous and provide fair and honest feedback. We want the individual to find the feedback meaningful and to not overwhelm them with too many items to fix. Selecting the top three themes and starting there will be the most helpful.
I agree with what has been written by many of the members of the class here. It takes time which we have little of. It takes development and training to do well which requires even more time. It takes the knowledge in what the goals are for the area being evaluated which as we learned last time in competence based education is a separate training arena altogether. What is needed is a way to generate meaningful narrative feedback without the need fro intense time, or faculty development of open ended written comments.
Faculty continue to struggle with providing written honest constructive feedback to residents and fellows. The fairly significant barriers are made of a combination of ignorance of education principles, lack of time and effort, a hightened sensitivity to the perceived effects of the comments, and a socio-cultural environment that encourages the respect of the privacy of others. I am sure that I forgot a few other reasons. Outside of the staff/resident-fellow relationship world, providing direct feedback is well established and accepted by employers/supervisors and by employees/subordinates. The responsibilities/job description of the latter makes it possible for the former to evaluate and quantify/qualify the level of performance. The GME world is following in that path, and the definition of the competencies, and now the milestones in each of these competencies will allow the evaluation part of the process to be completed more easily. Remains the harder part - How to convey the message? Compliments and achievements are easier to communicate. Deficiencies, not quite. One has to practice. First in clinic; a little at a time. Make notes, write down examples; suggest ways to improve on performance and correct mistakes; accumulate notes from multiple encounters and construct a balanced narrative evaluation. Yes - A lot of work! With practice, the process becomes easier and probably second nature, especially if it it reinforced by tangible results and appreciation from the recipients. Practice makes perfect.
Matt Celmar said...
A key point that Elias touched on is “a balanced narrative evaluation” in your feedback. One of the difficulties I have with feedback is being able to observe a trainee in high yield situations in order to provide a balance narrative evaluation. Throughout the day, I have no less than two students that I am directly responsible for to administer behavioral therapy to. What usually happens with verbal feedback is; I will work with my students along side of a trainee while they work with their students. I pay as much attention to their therapy as I am able to while attending to my students. I am sure that there are many subtle aspects of their therapy occurring, or not, that go undetected to my less than fully focused scope of concentration. Finding those in the moment, teachable, verbal feedback moments are difficult while managing students in a classroom. I also agree with many others that written feedback should concentrate on positives, modifications, and what to work on. I like to use the simple phrase “I would like to see you grow in…” to highlight areas of improvement and growth. I have referenced their previous areas of growth when writing future reviews. I think that it gives the trainee a very realistic progression of their development with their therapy. Verbal feedback for in the moment modifications that can be acted upon soon as opposed to, written feedback to show progression and thoughts for the future. Written feedback can always be analyzed and re-read for learning purposes.
Felecia,
Assessing need, evaluating and communicating feedback to an individual in training can be difficult. I believe the burden falls on the educator to communicate deficiencies in an individual/students learning processes. Giving humanistic reliable and acceptable feed back along with accountability measures are crucial components needed to fully engage the individual druing training. A folow up, constant monitoring with frequent updates on progress are needed to ensure success in an individuals training process. As most know, these measures can be hard to incorporate when there are numerous individuals to monitor. Sometimes five to ten minutes of constructive reviews with some is enough to them on track.
Post a Comment