Multi-Rater Feedback: A Few Thoughts
We are often asked to help companies implement multi-rater (aka 360-degree) feedback. This is a process where a person rates him/herself on a set of competencies along with his/her direct reports, boss, peers, and sometimes customers. Theoretically, seeing information from each of these perspectives provides the person with the insights required to improve. Before implementing a multi-rater feedback process (MRFP), you should consider four issues.
1. Insight, while necessary, is not sufficient to bring about change. You can’t have a successful
MRFP without post-feedback development opportunities available and a way to measure
changes in behavior.
2. There are two serious methodological flaws with most MRFPs that you need to address.
Among them are:
- Do You Really Know What You Are Measuring? No matter how well thought out your competency model is, or how behavioral your items are, most MRFPs can be boiled down into 2-3 elements of performance: Halo, Management/Leadership, and Technical Skills. Halo is when a person who is well liked get high ratings in a lot of areas because s/he is well liked (the opposite can also occur). What’s the best way to avoid halo and measure more distinct competencies? Ironically, use fewer items. This reduces rater fatigue and allows them to focus more on key elements of a person’s behavior.
- What Do You Get When You Mix All Of the Colors Of the Rainbow? Gray. Since we love bottom line numbers so much we like to give an overall score on MRFP dimensions (usually an average of the different rating groups). However, by doing so you are killing the best part of your data. For example, by averaging Bob’s delegation scores, you get a meaningless number that masks the most important ratings (his direct reports who know best whether or not Bob delegates effectively).
3. Be clear on the goals for your process. For example, is it to develop your top talent or
increase the skill level of oorer performers? The ugly secret of doing assessment for training
and development purposes is that it tends to benefit those who, arguably, need it the least.
Top performers get better while the lower performers view the process as a reminder of their
flaws. For poorer performers you should consider a more objective process that does not rely
on sources of data that have existing prejudices (real or perceived), such as an assessment
center.
4. Whatever you do, don’t make an MRFP part of your performance management system. It
ruins the objectivity of the ratings and focuses people on the process rather than using the
results.
To learn more about our MRFP programs contact Warren Bobrow.
Smart Choices Lead to High Payoffs
We want to acknowledge one of our clients for understanding the importance of investing time and effort in the selection process. When faced with the typical “excuses” of busy schedules, travel arrangements, and day-to-day work constraints, they took the high road. The task was to select a small number of management trainees for an international transportation company. Each candidate was pre-screened by phone and then had several individual and panel interviews. Using structured behavioral interviewing the client was able to paint a well-rounded picture of each candidate’s strengths and weaknesses before making the final choices. As a result, they selected talented individuals that will benefit from the company’s developmental opportunities and help our client prosper in the future. While some clients focus on the costs associated with selection and development, others focus on the payoffs which (as seen from the successful career paths of previous trainees) can be enormous.
If you are interested in improving the interviewing and/or selection process in your organization, please contact Kammy Haynes
For a PDF version of this newsletter, please click here. If you are unable to read pdf files, download the free Adobe Reader program.