By David Weisser
‘Human capital management’ is corporate-speak for an organization’s collective effort to manage its people’s activities. Survey research plays an important role in today’s human capital management. This article is focused on bias in organizational research, and how to avoid it.
Most successful businesses engage some form of survey research within their enterprise. Every large employer wants to reduce employee turnover. Attitude surveys identify what perceptions most contribute to an employee’s intent to stay. Research programs are designed to drive innovation in customer service. They are used to identify personality traits associated with a company’s top-talent, investigate communication strategies that most successfully promote them, and the list goes on.
Of course, capitalizing on knowledge gained through research depends ultimately upon data quality. The greatest threat to data quality is bias, in one of its many forms. So, what is bias? Where does it come from? And how can you escape it?
Bias is hard to define, but you know it when you see it. Bias is systematic. When a patient satisfaction survey neglects to ask items regarding the amount of time spent in the waiting room versus spent with the doctor, it is likely to yield skewed results. If an interviewer smiles and consistently shakes his head yes while asking questions, the survey is likely to yield fewer ‘No’s’ than it might otherwise.
These examples are obvious, but bias is usually far more nonchalant. Despite the best intentions of consultants and program managers, bias creeps into research programs. It’s more often not the deliberate mischaracterization of findings, or the creation of self-serving survey questions. On the contrary, it’s typically unintentional, yet remains an ever-present threat to quality.
Consider a patient satisfaction survey administered by a clinic’s office manager. The intention of the survey is to utilize patient feedback to improve service models – aid efficiency, reduce cost, increase patient retention, increase referrals and most of all improve the quality of care.
When the staff finishes the medical requirements of the patients’ visits, the office manager encourages the patients to provide experiential feedback in the form of a survey. But, when the office gets busy, the survey is the last thing on anyone’s mind. And the harder it is to meet patient needs, the less satisfied they become. These are precisely the patients we want to survey, because when the office is slammed, quality service is at its most vulnerable. And these patients experience it first hand, so they are in position to help the office learn how best to mitigate the effects on service. However they won’t complete a survey before they leave the office.
[youtube]http://www.youtube.com/watch?v=9SzJt-kg_ak[/youtube]
Whatever results are ultimately compiled will yield inflated service ratings. The design for data collection is biased in that it will, systematically, exclude less favorable responses. Service ratings will be higher than the service deserves. And, the clinic’s director won’t know how her clientele feels about the practice – until they decide to place their care in the hands of a ‘better’ provider. To expand this example further, note that the clinic’s staff has a direct short-term incentive to report only the high service rating they receive, as they may fear some type of retribution – no bonus, or termination – for receiving low ratings. The unbiased reader knows however that it is in the entire clinic’s long-term interest to analyze and act upon the more critical, more constructive, survey results.
This example depicts one of a million ways that bias can destroy a survey’s usefulness. So, how can managers and business professional’s best avoid introducing bias into their research efforts? Most influences of bias can be managed out of the project before the first survey is sent into the field. Consider these tips:
1) Hire an outside consultant.
The goal of research is to obtain knowledge. It is nearly impossible for someone who is close to a customer, or an employee, or a company program, to remain objective about it. A researcher’s independence is vital to a program’s success. I know you’re thinking, ‘Consultant, eh? Sounds expensive.’ I encounter this response every day. Many organizations choose to handle such projects internally, but soon realize that the time commitment required to do it right is more than they expected. So, they choose to do it quickly, and you guessed it: data quality suffers. Often, they end up doing more harm than good.
An outside consultant is free to provide stark opinions, feedback and criticism without fear of upsetting the status quo. He or she can promote the inclusion of survey items that serve the interest of knowledge and action, rather than the interests of inter-organizational cliques. Why let a plumber do your taxes when accountants are freely available? Trust me. Hiring a professional is a good idea.
2) Hire a ‘good’ outside consultant
Business moves fast. But, research moves slowly. Remember the last time you walked through a dark room feeling for the light switch? We conduct research precisely because we don’t what we’re looking for. Consulting is the act of drawing insight from research at the speed of business. That means, in addition to being well-educated and tactically skilled enough to handle software and statistics, he or she must also be well-spoken, well-credentialed and highly presentable. A scientist and a ‘people-person’ is worth his or her weight in gold.
3) Develop sound content
Take a 360-degree approach to content development. Get as much input as possible before putting a survey in front of all your employees or patrons. Leadership has primary accountability for the survey’s results. When they are comfortable with it, ‘pilot’ the survey with a handful of recipients. Or two handfuls or more. Encourage them to discuss the entire survey, its content, process and purpose. Ask them questions like, ‘What were you thinking about when you answered that question?’ Take notes, and encourage criticism. You, your respondents and your stakeholders will be far more pleased with the final result.
4) Commit to Transparency, Openness and Honesty
Before the survey process gets underway, it is important to make known to all stakeholders that it’s coming and what the timeline milestones entail. Explain the goals. Encourage input from all parties. Make sure leadership avails themselves to answer questions and comments as they arise. The best way to encourage honest survey responses is for the survey sponsor to make an honest appeal for them.
Oh yes, and introduce your consultant, and his or her firm. Be sure he or she is equally available to answer questions and concerns. Generally, people are more likely to provide open and honest, critical feedback to an outsider, rather than directly to their boss, or coworkers. This is especially true if anonymity of responses can be guaranteed (a much easier sale to make if using an outside consulting firm). It will also make it easier to report and present all results – not just favorable ones.
5) Take action on results
Opportunity for targeted action is the ultimate prize of research. No one wants to end up with a bunch of ‘nice-to-knows’. They want to know how their feedback will help you make their worlds a better place. And if their happy, you’re happy so it’s a win-win scenario.
First, publicly present the results. Encourage dialog about everything, the good, the bad and the ugly, the process the results and next steps. Then, plan and make adjustments and improvements based on the resulting discussions. Doing nothing indicates to your stakeholders that their feedback is unimportant. How rude!
Finally, and most-importantly, re-measure. If you’re interested in organizational change, and a consulting firm encourages a one-time survey, it’s time to find a new consultant. How would you measure progress without a baseline for comparison? And the best baseline for comparison is you’re last measurement.
The five steps listed above are discussed at a high level. But, it’s our belief that careful consideration of these components will help to ensure unbiased results in your research endeavors. And you might end up with a better business to boot.
About the Author: David currently serves as President and CEO of SpringBolt Survey & Consulting, Inc., an organization dedicated to bringing the power of survey research to the small business, not-for-profit, and religious institution – expertly and affordably. Visit them on the web at
springboltconsulting.com
.
Source:
isnare.com
Permanent Link:
isnare.com/?aid=455245&ca=Business
