Over the last ten years, we have enjoyed facilitating racial/ethnic climate surveys for many Christian organizations. We have discovered (the hard way) a few things that typically lead to failure and frustration:
1. Don’t collect data
The easiest way to fail at doing a climate survey is not doing it at all. Many Christian organizations express a desire to become more racially/ethnically diverse, but very few are collecting climate data to assess dynamics in their organization and barriers to progress. As a result, leaders often feel that dynamics are “pretty good” and “getting better”. When they collect climate data, they often discover that things aren’t as rosy as they thought. Climate surveys allow leaders to go beyond their own (often limited) view of their organization and gain a more objective understanding of the realities and barriers.
2. Don’t use the data you collect
We often hear comments like, “We did a climate survey three years ago, but I’m not sure what, if anything, we did with that data.” When conducting a climate survey, it is essential to begin with the end in mind and answer questions like:
- Who will analyze and report on the data?
- Who could benefit from access to the data?
- How will we apply what we learn?
3. Use an inadequate survey system
There are many online systems that make it possible to create a survey. Collecting data is typically the easiest part of the process. Making sense out the data is much more complex. Does the system make it easy to do complex reporting, comparisons, segmentation, etc.? Can you export your raw data for archiving or analysis in SPSS or other software? It is frustrating to find out the limitations of a survey system after the data is already collected and it is too late to make any changes.
4. Advertise poorly
We are all inundated by requests to “provide your feedback.” It is important for leaders to think through ways to overcome survey fatigue and help their members to see the importance of sharing their input. While the survey is being conducted, it is important to monitor response rates by various groups (e.g. students, faculty, staff) to make sure there is good representation. If groups are underrepresented in the data, it is a good idea to do targeted advertising to help bring up the response rates.
5. Don’t ask the right questions
The topic of racial/ethnic climate is enormous. There are thousands of questions that could be asked on a climate survey. Narrowing it down to the most effective questions is probably the most important, difficult, and time consuming part of the process. We worked with many leaders and experts across the country for over a year to select the questions on our climate survey. If you try to put together a survey in a few hours it is unlikely that it will ask the right questions.
6. Make it too long
Because there is so much helpful information that could be collected, it’s tempting to want to include everything (and the kitchen sink) in the survey. But, then you run into another big issue–people quitting after starting the survey because it takes too long to complete. It is tricky to find a balance between a survey that is comprehensive enough to collect key information and short enough to not drive participants away. We’ve found that 15 minutes is the maximum amount of time that a survey should take to complete.
7. Don’t use comparative data
The point of conducting a climate survey is to determine areas of strength and weakness and then use that information to lead effective change in the organization. But, it is often a big challenge to determine how good or bad dynamics are in the organization if leaders having nothing to compare it to. Let’s say a Christian college scores a 3 out of 10 on one item in the survey. The leaders will wonder, “Is this a big deal?” Comparative data can help to answer that question. If many other Christian colleges scored an average of 6 on that same item, then it is a “big deal” and it would be wise for leaders to consider why their campus’ score is so low relative to other Christian colleges.
We have learned to be very intentional about avoiding the “deadly seven” above with our climate surveys. We hope you can learn from our pain and avoid making these mistakes.