Download "Top 10 Mistakes in Market Research"

10 Steps of CSI Analyses – What is that statistician doing?!?

girl doing chemistry Is there a business out there that is not listening to the Voice of their Customer, and using that knowledge to guide improvements? Probably not! But how do you know what actions to take?  It’s crucial to allocate resources efficiently, so you want to know where to focus your efforts!  Sure, you can conduct a Customer Satisfaction study, look at the attributes where performance is poorest, and focus on improving there – but that may not be the wisest course of action.  Shrewd use of advanced analytics pays off. But how does your statistician come up with her recommendations? It can seem pretty mysterious! This blog post gives you a snapshot of the process.

Let’s learn from the best.  JD Power has been a leader for decades in providing companies with clear direction they can use to focus their improvement efforts efficiently.  We’ve cajoled Mark Rees, the statistician who built the JD Power CSI (Customer Satisfaction Index), to share his thoughts with us on how to create a solid CSI measure, and how to use the results of your Customer Satisfaction surveys.  For a full treatment, see his white paper on the JD Power CSI. You might also want to see our white paper on Loyalty Measurement. If you’d like to know more about the CSI process, and how you can use it to direct your improvements, contact Mark, or me.  My firm has done this work for many clients, and I’d love to see what we can do for you!

This is an abbreviated checklist of what the skilled statistician is doing – the reality is more complex, and we tailor it to each business.

Your basic ingredients are:

1) You need an anchor You’ll want to ask customers about their overall rating of your business. This might be a performance-based scale (excellent to poor), a satisfaction- based scale (satisfied/dissatisfied) or a NPS scale (likely/not likely to recommend) or all 3.   I’ll call it Satisfaction for simplicity.

2) And a battery of attributes You will have a battery of attributes, let’s say about 50, that describe what your product delivers. Building this set of attributes well is crucial (garbage in, garbage out).

3) …And a lot of respondents. Survey a robust number of respondents – several hundred at least – sometimes over a 1,000 are needed. Once you have your data….

Find out what matters:

4) Run a factor analysis to sort the attributes into groups (“Factors”). This output says that all the attributes in the Factor are pretty much measuring the same thing (highly correlated). There are many steps in the process that I am glossing over (rotating to remove multicollinearity is one). Factor analysis reduces your ~ 50 attributes down to a small number of Factors (could be 5 to 10, for example).

5) Run a regression analysis on the Factors against your measure of Satisfaction. The output is “importance” weights for each Factor.

6) Apply the “weights” to the Factors, and to the individual attributes. Some more manipulation is likely to be required but basically the output is something that tells us out of all your attributes, which ones are really driving Satisfaction?

Chart a plan for improvement:

7) Set a target for each factor and each attribute. This might be the ratings received by your top competitor, or your top performing retail branch, or some other rule. We advise on how to set those targets.

8) Define the gaps between where you are, and your target. Weighted gaps are created: the gap is the difference between your score on a Factor and the target you set for that Factor, multiplied by its Factor weight. You drill down and repeat this process at to the individual attribute level. Result: a weighted gap for each Factor and attribute.

9) Prioritize, based on the size of those gaps. You put those weighted gaps in order from largest to smallest, and you have a rough priority plan: the Factor with the largest gap is one you’ll want to look at carefully to see if you can improve – because improvements on that Factor’s attributes will have the biggest impact on Satisfaction.

10) Operationalize. How do you know exactly what to do to improve? You’ll need “diagnostics” on the specific attributes you’ve targeted to improve.  This is a separate study. Example: if the attribute you want to improve is, “ wait time between calling for a service appointment and bringing the car in”, then you have to know, how much time is too long a wait? By asking customers how long they had to wait for an appointment, and asking their satisfaction, you can figure out that waiting 2 days is the max – any longer, and satisfaction plummets.  Now you have a specific target to work against: get that wait time down under 3 days.

And there you have it.

Leave a reply