Download "Top 10 Mistakes in Market Research"

Lessons from Dr. Frankenstein on product testing


All Dr. Frankenstein wanted to do was create a human being. His goal wasn’t so different from that of the Marketer who just wants to mimic the “current product” with a “reformulation”. But oh my, Dr. Frankenstein’s “reformulation” didn’t work out so well. What will you do to ensure a better outcome, when your Brand’s product has to be reformulated – maybe an ingredient is no longer available, or business conditions mandate a cost reduction. How will you ensure that the new formula is an acceptable replacement for the current one? Here’s a quick checklist.

How will you research this?

  1. Begin with the end in mind. Gain alignment regarding what your Brand Team will do if the new formulation fails to meet the Action Standards.
  1. Start simple. Consider starting with a triangle discrimination test if that is possible for your product. This sensory test may be enough in some situations, and it may be a good first step if there are several reformulation options to evaluate.
  1. Be rigorous. Brand Teams often test the current formula and the new formula in a paired comparison test. There are several versions, but often a respondent uses and rates both products, one after the other, and then states a preference on attributes that might be affected by the reformulation or are key drivers of the product. The rest of these steps are about paired comparison testing.
  1. Test among users of the product. They’re the ones you seek to keep in the franchise.

What will you test?

  1. Mimic what’s on shelf as closely as possible. This means:
    1. Produce the current and new formulas under similar conditions, ideally plant-run.
    2. “Age” both products so they are more or less like what consumers buy (not super fresh).
    3. Give respondents the full size in normal packaging (not a “blind” study).
    4. Let them use each product multiple times in normal situations.

What will you ask?

  1. Set Action Standards. Typical Action Standards for a paired comparison test are: purchase interest, overall liking, alienation and a few key diagnostic measures. The reformulation has to be at parity to or better than current formula to be implemented. Too many Action Standards guarantees the new formula will “fail” on at least one.
  1. Focus. Keep the surveys short. Your respondents are completing 3 surveys: an initial screener, and then one after using each of the two products, that’s a lot! Focus on key drivers of product performance.
  1. Don’t add a concept. It is irrelevant here and chews up survey time.

What will you do with the results?

  1. Plan your analysis upfront: Test with enough respondents to assure a fairly reliable read on the results. Consider Reformulation Opportunity Analysis as an aid to interpretation of results and guide to next steps if the reformulation fails. Use the right stats – you are concerned about Type 2 error here, not Type 1. Beware “creep”: a product may degrade by tiny amounts in each reformulation, until eventually you have a product that is quite inferior to the original formulation.

The nature of the testing you do depends on what’s driving the call for product testing. This post is about testing a reformulation where the goal is to mimic the current product. The research is different if you are testing a product improvement, or a totally new product. Nufer Marketing Research has conducted a gazillion product tests, across many categories; please contact me for a chat on any aspect of product testing. Or to chat about monsters, that’s fine too.

Leave a reply