Answering the Big Research Questions

In this post, we’ll dive into two elements that are important to a successful product test. First, the test needs to deliver a clear answer to the underlying research question(s), e.g. “is my product better (or not) than the competition?” Secondly, the research needs to answer the question “why?” – e.g. “why is my product better (or not) than the competition?”

Understanding Action Standards and KPIs

While there are a lot of factors that come into play in successfully answering a research question (all those factors that make for “good research”), here we focus on one key element: action standards. Action standards are predetermined criteria that define the level of performance required for a product to meet specific business objectives. These standards serve as benchmarks against which the results of the research are evaluated. Action standards can vary depending on factors such as the stage of product development, target market, and business strategy. They help decision-makers determine whether a product meets the necessary criteria to progress to the next stage of development, launch into the market, or undergo further refinement. They are critical for decision-making and for building an effective research study.

In well-designed product tests, the action standards match up against key performance indicators (KPIs) to make informed business decisions after the research is completed. Common KPIs in early-stage product testing include overall liking, preference, purchase interest, and alignment with consumer expectations. Only at later stages do financial metrics, such as sales volume projections, become relevant. Based on our extensive database of early-stage product tests and our understanding of the most sensitive metrics, we can suggest numeric benchmarks that can serve as reference points for what constitutes a “good” score. However, we always recommend that clients refer to their own history of product tests when it comes to identifying action standards, or engage with us in mining their historical data to develop the best, product category-specific KPIs and benchmarks.

That said, it is not enough to determine whether the product does or does not meet action standards. The question is, why? What about the product contributed to its appeal? What attributes explain why it was less preferred? Insights into a product’s shortcomings guide product modifications, and insights into its strengths can suggest communication strategies when going to market. Here are three factors to consider that can enable answers to “why.”

Study design:

One of the most powerful ways for answering “why” questions is using an experimental design. In an experimental design, product features or attributes are systematically varied over several prototype formulations. For example, in the personal care category, this could mean samples of a mouthwash are varied in both flavor intensity and color. In a beverage, the product developer may have varied the level of a sweetener. Experimental designs can vary from the simple (only one factor is varied) to the complex (multiple factors), testing many variations of a product. P&K Research has conducted experimental design-based studies with as many as 20 product variants and more. In either case, an analysis of how a key performance indicator varies in response to the known differences in the products provides the most direct evidence for what is causing that variation.

In the case of experimental designs, it is the external information about the products (e.g. the level of fragrance in an air freshener) that helps drive insights. Another type of external information is that derived from trained sensory panels via descriptive analysis. Descriptive analysis quantifies the sensory product experience in an objective and reliable manner. The quantifications of sensory attributes (strength of fragrance, sweetness intensity, etc.) can be used to understand the basis for variations in the key performance measures.

Consumer questionnaire design:

The vast majority of product tests depend on answering the why question based on consumers’ answers to questions other than the key performance measures. These questions, often referred to as “diagnostic attributes,” are included to help elucidate the reasons consumers may be scoring a product high or low on a KPI. For example, in a personal care study on hand soap, the questionnaire may ask about the level of fragrance – was it too high, too low or just right? Additional questions may ask about the performance of the hand soap – how effective was it in removing grease? Obviously, the insights gained from this approach to answering “why” are only available if the right questions have been included in the consumer survey. This is where experience with testing in a product category plays an important role. One of the advantages P&K has to offer its clients is that in the thousands of products tests we have conducted, we have touched almost every category in food, beverage and personal care, with additional experience in many categories related to home care and consumer health.

Analytics:

Arriving at answers to the why question requires analytics that are tailor-made to uncover the relationships between a KPI and product features or attributes. Collectively, these analytics are referred to as “driver analyses” – since their purpose is to uncover what is driving variations in a KPI. There are many different kinds of drivers analyses, and they include some highly statistical approaches suited for the analyses of experimental designs or for correlating sensory descriptive analyses with KPI responses. At P&K, we distinguish between “micro” and “macro” level analyses. A “micro” analysis comprises analytics that are focused on a single product and aim to uncover what attributes account for high versus low scores on a KPI. “Macro” analyses compare the performance on a KPI across two or more products. P&K’s proprietary Variable Influence Analysis (VIA) is a macro level analytic approach that identifies those attributes that best explain the difference between two (or more) products, for example isolating those characteristics needed to close the gap between a client’s product and a competitor’s. Many of our clients have found VIA extremely useful and have selected VIA as a standard deliverable in our reports.

To wrap up, product tests need to be designed to inform business decisions. Clear and sensitive action standards serve an important purpose and are key to moving an organization forward following their research investment. Win or lose by virtue of those action standards, clients importantly also require an understanding as to why the product performed the way it did. Only with those insights into a consumer’s product experience can a client brand move forward with confidence in their product development and marketing efforts.

Considerations When Conducting Plant-Based Product Research

Consumers are increasingly interested in plant-based alternatives to meat and dairy products. The reasons vary such as reducing fat intake or for other perceived health benefits, for a concern with the environmental impact of animal agriculture, or for a variety of other personal reasons.

While the easy assumption is that vegans and vegetarians are the primary market for plant-based products, the reality is that most plant-based product consumers are actually “flexitarians” – people who are looking to reduce but not eliminate their meat and dairy consumption and who are open to trying plant-based alternatives. Let’s look at the considerations that plant-based producers should keep in mind when embarking on a research effort.

Proper representations

One of the key considerations for consumer testing of plant-based products is to ensure that the target audience is accurately represented. As the market grows, so too does the audience willing to consider these products. If your goal is a broad appeal, this means focusing on flexitarians rather than strictly vegetarians or vegans, as these groups may already be committed to plant-based diets and may not be as receptive to trying new products.

To effectively target flexitarians, it’s important to understand their motivations and concerns. These may include:

• A desire to reduce their environmental impact.
• A focus on improved health.
• Supporting animal welfare.
• A mix of some or all of the above.

Defining “Good”

Plant-based products are becoming a crowded category. With that competition comes the challenge of winning audience share among increasingly name-brand products. Ensuring the products themselves are appealing and satisfying to consumers becomes critical for repeat purchase consideration and establishing preference. Where does this effort start? It begins with conducting extensive taste testing and gathering feedback on texture, flavor, and overall satisfaction.

Proper testing involves a process of defining which characteristics to measure, how and why. Is the goal of a -alternative to mimic as closely as possible the meat-like experience? If so, attributes such as texture, flavor, and overall mouthfeel become paramount. For plant-based meat products such as ground beef, the focus on cookability may become equally important as the sensory considerations. Note that vegans and vegetarians may not be looking for a meat-like experience, which they have avoided — another reason to carefully define the consumer target for the product in question.

In addition to optimizing key attributes, it can also be helpful to use a competitive benchmark or normative score to assess the overall performance of a plant-based product. This can help to provide a frame of reference and allow companies to see how their products stack up against others in the market. An animal protein-based benchmark (dairy milk, beef, etc.) can also be helpful in establishing the gap, if any, in the product experience between the plant-based alternative and its animal protein-based version.

Proper Research Design

There’s a saying, “don’t try to boil the ocean.” In research, that means focusing on a controllable number of objectives to ensure your data is useful and actionable. Combining too many objectives into one piece of research typically doesn’t serve any of the objectives well. For example, often our clients want to compare several prototypes and in-market plant-based products (possibly a direct competitor), as well compare prototype performance against an animal-protein based benchmark. Our recommended practice is to split those two pieces of research, first conducting the research to select the best plant-based prototype and to gauge performance against other in-market plant-based products, followed separately by a comparison between the best performing prototype and an animal protein benchmark.

When it comes to selecting your research partner, experience in the category matters. Consider some of the things our experience conducting research in the category has taught us:

• For plant-based meats prepared at home, you need to understand the cues the person cooking uses to judge done-ness. Over or undercooking can seriously impact the experience.
• Cooking instructions need to be easily understood, and because consumers often don’t follow printed instructions, it is best if the preparation mimics that of the meat products which the consumer is accustomed to preparing.
• For plant-based cheese, the melting performance is important for hot preparation.
• For plant-based milks, there is no preparation, but their performance needs to be gauged in their use context. Are they consumed by themselves, or added to a beverage, or to breakfast cereal?

The Takeaway

Plant-based producers should keep in mind these three principles for insightful and actionable research in the category:

• Focus on proper representation of your target audience.
• Define what you’re measuring (taste, texture, cooking experience, etc.).
• Ensure proper research design.

These factors will guide your development efforts in achieving appealing and satisfying products that can win market share in an increasingly crowded market.