Product Testing

Product tests are consumer studies designed to evaluate and diagnose product performance. Two precursors of consumer product testing are sensory testing and employee panels, which are noted here solely for reference:

Sensory Testing: used to assess changes in ingredients, formulae, ergonomics, or composition under controlled laboratory conditions. Sensory tests guide R&D in improving performance, reducing costs, or developing new prototypes.: used to assess changes in ingredients, formulae, ergonomics, or composition under controlled laboratory conditions. Sensory tests guide R&D in improving performance, reducing costs, or developing new prototypes.
Employee Panels: convenience samples of “captive” respondents, but are not considered to be reliable, and results must be confirmed with external samples.

When Used
Surveys & Forecasts typically performs product tests (1) after concept screening or concept testing has identified a winning idea; (2) after a product development phase, in which R&D, sensory tests, or employee panels have identified a new product candidate; (3) at any point to assess consumer reactions to product variations (e.g., cost-reduced, improved performance, etc.); or (4) for competitive claims purposes.

Stimuli
The stimuli used in product testing varies widely, depending on the type of test and the number of product variations under consideration. Stimuli can range from conceptual product mockups (which are not handled) to fully functional, branded products that are evaluated in a real-world setting. To assess “pure performance”, products are exposed without extensive packaging graphics, branding, pricing, or other identifying information. If brand equity needs to be assessed, then branded information is included. In addition to the product, usage, preparation, or safety instructions are also generally provided.

Product Test Designs
There are two basic types of product tests: monadic tests, and comparison tests. In monadic tests, the respondent is presented with one product, much like a consumer would be in the real world. Conversely, comparison tests involve evaluating two or more products in either a head-to-head or sequential fashion and are often used as screening studies. Common comparison designs include:

Sequential monadic designs, where consumers evaluate products in sequential fashion but are not asked to directly compare them.
Proto-monadic designs, in which a product is given first and evaluated monadically, then a second product is given and comparison measures are obtained.
Paired designs such as preference tests between two products, or multiple paired comparison tests if there are more than two.

In monadic designs, measures often include:

Purchase interest, plus open-ended reasons why
Overall rating
Voluntary positives (e.g., likes, advantages) and negatives (e.g., dislikes, disadvantages)
Value, uniqueness, superiority, believability, relevance
Expected frequency of use, HH members who used/might use
Anticipated purchase frequency, purchase quantity
Usage occasions/situations used
Replacement vs. addition use
Product attribute ratings and product directionals
Classification questions

In comparison designs, many of these same measures take the form of preference questions.

Sample Frame
Unless a new product creates an entirely new category (e.g., when the product is a technological innovation), product tests are conducted among samples of existing category users (e.g., male blade shavers), with augmented samples for key diagnostic groups (e.g., males ages 18-24). Common sample sizes range from 100-200 per cell (or per product, if a comparative design). However, larger sample sizes may be needed to detect small product or ingredient differences.

Pros & Cons
Monadic Product Tests: Pros: The purest assessment of product performance, with extensive diagnostics. Cons: More costly, and test interpretation is more difficult if there are no control products or norms.

Comparative Product Tests: Pros: Cost- and sample-efficient way to screen or compare multiple products. Cons: Limited diagnostics, and relative (vs. real world) performance information.

Timing
Timing will depend on the number of products and screening requirements; excluding product preparation, from field start to an initial presentation is typically a few weeks.

Subsequent Steps
Concept-product fit testing, controlled store tests, regional test markets, or regional launches.