Designing Audience & Sample Size

How to Design the Right Audience & Sample Size for Your Highlight Test

Designing the right audience and sample size is one of the most important parts of product research. Audience definition and sample size are not independent decisions; they are tightly linked to your research objective. When either is misaligned, even technically sound data can lead to misleading conclusions.

This article is designed as practical guidance for designing sound research when testing with Highlight. You can leverage Highlight to better understand your target audience, optimize your product, make a go/no go product decision, gather ratings and reviews, and more. This guide helps you decide for your specific use case who to talk to, how many people you need in your test, what tradeoffs you’re making, and when results are “good enough.”

Step 1: Start With the Decision You’re Trying to Make

Instead of asking:

• “How many respondents do we need?”

Ask:

• “What decision will this research support?”

Your research objective should directly determine:

• Who to include: how broad or narrow the audience should be
• What level of precision is needed: directional vs. validation
• How results will be interpreted and used

Common Research Objectives

Step 2: Choose the Right Audience

Below are the most common CPG research audiences — from broadest to narrowest. In most cases, broader is better unless there’s a clear reason to narrow.

Common Mistakes to Avoid

• Using brand loyalists to estimate total demand or interest in a new product or concept
• Assuming bigger samples solve poor targeting or audience definition
• Over-screening early-stage innovation and narrowing your audience too early

Understanding Incidence Rate
The narrower your audience, the lower your incidence rate — which increases cost and timing risk. Incidence rate (or IR) is the percentage of people who qualify for your study out of everyone you ask. It tells you how common your target audience is within the broader population you’re sampling from. If you send survey to 1,000 people and 300 qualify by falling into your audience definition, your Incidence Rate = 30%. That means 3 out of every 10 people you contact are eligible.

A high incidence audience would be adults 18–65 who purchased shampoo in past 6 months, representing 60%+ of the total US adult population. A low incidence audience would be women 25–40 who bought sulfate-free shampoo in the past month, only buy salon brands, and have color-treated hair.

Lower IR = higher recruitment cost + longer timelines! You can use the Highlight platform to estimate your incidence rate and feasibility of your test by filling out the audience details of your project.

Step 3: Determine Sample Size (aka “base size” or “n size”)

Now that you know who you want to test with, you can determine how many testers you need.

Think about:

• Is this high-risk or early learning?
• Do I need direction or validation?
• Am I comparing product to product, or audience to audience?
• For physical product tests, how many samples or physical products can I produce for testing?
• Are there any subgroups I’m interested in, such as looking at those aware of my brand vs those unaware?

Practical Guidelines

Minimum Sample Sizes
The sample size ranges above reflect Highlight’s recommendations for physical and digital research. That said, you may increase or decrease your sample size depending on your study objectives, budget, timeline, and risk tolerance. Below are the minimum sample sizes we recommend to maintain various levels of statistical rigor.

Highlight Minimum Recommendations

Confidence intervals reflect the long-run success rate of the method. For example at 95% Confidence, if we were to test our product with many subsets of our total population, about 95% of those intervals would contain the true population parameter. This reflects how confident we are in the method over repeated tests, and its ability to yield an accurate result when working from a sample of consumers rather than an entire population.

• 80 or 85% Confidence Interval (Directional Reads)
  • 50n per study minimum
  • 25n per subgroup
• 90% Confidence Interval (Moderate Precision)
  • 100n per study minimum
  • 50n per subgroup
• 95% Confidence Interval (High Precision)
  • 200n per study minimum
  • 75n per subgroup

Think of this as a sliding scale:

Higher confidence → Larger base sizes → Greater precision
Lower confidence → Smaller base sizes → Directional guidance

The key is aligning your base size with the decision risk. Not every study requires 95% confidence — but high-stakes decisions often do.

Applied Examples: Audience & Sample Design in Practice

The scenarios below illustrate how objective, audience, and sample size choices work together in real-world research — including the risks and tradeoffs involved.

Final Takeaways!

All research is constrained by budget, timelines, incidence, recruitment feasibility, and operational realities. Running good research is not about eliminating constraints — it’s about making intentional decisions within them.

You may need to reduce sample size because you can’t produce enough units for testing. That’s okay! You can compensate with richer qualitative inputs — open ends, photos, videos — to increase depth of understanding even if breadth is limited. Tradeoffs don’t weaken research when they’re deliberate. They strengthen it!

Before launching your study on Highlight, ask yourself:

• What decision will this confidently support?
• Where might results skew — and in what direction?
• What conclusions should not be drawn from this data?

When you can answer those clearly, you’re ready.

Use this guide as your blueprint to test with confidence. And when you’re ready — launch on Highlight and turn smart design into decisive momentum.