Can AI Twins Replace Real Consumers? | Highlight

Can AI Twins Replace Real Consumers?

Pulse Launch Webinar -Trim | HubSpot Video

Gina Won, founder of The Yes Perspective, joins Highlight CEO Dana Kim to share what she found when she tested Highlight Twins side by side with in-depth interviews with real consumers: where the Twins held up, where real people still matter, and how she would use both across a project. Stay to the end for the launch of Pulse.

What you’ll learn

Learn more

Episode transcript

Please note that the following transcript was generated automatically and may contain typos or summarization.

Dana: So welcome, everyone! A huge thank you to Gina. Longtime Highlight supporter, friend, customer, partner, you name it. For joining us. For those of you who aren't familiar with Highlight, we help consumer brands test products with real consumers. We run digital concept tests, we run in-home physical product tests with our community of incredible Highlighters. But today, we're here to talk about something new we've been building, rooted in that amazing community, and that is Highlight Twins. So a small group of customers, has been testing Highlight Twins in beta. Gina is one of them. And stick around, because at the end we'll actually show you, a bit more about what it is, and how you, yourself, can get access. Alright, let's jump in, so that we can see, We can focus on the humans for a moment. I'm going to stop screen sharing. And we'll jump right in. So, before we get into Twins, let's talk about your background, Gina. Would love to hear a little bit more about, the Yes Perspective, your firm, and the brands that you work with.

Gina: Yeah, thanks for having me, Dana. It's been… we've worked together for, gosh, so long now. So I started the Yes Perspective, even before, I started working… or after I started working with you, and so I started about 2 years ago, after 15 plus years in CPG, the brand and innovation strategy consultancy, where I operate either as a fractional marketing leader. help build and execute marketing and research plans, help brands grow, and also build innovation pipelines, which is where, I primarily have been working a lot with the Highlight team. So after working full-time a lot on a lot of snack brands, you know, Simple Mills, Larabar. Belvita. I now happen to work a lot with beverages, as we were saying, so C2O coconut water, and Steaz Iced Tea, and Jarritos Mexican Soda, like, just a sampling of some of my clients today.

Dana: Such a cool portfolio, such a wide, variety of brands and products of all kinds. Yeah. You've run studies with real people on Highlight for the brands that you've advised, from in-home use tests to concept tests, to packaging tests. Tell me about generally how testing fits in with the way that you help brands make important decisions.

Gina: Yeah, it plays a big role, in the brands that I work with, but I think sometimes I can sound scary to, like, consumer research everything, right? But it's not a one-size-fits-all approach in the brands and companies that I work with, because you know, I really believe in right-sizing the decision-making and the research approach to the size, scale. and risk level of a brand. However, the non-negotiables to me, when it comes to research, is really two things. One. who you're designing a product for, so research to better understand your consumer. And then two, product feedback, right? So whether that's concept testing, sensory testing, tasting panels, and things like that. But what I often say is garbage in, garbage out. So, don't test just to say you got consumer feedback, and don't over-test just to test it to death because you can't make a decision, actually, is really what the problem is. But make sure you're testing with the right consumers. And you're asking the right things in order to make actionable decisions. And so, I always recommend a research brief, right? You wouldn't believe how much that helps, because that ends up being your North Star, so that you know what are you trying to learn, what are your learning objectives, and what decision will this research inform.

Dana: Yeah, yeah, awesome. Yeah, I think what struck me as you were listing some of the brands that you work with, and I know that's not even the full portfolio, is, like, those are very different customer types, and very different product types, and very different markets, and you have to really apply, like, research-first principles and fundamentals to every new brand you enter.

Gina: Yeah.

Dana: And we find the same. We work with brands of all sizes, across all verticals, and so making sure that we deeply understand our customers' problems, challenges, decisions to be made is, like, the foundation for partnership, which is… Yeah. …seems like we align on.

Gina: Absolutely.

Dana: Yeah, part of our portfolio, part of your portfolio, is making sure that emerging, disruptive brands, have the ability to win in a very competitive CPG environment. Tell me about where traditional research can slow brands down, and especially, like, smaller, faster-growing, ones that need to be more agile.

Gina: Yeah, I mean, the smaller the fast, and small fast-growing, or, kind of know that they've got a product that's winning and they're accelerating really fast, those are frankly, like. my favorite brands to work with, right? Because they, they move at lightning speed, but then suddenly they're like, I don't know what to do next, and so it's a complex but fun puzzle to solve. But I think that's where the challenge sets in, right? And so, oftentimes. time is always one of them, and so, you know, what's the saying? Time, money, and quality. You can only have… you can't have all three. It's like a push and pull, and you have to make some trade-offs, right? So there's those moments when you're trying to hit a retailer reset window, like, we know what, you know, what year the time is taking, or… you need to qualify a new supplier of a raw material ASAP, right? And so those moments are when it feels nearly impossible to take a pause and actually engage in traditional research, whether it be an IHUT or something like that, but what I like to say is sometimes you need to slow down in order to speed up, right? Because if you do nothing, and you launch without any feedback, you frankly might end up back at square one. Maybe negative one, right? Like, you might go even… you might backtrack if you did nothing at all. And then you're doing research retroactively, actually wasting resources and time to figure out, like, what did we do wrong? And when it was really, you didn't get any consumer feedback, so… what I like to think about is that research doesn't always have to be in the traditional sense, right? There are always ways to seek feedback. and get consumer input along the way, which I think is really just the most important. So don't get stuck thinking research takes too much time or is impossible. There are other ways to do it, and I think we're here to talk about one of those today.

Dana: Yeah, love it. Thank you for the tee up. And I love… I love the, like, the square one, square negative one, because we have.

Gina: Yeah.

Dana: Partners who come to us that are looking to be proactive and, you know, are de-risking their future plans. And then we have, unfortunately, sometimes partners who come to us saying, like, ugh, we need to course correct, or like, we need to figure out what went wrong, or, yeah, essentially go from negative 1 to 1, and then. And those are very distinct, but equally important, moments for research, so…

Gina: Totally.

Dana: Yeah, so, okay, so that's how you test today. Now let's get to the part, everyone is here for, which is Highlight Twins. So when we first invited you to try Highlight Twins, what was your first reaction?

Gina: I'll be brutally honest, I was skeptical. You know, always wary of the idea of replacing real consumers, real feedback. concerned about quality and how accurately it might represent consumer perceptions, but again, we've worked together for a long time, so of course, very cautiously optimistic, and excited to, get my hands on it and try it out.

Dana: Yeah, awesome. That reaction is common. I don't know that we've ever said, try Highlight Twins, and everyone's like, amazing, I trust it completely from the start. and so, I love that you were… thank you for trusting us and being cautiously optimistic and trying it out, but I love that you went in there ready to sort of pressure test it. So you didn't just try it out, sort of whimsically or without foundation, you actually set up a head-to-head. To ensure that you were testing the quality. So, tell us more about that. What was the project? How did you run that test? Thank you.

Gina: Yeah, so for a little bit of context, and I'm gonna keep it generic for the sake of, you know, the brands and the clients that I work with, but the project was a brand in a particularly crowded aisle and competitive set in the grocery store, and consumers switch frequently between brands, low brand loyalty, probably sounds familiar to a lot of you guys out there. So we wanted to learn, what does this category really mean to the people who buy it? You know, so foundational work, really. Attitudes, motivators, perceptions, behaviors, and then, frankly, who are they, right? So we started with quant, with Highlight. We did, like, a mini consumer segmentation, quantitative study, attitudes and usage, just to identify who the consumer groups were in this category. And then, and this is where we really engaged on this head-to-head, is we followed it up with in-depth interviews. So we engaged Highlight to do the live in-depth interviews with a moderator, and we went back and selected category superusers that had participated in our quant, which is one of the things I love about Highlight, the ability to go back and recruit actual members who had completed a survey previously. So specifically, we wanted to dig deeper through qual, from, you know, what's behind the numbers. Like, so a specific example we heard is. Do they want to reduce sugar in their diet. Like, that was clear from the quantitative results, but we wanted to dimensionalize that. So what does that mean? Is there a specific sugar quantity that they're sticking to? How does that impact their choices on zero sugar, sweetened, unsweetened? And why are they looking to reduce sugar? So, we did IDIs for a week, we conducted about 11 of them, and then concurrent to the week that we were fielding. the in-depth interviews, I was asking those same and similar questions to the Highlight team… Twins, like, kind of concurrently throughout that week.

Dana: So, so awesome. And we didn't even ask you to do this, you were doing this on your own, and… and then we heard about it, and we were like, yes! What about the results?

Gina: Yeah.

Dana: Now, when you put the answers side by side, tell us, what did you see?

Gina: Yeah, I will say, I think I voluntold myself, because I was like, you know, I'm curious about this, and I think cautiously optimistic and a little bit skeptical, kind of wanted to be able to compare it for myself and see what the results would show. And… qualitative research in general is always pretty fascinating, because, you know, after you've been in the back room for many years, even with a small subset, you always find similar, consistent themes that emerge. And I immediately found the themes that we were hearing in the live interviews to also be literally at my fingertips and immediate from the Highlight Twins, comparing them to the real people. So, again, to make this tangible, sticking with the reduction in sugar example, we found inconsistency in what the live, real consumers were saying, in what they said they did, and then what they… or what they said they wanted, and what they were actually doing when it came to sugar. So, they wanted to reduce sugar, but they bought sweetened tea, they bought, zero sugar options, which are Sugar alcohols and things like that, and they… but they also bought unsweetened And when we probed, the rationale was consistent among the live respondents, as well as the Highlight Twins. So it really had more to do with the sweetener type, or the moment of consumption, where, you know, yes, they're looking to reduce sugar, but it never replaces, like, ugh, you just need that. 3 o'clock sweet treat boost, or… so they're… they weren't so discerning or overly strict about their no-sugar-at-all rules, and that's just not a dimension you're ever going to get in quant. That's a dimension you typically get in qual, and it was… Really nice to see those nuances were still captured between the real consumers as well as with the twins.

Dana: So, so awesome. Such a great example of a complicated question that requires a very nuanced response, and… is a really, I think, a high bar for Highlight Twins in terms of can they parse their rationale, in a, a trustworthy way? I'm hearkening back to… I got my starts as a qual researcher, so I was in… On the focus group side of the two-way mirror for many, many years.

Gina: Yes.

Dana: And I'm, like, imagining someone passing me a note, coming around and passing me a note saying, like, hold on, that person said they don't like the aftertaste of Stevia, but they also want zero-calorie sweet drinks. Like, why? Tell us why. Totally. Yeah, and like, the fact that you were probing, you know, humans, real consumers and IDIs, and then probing Highlight Twins and getting the similar outcomes in a very specific use case is… Fantastic. So, thank you for running that pressure testing. Yeah. Was there anything, that surprised you, about the responses?

Gina: I mean, as a skeptic going into it, like, the similarities and the live responses and the Twins responses alone, like the… pure consistency was surprising. You know, I don't know what I… I don't know what I expected, to be totally honest, but I found that to be quite surprising. But then I… what I also really appreciated is that Twins shared outlier feedback in the same way that a moderator, formerly yourself, in real life would have delivered in, like, a wrap-up report. Like, Twins literally said, most participants said this, but one participant gave this interesting feedback, right? And again, you get that with a live moderator, with a live wrap-up report. I don't… I was surprised, because, you know, what my concern has been as AI research solutions emerge, is that an AI consumer, or an AI bot, or robot that's meant to represent your consumer flattens, you know, your entire consumer segment into, like, one person, and it lacks the dimension and the nuances that real consumers would give, because the reality is your target consumer has similarities, but they also have differences. There are similarities that pull them together to be your target consumers, but they're not all the same, and those… those dimensions are helpful to know, right? To scale and grow your business, to build innovation strategies off of. And I thought Twins did a really great job of even pointing out some of those nuances versus flattening them all to represent, you know, a single POV.

Dana: Yeah. Yeah, awesome. I'm glad, so glad to, to hear that you sort of discerned that and appreciated that. As we've been talking to customers about Twins and, fielding questions, answering, a lot of, questions around how are the twins created and how can I trust that they are, giving quality feedback, this, like, aggregation to the mean. is a big one, and the fact that you actually saw it play out in practice, give nuance, show outliers that match human outliers. Is… is awesome.

Gina: Yeah.

Dana: Cool, okay, so you, we talked about your sort of higher level, philosophy to supporting building winning brands and products. We talked about, how critical testing and consumer insight is in that process. We talked about, your experience with Highlight Twins and how that plays into, your, your learning plans for your brands. Now let's talk about, like. Twin research, human research, like, where… how do they live side by side, and how do you think about the interplay? Tell me about where… and we… and by the way, we at Highlight, we absolutely… we run human research, or, you know, research with real consumers, as our core business. We're launching Twins, we're very excited about it, but we absolutely see a time and a place for both. So, totally appreciate, that, you know, you ran real quant research, real qual research, and then twin qual research. Tell me about where, the live research gives you something that the twins didn't.

Gina: Yeah, And same, like, totally on the same page, too, right? I think there's always gonna be a role for live research, as well as A tool like this. And in this particular instance, the live interviews were helpful, where… so we… what we did is we shared the image of, like, a typical shelf set of this category as a pseudo, like, shop-along experience to get live brand and product perception feedback in the category. And I did find the live interviews to be a little bit more, descriptive and helpful in replicating that shopping experience, especially when it came to, like, nuanced questions about packaging design, like why they like the white, or why they did not like the logo, what popped to them, and then just, like. messaging hierarchy when you're thinking about claims and product attributes and designs and things like that. And I… and so I just found the live interviews to be a little bit more descriptive in the feedback there that I think, right now, I really appreciate for that feedback.

Dana: Yeah, cool, that makes total sense. And we've covered a little bit off on this, but why did it matter to test with real people alongside twins for you?

Gina: Yeah, I mean… I shared that I kind of voluntold myself to kind of test it out so that I could build the confidence and the trust that there was some reliability and quality in the data. But also, in this specific instance, we wanted to get brand perception feedback within the context of the competitive set, right? And so, we didn't want to spend the time to do actual shop-alongs and, like, walk along with them in store, but we thought it was helpful to do that shelf set exercise that I just explained. And thought live interviews with an image of a real store would be the best way to replicate that experience. But again, we were able to use the side-by-side comparison of all the other perception questions and probes that we spoke about from the sugar example earlier to really build confidence and dispel any myths or concerns that I might have had around quality of the responses for Highlight Twins.

Dana: Yeah, awesome. Okay, so when you're thinking about the full, sort of, product life cycle. So from initial concept, idea, all the way through to launch, all of the research associated with all the different, decision points and questions, that you're, answering and asking and making along the way. How do you think about using Twins in your research? And where do you still bring in real people with either concept tests or in-home use tests? Okay. this is going to be… we've been working with many of our early customers using Twins, and as we launch Twins to the world, this is, like, maybe the number one question, right, is the interplay, and the use case, and the where… where Twins were really.

Gina: Yeah.

Dana: So… Yeah. Would love your perspective.

Gina: Yeah, I mean… the area where I'm, like, most excited to get my hands on it now and dig right in, is I see Twins as being immensely useful for, like, early innovation. So when you're crafting a point of difference, or trying to massage a messaging hierarchy. what are the most specific, product attributes that are the most important? I know we've all designed a product that feels like it's got the kitchen sink in it. Does it need the kitchen sink, right? And, you know, I think the… many, many times, I fall into this trap oftentimes, like, brand leaders, innovators, product developers, we fall in love with the product, right? We're like, that's amazing. And we shortcut on the positioning, because we're like, let's get this to market, because we know it tastes really good. But positioning is just as important, because you can't have the repeat without the trial, right? So I think Twins could be incredibly useful, and I am really excited to get my hands on it, on that, like, back-and-forth engagement, and getting the real-time feedback on different positionings to refine and to design a concept and a messaging hierarchy. I know many of you have probably beaten a concept statement to death, like, and have 50 drafts of, like, 5 sentences, and you feel like your eyes are crossed, right? And so wouldn't it be cool if you could get real-time feedback from consumers Instead of feeling like you're like, oh, is this gonna test better, is this gonna test better? And really having 5 different ways in, and testing 5 different ways in to the same product, right? And so it really refines and makes your actual concept testing, which is where I wouldn't replace traditional research, real consumers, you're going in with what you feel like is your best possible concept, because you've kind of done the pre-work, the pre-feedback in advance to get there. And so those are… that concept testing is not the area where I would replace, real humans. I would… it's in the refinement phase. And then, of course, sensory testing isn't gonna go away, sensory testing on prototypes. Fortunately, robots can't taste for us yet, so, so sensory testing, I think, is immensely helpful, obviously, of course, to just get taste feedback, so that's where I would continue to use traditional methodologies for those as well.

Dana: Awesome, awesome. Yeah, I, and even in your, I, I very much… what your experience and your recommendation, where you think about where Twins fit very much align with some of our other, partners and how they're experimenting with it. The other use case that you had mentioned in your, you know, reduced sugar example was, was actually leveraging Twins to probe on an insight that you found elsewhere, maybe with real people.

Gina: Totally.

Dana: So that was… that's… and that's another, you know, frequently, cited use case of, I have an insight, and I need to, like, quickly, dive deeper.

Gina: Huh.

Dana: So there's… yeah, I love the early-stage ideation, the exploration, the refining of the human tests. Inherently, our first, you know, expression of twins is qualitative.

Gina: And so on.

Dana: All of this exploratory, innovation, early-stage work makes a ton of sense before you take it into quantitative human validation, whether it's digital concept tests, or in-home usage tests, and sensory. We get asked, do we see twins replacing sensory testing? And my answer is, like, there are worlds in which this is already actually being done, where there's, like. you know, synthetic panel being used to boost certain cells, or, you know, you can get super specific and nerdy about it, but inherently, like, is there really a twin that's going to tell you if Oreo A versus B tastes better? Everyone in, you know, product development and R&D appreciates the The mix of art and science, that makes the human supercritical in that work, so…

Gina: Totally.

Dana: Cool. Awesome. Well, thank you, Gina, for sharing your experience. It is, like I've echoed along the way, it really does align with the Highlight perspective, and that's why we were so excited when we heard about your perspective. In your experience, we were like, let's talk about it live! Because we agree. It's exactly how we think about it. Twins are not a replacement for real people, but they are tremendously value-adding, in certain moments, and they do help, you test more, test earlier, and then real people can take it from there. So. We are really, really excited to announce that starting today, everyone can get access to Twins, not just Gina, and not just a select few. And I'm gonna spend, a few minutes actually, sharing a few slides around what it is exactly that we're launching. Showing you a bit of the experience that Gina got a sneak preview of, and then we have 10 or so-ish minutes of Q&A, so… Let me share my screen. Alright… Okay, so something new from Highlight. Let me… share my screen. So, millions of data points, one clear pulse… We are introducing today, we are launching Highlight Pulse, which is the new way to put Highlight Twins to work. Gina talked a lot about interfacing with the Highlight Twins. We've refined the way in which you interact with them, the what you can do with our product, and today is the launch of Pulse. So, in Pulse, which you will see some screenshots of, you can start a discussion with an audience of Highlight Twins. You can ask your questions the way you would with a group of real consumers, and you can get answers in seconds. how does it work? If you, if you, well, actually, let me save, that for the next slide. Here you can see a screenshot of what Pulse looks like. So like I mentioned, you can start a discussion with an audience of twins, you can… get questions asked and answers, questions answered in just seconds. This is similar to the experience that Gina saw and had, in her, side-by-side. And the how it works… is, firstly, you pick your audience, you can tell us who is it that you would like to reach. Is it parents who are 25 to 44 years old, with kids under 12, who drink ready to drink tea weekly? Great. This is human-like screener criteria that now you can apply to Highlight Twins. You can then get into, the actual, discussion that you just saw. It looks like a conversation, and you can ask questions like, would you try a protein-packed sparkling water? The Pulse agent does run a discussion and bring back one clear summary. So sorry to switch back and forth, but as you can see here, the Pulse agent is essentially saying about 4 in 10 would try it. As a better seltzer, though, not as a protein drink. And carbonation level is the biggest risk. Price is the second gate. So, a summary of the 20 distinct twin responses. However, you can dig into each twin's individual answer. So, you can see that it says, read all 20 individual responses here. You can actually expand that and drill into, each twin's individual answer, and that is a huge point of differentiation and important, feature of ours. To Gina's points earlier, we are, absolutely, in support of understanding individual humans. We want to be able to parse that nuance, we want to be able to see the outliers, we don't want any personas that are averaged together and composites of no one. And this experience is really an expression of that. Why Highlight Twins are different? So there are lots of Twins products hitting the market today, coming from companies of all kinds. There are research agencies launching Twins. There are new companies founded that are entire Twins businesses, there are research platforms that are launching Twins. Why did Highlight choose to launch Twins, and why do we feel Like, we have the right to add value in this market. Three key reasons. One, we are built for CPG, we are built for consumer products, we have built this incredible, fast, quality research engine that is made for concept testing, packaging testing, flavor testing, positioning, everything, all of our core research that our product powers. Is entirely in honor of, of, you know, building better products and purpose-built. Secondly, we have, from day zero, been obsessed with data quality. Gina mentioned garbage in, garbage out. That has always been my worst nightmare. That has been our team's worst nightmare. And so we have put tons of, effort, time, care, dollars into building a quality Highlighter community of real testers all across the U.S. who are answering our, you know. Concept tests, product tests, to a high degree of quality. That… obsession with data quality, with tester quality, has now translated to twin quality. Each twin is grounded in a real Highlighter. There is a one-to-one, you know, Highlight Twin to real Highlighter in the community. And their survey answers, their profile questions, everything they've told us over time is what is fueling twin answers in the twin experience. So there is, data quality obsession from the start. And then continuous learning. We, as we, run real research with real humans in, you know, the core Highlight research platforms. We are continuing to make our Twins and our system stronger to be able to facilitate continuous learning throughout, a product development lifecycle, so… Our system continuously gets smarter. Our data stays rich, stays relevant, stays recent, and all of these things come… these three combine to… again, why we are here today, why we are launching Highlight Twins. We really believe that we can create something valuable for our customers, and that is, of course, our reason to exist. I probed on this with Gina, and I'll echo it again now. Twins are not a replacement for real people. They are a new step in your process, they are a new tool in your toolkit. They help you get answers, get direction, get insight in moments when, you know, real research with real people might not be possible. So here's a version of, you know, a product development lifecycle that we see in action, which is… You've got early-stage exploratory conversation with twins, you then, get to, you refine your concept, you refine your positioning, you refine your stimuli, your images, whatever it might be, and then you take it into that quantitative validation with real people. And then, great, you've got, maybe, refined, a list of 10 concepts to 3, and you need further refinement. You go back to Twins, you're actually in a brainstorming session with your colleagues around how do we refine these last three concepts? You're going to the Twins for instant iterative feedback. Great, you align on a concept, you decide to ship and produce that concept, you have now a prototype in hand, you are going to actually iterate on that product with an in-home use test with real people. There is, you know, there are moments when Twins can add real value, there are moments when real people will add real value. The beauty of, of the Highlight… the way that Highlight is set up and Highlight Twins is that you can, go to each audience and go to each tool at the moments that you need them.