Decision-Ready or Not? Tracy Adams and Keith Smith on How to Validate Synthetic Data | LIVE at TMRE 2026

In the third and final installment of our live coverage from TMRE 2026 in Denver, Signal & Noise correspondent Lucas Longacre sits down on the show floor with Tracy Adams, Senior Director of Research and Insights at the Advertising Research Foundation (ARF), and Keith Smith, Managing Director of the Marketing Science Institute (MSI), to cut through the hype around AI-generated respondents.
Fresh off their TMRE session, “Decision-Ready or Not? How to Validate Synthetic Data,” Tracy and Keith walk through the joint ARF/MSI validation framework and explain why the level of validation should scale with the risk of the decision. Synthetic data can shine in early exploration and concept generation, needs more rigor when you’re narrowing 20 concepts down to four, and demands multiple avenues of validation before it informs a product launch, segmentation or forecast.
They also get candid about where synthetic falls short. Synthetic samples don’t just average toward the middle; they often return nothing but the middle, flattening the tails, outliers and niche segments marketers most need to find. Tracy shares ARF survey findings showing AI confidence is sky-high while only about a third of marketers take the time to validate outputs, and Keith asks a provocative question: if human survey responses are already degraded by AI use, what’s the right ground truth?
Plus: why AI’s sycophancy is a research risk, the “pendulum” of AI hype and backlash, why most organizations hand employees a chatbot with no framework, advice for junior researchers on building critical thinking by pushing back on AI, and how research roles are collapsing and expanding at the same time.
A must-watch for researchers, insights leaders, marketers and anyone evaluating synthetic data vendors.
CHAPTERS
00:00 Intros: Tracy Adams (ARF) & Keith Smith (MSI)
01:58 Why TMRE: AI and the synthetic data wave
04:00 “Decision-Ready or Not?” and defining synthetic data
06:00 The Wild West: black-box vendors and “magic beans”
07:50 ARF survey: high AI confidence, low validation
10:00 Three decision contexts: exploration, prioritization, final decisions
12:35 What synthetic data is NOT good for
15:00 Rethinking ground truth when human responses are degraded
16:30 Can Reddit and Discord data improve synthetic?
18:10 When does using AI become just how we work?19:40 The AI pendulum: hype, backlash and the middle
21:30 The long road to ROI and the “here’s a chat account” problem
24:00 Advice for junior researchers: push back and think critically
29:15 Critical thinking is a life skill
30:00 Are research roles collapsing or expanding?
37:40 Where to follow the ARF and MSI
GUESTS
Tracy Adams, Senior Director of Research and Insights, ARF: https://thearf.org
Keith Smith, Managing Director, Marketing Science Institute: https://msi.org
The ARF/MSI synthetic data validation framework is available on both organizations’ websites.
Transcript:
speaker-0 (00:04.364)
All right, here we are at T what T R E? Market Reserve. Okay. Hello, here we are at T R E. I'm Lucas Longanker with Signal and Noise Podcast. Thank you for joining us. Tracy and Keith, do you mind introducing yourselves and how, you know, we should address you?
Thank you so much for having us. It's such a pleasure. so I'm Tracy Adams. I am the Senior Director of Research and Insights at the Advertising Research Foundation.
And my name is Keith Smith. Again, thank you so much for having us. I am the managing director at the Marketing Science Institute, which is a sister org with the Apps of Witcher Responding.
So just quick, you know, elevator pitch, what is it your organizations do?
Yeah, sure. so the Marketing Science Institute essentially is a non profit member organization and our goal really is to bridge the gap between marketing research on the academic side and marketing industry. So what we do for the most part is we identify interesting research from academics that really work with practical real world industry contexts and we bring that new innovative ideas, different methodologies to industry.
speaker-2 (01:09.619)
and we work with industry then to figure out where are the open gaps and and needs for research, and then we fund and sponsor that research on the academic side.
Awesome.
And the AirF as a nonprofit, it is based on evidence and research. The product that we provide is research. So we cater to agencies, advertisers, media companies, research companies. And the idea is really to identify the most pressing things in the industry, the pressing questions, the things that might need you know, like more of a meta analysis, so to standardize things, to provide insights and findings that are of course based on
Right for us to research.
Wonderful. And what really brought you here for this conference this year? What are you looking for? What are the kind of the trends that you're sniffing out?
speaker-2 (01:58.018)
Yeah, I mean there's a been a lot of change obviously in AI and the role that AI plays in marketing, market research and and marketing analytics as well. a lot of the work that we have done at MSI recently is around AI and marketing research and and I'll let Tracy talk to ARF, but I think it's similar on the ARF side, a lot of what we're seeing. And so we both want to kind of see some of the new techniques that are happening and then really try to
generate a a good perspective from the marketing and academic side and share some of those perspectives. So there's been a lot of work from academics as well in the last honestly three years around things like synthetic data and new AI techniques and marketing research. A lot of really cutting edge ideas and concepts. And so we wanted to really try and bring those together and share those with an intriel.
So same for you, Tracy, same for
Just add to that, this is my first time at TM three, it is super fun.
Same.
speaker-1 (02:54.072)
Yeah. it's huge. There's a lot of content and a lot of things going on at the same time, which is overwhelming, but in a
How are you feeling about yesterday? I'm like exhausted because I did not get a break. It was like one thing to the next.
managed to get all my steps in because the place is so very big you have to go up and down and all over. there is sometimes you know the issue where I have like four sessions at the exact same time that I want to see all of them so you know missed opportunities but it's been wonderful so far.
Yeah, that's c one of the reasons I like when everybody shows up and then they post. Like, you know, I I actually read out your LinkedIn about like the highlights you saw. I'm like, yeah, I missed that one, I missed that one. I did go to that one. so anyway, yeah, I I have to admit this event is enormous and supposedly it's the first year they're doing like three events combined into one. So, you know, I ever I've been hearing it from everybody essentially. so we're gonna dive a little bit into, you know, what your panel was about. So what was it called? Decision ready or not? That's the name of it. So can you just
You know, tell me a little bit about, you know, what that panel is or sorry, what your d you know, your discussion is there.
speaker-2 (03:59.758)
Yeah, absolutely. So our our panel and discussion was really trying to present a framework to understand synthetic data and how synthetic data is utilized.
I'm gonna stop you real quick. We should probably define synthetic data exactly.
Don't don't worry, don't connect. synthetic data essentially is utilizing and there's a lot of different definitions. I think there's still a lot of fuzziness in the field in terms of what it actually is. But I would probably define it as using generative AI techniques to simulate or create respondents in a market research style approach that didn't exist previously.
Right. So it's basically using those AI tools to create respondents that didn't exist previously. Do you wanna add to that at all?
Maybe just to add that the past week we had a huge discussion with ARF and MSI and our other sister company, Sim, exactly about what is synthetic and all the different definition
speaker-0 (05:01.902)
I've heard different definitions throughout this conference already, so thank you.
to maybe create some kind of really good, useful taxonomy that can really lay out, you know, the different definitions, what each one includes, how, you know, what kind of overlap there is, and so on. So that's like what I would add.
Yeah. And I'll even tease a little bit of that. Right now the plan is for the Three Sister Orgs to actually publish and put together a set of sort of definitions on that. So that's coming soon. you'll be able to see some of that.
Awesome, yeah. Maybe we'll do a follow up. Yeah. so yeah, tell me how the the how the wait, d you already did your panel yesterday? Yeah. So tell me how did that go? What was the kind of the reaction from the audience and everything? Yeah.
I think it was really enlightening to to hear a lot of people share some of their challenges and frustrations around synthetic. So I think one that I I mean, I don't know if you've walked the show floor, but pretty much every booth has AI can solve your problems, AI is magic and we'll fix everything for you. And I think what we wanted to really try and communicate through our panel was the idea that synthetic can be a useful tool, but right now the industry and a lot of the vendors sort of have these
speaker-2 (06:13.048)
black box solutions that don't really explain a lot of the complexity or the methodologies behind how they're actually implementing their synthetic data samples. Doke it's kind of the Wild West right now in terms of what people are trying to create and sell. And so what we wanted to do was really provide a framework to allow people, model managers, to actually assess whether it's vendor solutions or their own internal solutions, how to actually utilize synthetic data.
I just
speaker-2 (06:42.063)
and how to test that synthetic data against different use cases etc.
And are you both seeing a gap between like what vendors are promising and what's actually they're able to deliver? Okay.
I mean, I think there's a lot of a lot of hype around we can provide you perfect solutions and we can provide you I let me give you an example. There's a technique that's been used somewhat in this AI market mature space where they try and give you access to samples that are hard to get access to.
smaller consumers consumer segments that often you can't get out.
Makes sense though. I mean they need a solution for that. So then it's like we're promising you these magic beans that can get there and you know Exactly.
speaker-2 (07:32.328)
But but in overhearing some of the conversations that I've I've I've heard that you know on the show floor, people ask that question, Can you give me this access to this this sub segment? And the answer is Well not really.
I'm starting to see where like the value of what your organizations are bringing is like the sober reality of what it is, but also how to leverage it and make it useful, right? Exactly. Okay. Exactly. Yeah, exactly.
give you a data point. We did we did a survey at the ARF about how marketers utilize AI in their daily workflows. You know what what kind of percep perceptions they have, perspectives, what kind of use cases. So adoption is huge. Everyone is using AI, that's not a big surprise. But only thirty something, I want to say thirty-six are using it for synthetic. yes which is an interesting number. And then in terms of confidence in output
This is not just synthetic synthetic, but in general AI. Confidence in output is like ninety two percent.
Wait, so they're when they do use it they're very confident in it.
speaker-1 (08:32.929)
Base AI, again in general, not just in if they are super super confident, but in terms of validation, maybe thirty something are actually taking the time to validate.
Can I just tell you from my experience, I'm a product guy and I do a lot of like engineering product stuff. I'm just shocked at how many times I'll pr present an AI solution, assuming that whoever's gonna do it is gonna do check it as they would somebody a human's work, right? Like if your colleague gave you something, you'd at least want to do due diligence to make sure they got it right. Yet they will just blindly accept what AI told them. And I do think that's part of a just a a misunderstanding of the technology. They're just haven't used it enough yet to understand it.
but I so it sounds like it's very similar in that regard.
that and that's one of the points in our talk yesterday that AI output is so plausible and convincing it sounds so good you immediately say okay sure that makes sense you don't take the time to actually really dig into what goes you know what what's in there you don't take the time to push back and say well hang on a minute what are we missing we know that AI is very positive
So syncophantic.
speaker-1 (09:41.358)
So anything I'm gonna say it's gonna, you know, great job, Tracy. That's like the most impressive thing I've had. You know, and all these factors together make it so easy to use, you know, so and and so believable. And that's exactly I think where we come in, how very important it is to take the time to to validate.
Yeah, what what has been some of like from your perspective, some of the best use cases for synthetic data? 'Cause I do feel like it's has a huge value if it can be done properly. So I'd just love to see like where you see that going.
Yeah, I I mean some of that I think comes out in the actual framework that we shared yesterday and and is actually available on the MSI and ARF website. Yeah, so that'd be great. what we really see sort of there's three different I guess decision contexts where you really need to apply different levels of validation to effectively use Synthetic, right? And so I would say the the use case that has the lowest requirement for validation where we actually see a lot of use of
Which we can link in the podcast.
speaker-2 (10:40.814)
Amongst our member partners is in exploration, essentially. So you're really trying to start to begin to explore a space, right? Whether it's a product space or a service space or maybe a a new segment opportunity where you're really trying to just explore what's out there, it can be very effective in this kind of a use case. And so whether it's early concept testing of trying to generate new ideas, you're not really worried about the risk of making a wrong choice here. And so synthetics are great for that content.
when you get into more successively imp critical risk decisions essentially, it can get it really needs a higher level of validation. And this is where I think there's you know a caution that we would throw out to the industry essentially. and we identify really two other decision points. one we call really prioritization and what this is really doing it's it's starting to narrow down essentially from your previous sort of exploration into what subset we want to explore more deeply. Right. So this is a concept of where
you might have twenty different concepts and you want to narrow it down to four that you want to actually push forward into you know more involved product testing or product development or other types of context where you're really trying to narrow the decision space or the state space in order to explore. In those, again, Sympathetic can play a role, but kind of the level of validation that's required is a little bit higher. And then the most extreme for us is is really thinking about critical decisions or final decisions, things like product launch, things like
critical segmentation or a forecast. And in these cases, you really need to have some pretty extreme and and multiple avenues of validation, which we talk about a bit in our framework, in order to really trust that data. because you really kinda have to calibrate the level of of trust and risk to the synthetic data that you're using.
Anything to add to that, Tracy?
speaker-1 (12:34.958)
Think I think we need to think about those cases where we're looking for the unusual or the extreme or the tales, the heterogeneity, the nuance, right? Anytime you're looking for those, AI is not gonna reveal it, right? Synthetic is gonna average towards the middle. We talked about that.
Yes, or you
Exactly. It so it kinda glosses over all those wonderful bits of information that we really need to make a good marketing decision. Right. So I think the the flip side of your question, what is synthetic not so good for? Here we go.
Was the next follow up. okay. Yet. Yet. That's a good caveat. Yeah.
is all those kind of, you know, like subgroups maybe that we don't know enough about or the unusual things that we are trying to uncover. If there's a new product and you want to know, like or any kind of innovation and you want to reveal some kind of unusual response. You know, I'm thinking about all those cases where it's not the middle, the center, the obvious, that's what I would add.
speaker-0 (13:41.762)
Yeah, and this is a trend I've seen at this entire conference as well as, you know, I'm do other disciplines as well. And it's essentially the the real value we're seeing in a lot of these AI and large language models is in architecting, decision making, you know, at the beginning, the building and and and you know, scripting aspect of it. And then when you go into the rollout you start getting a little more nervous 'cause the idea is if you start with b something bad, all it's gonna do is make it acceleration it is gonna accelerate the bad. So anyway.
anything to follow up on that for stuff to be aware of?
I mean I I I think you know, more broadly it's it's really I'll echo part of what Tristan was saying about the variability and the the uniqueness and the niche the niche information that's in a in a data sets. we talked about this a little bit and Tracy handed out it, but the distribution of responses that you get in a synthetic data set, it's not just that you're getting the average, it's that you're actually not getting anything but the average. the distribution is is essentially incredibly narrow and you're losing
Okay.
speaker-2 (14:43.95)
all of the the nuance that exists. And so just to echo what she's saying, y you really do need to be aware of that. because it's a pretty critical piece to what you're actually I mean most of marketing is trying to explore those edge cases, right? Trying to identify the segments that you might be able to to targets you didn't think about before.
It's not the person that's already buying your product over and over again. You want to know who's where where can we head to, right? Which is the whole point of this whole process, right? interesting. Okay. what does it look like though down the line? Like you said, like yet, I do feel like there's a lot of solutions we could be putting in place knowing where the gaps are. So I'm just curious how you think what are some solutions that might be coming.
I I think again thinking about the validation frameworks that are in place are really important, right? So I I I agree with Tracy, I think it's just a matter of time before these techniques and the LMs themselves become more sophisticated and capable of handling some of these challenges. but at the end of the day I think it's a matter of putting validation in place and validation it's maybe a little bit different than what we would have traditionally used as validation, right? So you know from a
typical you know ground truth perspective, we would have said, look, the human response is the ideal perfect validation point, right? If a human responds in a certain way and that matches with how a machine responds, then maybe we can trust the machine. And I would argue that is one validation point, but I'd also throw out the idea that frankly if you look at modern survey responses, we're already seeing pretty degraded human responses in those surveys. The use of AI by individual
Respondents has created a scenario where we can't always trust what comes in from humans. So what actually is the better comparison point?
speaker-0 (16:31.914)
I have an interview later today about fraud. And I'm not we're not gonna go into it here because I feel like that's kind of the focus of the next one. But but it does bring up a a good kind of segue into we talked earlier, were we talking about Reddit and about all these other social media ones where anyway I was at a a panel where they're talking about how they were using memes to get like ex describe how it feels in memes, right? Or like you know, actually diving into Reddit responses and Facebook responses to the really
understand how people really feel, not just like the what they're filling out in survey. I'm just curious if that pollutes or or can enhance, you know, synthetic data as well. Are you pulling in these factors of like beyond just surveys, right?
think it can enhance it because at the end of the day you're just gathering more information for how humans engage with their daily lives in a way that isn't canned and standard and in a in a survey response, right? So I think there are opportunities to be able to tap into LLM responses that come from mind data on real human behaviors. Yeah. Whether you post on Reddit, whether you post in Discord and other types of communities. I mean that's a more honest human response, right?
As long as it is a human commenting, which is a whole other problem. That's we're not gonna get into today yet.
Yeah. so I think that there there is an opportunity for that to actually provide a more robust and sort of holistic perspective of of a human response. it's just I think you need to be really careful about how other ways you're valuing that data beyond just the human response.
speaker-0 (18:09.39)
Cool. Yeah. Anything to add to that?
Well, I wanna say I think it's maybe even theoretical question about how humans use AI because I use AI now so much that even for a LinkedIn post I'm gonna use it. yeah. For, you know, if I wanna comment on something or like a quick response, pretty much every single thing. And I think as AI kinda seeps into everyday life for everyone, right? You know, so in all different kind of
professions and occupations and people are just gonna use it more and more. At some point it's not gonna be considered at all fraud. It's just gonna be considered the way we do things.
That is a fascinating perspective. And I I'm gonna jump on that in a second with yeah. 'cause I'll say like how I respond to my colleagues depends on so if like they're like junior or much younger. I got like called out for my very blunt single sentence emails and responses. They're like, I thought you were mad at me. So what I do is I use AI as an interpreter for younger generations where I will say, make this sound human. Yeah. Which is an hilarious contradiction, but it's true. It's like
I was just responding because I was like, I just need to get this done. But it's like they have sensitivities and that are real. And if I want to be better l leader, I need to take that into account. So I go to AI, you know, send this and make it sound warmer, right? Make it sound more human. And to your point, is that AI? Is that, you know, is it me? Isn't it me using the tool? Right. Anyway, so it's this whole existential ide concept that I glad you brought up.
speaker-1 (19:39.946)
you know the line there it's very very blurred and it's gonna get even more I think. I heard in a recent panel someone said about synthetic as as a pendulum. That idea I'm it's Helen Katz who is amazing.
Okay.
speaker-0 (19:58.114)
Is she here is she here under
Okay. She said and I love that idea. It's a very so sociological concept, the pendulum. A new phenomenon comes up, people are like taking it to one extreme. So right now, like there's a huge hype with AI. Yeah. And then it's gonna go all the way to the other extreme, it's gonna be banned, people are gonna hate it, which we do see there. But then at some point it's gonna go back somewhere to the middle and we're gonna understand how we live with it, how we work with it, and it's gonna change everything.
yeah.
speaker-1 (20:29.74)
And then it's gonna kill us all.
Well, I always tell everybody like, don't believe the hype and also don't believe the doom the you know, the doom sayers. Like it's somewhere in the middle, it's probably the reality. where do you think we are in that pendulum right now?
think it depends. Okay. Right? I think so if you come to this conference, we're very much at the hype. but if you go to other you know, I'm thinking about like my community, family, friends, most of them don't use AI as much as I do. They definitely no don't realize the potential there. They maybe use it for like very specific things. such a stupid w we wanted to make a flyer for the school for like a fundraiser and someone was very quick to say, I did this without AI.
Yeah.
speaker-0 (21:14.028)
Yeah. It's a badge of honor.
In a very proud. So I think it really depends who are you looking at right now with the population. And sometimes I think in marketing and advertising, because we're such early adopters, certainly on the research end of the side things, you know, it seems like, my God, the whole world is AI, but actually not so much.
What do you think?
Just give a complementary perspective and think about it in terms of how companies are actually using some of this. So if you go back and you look at other technologies that have been disruptive and introduced into the workplace, things like PC computers, things like the internet, there was a lot of hype associated with them as well, right? And if you actually go back and look at that data, what you find is that most companies didn't actually find any productive improvement in their day to day until like 10 years.
Yeah, it was the tail was so long.
speaker-2 (22:05.256)
It took forever, right? and I think we're in a very similar place right now where a lot of companies are saying, You need to learn how to use AI, you need to use AI, but not a lot of them are actually providing frameworks or even scaffolds for how to do that. We've talked to a lot of our members and a lot of them say, Yeah, my company gave me a chat account, but that's it. So it's like they're not giving them, you know, guidance or insight on how to use the technology, on how to actually implement the technology in their day to day.
That's the second time I've heard that this this conference and I'm so on board with that. Because I do I've I've been experiencing it myself. Just like the the throwing the tools on the table and be like, hurry up and use it. And then the the pushback internally from especially larger organizations. And to your point, Tracy, so I've I've I've gone to like engineering and product conferences and already the fatigue is set in where people are like, Well, what's the ROI? What is the real value we have to this? Because we've been using it now for three years and what you know, we have to show a return on investment.
Whereas some of the other conferences where the tools are now finally getting online, people actually are now excited to play with it. So it's like you're right, I think it depends on where you are in the and also my friends and family, some of them are just like, it's gonna kill us all. You know, Skynet's coming. But yeah, it's a good it's a good point to make.
And we have actually academic research that has started to look at how productive are some of these actual tools and technologies. And there was an interesting piece that we we had presented earlier this year that actually showed different functions within the marketing organization and how different actual functions can leverage this technology more effectively and how they've actually produced an ROI. Yeah. some of them did not.
Yeah, not surprising. Okay, cool, cool.
speaker-1 (23:46.378)
Not to mention there's a gap between individual use and organizational use and it is very hard I think to bridge that gap and to really reach a place where in on an organizational level it is fully integrated. So on an individual level people can just use it, you know, daily. And it's I think e even easier to train them on it. But if you take that, you know, to the larger, broader level, that's a bigger challenge. And I'm trying to think of the numbers we have about training in the marketer survey that I said.
It ha there is a rise in training, so organizations do provide those tools more than they did in the past. I can't recall the exact numbers, but it wasn't you know, they weren't as comprehensive and as they should be.
Interesting. Yeah. So say you're a junior researcher just starting out, like what do you say to them at this point? I mean, any advice from people who might be like listening to this 'cause they want the new insights as they start their career.
I mean, so I would I think one of the things and this is in our framework and I would encourage everyone to take a look at it, but I think the one one of the things that's in that framework that I think gets glossed over a lot in when individual people start working with these tools is the systems are incredibly fragile and incredibly dependent upon how you ask the questions and how you interact with the AI. So I think a lot of people, especially junior junior people, are maybe in a space where
They ask an AI a question, they get a response back, and they assume, well, that's the right answer. and what we really encourage or or propose is that when you're working with these systems, you really need to push the system. You need to be asking questions in different ways, different formats, different you know, types of of text, even different order, and see are you getting the same response back? Because what a lot of people I don't think realize is these systems.
speaker-2 (25:39.054)
are and this has been in the site guys so maybe it is more familiar but these systems are designed to just give make you feel good, give you the answer you want, right? So if you look back to the social media platforms, you know those systems are optimized for engagement. in the exact same way, these AI systems are optimized for a version of engagement essentially that's trying to encourage you to continue to use the systems. And so
We get this sort of sick advancing comment, right? You said before, where they're gonna say yes to everything, they're gonna agree with everything you say, and they're not gonna actually interrogate the results in a way that you might want from these synthetic systems. So I would really encourage people to push back, ask the question multiple times from different perspectives, really try to explore the space rather than Exactly. I mean, you really need to apply critical thinking to the way that you work.
Be the devil's advocate.
speaker-2 (26:36.238)
I think that's maybe a secret weapon that that those of us that have strong critical thinking skills that have been developed in the pre-AI times have that maybe the current up and coming you know young people may not have as strong of a that that critical thinking sort of skill set developed. I'd really encourage you try and develop that skill set and engage with AI around that skill set.
He stole my hand.
I'm sure you can elaborate in some direction, but
I agree. I think critical thinking is the number one skill we should all be practicing right now. Because frankly, the more you use AI, the less you, you know, act on that. So your judgment is so very important. so double check, triple check, push back, you know, give it counterexamples, look for what is missing because it's gonna give you like a long list of things, but hang on a minute, what's not here and I should actually focus on.
yeah, and maybe just like to give them some hope, you know, it's just a minute in time. yeah. This two shall pass and you will still have jobs and it's gonna be amazing. but yeah, it's it's a rough patch I think now for people who are just starting their professional trajectory.
speaker-0 (27:52.302)
Yeah, and I'll say too that one of the things that I'm worried about is to have that level of pushback and deep thinking takes a career sometimes. Like I didn't get better at all this until, you know, years of working in it. So if they're not given the opportunity to do things maybe in a slower manual, more like, you know, how things were done previously way, like, are you ever gonna be able to have the full context?
So what
I think the one very, very good thing about AI is that it is a very intimate could can be. Yeah. Intimate one on one interaction. So no one is listening. You can say anything you want. You can push back as many times as you want. And you can go at it in so many different directions. And I feel like that is a huge advantage that we should
You know, a junior is not gonna push back against someone more senior. I need more information, I don't think this is the way to go. We would not do that. But if you're a junior, you'll push back against AI. That is like the one not the one, there's amazing things with AI, but that is one of the more amazing things that we should all take advantage of. Just push back and don't be afraid to call it names and get a a really good response.
Yeah, good point.
speaker-2 (29:13.174)
I guess I would just add, you know, there's no limitation on where AI is used. I understand your you know, your point of, you know, you don't have the exposure in today's work world necessarily to be utilizing AI and pushing back as much as someone who's developed it over an entire career, right? But you can use AI outside of work too, right? And so I have hobbies that I do and frankly I use AI extensively in those hobbies, not to get the right answer, but to explore different ideas and to
develop different perspectives on things and that is another context where you actually can can push back, develop that critical thinking outside of even or in addition to a work environment, where critical thinking is not a work skill, it's a life skill, right?
It's great quote. Yeah. well so kind of my final question, which is a pivot off that one, is you know, what a role roles are collapsing, you know, like now you're supposed you need to wear like five hats because you can in a way, because you have access to these tools now that you can like do more than you could have by just yourself. And so where where do you see your industries headed with that? As in like now the the the CMO is like, Hey, I did all this research, like I have this research study I just put together myself.
Which I heard people like gasping at like, you know, in some other sessions. But that that idea which I think is, you know, a good and bad thing, right? It's like now we now are not only able to do more, but we're expected to do more. So kind of do you see some of these roles merging or maybe like expanding in some way? Especially as a researcher, I mean
I don't think I think we're gonna see I s I should say elimination, but the reduction in certain explicit roles that can be done and not entirely by AI. Right? Things like analysis, things like you know, some of the more onerous tasks that are associated with research. But I think what's important to keep in mind is that we're gonna move into a place or we're already moving into a place where
speaker-2 (31:14.742)
a good employee or somebody who's really trying to develop as a good researcher is going to learn how to use AI systems to do the work that's been done previously and to be able to then ask a higher level of question. Right. So you're gonna have the time to explore things that you wouldn't have been able to explore before because you don't necessarily have to spend time on road tasks. Right. I heard a talk yesterday that was talking about the idea that you know
They can run way more projects now because of AI, but they actually limit how many projects they decide to run because the socialization of data into the organization is now the bottleneck. So it's not so much can I run the project, it's do I have enough time to actually communicate the results of this study to the rest of the organization and the rest of my stakeholders. So I think it's just gonna be a shift in sort of how we engage with the AI systems and the types of things that we focus our time on around what's
being given to the A and what we get back from the AI in terms of sort of how we think about our
Yeah, makes a lot of sense.
Think of this as a really positive thing. AI has opened up so many possibilities. So for me as a researcher, if I am more into qualitative data, I can now quickly run AI and get more insight into quantitative data as an example, right? as a researcher, if I see a presentation done by our marketing team and I don't exactly love the way it looks, I can
speaker-1 (32:43.96)
quickly run it by AI and just give it back to market and say, maybe we could do something along these lines. You know, it's like now my role can expand into many different things that are adjacent, but I never had the expertise before. Not that now I do have the expertise, not at all. I'm not gonna take anyone's job, but it will provide me the opportunity to like walk in their shoes for a minute and be able to think about new connections, right?
New things where we can help each other and contribute. and as a researcher, I think the wonderful thing about AI is that it does wonderfully recover so many patterns that you know, as a human it's hard to take all this data and like it's gonna take so long to so how wonderful. Let's just really focus on the wonderful positive aspects here and just push them and go with them in many, many different directions.
So some roles will collapse, other roles will just expand. yeah, and I and I'm less fearful of, now I just have to do so much more work. But actually to look at it as, you know, now I have the opportunity to be part of more projects and part of more things. I have more opportunity to engage with other like minded people or or not like minded people, you it's it's just gonna open up so many opportunities.
Expand, yeah.
speaker-2 (34:09.082)
And I would just sort of I think add to that as well, you know, in addition to creating opportunities to be able to interact with new people when you connect, like Tracy had mentioned, I think it also opens up the state space for you to actually explore the research itself, right? So in the past, you have a limited amount of time to be able to conduct an analysis on a piece of research. And so you would pretty early on clamp onto what probably made the most sense, especially if you think about like a qualitative trend analysis, right? or even a quantitative exploration of certain
Data sets. Well now AI can generate six different versions of a taxonomy or six different versions, way more than six, but a number of different versions of that. And then you can use that critical thinking to explore each of those in different ways and decide which ones actually make the most sense for this data set or for this use case or for this context. For those of us that love research and love playing around with data and finding these things, it actually creates more opportunities to be able to explore more.
Yeah, I mean I'll say from my my perspective, I asked you that 'cause I'm very curious what you think, but I'm totally aligned with both of you. So I had an in I interviewed Tony DeRoss, who is like the head engineer at Pixar years ago, and he was very frustrated that people would put like this idea that you were either like good at math and science and engineering or good at arts. And he's like, a a great engineer is a great artist. And that was his whole premise. And I was like, I completely agree with him. The idea that like we
By putting in these roles sometimes we we're shrinking our abilities, right? You're saying this is only what your responsibilities are. And the idea that suddenly you can have more, I think you you're gonna see people that are really good at doing that, like code switching and changing and using different part of you know parts of their perspectives and brains are gonna thrive in those environments. And maybe some people who are just wanna have that very rigid, simple task to do, like I'm sorry, like there's just not as much space in the the workforce for you anymore. Because you need to be like, you know, agile.
Anyway, so and I think that's not a bad thing in the long run. I think it's gonna be good. It's just maybe gonna be painful for while people are sorting themselves out and and trying to figure out how to, you know, let use the new tools that are presented.
speaker-2 (36:11.182)
Yeah, I mean there were barriers I think in the past around developing technology and developing sort of base skill sets to be able to explore art versus technology versus programming.
Did you learn word doctor? Yeah, like Excel, right?
Learn Python, right? Or you learn Photoshop or whatever the case may be. And now those things are all collapse, right? So now the creative people, the intelligent people, those that want to explore those spaces don't have those barriers anymore to be able to be limited by what's available.
But it's definitely something that you can do on your own.
So the organization that you're in needs to provide a safe space for you to explore. It needs to give you the tools and the guidance, you know, and the limitations. We don't want to do any kind of security breach for sure. But if you don't have the organization to back you up and really provide you that opportunity to explore, for us, how fun for us. We just received so many different AI tools and that was our job for a while. Go explore.
speaker-0 (37:15.096)
Go play. That's amazing. Yeah.
So many ch how fun, how lucky am I?
Well that is human we love play. And like the more we can make it play instead of now you like I said, hear this thing, use it for work. You know like, great. You know, but like go no, go play with these first. See what you can do with it and then maybe come back and if there's something useful, great. Yeah, that's a cool way of approaching it. All right. Well I'll say thank you that we're kind of running over and I want to make sure we're we're aware of our time. But so c how do people follow you? Like if they're curious about your research or what your com your organizations are doing, how do they locate you and reach out?
Yeah, absolutely. Tracy will we'll give her version of this, but obviously we're on LinkedIn and we'll share that information with you afterwards. I would also encourage you to go to MSI.org, which is our MSI website, where we have a lot of these programs and initiatives that we're running and really get exposure to to a lot of that work. we also do a number of conferences that I would encourage you guys to look for throughout the course of the year, and those are great opportunities to be able to see a lot of the research that's coming through.
One.
speaker-0 (38:18.531)
Thank you.
Pretty much the same. I'm I'm trying to be very vocal and transparent on LinkedIn. So I do post a lot. I think it's important. I'm trying to put out like, this report came out or this conference, you know, the key takeaways that I thought stru you know, something that struck a chord with me. and like Keith said, just go to our websites, you know, check what's there if there's a conference that you're maybe interested in. I don't want events team to like kill me, but yeah.
I'm trying more to.
speaker-1 (38:50.316)
Maybe we can make this happen in some way. and collaborations. We're always open to collaborations. So just
Thank you so much, both of you. Have a enjoy the rest of the conference. I'm sure we'll see each other ping ponging around everywhere. Thank you.
Thank you very much.





Comments