The Race to the Bottom: Media Quality, Attention, and Why Cheap Reach Is Costing Brands More Than They Think

Listen on Spotify

Watch on Youtube

For nearly two decades, digital advertising has optimized around one primary objective: finding the cheapest possible way to reach the right person. Programmatic buying, identity signals, attribution, and performance marketing have created extraordinary efficiency—but they may also have trained the industry to systematically undervalue one of the most important variables in advertising effectiveness: the quality of the media itself.

In this episode of Signal & Noise, Rio Longacre and Brett House welcome back friend of the podcast Erez Levin for his third appearance and his second full-length conversation. Erez is one of the industry's most respected voices on media quality, attention, and advertising effectiveness. A former Google executive and now Founder of Emmett Advisory, he recently co-authored CIMM's landmark report, Quality Matters: Navigating Quality in Media Buying and Measurement, an effort to establish a common industry framework for evaluating media quality.

The conversation explores a simple but powerful premise: Not all impressions are created equal.

While the industry has become exceptionally good at optimizing for identity, CPMs, and short-term outcomes, it has largely ignored the environments where advertising actually appears. As deterministic identity signals become less reliable and privacy regulations reshape digital advertising, Erez argues that marketers must begin treating media quality as a first-class buying signal—not merely a brand safety filter.

Topics include:

• Why optimizing exclusively for ROAS and CPMs can quietly destroy long-term brand growth

• How the industry's obsession with attribution created a "race to the bottom" in media buying

• Why attention alone is an incomplete measurement framework

• The difference between audience quality, media quality, and creative quality—and why all three matter

• Why premium inventory isn't binary, but exists on a spectrum of value

• The hidden problems with averaging performance across wildly different media environments

• Why CTV has an opportunity to avoid repeating the mistakes of display advertising

• How marketers should rethink media planning as identity signals continue to weaken

• Why procurement incentives often work against advertising effectiveness

• Practical ways brands can begin incorporating media quality into buying decisions today

• The future of AI-driven media buying and whether automation will improve—or worsen—advertising quality

• Why marketers—not platforms or agencies—must ultimately lead this shift

If you've ever wondered why digital advertising often feels more optimized than effective—or why brands continue chasing cheaper impressions while premium publishers struggle to monetize quality—this conversation explains why.

Whether you're a CMO, agency executive, media planner, AdTech leader, publisher, or simply interested in where digital advertising is headed next, this is an essential discussion about one of the industry's most important strategic shifts.

Watch the full episode and join the conversation.

#SignalAndNoise #AI #Data #MediaQuality #Attention #Advertising

🔑 What We Cover💡 Key Takeaways🎯 Why This Episode Matters Read the full transcript below.

---

Brett House (00:03.658)

Everybody welcome back to Signal and Noise. I'm Brett House joined by my co-host Rhea Longacre and we have a friend of the pod, very good friend of the pod, Aris Levin joining us again. Thanks for returning to Signal and Noise, Aris. Yeah, and for those, I'll reintroduce you even though we've introduced you before. This is your third appearance on Signal and Noise. So truly, including the last in Miami, if possible. But Aris is...

Rio (00:09.287)

You there?

Erez Levin (00:15.629)

It's great to be back with you guys.

Rio (00:22.555)

Yeah, this is this is think the third the third appearance overall. Yep.

Brett House (00:31.912)

Independent now, but he's one of the industry's leading voices on media quality, attention and advertising effectiveness. You spent a decade at Google in a bunch of senior leadership roles, both on the buy and sell side of the advertising ecosystem. So you have a pretty good broad perspective of how things work from the buyers and the sellers. Your firm is called Emmet Advisory, right? And you work with advertisers, and technology companies. That's the only shilling I will do on behalf of you, Aris.

Rio (01:00.197)

He's very prolific though. not not only he speaks in panels all the time, always put in a new report, which we're to get into, but he's a very prolific author putting on lots of great content.

Brett House (01:04.459)

Yep.

Brett House (01:09.984)

Yeah, yeah. And the thesis at the center of today's call, and I'm not going to steal Rio's thunder, but is this new SIEM, which is the Center for Innovative Media Measurement Report, which is owned, for those that don't know, SIEM is owned by the ARF, which is the Apprentice Research Foundation. I've certainly done a lot of work in the past with them. You co-authored it.

Rio (01:28.935)

Yeah, they do good work.

Brett House (01:29.984)

Which is, that's great placement. mean, to have that type of industry association as your backing and you're part of the authorship is pretty awesome. And it's called Quality Matters. Yeah. And that's Gabe DeRose, right? Former head of audience strategy and insights at the New York Times. Right? So he's, then John Watts, the managing director of SIM. Right? So.

Erez Levin (01:47.266)

Yeah.

Brett House (01:52.908)

That must have been a fun process. I mean, I've done a lot of these reports where you're gathering all the insights, you're doing surveys and inputs and qualitative and quantitative analytics, and then you have to kind of create a narrative around it. It's work, it's work.

Erez Levin (02:03.02)

And not even just us three, right? There were a lot of contributors. So there was the sort of multiple working groups within SAM and then folks even outside of SAM that we went and spoke with and got their feedback and interviewed them and made sure that sort of the direction that we had was accurate. So this is really meant to be sort of a consensus from many of the industry leaders on how to think about quality and how to incorporate it into the infrastructure, the ecosystem.

Brett House (02:27.2)

Yeah, I hope it wasn't pro bono. Was it pro bono? It was not pro bono. That would be like hundreds of hours of your life that you'll never get back, but you you come away with some good insights. But it's called Quality Matters, Navigating Quality and Media Binding and Measurement. And the central conclusion that I sort of assessed from it, and I read as much as I could, and then I summarized some of it, because it's pretty long. It's probably 30, 35 pages long, is that media quality, which you call MQ,

Erez Levin (02:29.613)

It was that, no. I was commissioned to do it.

Rio (02:49.944)

Dense report too, yeah.

Brett House (02:55.446)

which I think is cool, is not a soft preference or a brand safety add-on. It's a missing pricing signal. And if you use this quality metric or measurement correctly, you can shift budgets, displacements, where likely to drive both performance and brand effects. Publishers can earn fair value for high quality environments and CTV, which was a big part of the report, can avoid repeating displays raised towards cheap audience reach.

low quality audience reach which we talked that's a big issue in advertising in general especially in programmatic and you sort of don't want to see CTV follow. Yeah.

Rio (03:31.239)

Well, especially display, right? And the way I kind of interpreted the report too, I'll let Eric explain kind of the intent. it was like, let's not repeat the mistakes of programmatic in CTV, which is a newer, less formed channel. some of the same mistakes are starting to happen. I think this report is very timely.

Brett House (03:49.312)

Yeah, you even recommended, I'll finish this, you even recommended that media quality may ultimately need a formal governance layer, which I thought was super interesting, right? So like the GIC, the Joint Industry Committee, or the MRC, which is the Media Rating Council, structures, which I think is, so this is more than just interesting insights. It's like, no, this could actually be applied to drive effectiveness in advertising.

Sorry, long-winded intro. Eris, welcome to the show. Did I characterize the report and tell us a little bit about your experience with this larger team putting this together?

Erez Levin (04:22.343)

Yeah, you did characterize it well. know, the context is quality has been, you everyone's been talking about quality for a while. It's been this sort of buzzword in the industry, but just a buzzword because no one defined it. People were talking about it. Maybe they had their own definition. There was no consensus. And that was part of the impetus of putting this, you know, thing together. Like, hey, can we get consensus from the industry, move the ball forward, get shared language, definitions, principles.

something that everyone could align on and then we can all move it forward because we can't get to any sort of governance or anything else, any standardization until we get to at least agreeing to what we're talking about here. So that was the main goal. Just maybe put a little bit simpler sort of the summary and what was really important from the beginning to get everyone aligned, right? All of our working groups aligned on sort of how we were thinking about this. I was like, let's get one vision, one sort of statement that everyone agrees to.

And it was this idea that not all impressions, not all media are created equal. And that's something that everyone agrees with. And we sort of saw that as almost like a first principle. And so once everyone agreed to that, then it started to open up a lot of questions. Like then how do we value media? And yes, sometimes it is with audience. Sometimes it is with a cookie or identifier, but not only. Just because I'm reaching Rio on a CTV ad.

Brett House (05:22.357)

Yeah.

Erez Levin (05:42.243)

or a mobile ad, a small video, like those are worth different amounts. And so what are those sort of qualitative dimensions and can we sort of build those frameworks to understand how buyers can assess value, incorporate those quality measures into their buying measurements.

Rio (05:59.463)

Yeah. Well, I like, I like that. there is, mean, I think one thing I found like interesting, very intriguing about the report. And I, and I liked this a lot. It didn't really seem like it was an argument like against scale, which is kind of the, offering of besides speed of programmatic, right? Didn't seem like it was really saying we shouldn't focus on that. You know, I think it was saying there was a time and a place for it, but it seemed more like an argument. Like we've become way too comfortable over the past. And for those who aren't like,

haven't been working digital media for a long time. Over the past 15, 20 years, ever since the emergence of programmatic, we've built this incredibly complex ecosystem with tons of different layers. That's been very good at identifying the, like, how, like to your point, how do we value, how do we value advertising? Right. And there's just been this like very myopic or like maybe like one dimensional focus on like identity, right. And on, you know, which we glean from cookies and other signals, which aren't always super reliable. Right. So it's like,

And that has been the foundation of our buying decisions and how we've been deciding how much we pay for inventory, where ads are served up and what tools we use to do that. maybe we've become too comfortable in this and as identity signals fade with some of the privacy laws being implemented, which some of the discomfort people have with this, maybe it's time we take a fresh look at this. your thesis seems to be that media quality, focusing on our quality, the characteristic of the environment where the ad is actually being served up.

This has probably been undervalued over time and this is probably the most important signal that should determine how we value buy and then ultimately measure media. let's just, I think that's maybe a good place to start digging in there.

Brett House (07:35.445)

Yeah, yeah. I think, to your point, media environments are important. So like where the media is actually being shown and then exposure conditions. these are, you know, the report basically states these are predictive, meaningfully predictive of recall, brand association, all your key purchase intent. But they're being. Yeah, the things and they're in a lot of these exposure plus media environment metrics, which we have access to.

Rio (07:50.703)

Yep. The things that drive decisions, right?

Brett House (08:03.956)

are being basically ignored, Is that the, am I reading this right? Right, both from the targeting in media buying perspective, but also the measurement of the impact and they're kind of missing in this flow. that?

Erez Levin (08:17.48)

Yep, that's it. And I would say there's one other really important element that was another fuel to the fire that brought this race to the bottom. And that's this idea of what we've ignored. There's a lot of marketing fundamentals, these principles, and we bake them into the paper. We talk about them a lot. And I think there's one that's more important than any other one. And it's this idea that marketing, most advertising, has an impact over a dual time horizon. It works in both

the short term and the long term. When you serve ads on TV, there's probably 5 % of people that are in the market for that good at any given time and are gonna be more likely to buy your product because they were in the market when they saw your ad. The other 95 % of people, it wasn't wasted. They were more likely when they are in market in two months, three months, six months, when they remember that they need to buy that paper towel or the car or whatever it is.

They're gonna be more likely to remember your brand. And so when we have obsessed too much with attribution, which is gonna favor things that you can measure in the short term, when you favor identity, when you just focus on, what can I do in this quarter? You end up sacrificing what's in the long term. Now, maybe it did work actually for real in the short term. It's not all sort of for not, but that often comes at the expense, right? It gets into this performance doom loop.

and that is one of the fatal flaws in this idea that you need to balance short and long term together. Media quality is really critical for that because I'm serving a million ads, a billion ads. Okay, I'm gonna get X number of people to buy my product and hit my number, but for all the people that didn't buy my product, I wanna make sure I have maybe a minimum level of quality or I just wanna get as many good exposures as possible for the future buyers for when they do enter the market. And that's where media quality as a

pricing mechanism as a predictive level of efficacy and then a pricing mechanism with that can help you sort of maximize that effectiveness.

Rio (10:21.383)

One report such arguments is that the industry thinks it's very good at understanding who saw an ad. I mean, even that's debatable. You look at the quality of some of these signals, right. And like we covered that the last couple of episodes, right. Brett was Scott McKinley, especially, right. That maybe the signals aren't as good as we think they are, but the industry at least thinks it is. And that's been the premise of, of, of advertising is right. We can tell you who serves, who's going to see an ad. We can serve up the ad. Right. But as

Brett House (10:37.729)

Yeah.

Rio (10:49.711)

identity signals become less deterministic, right? And more probabilistic, is certainly happening. Like seems like we're entering a period where media quality should become a much more important when it comes to actually the targeting mechanism itself. Love you could talk about that for a

Erez Levin (11:05.169)

Yeah, that's how I look at it. There's two sort of ways that buyers can value media. One is with identity signals and they use that for targeting and attribution. And then the other one is these media quality signals, things that are sort of detached from any sort of identifier. Both of them are valuable when you have both use them in combination with each other. But now especially and in the future, more often than not, not that you won't always have the deterministic signal of

Brett House (11:21.228)

Yeah.

Erez Levin (11:31.751)

sort of potentially an identifier and be able to, but then you have to map that to all of the other sort of exposures that someone has. And that is going to be very rare. And so we're constantly, the models are getting weaker. They're becoming more model. They're becoming even less certain and less accurate than they were before, which isn't to say they aren't a helpful signal, and to like signal and noise, they can be helpful signals.

Brett House (11:39.947)

Yep.

Erez Levin (11:55.866)

But if you sort of assume that they are fully accurate, you sort of ignore that there's a lot of noise in there as well. And I loved your conversation with Scott McKinley. Very much touches on the same thing. And to your point earlier, Rio, I don't know that, I debate this with Scott, I debate this with folks on the creative quality side. I don't think, I wouldn't say media quality is the most important sort of signal of the sort of three-legged stool of media quality, creative quality, and audience slash data quality.

They're all equally important. There are situations where you have to have a higher level of one or the other, but in all of them, you have to have a medium, at least a middle value of quality and be pricing that accordingly.

Brett House (12:43.168)

Yeah, and you talk about, let's define media quality, because I think that'll be helpful for the audience. We probably should identify deterministic versus probabilistic in identity, but we'll get to that in a moment. So media quality, I think of it as sort of placement attributes that are independent, as you guys mentioned, of user identity, but they're helpful in predicting advertising effectiveness. How would you define media quality as you defined it in the report, and why is it important?

Erez Levin (13:10.424)

Exactly as you just said it. the placement, the characteristics of a placement of sort of groups of media and impression not tied to the individual impression or the identifier that predict the like the relative value given its likelihood of driving an outcome or a set of outcomes.

Brett House (13:21.227)

Yep.

Brett House (13:29.12)

Yeah. what other characteristics are you tracking? What other metrics are you tracking to really get that media quality? So you guys talked about a quality trifecta, which was sort of, you you take creative, you take media, you combine that with audience quality data and you get kind of a broader read of potential effectiveness.

Erez Levin (13:52.067)

Yeah, we slice and dice this in different ways in the paper and the lens is mostly on media quality. so thinking about media quality, I'll just like go explain that because the definition is still a little bit too dry. So I like to bring it to life with some examples. And so I think there's two main categories of media quality. One is what everyone calls attention. It's really the prominence of a placement, right? The different attributes that make sure, you know, it's more likely to be seen, noticed, remembered, the size of the ad, how long it's on the screen, whether the sound is on or not.

Brett House (13:58.208)

Yeah.

Brett House (14:04.321)

Yeah.

Erez Levin (14:20.176)

Then you have the, what I call receptiveness. This is the situational context. This is how likely is a person to be receptive to that message. They might be attentive the same amount at 5 a.m., 5 p.m. It's a McDonald's ad. They're gonna see it right before they ate dinner, right after they ate dinner. They're gonna be a different level of receptive to that message if they're hungry or not. And so the marketer might determine they're willing to pay more for, or they wanna prioritize more impressions at certain hours of day or not.

Brett House (14:29.633)

Yeah.

Brett House (14:49.706)

And what are the predictors of receptiveness? mean, you mentioned sort of a timestamp almost time of day could be seasonality. What's going into that sort of?

Erez Levin (14:54.181)

The day of the week, could be the seasonality, it could be the geography that somebody is in because it's cold and flu season. And these are very campaign specific. And so it could be the device they're on and somebody to be more receptive to a message because they're on their phone holding them right in front of them or they're with their family watching the TV. And so there's somebody.

Brett House (15:00.096)

Yeah. Yep.

Brett House (15:10.946)

Yeah, had like a, there was a cold medicine example that you guys brought up in the report, right? And that was probably seasonal specific, et cetera, et cetera.

Erez Levin (15:13.657)

Yeah, yeah.

But it doesn't always have to be seasonality. worked with a, it was an energy drink company and they wanted, they were doing CTV ads and they wanted to just sell more of the energy drink and they wanted to optimize to attributed conversions. And I was like, that's not going to happen. We're not going to be able to get granular enough data to optimize. Why don't you give me a quality signal that you know is higher sort of value for you? And they gave

Brett House (15:35.702)

Yeah.

Rio (15:39.972)

When do you think people should be watching your ads? Right? I mean, that's kind of it, right? mean, I...

Erez Levin (15:41.767)

Well, we did something even simpler. This was a geo. They knew there were certain DMAs that they had higher market share for that energy drink. And all they wanted to do was move more product off shelf. And so they just tweaked their custom algorithm to prioritize slightly the ads that they could serve into those DMAs. They were willing to pay a little bit more because they knew it have more of an impact. And they did. If they serve more ads there, they were able to buy more products. Otherwise, would be nationally distributed sort of across.

Brett House (15:42.752)

Yeah. Yeah.

Erez Levin (16:09.306)

But the interesting inverse, imagine they were trying to grow their market share, then the algorithm would be the inverse. They would find of DMAs that have a lower market share. Exactly. they would change the algorithm to prioritize DMAs where they have a lower market share if they were trying to grow their market share. And so it really depends on what the objectives of the campaign are.

Rio (16:16.869)

Yeah, you target the places where you're underrepresented. Yeah, yeah, yeah, makes sense.

Brett House (16:20.287)

Say that again, say that again. They got lost in the concurrent company.

Rio (16:34.439)

Yeah. Are you trying to grow your share of wallet or are you trying to like grow your, you know, grow your market presence, right? You know, by, by finding, identifying you to brand customers. So it's, so I really liked that because it's basically saying like the value, how much we're willing to pay for this. It should not just be who we're targeting, but it should be where we're targeting and like, and all these other things. could be the time of day. could be the DMA, you know, which is, you know, proxy for geo could be there. All these other things we should be considering. It makes me wonder like,

Erez Levin (16:37.859)

The pool of audiences, yeah, exactly, yeah.

Brett House (16:40.896)

Yeah.

Yeah.

Rio (17:05.095)

You know, you've, you've heard people at Jeff Green sale, the auction is the best way to get the most value for the publisher and the most value of the advertiser. This is almost saying like maybe not, right? Maybe there's a better, maybe I'm not saying you don't have an auction, but it's like, if you're just running an auction and you're only using identity, right. As your, as your way to target, then that seems like it's just not going to, it's not going to give you the result that you're suggesting you could get here.

Brett House (17:14.284)

Yeah.

Erez Levin (17:28.036)

Well, definitely not with just identity, but I think the auction, you have identity and media quality signals together, that is the most powerful way. And so I'm all for it. I built those sort of custom bidding, you know, within Google and incorporating all these media quality signals. do think, imagine, and this is I talked about like geo.

You don't have to break out your line items into different geos that are DMA targeted because at some point you hit diminishing returns to try and spend in a certain area versus an algorithm that knows dynamically all the supply available, who's bidding what, can predict it pretty well. And it's going to be able to find that most effective way to distribute according to these different priorities.

Rio (18:07.195)

would this work better in like a private marketplace than just using open exchange? Right? Go ahead.

Brett House (18:09.31)

Yeah, well, and I was going to make that argument because isn't there an argument that programmatic displays already sort of repriced towards quality? And something said, I think it was a report that I read in 2024 that said 80 % of programmatic displays is now happening in private marketplaces or programmatic guaranteed, right? Which is a shift kind of reversing that sort of open auction dominance. mean, is that accurate from what you've seen?

Erez Levin (18:35.295)

skeptical? I'm skeptical because I think a lot of what drove that shift to PMPs was people thinking the open auction was bad and it is and thinking PMPs would be safer and they're not. And so you're usually paying a premium and getting kind of the same thing. Maybe it's a little cleaner, but you're still buying fake instrument. Yeah, you think it's a trusted person you built a deal with, but it's not. You bought it off the shelf. But ultimately, are you still buying, you know,

Brett House (18:47.02)

Yeah.

Brett House (18:51.298)

Kind of slightly different packaged, yeah.

Brett House (18:58.741)

Yeah.

Erez Levin (19:00.772)

muted video that's calling itself in stream and paying for it as if it's in stream. Yes, you're doing that in deals and you're doing that in open auction. That is still really inefficient in the market and that's because media quality isn't valued. You're chasing the user and the cookie on the recipe site.

Brett House (19:14.33)

That's a key point. It was almost like reposition and repackage because it was being asked for, but it doesn't get to the heart of what we talked about with Joe Root Rio, right? We need one higher signal value so that we know that we're reaching the right people, but in premium environments, right? And he encouraged sort of direct buying.

because you're more likely to get premium inventory, premium audiences, that's not just your typical sort of broad open ecosystem reach, right?

Erez Levin (19:45.3)

And that's where I think there's so much scarcity in that market of really premium media with first party audiences, let's say, attached to it. That is not gonna get bought in open auction. That's probably never gonna get bought in PMPs. That's gonna move into more guarantees and reservations because it's gonna be a seller's market there. They have a really scarce good that most of the buy side wants. And so, yeah, exactly.

Brett House (19:59.458)

Yeah.

Yeah.

Brett House (20:05.898)

Yep, low supply. Certainly, yeah.

Yeah, no, that's great. So you made it really interesting because there's been a really interesting point in the report that sort of attention metrics, everybody says the attention economy, you hear that about as much as like data is the new oil, which I'm going to fall off my chair if I hear that again. But like attention metrics and the value of these, which has been the talk of the town for at least the last couple of years, it's useful, but you suggest that it's bounded, that it's limited. Can you talk about how you guys sort of

came to that conclusion in the report.

Erez Levin (20:40.216)

Yeah, mean, attention is super valuable. It's probably the most mature of all of the quality categories, but it doesn't tell the full story. The example I gave earlier, like somebody at 2 a.m. versus 2 p.m. versus 8 p.m., like they might be equally attentive. I'm not saying that they are. Or somebody in a different city might be equally attentive to the media, but they could be more receptive because they know, you know, there's a social contagion effect. More people drink this type of energy drink around me. And so.

Brett House (20:52.257)

Yeah.

Erez Levin (21:04.75)

there's all of these other variables that attention metrics, the traditional ones from a media measurement standpoint, couldn't capture. So I think that was one really important point of sort of broadening this. This is why years ago I sort of went from like focusing on attention to broadening its equality.

Brett House (21:13.601)

Yeah.

Yeah.

It's like what kind of attention it needs to be calibrated to context, to channel, to a bunch of different things, right? And I think you mentioned an example around CTV of like that ad being served 2 a.m. right on a non-television device for cheap CPM, which you know, that's context, that's channel Intel that would say, hey, this is probably not the type of attention we want to be seeking.

Erez Levin (21:41.955)

And something really important, you we talk about the different attributes of sort of these quality characteristics of sorts. And one was sort of this non-binary, really important. And the other one, it's all relative. Most value of media is relative. It's probabilistic. And so I'm not saying this CTV ad is worth exactly $12, but I'm saying this CTV ad at 2 a.m. is worth some amount less than that CTV ad at 2 p.m.

Brett House (22:10.73)

Yeah. Yep.

Erez Levin (22:12.267)

the marketer is going to deter, well, unless you're, you're, you're Taco Bell, right? Then maybe that middle of the night slot is actually good for you, but this is where, different marketers can assess what that relative value is. And this is to your point earlier, Rio about scale. This is not a scale thing. I think when you set a binary, when you say this is premium, this is not, this is what we're willing to buy and not, you're going to cut off a lot of stuff that probably is worth it at the right price. Cheap, cheap reach is fine if it's cheap enough.

And so this is where we have to sort of introduce value, not just think about cost, think about value and sort of overlay that with price.

Rio (22:52.485)

One thing I liked in a report is you tried to, actually, it was an attempt to have a methodology for evaluating like the, scoring, let's call it, like how much, in terms of its impact on people and how it, and how much attention it's actually garnering, like how much would this actually be worth? And you had a scale of one to 10. And I think I'm looking here sound on in-stream video, which is, you know, this, so this would be like, you know, let's say you're watching like a

Brett House (23:15.148)

yeah.

Rio (23:18.661)

Some of an Amazon prime video, like in the ads that come up, those would be called in-stream cause it's while it's streaming, right? Sound on scored 8.2 out of 10. And you were comparing that to muted out stream. That's, know, there's videos that, you know, that play, they're not, they're not part of a video. They're just popping up, you know, maybe in the body of your screen, muted out stream, which is certainly not, definitely not premium inventory. Those placements scored around a one point through it's a huge gap, right? Between, between it to, which I think, which I think makes sense to me.

like what does this tell? mean, that should be a signal that we're using to understand how much to your point we're going to be bidding on an inventory. It's going be a lot less of that sound on in-stream relative to the sound off out stream, right?

Erez Levin (24:00.908)

much less. mean, this is like a finite amount. Dr. Fu, right, talked about it like this is there's a finite amount of time and attention and people are actually watching, especially video ads with the sound on. And so there's a big delta there. And this is what everyone would agree. They're not worth the same. I'm not saying the upstream is worth zero. It's definitely not worth zero. And for a lot of advertisers, but they shouldn't be paying the same price. And that's what we saw. Unfortunately, what I saw, unfortunately, is the amount of spend that went towards the small muted video ads.

Often that called themselves in-stream, to be fair, but the same tons of spend was going to them and buyers were paying the same CPM for those small muted video ads as they were for sound on large players.

Rio (24:42.279)

Well, and that was just a problem in the DSP that would just be like inventory. They would just, they would just pick that audience and it would just write again based on identity and.

Brett House (24:43.424)

Yeah, and if you can...

Erez Levin (24:48.098)

They check a box that says in stream and then they just, they look at is viewability and completed views and maybe attributed conversions.

Rio (24:54.875)

And I think...

Brett House (24:57.504)

Yeah, no, just to set metrics around very specific media types that you're literally saying these impressions are not the same, they shouldn't be valued the same, and they shouldn't be measured the same. When you start to have weighting in your measurement algorithms, you've got to push certain things down and certain things up. How do you actually integrate that from an actionability standpoint into the media planning, the media buying ecosystem? What are your recommendations on that? Great.

that now, how do I actually make that happen as a marketing leader, as an agency?

Erez Levin (25:32.165)

Ultimately, mean, these things need to be sort of experimented with. I think one of the big risks is because we can't do perfect attribution for the vast majority of these instances. Like if there's ever an exception, it's an exception, right? And for the most part, one of the big issues that I've had since the beginning is most of these channels and formats, there's a big range of quality within them. You have some really high quality stuff and then you have some a lot, a long tail. These are not monoliths, these channels.

Brett House (25:46.22)

Yeah.

Erez Levin (26:01.518)

And the problem is if you just average them out, it looks like it, maybe it looks like it performs well on average, but you're actually just filling with a ton of this low quality stuff that maybe costs a little bit less and lowers your average CPM, but it's not actually performing well. And so what you want to do is sort of break apart this media into homogenous sort of formats and channels and at least test on that regard, right? Because maybe that out stream, that small commuted video ad on a recipe site.

performs a little bit differently than on some other sort of on a news website, but they're probably within the same ballpark. And so starting to assess these things and again, going back dual time horizon, if you only look at short-term results, it's not gonna tell that full story. And so you have to find different proxies for that.

Brett House (26:45.354)

Yeah, let's talk about because the averaging out I think is pretty interesting, right? Because you talked about how

Rio (26:51.463)

That was a big thing, Doctor, if we said Brett about having, would think the average answer is a problem, right?

Brett House (26:54.964)

Yeah, the numbers sort of lie and they impact your MMM, your media mix modeling, right? When you're trying to implement a solution, right? You've kind of said that MMM really needs to evolve so that it starts to see all of these parameters so that when it gives you a prediction of like, you should, or a budget allocation change or shift, it should be accurate and not based on sort of aggregate or average data, right? Otherwise you're going to be, yeah.

Erez Levin (27:18.467)

Now when it's a big range, right? So if you're averaging and it's sort of within the close bound, then it's probably okay. But when you have a really big range from high quality, high value to low quality, low value, and a huge range in the sort of CPMs, let's say, within them, that can be really, really problematic to average out.

Brett House (27:38.135)

So it sounds like this is something that you could implement or integrate into workflow because a lot of this is happening both either in planning platforms, the is integrated directly into the planning platform or into the AI decisioning workflows.

Erez Levin (27:59.694)

That's exactly, quality has to sort of be within those. that can, it can come out, but I think it's just much harder when you have this sort of broad range of a channel. If you're sort of, you you're buying too broadly and you have like, you're just looking at all OLV, which includes that really high quality and the really low quality, that average is almost meaningless.

Rio (28:23.409)

So your report in a lot of ways, like I said earlier, kind of seemed like it was a CTV paper that also happened in a lot of ways to talk about quality. mean, which I thought was good, right? And if you look at some of the stats in there, I'm going to call out a couple here. A couple jumped out at me, like 80 % of programmatic display spend now transact through PMPs and programmatic guarantee or PG deals, which I thought was a little surprising. I don't think it would be that high. And then I think that

You kind of brought it full circle when you said that that recent ANA transparency study found that CTV buying is already 59 % PMPs versus only 40 % open exchange. So most of it's biddable, but there's certainly a trend. So does this suggest that maybe like we're already, the market's already rewarding transparency and rewarding quality or not really?

Erez Levin (29:07.054)

I don't know yet because I think sometimes the way these things are classified is misleading. So it says CTV, but like what was considered CTV is that things that were declared CTV and bought a CTV, like how many bars, TVs and things like that. So there's a lot of mislabeling. Some of these categories are way too broad. I do think that there's just like natural differences with CTV and it's important, right? That's why we put these in the paper.

I approach this stuff from a very sort of principled, universal, thinking about marketing effectiveness, how it works. This stuff should apply everywhere. Why it's really important because CTV, this new channel, it's emerging very quickly. It can't afford to get things wrong like digital display. And I just don't think it would even work. There is no attribution just does not work as cleanly. The CPMs are really high. Like it's not gonna be able to take credit for enough outcomes to offset.

Brett House (29:56.598)

Yeah.

Erez Levin (30:06.692)

the price that it is and so and the scarcity of the supply and so we're just going to see very different dynamics but you know I thought one of the real important reasons that come at this from a quality lens is to move away from that binary to move away from the premium narrative that sort of says like we call this premium and everything else is not that binary is dangerous not just cuts off scale it sort of makes it much more subjective.

where there is value here and so thinking about that range, I think it gives the opportunity to think about the long term, short and long term value. There's gonna be room for sort of more performance driven CTV that's like going to also be measured on a short term impact like TV is today, maybe a little bit more granularly. But for the most part, most of its value is still going to be for that long term brand building.

Rio (31:02.031)

If we're moving from, say an identity first, like outcomes focus to more of a quality first, like attention era, like what are the things that you think are going to matter most? Like, cause I, I think I agree with your thesis. We're entering this, we're entering an era where people care about these things more. It's overdue, probably overdue by a bunch of years, but it's, it doesn't mean it's not, not a welcome change, right? In my opinion. So what is going to matter most as we go into this, like, like how do things like

Like what signals are going to be important? Like how is this going to impact attribution and different measurement tools we look through.

Erez Levin (31:38.402)

I think it can probably predict what are some of the signals that are going to be most important, but a lot of its combinations. Ultimately, it's not pricing all media the same. And so what you're going to see is much more of a distribution across times of day, maybe across geo and not for every single advertiser before a lot of campaigns, they're going to say, I don't want to just distribute nationally sort of equally according to the population. The size of the ads, right? A lot of those more sort of attention prominence type of metrics based on different devices.

I think that's what's going to come into play and what we're going to see because right now if people run reports and they say, you know, how was my CTV spending by hour of day, they're going to see they're spending 30 % of their budget overnight, probably 80 % of their budget before prime time and spending the same CPMs, paying the same CPMs for every hour of the day. Like that is the issue. That's the core sort of call to action to marketers. Stop overspending on and or overpaying for lower quality media.

And I'm not saying that there's a right price or the right exact allocation. It certainly don't trust the algorithm, the platforms, know, algorithms to just spend, to fill your budget without accounting for some of these quality dimensions that you have to tell it sort of how to value those on a relative basis.

Brett House (32:53.216)

Yeah. And you talk about like in the paper, short-term efficiency can punish quality, right? That high quality inventory might look less efficient when you look at the ROAS or the CPA metrics, right? But I think you brought this up in the very beginning that it's actually critical not to lose focus, that it can really contribute to brand recall. All of those classic sort of halo effect brand awareness metrics that might, you especially with products that are sort of long-term decision cycles like car buying.

Right? Right. So talk about that a little bit about about quality and how short term ism can punish that.

Erez Levin (33:29.106)

Yeah. And this is where like I embrace proxy. Somebody's like, no, we have to move away from proxies and move towards outcomes. Like I reject that entirely because it's inherent marketing is essentially always going to work. Not like there's exceptions. You're an SMB. You want to run on, you know, this person was on your site. You want to remarket to them that can work in the short term. There are exceptions that prove the role, but for the most part, you're trying to impact in market and out of market customers and sort of build up that mental availability in that brand equity. And that brand equity is a proxy.

Brett House (33:56.46)

Yep.

Erez Levin (33:58.881)

to pricing power. It's a proxy to profit, to long-term profit, to long-term growth. The number of buyers that sort of entered that doom loop, whether it's Uber and Nike and Adidas and Airbnb and all of these marketers that we've heard all of their case studies that fired their CMO that sort of had to make these drastic changes because they chased this sort of short-term performance.

Brett House (34:01.249)

Mm-hmm. That's a good point.

Rio (34:22.381)

stop spending money and so on, no impact, right?

Erez Levin (34:25.077)

They focus on performance. They lost that pricing power. They lost that sort of competitive edge that they had that people were willing to pay a premium for their product. And this works with CPG. So it's not just the long-term. It certainly works with autos. Very important because most people are not gonna be in market and you have to keep reminding them. But even more people, and there's so many studies to show this, you see certain ads for certain candy bars and things like that. You're gonna be more likely to buy that product.

Brett House (34:25.632)

Yeah, yeah, or drive, yeah, or drive the, yep.

Yeah.

Erez Levin (34:52.503)

versus the generic or versus some other category product.

Brett House (34:57.078)

Yeah, yeah, especially if you're targeting people that are not not already customers, right? The movable middle like they call them that are category buyers, but might be competitive category buyers where you're getting incremental purchases that wouldn't have had happened without media exposure. That's the audience that you're really trying to reach. It's those people that are buying in the category, might be buying it from a competitor and aren't existing customers because because otherwise you just don't have incrementality, right? You're just you're just you're advertising to people that are already already have the propensity to buy your product, right?

Rio (35:21.543)

What? yeah.

Rio (35:26.267)

You think about for cars too, people are going to buy a car once every what, seven years, right? So that's why I think television advertising when you're, you know, it's broad-based, right? You're focusing on just households within a specific audience. You know that like a certain percentage of them are going to be in market within an extra several months. You hit them with those ads. like, mean, it makes sense to me, like looking at like this, okay, so we switched to more.

You know, from the old approach, identity and outcomes, which was just something that's more focused on attention, focused on quality. Like what does this do for the open web though? Errors. mean, I know you've been a champion of the open web and also a critic, right? But like it's the world gardens are effective because they have identity nailed a lot better than open web. As we see, remember the Scott McKinley report last time, Brett, like what do you have 13 % match rate to actual like, you know, bait, you know, to actual. People, which is crazy. Right. So a lot of the signals are terrible. Right. So, so let's say you have like.

Brett House (36:09.324)

Yep.

Brett House (36:17.888)

Yeah

Rio (36:22.567)

Does this maybe create an opportunity for the open web to show that, okay, like, should be spending your money here, you know, as, as, as it may be not as opposed to the walled gardens, but in addition to the walled gardens, right? Cause you can see why it was so attractive to spend money. can show attribution, everything's closed loop. They can evaluate where the content show. can, they can tie it directly to outcomes much easier. So thoughts on that.

Erez Levin (36:43.586)

Yeah, well, they can show attribution. It doesn't mean it's incremental. It doesn't mean it's accurate. Rick Bruner, or no, sorry, it Rick Bruner. was Chris Williams, but Rick Bruner reposted it, but he was talking about multi-touch, MTA should be multi-touch analysis. Multi-touch analysis, I'm like, I was like, oh yeah, obviously that's the much better acronym. So yeah, so I don't trust what they have. Certainly their audience data is much more accurate. Multi-touch analysis.

Brett House (36:48.097)

Yeah.

Brett House (36:57.67)

my god, was... Yeah, I was gonna ask, that was my next question.

Rio (37:01.308)

I love it.

Brett House (37:08.275)

what was the acronym? What was he calling it? I'm sorry, I missed that. As opposed to attribution because it's technically not attribution because you've got a loss of signal, you've got a loss of identifiers, right?

Erez Levin (37:12.992)

Yeah, exactly. Yeah.

Yeah, it's a good analysis. So it might hint at some things for like sort of what to attribute to, but let's not call it attribution because it's not actually telling you how to attribute. It's maybe giving you some signals towards that, some nudges. But the, the walled gardens absolutely have very good audience data. And so on that, the open web broadly can't compete. Now my definition of the open web, and I think it is the better one and nobody has presented an alternative that I think is principled and thoughtful.

Rio (37:18.491)

Yep. It's a guess.

Erez Levin (37:46.668)

But I include CTV in the open web. The open web to me is everything that is not the walled gardens. And the walled gardens are defined by they have a buying platform, the corporate entity has a buying platform and only they can buy their own owned and operated media. So even though they buy externally what they buy in the O &O, that you can only buy via that platform, that's what makes them walled gardens. And so Netflix is not a walled garden.

Brett House (37:53.505)

Yeah.

Brett House (38:12.96)

Yeah, and some of the CTV fits into that category, right? I if you're a Netflix or you've got the majority of your inventory behind those walls, you're still functioning like a wall garden. Yeah.

Erez Levin (38:20.801)

Netflix is open, You can call them a hedged garden, but they're open. Exactly. They're open. It's only the big three are going to be in this walled garden where their O &O can only be bought from them and then everyone else. I like this expansive view because the open web has some really high quality stuff.

Rio (38:25.457)

You can buy from Amazon DSP. You can buy from Netflix. From the trade desk. Yeah.

Brett House (38:28.031)

Yeah.

Brett House (38:35.318)

Okay.

Erez Levin (38:42.505)

Netflix is one of them, but all CTV and then plenty of other stuff that's on the web. Even you're talking about like news publishers that are high quality news publishers. You're talking about like the newspapers and things like that. They are high quality. They're generally undervalued today because everyone is just measuring according to, you know, attribution and short-term outcomes and things like that. So those should all benefit when buyers start to value media quality. There is also a really long tail.

Brett House (39:08.737)

Yeah. Yeah.

Erez Levin (39:10.931)

of media that is very low quality in the open web. MFA and things like MFA.

Brett House (39:14.816)

Yeah. But yeah, and if you apply an MQ and some of these metrics to kind of filter out or separate the wheat from the chaff, right, that there's a way to say, hey, we can, you might reduce scale, but you're really focusing on real reach.

at a very high quality across a bunch of different dimensions so that you know that you're going to just have increased receptivity and you should see that in your end results. So this to me sounds like a good solution for the open web. It's like, if we can actually confidently say we're separating out the chaff, we're sort of focusing on quality content, CTV or otherwise. it's also sort of you kind of redefined how people think about the open web. Often people think display inventory.

Erez Levin (39:39.609)

You would absolutely say it.

Brett House (40:01.131)

Right? Programmatic display versus CTV, which is a display inventory, but it's, it's categorized as on the television class, much higher quality of sort of sound site motion. Right? So it's, it's a stronger inventory, arguably for recall.

Rio (40:15.495)

Well, but, who was saying, obviously remember Brett, thought that was interesting. Like, he, he actually told us, you know, if you listen to episode errors, but he said that he did not consider legit CTV as open web at all. He said, there's a lot of CTV that's open web, but that's right. That's bullshit. Right. And then, then, and then, and then, sorry. So then he, then he also said to me, remember he said almost like don't buy through DSPs because he said only buy places, the places where you have owned and operated, you can buy through an advertising portal. That was kind of his, and that was kind of, I thought it was provocative.

Erez Levin (40:21.984)

Yes.

Brett House (40:27.183)

That was my point! That was my point with that-

Erez Levin (40:28.168)

and I and and I

Erez Levin (40:45.055)

So don't disagree on that point. I'm not saying you should buy this stuff programmatically, but I still think the definition making that part of the open web as in not in the walled gardens is a stronger, more expansive definition. Cause otherwise everyone's drawing an arbitrary line. This is open web, it's web, it's display, stop, it's display. It's not app. That's arbitrary. Let's make it really clear, which is basically walled gardens versus the rest of the web. Now, whether you can buy it programmatically via DSP or only direct, that's also drawing an arbitrary line.

Rio (40:45.479)

Thoughts?

Brett House (41:01.312)

Yeah

Erez Levin (41:15.229)

And so if Netflix decides there's nothing going to be open, they're still part of the open web of advertising that you can buy, you know, potentially through a DSP and a PG deal or via them directly.

Brett House (41:23.734)

Yeah, well.

Yeah. And my question around Netflix and the other kind of big CTV players, the Hulus, et cetera, the Paramounts, you know, what percentage on average do we think their inventory is direct buy only? So it's closed off to the open web or to the DSPs versus the inventory that's actually sent out to the buying platforms.

Erez Levin (41:46.975)

I'm 90 plus percent is bought one to one direct, maybe in a PG deal of sorts. I assume there's still so much more demand than supply for the industry.

Brett House (41:51.244)

Direct.

Brett House (41:55.724)

Yeah.

Brett House (42:00.215)

Yeah, and you think they have sort of a dynamic way to manage that, right? For unsold inventory, they can push that inventory to the DSP to fulfill.

Rio (42:06.929)

So it's basically Remnant would then go out to the DSPs that you could buy through like the trade desk or Amazon or anywhere else. Okay. No, you weren't.

Brett House (42:11.266)

Yeah. So I wasn't that off with the Netflix comment, right? I mean, it's 90, 80 to 90 % of their inventory is actually direct buy. But to your point, yes, it is still part of the open web. It actually strengthens.

Erez Levin (42:22.576)

It could be 95 and 99%. Like it could be that high. They could have almost no remnant. And honestly, I think a big factor here is the quality control on their side. The way that Disney, from what I hear, they hand review every single creative that's being served on their platforms. Netflix might not do the same, but I think right now the scrutiny for these ads...

Brett House (42:45.856)

They hand review. That's that.

Erez Levin (42:46.897)

Yeah, like human review for every single one. I don't know if they have an animator looking at it, but they have a person looking at them. I think the quality control for them is too high because my view is if you're watching TV and you're seeing all these repeat ads, you're actually, I'm less mad at the advertiser. more mad at the publisher. Maybe I know too much about the space, but I'm kind of like Netflix, like your ad experience sucks. I'm not mad at Bounty.

Brett House (42:51.201)

Yeah.

Brett House (43:05.25)

No, I know.

Yeah.

Rio (43:09.935)

It's terrible. That's the worst. When you have four or five times like.

Brett House (43:11.81)

Yeah, no, and you see this, oh yeah, you see it multiple times and I'm finding, I'm finding unfulfilled, let's say remnant inventory, like when I'm watching Prime especially, from a sports perspective, when they have the Thursday night games or some of the playoff games, I'm finding there's periods, I've had periods of both on Peacock and Amazon Prime where they have no ads and you're just getting that sort of to be continued or come at, and you get some elevator music in some generic background.

Erez Levin (43:14.693)

And so it's their reputation, yeah.

Erez Levin (43:34.747)

Mm-mm.

Brett House (43:38.53)

Or you see the same ad over at literally every ad break every ad spot

Erez Levin (43:41.809)

And I suspect a lot of that is just like there's enough demand, but they don't have it that meets the quality criteria that the advertisers or the publisher has. Like the competitive separation, all that kind of stuff that's built into TV that they don't have. And so like they just, those slots go unsold, even though there's plenty of people that would love to serve their ad there, they just can't connect those pipes or they can't match the price.

Brett House (44:03.232)

Yeah. Yeah. And it was actually a pretty jarring experience. I was on Peacock and it was a playoff game. I think it was the first round of the NBA playoffs. And I said, I'm going to turn on Peacock. I just happened to have the Apple TV on. So I went to Peacock to watch it. And I kind of experienced the ad experience there, which was one, a bunch of blanks, right? A bunch of repeats.

some questionable quality sort of breakage. And then I was like, wait a minute, this is on linear television as well. So I went over to NBC and it was just a radically different experience. And I was like, wow, there's a huge dichotomy or huge difference in the quality of ad delivery and also how diverse the ad delivery is on linear versus CGB today. maybe that's just a peacock problem, but I'm seeing this on CGB across a couple of different channels.

Erez Levin (44:51.707)

I'm here at the everywhere just to some degree. I wanna go back to one point that you were asking about Rio, like the quality versus outcomes sort of piece too. I believe a lot in outcomes. We have to be able to prove the effectiveness and we wanna call it outcomes. I just don't like the word outcomes and especially sort of setting that as a North star. think it's too easy to game. think that's what we've been doing for two decades. What I like about quality is you kind of have to prove, you have to show the math.

Brett House (45:07.169)

Yeah.

Erez Levin (45:17.581)

I can't just go to my CMO, the CFO and say, look, I bought this quality. You then have to show the receipts of how that quality translated into short-term outcomes, long-term outcomes, versus when I go up to them and say, look, I got outcomes, I got this ROI. Sometimes they're like, good. They kind of like, that's good enough. So what I like is qualities defined by its likelihood of driving outcomes that we can prove in this sort of probabilistic way.

And so I thought that was sort of a critical piece. Now I listened to you guys had the guy from Newton research on and something I've been really adamant about for a long time, Ethan. Yeah, exactly. From the agentic side as well as from the MMM side. And this is for measurement in general. What I think is really critical for any of these more automated systems that we're building to prove the effectiveness of media.

Brett House (45:50.713)

yeah.

Ethan.

Erez Levin (46:11.377)

they generally for enterprise marketers have to do it on two time horizons. So they have to see the short term impact and some proxy to long term impacts for the same campaign, not every single campaign, but you have to have some sort of proxies for short term and long term impact at the same time, because otherwise you're just going to prioritize most likely the short term. And that is most often going to be at the expense of the long term.

Brett House (46:22.839)

Yeah.

Brett House (46:34.636)

Yeah.

Yeah, and it's a nice.

Rio (46:37.617)

Yeah. How's that any better than prioritizing ideas over non-IDs, right? Which is, guess, one of the criticism of like the current way a lot of programmatic is bought when you look at it at DSPs. looking at DSPs, I mean, the Joe Root episode, covered some of this as well, is about how like, you you look at all the rich contextual signal. Remember that bread he was talking about, and like how only just like a tiny fraction of that is actually makes it over to buy side, you know, to actually make these decisions. Right. And then because of that, that's why they prioritize based on ID, because it says, well, at least we have the ID. We know, we kind of know who this is. So.

Brett House (46:54.592)

Yeah.

Brett House (46:58.22)

Past. Yep.

Rio (47:07.557)

anything, not idea. We're not even going to bid on that. So looking at like what you're like, all of this, these are new signals you're proposing, right? And clearly, I guess you could build algorithms to look at that. But how would this, how would this actually be like implemented at the, let's say at the demand side within a D are there DSPs that are doing this? Are there, are there, are there like, how, does this actually, like, where does the rubber meet the road in terms of making this reality?

Erez Levin (47:32.303)

Yeah, I mean, these are custom algorithms. So, know, DB360 has that baked into DSP. You've also got the Chalice and the CyBids. And so that's what they specialize in. They were, I think all of these products originally were focused on optimizing towards conversions, towards some sort of identity, some sort of conversion. And this is where I came in at Google and said, like, we need to build in the quality signals as well, because we're not always going to have the identity and the conversions and there's going to be value to assign.

And actually, Brett, liked MQ as the acronym. It was IQ. It was impression quality. I was like, so genius. And so the idea was bringing in these quality signals as well for buyers to either just use those or use those in addition to the conversion signals, the floodlights, or any of the identity signals as well. So that exists from a custom bidding standpoint.

Brett House (48:07.298)

Yeah.

Erez Levin (48:25.541)

But there are other ways, even within, you the example I give in the paper was the hour of day on CTV. In a biddable environment, you can tell the algorithm the set, you know, certain priority relative value for overnight daytime and then prime time. And then it's going to seek that out, price it accordingly, shift some of your spend. But in most of these sort of, you know, premium scarce environments, there's not going to be biddable. This is where the opportunity to go and do that negotiation, just like in the broadcast era, the broadcast era, the quality controls was

don't run my ad overnight. Like that was a quality control. Now I don't advise for most buyers to say no overnights. But what you can do is go negotiate the deal and say, I don't want more than 20 % of my budget to run overnight. And I want at least 30 % of my budget to run during prime time. Now as a seller, if I'm the publisher, I'm gonna say no problem, but you have to pay me more, right? That stuff is more expensive. And I, as the buyer have to say, that's worth the price to me.

Brett House (49:19.648)

Yeah. Yeah.

Erez Levin (49:24.986)

I'm not gonna just give you a million dollars and you just buy the ads at any hour, any geo, or anything else that's unsold that's sort of the best margin for you. I'm gonna have to be willing to pay a premium for the stuff that I find more valuable.

Brett House (49:39.597)

So this gets to the point of we talked about how that applies to the buy and the sell side. Let's talk about the people that really control the money around here, the marketers. What should marketers do? And after reading your report, I came down with a handful of what the report was suggesting, which I thought was really interesting. I love your...

Rio (49:50.787)

The brand marketers, yeah.

Brett House (50:00.291)

Errors your feedback on this one is sort of and I think you just hinted at it audit your current exposure to quality variation Your current exposure, excuse me to quality variation, especially as it applies to CTV You've talked about that a ton today, right time of day content type geography ad load format CPM So so how would a market if you're if you're a CMO at a big brand? How would how would you begin that audit? Process is this a collaborative effort?

across your partners plus your team? Do you need to have a data science and analytics team behind this to actually get to the metrics that matter most to be able to sort of proxy quality and see what's quality versus not?

Erez Levin (50:42.805)

Yeah, so, you know, depends on the resources you have and everything else, but like the way I like to red pill everyone because it's so easy is just pull a report of your spend and CPMs by hour of day on CTV. And if that curve looks like, that's I don't want to be spending this much of my budget overnight and throughout the day and not have anything left at prime time and your CPMs are flat the entire time. That's like a wake up call. That's a hey, that's

probably not ideal for me. I'm not saying it's the worst thing ever, but are there things that I could do? Do I want to now inform my algorithm or in my deals and say I'm willing to pay more or I want to spend more of my budget during different hours of the day? And that's just the first example, right?

Brett House (51:30.474)

Yeah. And that's a simple place to start, right? Because that's oftentimes like, what is the starting point? So how about marketing media quality? I almost said marketing quality, which I guess is the end result here. Media quality as a continuous pricing and sort of allocation variable. Because I think a lot of people might think of it as a hygiene filter, right? Like I want to be associated with quality content at quality times of day for max intent or engagement.

So where would they start starting simple when it comes to that sort of continuous pricing and allocation sort of variable?

Erez Levin (52:07.2)

So, know, I again, I think the like time of day is a good place to start because it just like opens up the door of what else we can go for. And then this is up to the buyers to then determine how they want to look at these things, because it really is up to the campaign to evaluate it. Does geo matter? Right. Do I care if I'm spending nationally distributed across the population or are there reasons that I want to prioritize for seasonality or other purposes? There's other, you know, somebody on the TV or certain content.

Do I want to be not in children's content? If I'm buying a ton of YouTube or I'm buying a ton of whatever that is and I don't have transparency into what I'm buying, that's sort of a concern. I do have transparency, but I want to look and see and I want to make sure I'm not spending too much of my budget on children's content. I'm not paying the same CPM for the children's content. And so that's part of this audit is really just to say, am I overspending on and or overpaying for things that...

are probably lower quality, because I think that's sort of the low hanging fruit.

Brett House (53:05.986)

And should they require it in the RFPs that they're sending out to either agencies or otherwise? I how do you enforce this at the RFP level? And are our brains currently doing this today?

Erez Levin (53:14.528)

Absolutely. They're doing it with other things, right? Like there's plenty of things that could be called quality, right? Like all of the verification vendors and fraud and viewability, those are quality too, but those are filters. And this is why I like moving to this continuous range where I'm not saying middle of the night is worth zero and you shouldn't buy it overnights. You're gonna hurt your reach and you're gonna miss out on some potentially good inventory, but pay less for it and don't spend too much of your budget on it. And so this is moving beyond those sort of hygiene filters.

Brett House (53:20.15)

Yeah.

Yeah.

Brett House (53:26.101)

Yeah.

Brett House (53:38.849)

Yep.

Erez Levin (53:43.493)

into this sort of a qualitative assessment of value based on the predicted likelihood of that sort of driving various outcomes you care about.

Rio (53:52.551)

Yeah, it's a good point. Yeah, I think you made about brand safety, brand suitability and some of the other filters. They're blunt force objects, as we've just talked about a few times in this bread, you where it's very, you know, they'll block entire domains. They'll block keywords. News publishers have certainly not had a good outcome because because of a lot of this. But one point I like, though, is how like you were saying, well, these things like brands have relied maybe too much on these things, almost like a crutch. Right.

Brett House (53:59.553)

Yeah.

Rio (54:18.075)

This hasn't given you, this hasn't guaranteed you more quality, right? It's just prevented you from serving up ads on some certain pages or sites, but it's given a false sense of assurance, I think almost. And that seemed to be one of the points you're driving. Is that fair to say, Iris?

Erez Levin (54:29.814)

Yeah, it made me think of the Tommy boy, you know, which line I'm thinking of. You put a guarantee on a box, right? And so, yeah, it gave a false sense of security, I think, to buyers. And again, it's not false sense of security. It did ensure that they weren't spending on things that they said that had zero value or too much risk for me from a brand safety standpoint, suitability or viewability. Like, yes, but that filter just said everything above that is worth the same, and it's not.

And so everything above that still needs to have a relative value, which isn't just determined based on the audience.

Rio (55:06.449)

think it's sometimes maybe almost had the opposite effect. They ended up not serving up ads in a lot of what I would argue are high quality premium publishers. And they ended up prioritizing a lot of MFA and you know, another recipe sites.

Erez Levin (55:16.97)

That's a separate story that I totally agree with. That's a risk level that I think is a little bit hard for me to give abroad. Some marketers overly risky or overly risk averse and avoided things that they probably would have been fine. Actually, probably a lot of them, but I understand your finance and you're just like, no, by zero margin for error risk on this.

Brett House (55:17.674)

Yep. Yeah.

Brett House (55:23.201)

Yeah.

Brett House (55:38.986)

Yeah, yeah. And then the risk associated with news media. just talked about that with Vanessa Otero, the CEO of Adventist Media. Yeah, there was there was a lot of a yeah. Yeah, a lot of aversion. Yeah.

Rio (55:44.204)

That's a good one. I'm excited for that one to come out, Yeah.

Erez Levin (55:45.972)

that she's talking about media quality, but that like, I'm a huge fan of what they're doing because that is another way for some buyers to say, I'm going to value higher, whatever, integrity, lower, more neutral media versus this super polarizing, much more sort of partisan, much less authoritative media source. And I'm not saying don't buy them, pay less for them at the very least.

Brett House (56:10.048)

Yeah, would you? Yeah.

Populist on both sides and I think the suggestion, it's just smart because you're taking a nuance to your point Rio about the blunt force instrument, the keyword blocking and all of these techniques with media quality. Visibility doesn't mean, or viewability doesn't mean media quality. Down funnel metrics like clicks don't mean media quality. There's all this other nuance that goes into assessing what real media quality is and it should have a down funnel impact on

on your measurement, right? Like when you start to see, cause we've all been fooled by ROAS metrics at least once or twice in our careers, right? Where you're like, the ROAS looks phenomenal. Like this is incredible. And then when you, when you, and I've been fooled on this, at the, at the B2B side, then when you actually look at your, technical attribution and you start to try to tie actual revenue earned back to the terrific ROAS metrics, the medium team was showing you, and you just can't do it.

and then nobody believes you, you lose all credibility. Right? From the measurement perspective.

Erez Levin (57:13.993)

Well, listen, there's easy to fool on ROAS, but like multi-billion dollar companies were just chasing ROAS, which is easy to gain. It's very easy to turn the dial and increase your ROAS at the expense of the long-term. And too many companies did that. Fortunately, some of them saw the light and have turned around and realized the errors of their ways.

Brett House (57:24.271)

yeah.

Brett House (57:37.238)

Yeah, well.

Rio (57:37.285)

Yeah, there's the Nike example we've talked about a few times in this pot. How they just, dumped everything into performance. They just chase ROAS. ROAS is super high probably for several years. And then they realized like all the things you mentioned earlier, Brett, right? Like the, like the, you know, the memory of the brand awareness of its new products. All of those things just fell off a cliff and, you know, and they, they really got hammered over time. So I was looking at like, if we are going to...

Brett House (57:42.113)

Yeah.

Brett House (57:57.577)

Yeah, yeah.

Rio (58:02.535)

Like succeed in this new era. what needs to change? Brett, you mentioned the brand marketers. Like is it, is it there? Do they need to start? Does it, does the change need to start with them? Does it have to start with media planners or buyers? mean, I actually have a lot of confidence in the tech. whether it's, know, again, we had the, you know, that you've had some really cool AI companies on here, hearing the new features, even some of the legacy companies coming out with, and there's some pretty cool stuff going on. Like, but what needs to change to make this happen? Like what behaviors, what behaviors need to start evolving?

Erez Levin (58:29.844)

The marketers, absolutely the marketers. There's sort of two things. One, they have to be able to be willing to and able to and figure out how to price the value, the quality of media. And two, they have to do that not by averages, especially these sort of like wide distributions of averages. as long as they... Absolutely.

Brett House (58:49.666)

Yeah, and it gives them it gives them negotiation leverage, right during. Yeah, and that's a huge play. It's like, no. And then and then you start putting those demands in your in your RFP start there gives you negotiation leverage and you say prove it, prove that you're delivering quality inventory that's going to try.

Erez Levin (59:03.57)

You know, I worked with an advertiser, this isn't like in the report, but I sort of present this. I worked with an advertiser and they did a, we built a custom bidding algorithm and we had a high, medium and low quality sort of video. And we defined this with like, you know, time on screen and the audibility and the player size and things like that. And we built these algorithms and we prioritized for them to spend more on the higher quality stuff. And the net result was they shifted a lot of their, what was low quality media, they shifted it towards the medium and high quality media.

were able to buy that stuff still at the same price. But because they bought less of the low quality media that was a little bit less expensive, their average CPMs went up and that was a red flag for them. And so that is sort of the big risk here. So if you only care about cost, but if you only care about cost and not the value and the quality and you sort of look at these things superficially, it's really easy. And so this is where it has to start with the marketers.

Brett House (59:47.69)

You get what you pay for. mean, come on, you get what you pay for.

Rio (59:49.286)

Yeah.

Brett House (59:53.28)

Yeah.

Erez Levin (01:00:01.239)

And it's the marketers having to work with their finance, the procurement teams to help explain to them, hey, our average CPMs are gonna go up. We're gonna prove to you why it's worth it, why this is actually gonna deliver you not just better short-term results, but also brand health, brand equity, whatever the sort of other proxies to long-term value that they're actually going to prove as well.

Brett House (01:00:24.064)

Yeah, and you can automate a lot of those.

Rio (01:00:24.391)

Yeah. mean, the agency should be, they should be able to, like, they should be, I mean, they should be like able and willing to do that. Right. I totally agree with you on that, but I think, yeah. But I think that.

Brett House (01:00:28.074)

Yeah. Yeah.

Erez Levin (01:00:28.411)

People are doing this. People are doing this today.

Brett House (01:00:33.398)

Yeah, and it's easy to automate the proxies for the upper level metrics. You can do that through automated surveys. There's little pop-ups. And Cantor has been doing that for decades. There's ways to get reads from a statistically significant sample of people being exposed that unaided brand awareness and recall. These go back to the foundations of marketing, the Byron and Sharpe view of...

Erez Levin (01:00:44.303)

It exists.

Erez Levin (01:00:57.924)

Byron Sharp, yeah, exactly. yeah. And I said, I said, all that, listen, the quality error is here. It's just not evenly distributed. There are people doing this end to end really well. And there's plenty of people doing this sort of piecemeal. We need to get sort of more folks going and doing sort of end to end measuring well, buying well, especially in the digital era. This works well on TV. A lot of folks are doing this well and right in TV. Digital just fragmented so much.

And we didn't have the quality language and frameworks of how to assess the uniqueness in these broad channels and helping understand how to actually value that media.

Brett House (01:01:26.144)

Yeah.

Rio (01:01:36.071)

Yeah. I think a lot of it is like bad behavior, like from what I've seen is being driven probably by procurement, right? Cause they're saying like, we're going to evaluate based on CPM, CPMs with major pubs. That's it. That's, that's the, know, and then, you know, after that, then we'll then look at different things. So I think that the way that these account reviews are handled by procurement, the fact that the agencies, I mean, the agencies complain it's a race to the bottom, then, you know, they're, but then, you know, they'll try to win based on just having every employee totally.

Brett House (01:01:36.321)

The.

Brett House (01:01:49.567)

and

Yeah.

Erez Levin (01:02:01.272)

Everyone plays the game. Everyone has to play the game.

Brett House (01:02:02.613)

Yeah, they're incentivized to maximize the budget spend. They have to spend the budget. And if that means cheap reach, that means cheap reach.

Erez Levin (01:02:08.386)

Ev- ev- ev- everyone-

everyone's incentives are misaligned and I don't like to like point fingers because especially without knowing like broadly, right? Sometimes it's the marketers that didn't push back enough on procurement, right? And sometimes it's the procurement that just wouldn't listen and just were set in their ways and said, I don't care, just get it cheaper. Like there has to be a balance that everyone has to be more courageous and push back and say, this is how marketing works. We're going to prove it, not sort of a sacrifice on their values.

Rio (01:02:22.886)

Yep.

Brett House (01:02:34.914)

Yeah. And to me, yeah, well, and then, they have to be able to prove it though. The marketers have to, and it comes down to incremental incrementality testing, testing, experimentation and incrementality testing campaign reporting. That's actually accurate because you know, you've had CMOs that get up in front of a board and present ROAS metrics and, and, you know, are super proud of what their teams have done.

Rio (01:02:35.685)

Yeah. Well, trust us. the marketers. You have to trust us. We know what we're doing. And like, but I Yeah.

Brett House (01:02:58.21)

And then yet they're not seeing the bottom line impact, right? So how do you embed quality into everything from your strategic planning, which is your MMM, down to your incrementality testing and measurement? Because without that, you're still going to be presenting kind of false numbers or false indicators, right?

Erez Levin (01:03:15.3)

Yeah, and again, there are marketers that are doing this, so it's possible. I think the challenge is that everyone sort of figured it out for themselves as opposed to had like industry standards and best practices. And this is what we're trying to do. And so I'm hoping that this paper creates that sort of shared sort of language frameworks. We all agree. We have a consensus on how to think about these things. And then we can start the conversations on how to build these things into governance. do other folks that might, you know, we had consulted with so many different people.

Brett House (01:03:28.119)

Yep.

Erez Levin (01:03:44.716)

many of whom have different quality-based solutions, whether it's on media or creative or audience quality. And my goal is like, this paper may a thousand quality initiatives bloom from this, where companies can say, yes, this is sort of this idea, this is where we plug in. And we have sort of something that's overarching that now everyone's like moving in the same direction. So we move the ball forward, we put on the field and now everyone can sort of move it down the field.

Brett House (01:04:11.798)

Don't you love this about Eris? He's always championing some great cause, right? Whether it's taboos, right? I'm serious, because I'm inspired by you. You're super passionate. You're super analytical and data-driven, which means you've got reasoned arguments that are provable. You're not just pulling shit out of thin air, right? But no, because you really do care.

Rio (01:04:12.197)

I made it.

Rio (01:04:33.735)

And Brett, he's not just, he's not, he's not just, he's not just saying this is like, he's not just calling like the baby ugly too. He's also saying like, I have in, this is how you could potentially fix it. don't have all the answers, but here's, here's my, here's my thesis or hypothesis about what we should probably be looking into. I think that's a smart way to go about it. Right. Cause there are a lot of people who can, it's easy to point out all the things that are messed up in this industry. Right. And a lot of it, I do agree with what you said a minute ago, is outcome is.

Brett House (01:04:39.371)

Yeah.

Brett House (01:04:54.453)

Yeah.

Rio (01:04:59.141)

based on incentives, right? There were incentives to spend the budget, there are incentives to have low CPMs, there are incentives to do all sorts of things that drives very many bad behaviors. So I like that, yeah.

Brett House (01:05:10.304)

Yep, yep. Yeah, and there's only so much of the like the black hat. I use that all the time. But like the I won't call out Dr. Fu, but just the you need people there to throw out to say like, you know, don't like to throw out the bad stuff. But but you also have to say, OK, well, what are the the the

the solutions that we can start with on day one to actually be able to incrementally make change. And that's oftentimes the hardest part is that you're hard to make that shift. And sometimes, just like with AI adoption, it just takes the first baby steps in order for you to get to a point where you're more comfortable with this and that you start to demand the same sort of...

You know the same sort of quality from your vendors from your agencies right your your the accountability becomes a big issue

Erez Levin (01:05:57.396)

And this is real, yeah, and realigning the incentives to your point, like that I see it all, right? This is a systemic problem. It's an accountability sink and we have to realign the incentives. And I think that starts with the marketers where they spend their budgets, how they spend their money, what they check or what they don't verify and sort of what they prioritize in value. That's how we got MFA, right? Like just like fast forward, MFA is the result of marketers caring about quantity and not quality.

And so there's various other examples just like that. And so as soon as the marketers create different incentives and say, we are not going to overspend on and or overpay for things that are lower quality based off of the time of day or how many ads are on the page or all these various other media quality dimensions, just because a cookie was there, that's not enough for us to spend a ton of our budget and pay the same CPMs for. As soon as enough marketers do that, this is going to change. Not only because those marketers will create incentives for

for the rest of sort of tech to offer those solutions and agencies and everyone else to sort of meet their needs. But it's also going to create the incentives for other marketers to follow suit. Because if you have marketer A in a certain category, if it's Coca-Cola that decides they're going to start buying with quality and things like that, and Pepsi doesn't follow suit, Pepsi is going to lose market share. Like it is just objectively true. They're gonna keep spending too much of their money on low quality. They're gonna waste it. And so...

Brett House (01:07:12.374)

Yeah.

Erez Levin (01:07:17.59)

they're going to have to follow suit. And so we just need to get a couple more of these marketers to sort of start that snowball. And I think it's going to force everyone else down that path as well.

Rio (01:07:27.365)

Yeah, well, could certainly, I certainly would not, it would be overdue in my opinion. I give this example a couple episodes ago about how, people like to blame agencies for a lot of this, but I think you're right. lot of the accountability really needs to start on the, on the brand. So like, remember I saw one RFP. It was actually an O like one of the auto OEMs and like, they were basically asking for close to zero margin for the agency, basically saying like, we expect you to find the money. It's basically we expect you to steal from us and like, go ahead and do it. Like, in like, we, like, we're not like,

We, and we're going to value it. This just based on it. And I had another one where they actually, I remember we were trying to help them with some in housing work. And then they said, actually we're going to send it all to India because they'll do it cheaper. It's like, that's the mentality of this driving this, this, this driving. Like that's what, that's what made the MFA industry bloom. Right. There's like the, just the quest for local. Totally.

Brett House (01:08:15.168)

Yeah, it's the low quality, it drives the low quality reach, right? And the low quality reach can be just, you may not even be exposed to human beings, right? So people don't realize how much that's been gamed. And there's a gray economy where people are gaming that system to...

Rio (01:08:24.807)

Probably a lot of it's not.

Brett House (01:08:31.49)

Yeah, and it doesn't result in other than the ROAS metrics. Right? It doesn't really like that's why like incrementality testing and measurement is so critical that pulls in this nuance. Because if you're not getting that right, you're not able to prove that like, hey, listen, procurement, need to drive, we need to spend 50 % more on our CPM rates across these channels, because otherwise, the results aren't there. And I can prove it both.

Rio (01:08:37.788)

Right?

Rio (01:08:58.823)

Yeah, because even if you are serving up to humans, it's some dude at 2 a.m. in Bulgaria on his his Android, right? mean, is that is like that's worthless, right?

Erez Levin (01:09:03.294)

E E

Erez Levin (01:09:07.751)

It's the performance zoom loop. like, even if you get those folks to buy, it's not going to build your brand equity. And so that is, you're just going to have to keep paying for every single customer. And that's not sustainable when you have competitors that are actually building their brand equity and people are going and buying from them by choice, by name, not always just in response to an ad that they happen to get served when they were in market for the product.

Brett House (01:09:31.093)

Yeah.

Rio (01:09:31.377)

How did, how did some of these newer formats, like I'm really like, like, there's been this big focus in, real life. And I think like out of home and like, and I think audio is the same category. I think category, these are, these are formats that are like undervalued in terms of the overall spend going there. And part of the reason undervalued is hard to, it's hard to actually like calculate like incrementality or, or look at, or even look at ROAS or even understand the effectiveness of some of these. I, but I think there's some really interesting

Like, are you seeing anything interesting in the quality space related to some of these newer formats?

Erez Levin (01:10:03.314)

You know what, I haven't focused there too much because they feel to me like way too much of the Wild West because the standardization of the classifications for those formats isn't there. And so with audio, like how many, I did a survey a while ago asking people like who skips the ads in the podcasts they listen to.

Rio (01:10:13.105)

Yeah.

Erez Levin (01:10:27.123)

Right. And I asked, you know, like questions like that. And it's a lot of people, but there's no signals. We don't know that right with video, you know, when there's a skip rate, we don't know on podcasts. so yeah, 75 % of people are skipping. don't know. I'm making up a number and the more, the more ag load goes there, but there more are get skipped. so the attribution is broken and weird. That's not to say there isn't value there. I just think there's a lot of big challenges with every platform having different skip features or not. And then without a home.

Rio (01:10:33.575)

We don't know.

Brett House (01:10:36.108)

Yeah.

Brett House (01:10:56.382)

There's no standardization across the yeah across the audio streamers

Erez Levin (01:10:57.194)

Yeah, and without a home, sort of the same thing. And so like, you know, I see a subway car with like a million digital ads all over it. That's great. If Apple is, you know, it was Apple that was bought that first one that I saw in New York and they did a takeover. And so they're buying that via a deal and that's great. But imagine that was bought programmatically and all of those different ones and who knows how many people saw those, each one of those little ads. So there has to be some quality measure. think traditional out of home had those, right?

Rio (01:10:58.779)

That's tough,

Erez Levin (01:11:24.636)

It knew what angle to the highway it was on and how many people were driving at certain hours of day. And so that was baked in in the digital world.

Brett House (01:11:30.603)

Yeah.

Yeah, like to the total exposure. Yep.

Rio (01:11:33.401)

Yeah, they would measure the traffic. They had those little lines in the road, they would see how many people are going. Yeah, you're right, there was a science behind it.

Erez Levin (01:11:36.906)

Yeah, so I'm concerned in digital. I'm concerned in digital, it's gonna be a lot easier if the buyers don't apply the same quality controls, ensure that sort of the, it's verified of how big was that ad, what time of the day was on, like all these kinds of variables. So I believe it can work. It definitely works. I believe in out of home, I believe in audio a lot, but I think there's a little bit of a too much of a wild west in the digital space that especially outside of a trusted deal.

Brett House (01:11:39.189)

Yeah.

Erez Levin (01:12:03.816)

with a known publisher, that you have a trusting relationship, you know they're not gonna risk the relationship for some short-term gain, potentially, that's where I think it's safe.

Brett House (01:12:12.982)

Yeah, like off-site targeting and things like that. Yeah. And I think the key point here is that make sure that the CTV ecosystem doesn't get completely polluted before it's too late. Yeah.

Erez Levin (01:12:23.627)

That one is clear. That one is actually the easiest, I think, to get right because it's almost a one-to-one of the TV space. So it's probably a little bit easier for us to get right first.

Brett House (01:12:30.945)

Yeah.

Brett House (01:12:37.14)

Yeah, but it's the open ecosystem where you're going across these hundreds of thousands, millions of sites where it's very, very difficult to control. Yeah.

Erez Levin (01:12:45.674)

You shouldn't be buying on that many sites.

Rio (01:12:46.587)

But you wonder how all this is. Yeah. I wonder how all this gets a FIMPacted though. I mean, clearly referral traffic is falling everywhere. Right. I mean, I definitely look at my own behavior, right. Anecdotally I go to far fewer websites that I used to still go to some, right. But I mean, I spend a lot of my time now within the chat, within the chat, prompts, the chat, little chat tools, whether it's, you know, Gemini or Claude or a chat GPT, right. So I'm definitely going to fewer sites and even like, I even like,

And I finally at possible, I had this dinner, was great dinner with Hearst. Right. And then like we were talking about, we were just spit balling, like ideas would share. then like some of the people were lamenting all that, you a of the younger, young audiences that they're just getting a lot of the content from, tick tock and from, let's say from shorts. Right. And, but, but then my comment was, but those same people was through a three hour podcast. Right. So just think content consumption patterns are changing so dramatically too, in ways that we, you know, that

that are hard to predict. I thought so. mean, I don't know if there's a question or a comment, but like.

Brett House (01:13:43.925)

Yeah. Yeah. And they're more nuanced. Yeah. It's more nuanced than I think we're giving it credit for. Right. mean, you agree with that? it's not just because like, I think we rushed to the thing of like, we have no attention spans. Everything's TikTok. Everything's YouTube and Instagram Reels. And it's all like quick short feeds. And I think Rio's point is that's not necessarily the case when you look at some of the long form content that is performed well, like podcasts.

Erez Levin (01:13:51.532)

Peace.

Erez Levin (01:14:10.652)

Yeah, one of the core attributes of media quality is it's multi-dimensional and it's dynamic. So it has to evolve over time, right? The media landscape changes, the content consumption changes. And so I totally agree. And this is where three-hour podcasts, I have one thing that I didn't put the image of it in the paper, but quality, when I sort of look at media quality, defining sort of media quality, there's two lenses and they sort of compete with each other.

You have the advertiser lens, how they assess media quality, and you have the consumer lens and how they assess media quality. And even for the consumer, it's a balance of what they're getting and what they're paying. So when you get something for free, you accept more ads, maybe even more intrusive ads. And so it's this balance for the app for the user. And it's this balance between the so the user, I love the podcast that has no ads on it. It's great. It's worth zero for the advertiser.

And so we always have to think about both of those lenses. And that's something that's just going to evolve because each of these platforms are going to have to monetize in different ways. I'm a big believer in more of that ad-supported web where somebody has to consent and watch an ad, a pre-roll, like a YouTube experience for more premium content and be able to pay for it that way as opposed to always being asked for an email or a subscription.

Brett House (01:15:31.458)

And speaking of context, was a report that I read, the 2024 FAME study by The Guardian, that found that low ad environments generate higher trust than high ad environments, right? And then the resulting brand favorability, right? Which is not surprising. And it just shows you that that's a metric. Like you've got it, like volume of ad exposure, yeah.

Rio (01:15:46.309)

That's not surprising, but it's interesting.

Erez Levin (01:15:52.874)

That's the balance. That's the balance, right? So it's like, you can have my trust with zero ads, but then it's worth nothing. So it's like, how do you get like to diminishing return? And also the advertisers have to be willing to pay for that trust. And so if they say, no, I just care about the number of impressions, only quantity, then how much trust somebody gets on a website because there was only one ad matters nothing to that advertiser.

Brett House (01:16:00.236)

Yeah.

Brett House (01:16:17.729)

Yep.

Rio (01:16:18.371)

Or, or, or do you like completely and shitify the experience? there's that, that retail media discussion, right? I mean, okay, you can keep like the merchandising people are like, no, we want fewer ads because like we want people to have a great browsing experience. They can find the products and get the product, but there is value to having ads clearly because you like people like it. If someone's looking for something suggesting the right product can be incredibly helpful, but then you can get overloaded, right? If there's too many ads and then then then eventually your, your, your revenue per revenue per.

The revenue per user collapses at a certain point, right? There is a lot.

Erez Levin (01:16:48.116)

And that's

Brett House (01:16:48.67)

yeah which is it's a pretty bad that frequency management issue that threshold of a certain number of exposure to the it's about seven before you just see it plummeting in performance because people just get at overloaded from a particular brand

Erez Levin (01:16:59.913)

And sometimes I like retail media, like in theory, there's a self-policing mechanism of sorts, because if they annoy their users too much and show them too many ads or bad ads, that actually hurts them, like they're actually money-making, right, from the core. And so they don't want to risk that too much. That doesn't exist everywhere on the internet. And everyone's just like, I'll just make the buck that I can make today. I don't care if no one ever comes back. And so this is where we see a difference between publishers, know, content.

Brett House (01:17:13.536)

Yeah.

Erez Levin (01:17:29.352)

that wants to build a relationship, that build that trust, have people come back and like the experience they have there versus ones that don't care. They just want transient clicks and impressions.

Brett House (01:17:30.55)

Yeah.

Brett House (01:17:39.54)

Yep, yep. So should we go to quick hits? We're moving on an hour and 17. We're approaching record time. But what do think, Raph? Yeah?

Rio (01:17:49.383)

Yeah, we could, we can do quick hits. did want to touch on it. I had long errors has I did want to touch on maybe ads and AI a little bit. pushed that article. pushed that. Yeah. I pushed that article out. You know, I don't know if you had a chance to read it or not and love your thoughts. mean, I can't, the thesis of the article was that like ads and AI are going to be a big revenue stream, but I don't think it needs to be as big. don't think it, what number one, I don't think it needs to be as big as it. I don't think there's controversial. I think it needs to be as big as it was for, let's say traditional searches because these.

Brett House (01:17:54.944)

Okay, so let's let's we'll do that and then we'll do quick hits. Yeah.

Erez Levin (01:17:55.049)

Let's do that. Let's do that. Yeah.

Rio (01:18:17.703)

people are already paying, a lot of people are the freemium versions, but so many people are paying $20 or $200 a month for these services. These are massive companies, even without advertising. But I think advertising makes them bigger and more successful, more profitable. And I think the second point I had was that we don't know what the final form of the ads will look like. We're starting to see it maybe a little bit, but I think it's TBD what they actually look like. There's going be a lot of experimentation. So people like that, know, there's that's Claude ads, mocking ads, right within AI.

I think that was assuming ads would be a certain way, right? Which is actually kind of, you know, would not be a great experience, right? If they're trying to fool people and then trick them into doing things. So I don't know, thoughts.

Erez Levin (01:18:55.817)

So I don't speak to the financials. It's more of the like what I'm understanding is like ads have to be a really big business for them because they're just not going to be able to grow at the scale that they're expected to sort of like the markers that they want to hit. It's a huge one.

Brett House (01:19:11.926)

So they have to offset, it's a balance sheet issue. They've got to offset the costs, the debt that they're leveraging to build out the data systems, right?

Erez Levin (01:19:19.96)

the financial side and then people are just saying, no, this is going to be the next search because the intent is so high. And I just don't think he can come anywhere near there. So that's the part of like where my criticism is like, it's just overhyped to like a 10 X. Like it'll be like one 10th of search, which is still a massive. So I agree with you. It can be a really good, healthy business, but if they're sort of $1 trillion valuation that they're going off of is based off of it being as big as search today or even bigger, well, there's a big sort of mismatch in expectation on the.

Brett House (01:19:48.587)

Yeah, and really you make a good point is that is that needed? Do you need to be as big a search because of the subscription revenue, the efficacy that LLMs in these chat bots provide to consumers? There may be enough demand. Right now, just from a numbers perspective, Reuters reported 13 billion in 2025 revenue for OpenAI subscriptions. it's expected the longer term projections suggest 280 billion, so almost 10x by 2030.

It's not the trillion dollars. It's not covering their debt repayment problems. Yeah, the data center build out is so massively expensive. And you would assume that you would start to see decreases in data center mobilization and builds over time, which would affect sort of the balance sheet. I think to your point, Rio, I think that ads are going to be supplemental.

Rio (01:20:23.739)

The data center build out, right? It's not covering that.

Brett House (01:20:45.879)

But they're not going to be as necessary in terms of a pure revenue driver for a lot of these guys.

Erez Levin (01:20:50.857)

And this is where I think to dovetail, one is if they're not really big, then the question is, are they actually having a negative experience? Because at some point they are, they have a negative experience. For a trillion dollars is okay, you pay that price. But if they're not adding enough to the balance sheet and they're causing more headaches than they're worth or attrition or things like that, and you can have anthropic hitting you with ads and actually pulling people away, at some point that cost is meaningful.

Brett House (01:21:01.889)

Yes.

Erez Levin (01:21:20.648)

And then on the point of the actual formats, can you get it right where it's a net positive for consumers? I think that is very hard to do on its skin. Yes, yeah. And it's tight because it's tiny, because I think it's so small and the incentives aren't totally out of whack yet. But at some point, it's going to be really hard for them to serve ads that are relevant, but not answering the question. It's really hard.

Brett House (01:21:28.564)

Yeah, is Gemini getting it right? Is Gemini in Google? And from my own personal experience, you think they're getting it right?

Brett House (01:21:49.889)

Yeah, and it really does. It does kind of facilitate the search experience. Like you get this expanded search bar now, right? So it's just at least when you enable Gemini, right? So it's a little bit wider, more like a prompt, right? And then you get your answer to the question that you've asked, which is your AI response, which is oftentimes what you're looking for. But then you're within that page environment without it being interruptive.

Right. You're exposed to, I want to go because you just automatically know we've been dealing with Google for most of our adult lives. Right. You scroll a little further down for maybe, maybe you're in a retail mindset and you would have a recommended product and you get that classic search sort of a paid market, paid me paid search listings and organic listings experience. I actually don't find it very, I don't, I think it's a pretty good experience. And my question would be, you know, is Claude and open AI and the rest, are they going just to

start to imitate and to start like are they going to be like is it really are we seeing a convergence of search yeah it's like

Rio (01:22:44.389)

If anyone nails it, if anyone nails it, it'll be Google. you know, they know them.

Erez Levin (01:22:47.046)

It'll be Google. It'll be Google, except now the 80 % of queries are gonna be answered at the top organically. And so only somebody that scrolls past and isn't even looking for the answer. Like I looked up for the recipe for eggs and then below was an ad for a hot sauce. that's nice, I've never heard of this hot sauce. It's great, I don't lose trust that they serve me this ad, it seems relevant. But how many people are gonna say like, I was looking for eggs, the hot sauce is not relevant, like I've got a fridge full.

Brett House (01:23:03.594)

Yeah.

Yeah, yeah,

Erez Levin (01:23:14.792)

And so it's great. It's nice. It's a little bonus. It's not a huge moneymaker. And then where it can be a moneymaker, think they have one ad, it's like you look for headphones and it shows you the retailers where you can buy them. And then those can be ads. But my question is, if that, like, it can be bought via a store that isn't willing to advertise. Like that's when I lose the trust that...

Brett House (01:23:38.464)

Yeah.

Erez Levin (01:23:40.23)

Google or whatever LLM is not serving me the truth. It's pay to play and I expect my AI to give me the best possible answer for me, not the one that they have a profit motive to show me an ad for.

Brett House (01:23:40.77)

Yeah, yeah, yeah. It's a pay to play. It's, well, there's always a-

Brett House (01:23:53.655)

Yeah, so the difference between AI and your classic paid search listings, is, yeah, one is giving you a real answer, you know, but if you bifurcate those things, which Google, think, has done effectively, then you're...

Erez Levin (01:24:05.456)

If it's below the answers, that's okay. You can get away with a lot of stuff below, but not incorporated within, which just means like it's lot fewer people are going to potentially click on that.

Rio (01:24:15.831)

Well, the experiment and I guess my point is that, within classic search, they figured out you can give people, you know, and part of it is because they have a monopoly, right? Or based on normal search, they can give you like two pages now of the blue links, right? Which, you know, maybe they couldn't have gotten away with if there's actual competition search. don't know. And for sure, like within AI, there is much better competition, right? You know, it's definitely a three horse race right now. And then a three pony race, maybe four.

Erez Levin (01:24:41.54)

You know, I have a theory on this. like with Google, you're not asking for the answer. You're asking for like a prioritized link, like list of the internet to click on. And so the blue links at the top that are paid versus the organic ones, the organic ones aren't great, especially now Google has allowed the internet to inshitify so much. You look for a mattress and the first 10 links are mattress review sites. It's not like this is the best mattress to buy for you as a side sleeper.

Brett House (01:25:06.028)

Yeah.

Erez Levin (01:25:09.274)

And so it's not even giving you the answers that the ads are competing with. The ads are competing with like another site that you don't really want to go on because it's going to be loaded with ads and a lot of content. This is going be very different with AI search, especially for these commercial queries.

Brett House (01:25:22.785)

Yeah.

Rio (01:25:22.983)

Yeah. And one thing I call it in the article too, is Google's generating between three and 350 per month per user. So you look at, you know, the ARPU for them. mean, it's, you know, these are billions. It's not super high, but these are billions of people who are using it. So it really is a money, money. don't think many people realize that, you know, there's a reason why people call it the greatest business ever invented because in a lot of ways it really is. But then you look at what are people paying for, you for AI now it's, know, let's say many people use it for free, but then.

Most people I know at least pay the $20 a month for one of them. many people will pay $200 a month for multiple ones, right? might pay for, you they might pay for, you they might have a Claude's Claude subscription and then maybe they have a Grok or or a, or chat GPT. So people are paying a lot of money for these, but yeah, they're paying because they're really good. mean, like it, it, it, it makes

Brett House (01:26:12.674)

I'm not sure anybody's paying for Grok.

Erez Levin (01:26:15.017)

Listen, professionals, white collar workers, they're paying for them out of pocket or the companies are paying for them and that will be a business. I just don't think it meets their balance sheet needs from what I've heard and understand. yeah, TBD, think we'll see. We won't see huge advertising numbers coming from these numbers anywhere near what people are hyping up. I don't think these companies have necessarily said what they're expecting to make.

Rio (01:26:15.141)

Some people are. mean, you go to Twitter and some people are.

Erez Levin (01:26:45.191)

is it'd a very slow burn to even get to whatever that ceiling is.

Rio (01:26:50.311)

Yeah, the one number I cited in the article is I think like 13.6 % of overall search spending by 2029, which is, mean, I think that's kind of agreeing with your point is it's going to be meaningful, but it's not going to be, it's still going to be a fraction of even of search itself. So we look at like how much of overall media spending is going in there. It's going to

Erez Levin (01:27:05.193)

And for Google, it cannibalizes Google. they're finding another monetization stream here. But there's big implications for them as well.

Rio (01:27:16.935)

Yeah. And then I had three categories. There was the skeptics, the optimists, and then the purists. I think you were maybe arguing a third category was kind of redundant with the first, right?

Erez Levin (01:27:24.441)

There was overlap. It wasn't redundant, but there was overlap. Let's go to quick hits before I gotta go pick up my kids from school.

Rio (01:27:27.109)

Yeah. Yeah.

Brett House (01:27:31.966)

Yeah, yeah, we're three minutes from the record though. If you go over an hour and 30 minutes, mean, my God, like the Ares has now achieved the most points in this video game. So.

Rio (01:27:32.807)

Sounds good.

Erez Levin (01:27:34.505)

OK.

Rio (01:27:36.327)

Quick kiss will take three minutes, but we don't want to keep you from picking up your kid though.

Erez Levin (01:27:42.456)

I love it. Who am I beating? That's the most important.

Brett House (01:27:47.294)

You are beating Scott Brinker. Yeah, that's somebody that's, yeah, he's up there on the Mount Rushmore here.

Rio (01:27:47.697)

Scott Brinker. Yeah.

Erez Levin (01:27:48.92)

Okay, perfect.

Rio (01:27:54.873)

And that was also on a Friday. I think we got him on when he had a lot of time. Finally, again, was a guy I had to go. Yeah.

Erez Levin (01:27:58.224)

Fridays are good. Fridays are good, yeah.

Brett House (01:28:00.179)

Yeah, we were mentally exhausted by the time we finished that podcast because we'd all just like it was everything we had. We gave it all. I don't think we're giving it all right now. And you got to pick up kids from school. So I'll start with number one for what's more dangerous for marketers optimizing for CPM or optimizing for ROAS?

Rio (01:28:08.539)

Yeah, for sure.

Erez Levin (01:28:19.27)

I'd say CPM. ROAS might actually have like something behind it. CPM is like as arbitraries can be at this point.

Brett House (01:28:23.521)

Yeah.

Brett House (01:28:29.31)

Yeah, yeah, ROAS you can actually put in other metrics to make sure that it's the

Erez Levin (01:28:31.782)

probably something. It's not like totally made up. It's just like off maybe even by like, you know, order of magnitude, but it's still something.

Brett House (01:28:41.558)

Yeah.

Rio (01:28:43.291)

Will AI improve media quality or accelerate the race to the bottom?

Erez Levin (01:28:48.484)

my God, it's a, you know, both, race to the bottom. Like there's gonna be so much slop, but I think it's gonna force marketers to value the good stuff because they're gonna say, the good stuff is so scarce and I don't wanna buy the slop because it doesn't work as well. And so that's what I'm actually hopeful for that like the Delta that you can't average out high quality and AI slop.

the way that you could average like, you know, pretty good quality and like, you lower quality.

Brett House (01:29:20.736)

Yeah, back to one of the earlier questions. you just just the interplay between media quality, audience quality and creative quality? Three very key drivers of incrementality will say, which one are you betting is going to be the most important? mean, you just wrote the report, so you're going to give us a biased answer. But how do you maybe talk about the interplay of those three?

Erez Levin (01:29:43.044)

I think there's nuances. I talk about the interplay of media quality and audience quality, and then I talk about the interplay of media quality and creative quality. I don't talk about the interplay of all three. It's just easier to sort of like break it out because it's a lot easier to look at these two by twos. And so when you have high quality on all three of them, that's the best, or two of the three, that's the best. It's all a matter of pricing at the end of the day.

Brett House (01:29:54.475)

Yeah.

Brett House (01:30:00.075)

Yep.

Brett House (01:30:05.292)

Yeah.

Erez Levin (01:30:09.286)

And it's a matter of matching up, right? It's the orchestration layer. So you could have a really high quality creative, the highest quality creative, but it's running in a muted slot where the audio, you got Samuel L Jackson to do the voiceover. It was worth almost zero, right? And so, but you could have had it in a medium quality environment that at least had the sound on and it could have been to, and this is where like me and Scott McKinley, like he thinks like you, that.

Brett House (01:30:25.568)

Yep.

Erez Levin (01:30:35.428)

The audience does matter very much so, but it's probabilistic. If not always the exact person. I'm still running at a TV show at a certain hour of day and there's families watching. And I know it's 60 % female and 40 % male or vice versa, whatever that is. There's still value there in serving these ads to broad cohorts of populations. And then making sure that on average it's performing well and reaching the people that I want it to reach. so lots of nuance there. Definitely not a quick hit as a...

Brett House (01:30:42.22)

Yep.

Brett House (01:31:02.55)

No, that was great. That was a great answer. That was a great answer.

Rio (01:31:07.111)

All right, if there's one thing media buyers should stop doing tomorrow, what would it be?

Brett House (01:31:07.892)

Last one.

Erez Levin (01:31:13.017)

It's probably like the average, like focusing on averages, average CPMs, average value, average quality. So yeah, if you just look at sort of break down on any of these quality dimensions, time of day, make sure you're not averaging out across everything and only looking at it through an Excel spreadsheet. I think you're gonna see, hey, there's stuff that I don't wanna spend too much of my budget on or overpay for.

Rio (01:31:39.655)

Okay, that's fantastic.

Brett House (01:31:39.948)

That's awesome. Well, thanks, Aris. This was awesome. This was a marathon session, and we are now, you now broke the record. Hooray! We should have like a, I need like a, I need some of those sound, but yeah. So everybody that made it this far, thanks for joining us again. You can find us at dev dev dev.signalinoise.ai on YouTube. Lots of shorts, shorter form.

Rio (01:31:46.362)

You

fireworks or whatever, yeah, yeah, yeah.

Brett House (01:32:01.79)

as well as Longerform, which we love, as well as Spotify and Apple Podcast. And we will see you next time. Thanks.

Erez Levin (01:32:09.721)

Thanks, guys.