Own Your AI Advantage: Spencer Gusberg on Open-Source Models, Data Sovereignty, and When to Rent vs Buy

If anyone can get the world's most powerful AI models for $20 a month, where does a company's real advantage come from? And when you feed your prompts, corrections, and proprietary data into someone else's model, who ends up owning what you know?
Spencer Gusberg, Founder & CEO of e52, an independent AI engineering firm in Cambridge, joins Brett House and Rio Longacre to dig into one of the most consequential decisions companies now face: what to rent from the frontier labs, what to own themselves, and how to protect the know-how that makes AI useful in the first place.
Spencer brings a practitioner's view. He started his career in options trading, earned his MBA at MIT, co-founded P2 Analytics (acquired), and has spent 15 years building data tools and AI systems for companies from startups to the Fortune 500.
The conversation starts with e52's new research, a review of 30 tech-forward companies that found 19 AI systems in production or close to it. Many of those systems run on models the companies control. Spencer explains why API spending charts miss much of the open-weight usage happening on-prem and on rented hardware, why open models are roughly four months behind the frontier, and why that gap may matter less than people think.
From there, they get practical. Spencer explains the difference between "context" and the "harness" around a model. He walks through how Atlassian splits work between frontier models and a fine-tuned small model, and how LinkedIn reported roughly 75x lower serving cost with a specialist model. He also makes the case for splitting AI spend by job: rent the frontier for broad, judgment-heavy work, and own the high-volume, repeatable, measurable tasks.
Then comes data sovereignty. What does "we don't train on your data" actually promise? Where does your core IP really live? And why are the frontier labs under so much pressure to find proprietary data?
And yes, there are detours, from New Haven pizza vs. New York pizza (and a confession that Spencer is a Mets fan) to Star Trek, Mad Men, and a modern version of the Solow productivity paradox.
In this episode:
Who owns your AI advantage, and why the model itself is now a commodity
What spending charts miss about open-weight and open-source adoption
Context vs. harness, explained in plain English
When a fine-tuned specialist model beats a generalist frontier model
The LinkedIn and Atlassian examples, and what made them work
The hidden costs of owning: engineers, upgrades, evaluation, and support at midnight on a Saturday
Why "golden data sets" and evaluation frameworks come before fine-tuning
Data leakage, retention policies, and protecting proprietary IP
Why AI hasn't yet shown up in firm-level productivity data
Why Spencer has never seen a project fail for technical reasons, only organizational ones
The one AI decision every leader should make next
Spencer's bottom line: context is king. Show the models the right context at the right time, build a framework for choosing the right tool for each job, and get very good at continuous innovation.
This is a Signal & Noise conversation for any leader trying to make AI pay off without giving away the knowledge that makes their business unique.
Watch, listen, and subscribe: https://www.signalandnoise.ai/podcast
Follow Signal & Noise on YouTube, Spotify, Apple Podcasts, TikTok, Instagram, and LinkedIn.
#SignalAndNoise #SpencerGusberg #e52 #AI #OpenSourceAI #OpenWeights #EnterpriseAI #AIStrategy #DataSovereignty #LLM #FineTuning #GenerativeAI #MachineLearning #DataScience #AIGovernance #TechPodcast
Transcript:
Brett (00:00.866)
Everybody, welcome to Signal and Noise. This is Brett House joined by my co-host Riel Longacre. And today, joining us from Boston or maybe Cambridge. I don't know which part you're but you in Cambridge.
Rio Longacre (00:06.799)
Hello everyone.
Rio Longacre (00:11.779)
Cambridge. But for most people in the country, Cambridge and Boston, Cambridge is Boston.
Spencer Gusberg (00:12.754)
Yes, Cambridge.
Brett (00:17.824)
are kind of the same technically two city. Yeah, yeah, yeah, exactly. So so from from Cambridge, Spencer Gusberg, who is the founder is the founder and CEO of E fifty two, which is an independent AI engineering firm, in a t in a town that I know very well being a BU grad and certainly Rio and I and we all talked offline about having spent a lot of our college years. Rhea went to school there. I went to my MBA later on, a little bit later on in life.
Spencer Gusberg (00:17.97)
Yes. Yes. Out of towner.
Brett (00:47.234)
But we spent a lot of time in Boston, so I think we've got that in common. And and Spencer, I think I confirmed that you're you're from Connecticut. Right? And but you are from the Red Sox portion of Connecticut, not the Yankees.
Rio Longacre (00:49.273)
Fun town, definitely a fun town.
Spencer Gusberg (00:54.984)
Yes, from New Haven.
Spencer Gusberg (01:01.604)
It's it's it's it's pretty much the line of where you have Red Sox Yankees fans split. I am actually a Mets fan, unfortunately, so so I don't I don't usually admit that that's some breaking news for the pod, but bam family. My grandma was a huge Mets fan, dad is a huge Mets fan. So
Rio Longacre (01:01.807)
Well, the Red Sox Patriots, not the Yankees Giants portion,
Brett (01:11.914)
Yeah. you're a Mets fan. Interesting. Yeah, yeah, interesting.
Rio Longacre (01:16.525)
How did that happen? How did the Mets fan happen? Family. That'll do it.
Brett (01:19.256)
Yeah.
Brett (01:24.886)
Yeah. And and are you a are you a total pizza snob? Because I know Frank Pepe is in the the pizza culture of New Haven.
Rio Longacre (01:29.699)
You're having pizza, yeah.
Spencer Gusberg (01:30.926)
I I would say I'm not a pizza snob. I'm open to the fact that there may be pizza that's made outside of New Haven, but I c haven't confirmed that. yeah, exactly.
Brett (01:42.262)
You're not you're not open to the fact that any of that's any good. Yeah. I mean
Rio Longacre (01:46.128)
Well, Brett, that's like me with bagels, right? So like I know they had bagels in Denver. I would never get one, right? Even though some people say they're pretty, they're okay, just because growing up in New York, right? Long Island had the best bagels like back in the day. I mean, I still think it does. And there's some places in Manhattan. I mean, it's harder to find amazing bagels in Manhattan, but they still obviously exist. But yeah, the bagels don't compare anywhere else. So I hear what you...
Brett (01:58.732)
Yeah.
Brett (02:08.159)
Yeah, it is it is hard to eat it is hard to eat pizza outside of you know, and I'm gonna put New York, New Jersey and New Haven at the top of the pizza world. No matter where I travel anywhere in the world, including Chicago. I'm always like yeah, and whenever I go on on like s yeah like ski trips out west, one o Yeah, totally. It's not yeah, whether or not y it's more like a quiche.
Rio Longacre (02:20.685)
No, that's not pizza. It's like deep dish casserole. It's like a pizza casserole.
Spencer Gusberg (02:22.387)
That's not.
I have to say I agree with Rio.
Rio Longacre (02:28.803)
It's like that thick, like that thick crust that's like hard to chew and like.
Brett (02:33.462)
Swimming swimming in cheese. Yeah, and I
Spencer Gusberg (02:34.123)
it's it's a different animal, yeah. my so my wife is actually from New York and when we first met I was, you know, giving her the Spiel on New Haven and she was like, No, you have to come to New York. There's the best pizza place ever, it's in Yonkers, it's called Peppy's Pizza. And I said, Have you ever have you ever looked at the sign? Yeah, it said the sign says New Haven Pizza.
Brett (02:51.479)
That's a new Haven Pizzeria!
Rio Longacre (02:52.527)
That's so funny. is so... Yeah.
Brett (02:56.171)
Yeah. The white clam pizza, right? Yeah, it w it's f whenever I go on like like ski vacations at West, a buddy of mine who's also from New England would be would say, Let's you know, let's do pizza one night and I'm like, We're going to be disappointed. Like, by definition. And we always are. A I've rarely ever been impressed with pizza outside of this larger region, right? And New Haven definitely included, which is
Spencer Gusberg (02:58.397)
Yes, good man.
Rio Longacre (03:17.743)
That's really good pizza. I still think New York style, the New York crust, you could fold it. And look, I do appreciate other types of pizza, like the thin crust Italian, I'm sorry, like California stuff, I appreciate it, but I just think New York pizza.
Spencer Gusberg (03:26.885)
sorry. It's
Brett (03:33.122)
Well we have we have every type of pizza though. We have thick crust Roman style, we've got yeah, i or no but th they call it Roman like Roman. There's a thick it almost is is like a crunchy, flaky. Yeah, but yeah. Yeah, yeah.
Rio Longacre (03:37.271)
or Sicilian. Well, but Detroit, funny.
Rio Longacre (03:43.459)
No, but that's what we called, that's what we called Sicilian growing up, right? Which is like the square pan, which I know it's funny. Detroit calls it Detroit pizza now.
Brett (03:51.532)
Yeah, and then and then you yeah, then you've got the Neapolitan pizza, which is more common with with New Haven. You've got the thin crust with like thin crust pan pizza with no crust, where the cheese caramelizes around the edge. Right? That's you know, there's a lot of different styles. When Europeans ask me what kind pizza I'm like, we've got like every fundamental style of pizza in the world. So I'm not really sure what New York pizza is because it's a blend of different things and they're all really good.
Spencer Gusberg (04:04.744)
Yes.
Spencer Gusberg (04:12.499)
This is the truth.
Rio Longacre (04:18.511)
Now New York pizza is like the fairly thin crust. You can fold, right? You've got, yeah. Big, big pizza, big slices. Yeah, it's in New York. It's good, I mean. They cover the board and stuff,
Spencer Gusberg (04:19.26)
Sure.
Brett (04:20.417)
The folding.
Spencer Gusberg (04:22.897)
Yeah, the big slices you can fold, yeah.
Brett (04:26.987)
Yeah. Yeah. Well back on topic. Now now that we've covered that and that was the that was that yeah, today we're talking about New Haven versus New York pizza. so so E fifty two is you know, w which you're running with well, you have a partner, right? Chuan Zhang, right, who's a former BCG consultant of w a former meta engineer. you've got a team of senior engineers is what I've read. And you've got a bunch of really high profile i largely MIT oriented.
Spencer Gusberg (04:31.911)
That that is the topic, right?
Spencer Gusberg (04:43.506)
Yes.
Brett (04:56.917)
advisors, right? Demetrius Bertzimus, right? He was a he was an MIT Sloan professor, right? Alfred Spector, who was a a CTO of Two Sigma and a research VPG, did Google and Kate and is it Katie or Katia Walsh? Did I I think I missed him. Katia. Katia Walsh. Walsh who was a former chief strategy and A officer at Levi. So I I I saw your board and I thought really interesting. A lot of MIT affiliation. You were an MIT
Spencer Gusberg (05:03.089)
Yes.
Spencer Gusberg (05:14.347)
Katya. Katia.
Brett (05:26.322)
master's or master's degree? MBA?
Spencer Gusberg (05:26.833)
Yes. Yes, MBA. So I I actually started my career in options trading, found my way to MIT for business school thinking I wanted to start a hedge fund. And I actually met Dimitri Bertzimus, our advisor there who was running operations research. I met him at Sloan and he convinced me that it was the data science side of what I was doing that I loved and not the finance side.
Rio Longacre (05:44.911)
You see my name at Sloan?
Brett (05:53.141)
Yeah.
Spencer Gusberg (05:54.301)
and he was right, so I gotta give him credit for that. He was right. It's been great.
Brett (05:57.218)
Yeah, and now you guys are doing what? Technical strategy for management teams. You also still kind of affiliate yourself with with boards and investors when they're going through diligence, from what I've read. Right? Data foundations, which is critical and that's a cle a key use case for for AI, right? Whether it's pipelines or entity resolution or data consolidation and and
Spencer Gusberg (06:06.831)
Absolutely. Absolutely. We
Spencer Gusberg (06:12.655)
Exactly. Exactly. We d
Brett (06:18.893)
cleansing, which is often a a a major issue. And then kinda things like applied AI, forecasting, pricing, allocation, scheduling models with the you know so it sounds like you guys are doing some really heady, difficult stuff from an AI perspective as E fifty two.
Spencer Gusberg (06:32.721)
Yes. Absolutely. We do I like to say we do everything throughout the data science value stack. So everything from data engineering to AI and ML research, implementation, deployment, and also as you mentioned, the strategy side, which is so key these days, right? How do we stay on the competitive frontier given you know the specific niche of a given client?
Brett (06:59.915)
Yeah. And one thing we wanted to talk about today was a research paper that you're putting out, I think in the near well, by the time this this pod is live, it should be out there in the ecosystem. Right, where you reviewed thirty top tech companies, you found I think nineteen real AI systems in production, and then you looked at those systems to see how many had you know, did the companies actually control where they were using their own open source models that had been that were sort of behind
Spencer Gusberg (07:05.265)
Yes.
Rio Longacre (07:09.228)
It should be out, yeah.
Spencer Gusberg (07:10.406)
Yes.
Brett (07:28.707)
a firewall so to speak versus frontier models, which, you know, they were accessing externally, and the cost comparisons, et cetera, and ownership kind of nuances between those things. So I thought that was really interesting. We've certainly had conversations like that in the past with
Spencer Gusberg (07:43.666)
Yes.
Brett (07:46.59)
we mentioned before when we were talking Eddie Drake from Snowflake, who did a really interesting paper that was probably a little more oriented around it was called Why Marketers Need to Own Their Context Layer and it was really around brand IP Yeah, it d data and AI sovereignty and in in in brands,
Spencer Gusberg (07:50.856)
Yes.
Rio Longacre (07:59.139)
data set sovereignty really.
Brett (08:05.739)
realizing or or at least signing contracts that that are to their advantage when working with tech or AI vendors. otherwise they might be be literally training their competitors with the AI that they're renting. and so how many how many vendors or for or brands are fully aware of that and what steps do they need to take to ensure that they've got this stuff behind closed walls so they're protecting what is fundamentally their brand IP. So I know that's not exactly what your research kind of uncovered, but I think it's a it's a it's a
Spencer Gusberg (08:18.509)
Absolutely.
Brett (08:35.987)
a a a related topic for sure.
Spencer Gusberg (08:36.112)
I absolutely. It's it's absolutely a relevant piece. So, you know, we get this question a lot about, you know, it used to be who's better for this task, Chat GPT or Claude? Right? and then the question has slowly become, you know, should we be considering open weight models? And I think, you know, what you mentioned in terms of yeah, open source, open weight.
Brett (08:51.127)
Yeah.
Rio Longacre (08:58.895)
Yeah, open stores, open weight.
Spencer Gusberg (09:02.96)
the the owning the context layer, how important that is. You know, we see this these stories about like the Millennium Math Price problem where, you know, OpenAI can't really say for sure if they've used the IP of users in you know in their solution to that problem. And it begs the
Rio Longacre (09:24.035)
Well, they don't know. If the information leaked to LLMs, I mean, it's kind of a big mystery to everyone, right? It's black box.
Spencer Gusberg (09:30.202)
Right. So, you know, it's it's it's a big question for every company who's has a proprietary context layer, which is most companies, and how to effectively stay competitive and operate on the frontier of the technology without necessarily always using frontier models to get their machine intelligence. Right. And you know, it's
Rio Longacre (09:53.679)
Well Spencer, this piece raised a bunch of questions we definitely wanted to dig into and that's part of the reason why we invited you on the pod. I mean think Brett, you ran into them recently too, which kind of precipitated this, but the question really I think boils down to like who owns your AI advantage and what does that even look like, right? And I think you're...
like a couple of your pieces, there was a study itself and we sent over a couple of, we looked at a couple of the pieces you wrote in your company blog, which is actually quite good for anyone who wants to check it out. And you know, one asked what, like when companies rent intelligence and when should they rent it versus when should they build it on their own, right? I mean, I think that's something we want to dig into. And the other was what happens to the data and know-how that they hand over along the way to the frontier models themselves, like what happens, right? Or to other AI companies. So maybe we start with like,
What prompted the research? I know you dig into it a little bit, what was the impetus behind that?
Spencer Gusberg (10:46.896)
Yeah, that's a great question. So a big part of it was this conversation around what is the right approach to machine intelligence, right? How should we approach keeping companies on the frontier? What does it mean in terms of what tech we build, what we rent, what we own, and how do we get at an answer for that? So as we thought about it, we said, Well, what do the most
Tech forward companies out there do? What does Ramp do? What does Atlassian do? What does LinkedIn do? You know, these companies that publish on their engineering blogs, models that they've created on their own, workflows that they have, how they've used models, let's review those and get a sense of what the smartest companies do to capture machine intelligence and stay on the forefront of where we are with AI.
and that was really the impetus for this research study was to get that information you know, for our clients, but also, you know, we thought it was worth publishing because it's an interesting thought piece. And, you know, the conclusion may not be super spicy in that you you know, at the end of the day, you want to choose the right tool for the right job. but what that means for different companies and in different contexts,
Is where the rub really is.
Brett (12:17.099)
Yeah, and and your paper says that spending charts and engineering disclosures s often tell s different stories. Right? What does an API spending chart fail to capture that like w the work for the the compared to the work the companies are actually doing?
Spencer Gusberg (12:23.247)
Absolutely.
Spencer Gusberg (12:29.399)
Absolutely. So if you look at API spending charts, they're very much skewed towards rented frontier APIs, right? you know, what for a couple reasons. One, they're looking on a nominal basis of cost. So when you think about usage, okay, using the frontier models is much more expensive. so that is gonna skew it in the favor of the frontier models. but also a lot of usage, token usage of
Brett (12:38.018)
Yeah.
Spencer Gusberg (12:59.301)
open weight open source models happens outside of APIs, right? They're on you know on-prem for some people or on their own hardware or hardware systems that they're renting. so because of that you miss out a lot on a lot of usage. And the biggest trend that we thought the you know the biggest data point that we thought was relevant to that was if you look at open router who's one of the biggest hosts of
open weight models. open model token usage is now about a third of their volume and has been growing as opposed to, you know, the headlines on token spend, you know, inflating drastically at Frontier Labs. I think the number in that Menlo study was like eighty eight percent of dollars are spent at the Frontier. and I think the reality is is not that high. And again, that's not to say that there aren't great use cases.
Brett (13:48.908)
Yeah.
Rio Longacre (13:55.503)
So you see a lot of the tokens, a lot of tokens are being accounted for because they're not through these APIs and so overall token usage. What is the current split? It seems like the vast majority of tokens are still consumed by the frontier models, right? But open source is definitely picking up. I more and more companies are adopting this because of the heavy charges, right? mean, this is not, if you're consuming billions of tokens, it can really add up.
Spencer Gusberg (13:59.526)
Yes.
Spencer Gusberg (14:22.563)
Absolutely. So the exact numbers are hard to know with certainty, because so much of the token usage on open white models happens outside of these frameworks where the public can get data on it. but I think open router is a good proxy for what what the usage is and it's about a third open source.
Rio Longacre (14:45.838)
So a third of tokens or a third of spent? Okay. Okay.
Spencer Gusberg (14:49.029)
Third third of tokens, which is a much much better measure, obviously, because of the the nominal aspects of the price.
Brett (14:55.403)
Yeah, yeah. And do you think this is gonna have an impact on the economics of of the big frontier models, right? Because it's clearly think of it almost as a it's a challenger model approach as opposed to a channel challenger brand approach, right? Where where why not do your your your sort of initial let's say you some initial work with the frontier model
Spencer Gusberg (15:02.276)
Yes.
Spencer Gusberg (15:08.12)
Abs absolutely.
Brett (15:14.665)
And then figure out how you're gonna break up a project into different work streams and then apply lower weight, lower cost models that potentially could do. And I think some of your research actually suggests that some of the models that are being trained to do things, lightweight or open weight models that are being trained to do things that the frontier model started to do actually perform a lot better. and for for you know, an order of magnitude less cost. So I mean, how do you think, you know, i is that the case? Did you give us some examples around that? And then how do you think that
Spencer Gusberg (15:34.609)
For sure.
Brett (15:44.127)
affects the overall economics of model usage in the in the big frontier models.
Spencer Gusberg (15:48.386)
yeah, absolutely. So to Rio's earlier point on why you would go the open weight model route, it's really twofold. It's it's it's cost well I'd say th threefold maybe. Cost, flexibility, and then also the ability to protect IP, right? Those are three big items. And one thing we always stress is regardless of what
Brett (16:12.363)
Yeah.
Spencer Gusberg (16:18.147)
model you're gonna use, whether it be rented, owned, you always wanna be model agnostic in your framework, right? And right now the c the frontier is is very competitive, to your point, Brett, right? Like the the labs with each other and then also the pressure of open weight models. You know, the general thinking is like open weight, open source is about four months behind the frontier in terms of performance. Now there been
Brett (16:27.169)
Yeah.
Rio Longacre (16:46.542)
It's not a long time, but it is a lot of capability that deploys in four months, right? I guess that's the interesting paradox here. The four months is nothing, right? But when you look at all the advances in AI, they can get shipped over one or two models, which is, I don't know what the current turnaround time is, just a couple of months per model. That's a big difference, right?
Spencer Gusberg (16:48.297)
It's not
Spencer Gusberg (17:05.135)
Yeah, it's true, although there is some nuance even there in terms of like you know, people throw around this term of the harness around the model. And you know, I think you know, as I've talked about before, you know, horrible term. whoever coined that was, you know, not a a branding expert. But when you call Claude, for example.
It's not just a single call to the underlying model, right? It's it's it's a model with a harness around it. So it may run you know, a query to the model, a loop that checks that, another query to the model i in a framework that they've designed and they've innovated on that. So you really have multiple levers that can improve to really improve your outcome. So it's not just the underlying
model technology, but the harness around it, basically the software system around it that helps utilize the model.
Rio Longacre (18:06.35)
Could you, Spencer, could you maybe define hardness? Because I think we know what it is, about it, yeah. Well, maybe define just two terms, maybe helpful to have like as an expert to have you define. One would be hardness, the other would be context there, because that's another one that's thrown out so much. I don't know if many people really are saying the same thing. And some people I hear using, I don't think they even know what it is, right? So, right. So how would you, what could you define the two and like your definitions and like how do they differ?
Brett (18:09.068)
It is the it is a term that's being thrown around a lot, yes.
Spencer Gusberg (18:13.152)
Yes.
Brett (18:25.964)
Yeah, or they think it's synonymous, yeah.
Spencer Gusberg (18:26.841)
Yeah.
Absolutely.
Spencer Gusberg (18:33.932)
Absolutely. So they're they're they're very tightly linked. You know, part of the reason why the harness is so important is because it's important to show the model the right context at the right time to get performance. So in a very kind of traditional way, when you think about context, it is when you think of a prompt that you're feeding into a model, it is the piece of information that's fed in that is not directly the user prompt. Right? So it may be
let's say you have a your signal and noise voice that you like to speak in when you create documents. And you may have context in the form of a file that always goes into every prompt that says that we like to talk in this voice, we we always you favor clarity and honesty.
Brett (19:20.234)
Yep. Our marked it's marked down files or skills that we've built into into Clog, for example. Yeah.
Spencer Gusberg (19:25.238)
Exactly. Exactly. And one of the things that Claude is very good at is saving context in different places so that it's not fed in necessarily to every prompt. But when there's a prompt that's relevant, it knows how to pull the right context into the prompt. And that is all that process is what the harness is doing, right? Is that software around the model itself where it's saying, you know, grab this context from here, feed it into this next.
Brett (19:46.218)
Yeah.
Spencer Gusberg (19:54.841)
model call. Okay. Receive back that response before you show it to the user, loop that again and and d you know, check that with
Brett (19:55.425)
Yeah.
Brett (20:01.642)
That's a very good way to understand it. Yeah. So so the context is the actual instruction, so to speak. Like it could be in this case the voice that you wanted to use to procure create a specific type of document or rules of engagement. Yeah. And then and then the harness is the actual capability of saying, okay, this needs to be placed here, this is how it's going to be used and how it's going to be applied to this this overall system, which is software. You called it software in a sense.
Rio Longacre (20:11.586)
Could be marked down fire, could be skill stockings, anything, right?
Spencer Gusberg (20:24.621)
Exactly.
Exact exactly. So if you think about, you know, general, if you go to claude.ai and and chat with that model, it's you can chat with the same underlying model that's used in Claude Code to write code. The difference is that there's a coding assistant harness around the model for Claude Code that is designed to allow that model to interact well with the information it needs and to to have the right context for how to generate code.
Brett (20:47.745)
Yep.
Brett (20:55.606)
Yeah, and would that would that allow that model to operate more efficiently and use less compute based on the the task that's being asked of it?
Spencer Gusberg (21:03.126)
Absolutely. So there's there's a lot of innovation to do there. So what what made me bring it up was, you know, Rio's point on the gap of four months in model intelligence. You know, that that can be a significant gap, but it also may be driven not only by the raw model intelligence, but it might also be driven by harness innovation that the labs have done. and there are there are now open source harnesses as well that are that are pretty effective.
Brett (21:14.666)
Yeah.
Brett (21:27.061)
Okay.
Spencer Gusberg (21:33.393)
but it's i you know, it's a dynamic environment as you guys know. So month to month, you know, the dynamics can change, of course.
Brett (21:37.61)
Yeah.
Brett (21:42.1)
Yeah, your your research I mean you talked about 'cause before you before we went on that sort of tangent
You were we were kind of talking about how you've how or I think you were mentioning in general, how you can fine tune kind of a rented frontier model for kind of an a a a narrow work stream or a narrow use case or evaluation job, right? And I think in the paper DoorDash was an example of that. Can you talk about that example and then also look at well, how does that apply to open model usage? and using them for the same sort of narrow evaluation use cases? I mean, can you kinda
Spencer Gusberg (22:01.295)
Yes.
Brett (22:15.891)
Talk about those two sides of the fence.
Spencer Gusberg (22:18.384)
Absolutely. So
Spencer Gusberg (22:23.064)
fine tuning piece is an added layer to what we talked about in terms of that decision that you make and the factors regarding flexibility, cost, protection of data. Fine tuning can actually allow you, as you said, to gain performance on a very specific focused task, right? Like when we think about these frontier models, they are absolutely generalists, right? Like
Brett (22:37.29)
Yeah.
Brett (22:45.087)
Yeah.
Brett (22:50.538)
Yeah.
Spencer Gusberg (22:51.606)
It has the capability to draft an email, use an API, pull information on law, design a PowerPoint. W it you know, at a at a very impressive aptitude. But sometimes one, you don't need all those capabilities, oftentimes you don't. and two, sometimes it's actually better.
To not have those capabilities, you when you think about it from like a security standpoint, right? Like maybe you don't want your model that's detecting fraud to also be able to send emails. you know, as an example. so you know, this question of like if you have a very focused task and you have a very good data set on how to evaluate that task.
Brett (23:23.808)
Yeah.
Rio Longacre (23:32.803)
Yeah.
Brett (23:33.653)
Yeah, that's a pretty good example.
Spencer Gusberg (23:48.26)
like performance, a labeled data set where you say to use the fraud example, you have a great data set where you've labeled the cases that have fraud, so you can train a model against it and also measure a model's performance. So like have a good evaluation framework, as people say, for as you tune that model. you can it's been shown, you know, the paper has a few cases of it where people have tuned models that not only are
lighter weight, more cost effective, but also significantly more performant than using the generalist frontier to perform those same tasks.
Rio Longacre (24:27.544)
Spencer, looking at the costs, I think in your paper, I think I remember something about 75X. For those who don't know, who are listening, what I'm referring to is the cost differential between the frontier models. And again, when you look at all of the revenue they're bringing in, and Claude, Anthropic filed this S1 the other day. I their numbers are just mind-bending how quickly they've ramped up from a few million to tens of billions of dollars. I mean, no one's ever seen ramp ups like this.
in history of industry, it's absolutely remarkable. But a lot of the reason we're making so much money is those tokens really do add up. So I have a couple of questions. Number one would be looking at the cost difference. The reason why a lot of people are leading to open source is not only can it be, I think, tailored to be more specific to specific needs, for which you maybe don't need a frontier model capabilities, right? But also the difference in cost. I think I remember seeing 75X, difference between open source, open weights, and a frontier model in terms of the overall cost per token. Is that accurate?
Spencer Gusberg (24:56.493)
Same, yeah.
Spencer Gusberg (25:02.703)
Absolutely.
Rio Longacre (25:23.511)
Or is it different?
Spencer Gusberg (25:24.077)
Yes. So it it the details matter in terms of you know, how you're consuming the open source model, which model you're using, all of those elements. But there is a case described by LinkedIn where they were able to achieve seventy-five X with an open source model that was able to increase performance as well. and that's likely because they were able to to
serve it themselves, you know, host the model themselves, where the costs were basically
Brett (25:58.131)
So it was one seventy fifth the price? Is that what we're saying? You see seventy fifty what yeah, we'll just one seventy fifth of the cost per token. Yeah. Yeah.
Rio Longacre (26:00.814)
Yes, it's not one tenth or one seventh, it's like one seventy-fifth. It's literally seventy-five more.
Spencer Gusberg (26:07.117)
Yeah. Yeah, in total to run it. And, you know, when you think about that it it starts to become clearer the type of tasks that will make sense to put on that type of framework, right? Like tasks where you're making, you know, thousands of calls to it a month, you know, repeated tasks, all of these things where you might not even engage in that workflow if you had to pay tokens to a frontier model for that.
so the cost you know, a good place to look to get a sense is you know, we have no affiliation with open router, but it's the you know, one of the biggest sources of hosted open weight models. And you know, in that the costs will you know i it's it's you know well cheaper for any open source model than than what's on the frontier.
by you know many factors
Brett (27:10.463)
Yeah, and it and it seems like a lot of the examples that you brought in, I think it was the Atlasian example where they use frontier models but they also use smaller open weight models and different parts of their kind of core product, right? How d how does how did they or how do companies just in general decide to split that work and and do you actually create maintenance headaches? I mean, from the teams internally that are trying to manage these processes, how complex does that make the overall kind of workflow?
Spencer Gusberg (27:39.907)
Yeah, that was a really great example and an interesting one to see how they did that. So they have this chat assistant, which helps companies basically talk to their data, ask questions that will give a picture of you know very the company specific context. And what they did was they use frontier models for the general chat interface, but they fine-tuned a model.
Brett (28:06.613)
Mm-hmm.
Spencer Gusberg (28:09.486)
To do to create the specific queries for the data in the database. Right. So like if you think about it, that the kind of generalist task, which you know, someone might come to this chatbot with a very broad range of questions for which it would be hard to fine-tune a model that could you would be confident would capture everything. But for this specific task where it had a narrow focus of creating a database query.
Brett (28:15.925)
Yeah.
Spencer Gusberg (28:39.252)
That was focused enough that they were able to fine-tune a model that was very effective at doing that. and probably also touched on the security concerns we talked about before, where you might not want a large generalist model querying the database for various reasons. You might want reduced capability in certain ways.
Brett (28:58.058)
Yeah.
Yeah. And and how effective are those large generalist frontier models, you know, within and I certainly do this within Claude and and OpenAI Chat GP chat ch Chat GPT, in sort of can you block off and make it a private, sort of internal only type exercise versus, you know, publicly
jump sharing your information, you know, right, 'cause I think there's a there's a a manual mechanism where you can basically say, do not use my information to train the larger OpenAR Claude model. I'm not sure that radio button that I hit how much I can trust that, but do you have any evidence or any knowledge of of you know that being a concern with especially with brands?
Spencer Gusberg (29:22.796)
It's Yeah, it's
Spencer Gusberg (29:44.375)
Yeah, I I think it is a concern. You know, there are multiple concerns. One is the question you raised of like, does having the right controls in place even protect you? Like we don't know. You know, this we hope, you know, there this right, the this Navier Stokes math problem where researchers were, you know, using codecs.
Brett (29:57.001)
We hope.
I checked the box.
Rio Longacre (30:02.466)
Yeah.
Brett (30:06.515)
Yeah.
Rio Longacre (30:08.024)
Well, yeah, they just brute forced it. mean, they had, you know, tens of thousands of agents doing this for tens of thousands of hours. I mean, it's incredible what they did, right?
Spencer Gusberg (30:15.948)
Well they did that, but they also initially there were there were researchers that were on the cusp of releasing their research on how to solve this problem. And
Rio Longacre (30:26.155)
they think it was probably trained by the actual AI, differential model usage by the researchers? that the theory?
Spencer Gusberg (30:30.814)
So yeah. Yes yeah. So OpenAI initially came out and said we absolutely did not train on, you know, any of their data relevant to this this problem. And then they walked that back a bit later.
Brett (30:46.628)
wow, wow. And well just for people that don't people that don't know the Navier Stokes problem, because we actually brought it up in our advertising week interview with with Matt Krebsick, for those that haven't heard it, it's fluid dynamics, right? Air and water. and it is the second law of motion from from the Isaac Newton, Sir Isaac Newton. And I think it's
Rio Longacre (30:47.467)
but but you
Spencer Gusberg (30:53.528)
Nice.
Rio Longacre (31:05.454)
Yeah, but Brett, there were like a couple hundred different things they actually announced at the same time. This is the biggest one, right? Which is like this one, they cracked this one.
Brett (31:11.516)
Yeah, but the biggest one, I mean, at least yeah, the biggest one, at least what cr Matt Krepsic, who we interviewed, said he's the CEO of a company called Media Radar, was that for a human application of trying to solve this particular problem, meaning all human mathematicians on the planet, brute force mathematics, it would take ten thousand years to solve the problem. And open AI did it in a matter of weeks, right? But you're saying that it potentially trained on existing
mathematical work so it's not like it it started from scratch.
Rio Longacre (31:44.879)
Well, yeah, Brett, that's a data leakage thing. So, I mean, they're claiming that if you prompt it, they don't actually store your prompts, which could be true, right? Let's assume that's true, right? But if you show it something new that it didn't know before, you actually teach the model and it becomes smarter. And like the idea that you gave it, it does actually keep in its model. So you actually, that is data leakage. It's not, again, it's not by indexing your actual...
Spencer Gusberg (31:46.742)
Exac right.
Brett (31:53.032)
Yeah.
Spencer Gusberg (32:08.184)
Yeah, for sure.
Rio Longacre (32:10.639)
the prompts themselves, it's by these things are actually, this is intelligence, it's actually getting smarter as you use it.
Brett (32:12.625)
yeah.
Spencer Gusberg (32:14.646)
Right. And there are a lot of caveats too to what they can store even if you do that. You know, anything they label as abuse, potentially abuse, which you know, can be there's no clear definition of like what they can categorize as as that and worth them storing. we actually internally
Brett (32:16.2)
Yeah.
Rio Longacre (32:35.512)
So they could categorize something that's not really abusive, they want to distort and distort your say.
Spencer Gusberg (32:38.936)
Right. right, exactly. Or their their their framework for that could be very broad, right? Like, well, we need to review all of this category of things, and if we label it as that, then you know we can store and and you know work with it. You know, they also if you look at the history of their policies in terms of whether by default it's zero data retention, why whether by default it's no training, what that means.
If you look at that over time, the number of times in which that has changed has been a lot. and this is, you know, as we said, this is a very dynamic environment. I, you know, I have no idea what you know OpenAI's intent or anthropic's intent was, or but the the point is that like there's a lot of complexity to these systems, right? And if you're feeding in your IP.
into these tools. Even if you think you have the right controls in place, which if you have a team, also that's another challenge altogether, right? To make sure that everyone is being compliant.
you still might not be protected and that's a pretty big risk to take with your core IP, right?
Brett (33:56.093)
Yeah, well and I and I wonder if if you if you do take legal recourse because you feel either through pattern you know, the training aspect of it patterns that were being developed or data leakage, yeah, i like is even plausible. Like 'cause they probably have d d plausible deniability.
Rio Longacre (34:03.875)
Can you?
Rio Longacre (34:10.36)
think in terms of service, you've it away in terms of service. I'm sure. never read that. I who does, right?
Brett (34:14.824)
Yeah. Well and and they could probably deny it and find some way to obfuscate your ability to find out what really happened considering the complexity of these systems. You would think. There's plausible deni deniability options, I think.
Spencer Gusberg (34:23.413)
Yeah. Yeah, I Yeah, for sure. I I don't pretend to be a lawyer, but you know, it I don't think there's a lot of precedent for most of this stuff, so it's really an unknown. which is a a scary thing and something that, you know, a lot of companies are trying to navigate now, right? How do you stay as competitive as possible and take advantage of these tools while also protecting core IP?
Brett (34:34.782)
Yeah.
Brett (34:50.888)
Yeah, yeah. And co and Core IP, where where do you think Core IP lives? Do you think it exists? It lives in that sort of context layer, in the harness layer, in the actual model training, right? Because the model's getting smarter based on what you're giving it, whether it's it's it's knowledge context from the organization or the people that work there, or if it's market context that's being provided, for example, in an advertising ecosystem, it's getting continuous inputs on performance and
Rio Longacre (34:51.896)
Suspect.
Brett (35:18.45)
Right, of advertising, say in various channels and it's optimizing to help you drive better marketing outcomes, whether you're an agency doing it for brands or brands doing it for themselves. Like, where does the brand IP live and what should they be thinking about when they're saying, Hey, we need to keep this within our closed walls because if it goes and trains external of us, that information's gonna be used by our direct competitors or others in the space. We
Spencer Gusberg (35:20.854)
Yes.
Spencer Gusberg (35:45.912)
Yeah, I think one of the big challenges is right now for most companies it lives in a lot of different places. Right? It's it's not unified in one place that is easy to govern. Yeah, it's
Brett (35:52.863)
Yeah.
Rio Longacre (35:56.879)
Nothing ever is in enterprises, right? Like nothing ever is everything. Even the customer data, like you store 15 different places minimum. yeah, I can see, but Spencer, getting really, before we jump into the open source open way, I really want to do, I do want to dig into that and like how I think it'd be instructive for you to hear how that works. I do have, do want to maybe really briefly go to the economics of this whole thing. So I mentioned earlier about anthropics, like the meteoric, like
Spencer Gusberg (36:04.79)
Yes.
Rio Longacre (36:22.062)
the cash machine they've become just to throw some numbers out there. So, an anthropic one from 386 million in 2024 to 4.6 billion in 2025 to 16.3 billion in just the first six months of 2026. That's a 42X revenue jump in 18 months. mean, that's unheard of and a lot of that has just been this, how much AI, how many tokens are being consumed and the cost of tokens at these frontier, from these frontier labs. And there's another study that came, total revenue numbers, right?
Spencer Gusberg (36:22.797)
This one.
Brett (36:46.962)
And that's too those are total revenue numbers. Those are total revenue numbers, yeah, because yeah, 'cause it can't be recurring by definition.
Rio Longacre (36:51.598)
which is just a, but that's mind boggling, right? And it continues to ramp up at this incredible rate, it has to slow down. but my question is related to that. So another study came out that I thought was really interesting that showed that 90 % of AI spending is coming from 1 % of companies that are using, that are actually consuming tokens from these frontier models. And 80 to 90 % of them are unprofitable startups.
Spencer Gusberg (36:52.298)
Yes.
Spencer Gusberg (36:57.883)
Yeah.
Rio Longacre (37:18.51)
So I thought that was wild. if they are really this expensive and people are then saying it's too expensive, we're going to do open source, it's already 30 % of token use minimum, right? But it could be more because of the cost. I mean, they've just put out the rest one looks impressive, but that just seems kind of risky to me, right? If it's that expensive, mostly unprofitable startups using it in the kind of the circular buying thing, right? And it seems like these tokens from...
Spencer Gusberg (37:21.355)
It is interesting.
Spencer Gusberg (37:36.343)
For sure.
Rio Longacre (37:44.938)
open source models are picking up and are one seventy fifth the cost. I thought so, Matt.
Spencer Gusberg (37:49.751)
Yeah. Yeah. I think that's a super interesting area. You know
Spencer Gusberg (38:00.693)
If you look at the economics of the frontier labs, as we talked about before.
The space is unbelievably competitive, right? When you take into account competing labs, open source.
These companies are built upon and are raising money on the premise that they are gonna grow an incredible amount. And
you know, I'm not inside those companies, so I I d can't tell you exactly what's going on, but I I think they're under a lot of pressure to figure out ways to separate themselves from the frontier, from the rest of the frontier. And Anthropic's done a nice job with creating a nice ecosystem around its models. But like when you're talking about
the numbers that Rio just mentioned, and then we talk about their free models that anyone can download that are only four months behind. You know, that's a that's a a scary proposition and you know s serious
Brett (39:11.58)
In a serious competitive threat to anthropic who's about to go public and they're going public now just announced or at this when this pop pop right, they are not going to be bringing on advertising. They're not gonna bring in advert advertising revenue. Right, which obviously open eye stands on a different side of that fence, right? Realizing that it might be necessary. I you know, I actually think it's a it's a you know, I I sort of salute
Spencer Gusberg (39:19.467)
Yes.
Rio Longacre (39:19.662)
Yeah, the S-Wave just came out.
Brett (39:37.128)
that, even though I'm in the advertising industry, but that Anthropic says, hey, we've found another monetization model considering their growth or s set of monetization models that are gonna help us grow at the rate we need to grow.
Rio Longacre (39:47.15)
Yeah, that sounds like, Brett, sounds like that monetization model is just startups, Who are AI companies anyway, are consuming 90 % of the tokens.
Brett (39:52.691)
Yeah. Ninety percent of the tokens and it's and it's you said it's ninety percent of the token usage is being used by one percent of the company?
Rio Longacre (40:01.068)
by 1 % of companies and 80 % of them are startups that are unprofitable.
Brett (40:04.69)
Yeah. Right. And if you think about that, talk about a an AI w like it like it like if there's gonna be a bubble that bursts, right? And you've got a bunch of of these startups that that go out of business because of unprofitable models. Yeah. There are a lot of threats to to anthropic that model.
Spencer Gusberg (40:06.101)
Yeah.
Rio Longacre (40:16.718)
Eventually the VC money runs out, right? Eventually.
Spencer Gusberg (40:19.562)
Yeah, for sure. For sure. Yeah. There are a lot of yeah, there are a lot of threats. And if you think about how you increase model intelligence, like their core business, having the most intelligent models, okay, you need data to do that. You need proprietary data. These models have already been trained on everything that's publicly available, right? So how do you keep increasing model intelligence? Well it it's having unique user data sets, right?
Brett (40:41.321)
Yeah.
Spencer Gusberg (40:49.152)
So what you know, I know how Google I know what their proprietary user data set looks like, I know what Meta's user data set looks like, I know what Amazon's
Rio Longacre (40:58.382)
So basically proprietary user data comes from the users. In other words, the source, right? You're the battery.
Spencer Gusberg (41:04.682)
Right. It's well I you know, I mean right. It's like so s I mean, they're I would say that they're under incredible pressure to figure out how to monetize the prompts that we feed in. Right. you know, it's just the reality of it and
Rio Longacre (41:17.646)
Well, know, it's wild. We probably get hit up by some rando emailing us. Some of it's probably just fake, but some of it think is probably real emailing us. I've got three emails this last probably two weeks, Brett, from someone emailing us saying, we'll buy all of your video or all of your archive podcast just because we want information. We'll pay for it.
Spencer Gusberg (41:31.667)
Yes.
No, this this is huge. The data I mean, proprietary data has been important forever, but you know, the the pressure on that market right now is being really turned up because of the need for data. I I read an article, I I'm forgetting the exact source, but that was talking about a lot of businesses are gonna have this existential question of like, should I shut down before I'm really ready to shut down? Because all of my data is way more valuable than
you know, what my actual business line is.
Rio Longacre (42:07.758)
Well, it was the frontier air, which one went bust? And then they ended up selling their data for tens of millions of dollars, right? There was that one airline, it Spirit? I think Spirit went bankrupt, filed chapter 11. They liquidated the company, but then they were able to sell all of their data. I why would you want data on a failed airline? But apparently it has a lot of value. I think it was 10 million, maybe. It's a lot of money. We can look it up.
Spencer Gusberg (42:18.801)
was it spared?
Spencer Gusberg (42:32.106)
Abs absolutely. I mean, these models are
Spencer Gusberg (42:39.274)
Not to get too in the weeds, but like very brute force outcome-based models. Like we're way past the point of where there's real theory attached to to why they work. We just know they work. Right. You know, similar if you go back to j you know, deep learning and neural nets, like prior to deep learning really catching on and compute catching up.
Rio Longacre (43:03.73)
yeah, by the way, $10 million, they sold it to Google. We just looked it up, so sorry to interject, but you should keep going. Just for the data, to Google, yep. So Google learned how to build a failed airline.
Brett (43:07.38)
Ten million dollars just for the data for spirit.
Spencer Gusberg (43:08.97)
For the for the data.
Wow. Wow.
But think but think about all the unique human data that you have in there. You know, whether it's, you know, ordering behaviors on flights or whether it's how people purchase tickets, you know, all this information that can help you tune a model which is, you know, when you talked about like the anthropic numbers, right, their business is that having the most intelligent model.
Brett (43:16.029)
Yeah.
Spencer Gusberg (43:43.52)
So when you're talking those kind of numbers, ten million dollars to have insight into something no one else does, that's you know, it's pocket change for these guys.
Rio Longacre (43:54.402)
Yeah, for Google, they'd make that in like 10 minutes, that's for sure. Less than that, probably five minutes.
Brett (43:54.558)
Yeah yeah.
Spencer Gusberg (43:56.94)
Yes, it's true.
Brett (43:58.953)
Yeah, and it was and and it's funny 'cause Anthropic said that advertising, it it sort of biases or distorts an assistance or an L L M's incentives, right? towards engagement, right, which is the complaint around social media and that entire sort of we'll call it the the phase two of of sort of the internet revolution, with phase one being sort of when we just got online, social media and communication between all of these systems and between people, that they gained this system of driving
Spencer Gusberg (44:10.165)
Mm-hmm.
Brett (44:28.934)
engagement, whether it was a downward funnel into some sort of you know doom scroll or not, the idea was get people to engage with content and serve them more of that content. And and it sounds like anthropic sort of standing on this kind of pedestal saying we don't want our our answers to be distorted that way, but in a sense they've built the perfect mouse trap, right? I mean the LLM in terms of just just as it is.
Spencer Gusberg (44:41.226)
Mm-hmm.
Brett (44:55.976)
without being you know incentivized, let's say, by advertising, is kind of the perfect mousetrap. It's a conversational engine, it's something that can can answer any number of types of questions across the entire span of human communication and knowledge and psychology, right? So i i y I I could understand why you say, do we really need to do this? If we've got
Spencer Gusberg (45:11.708)
Absolutely.
Brett (45:18.428)
large B to B growth potential, right, within within businesses, right? And I'd love to see what the numbers of those of of that type of growth is and projected growth for anthropic, plus our consumer sort of mousetrap, which is highly effective. wh why bother? Right?
Spencer Gusberg (45:37.577)
Yeah, I think that you know you could argue that we are still very early in the diffusion of these models from an enterprise standpoint, right? Like we're we've made some great advances in terms of the modeling itself, but in terms of how to properly utilize these models to gain advantage to deliver value for companies, that's still very much an
ongoing area of study, exploration. you know, when you initially introduced E52 as like an engineering you know, as we sell basically engineering, I always like to say to people that engineering is one of the tools we use. But we're really what we're really trying to do is deliver value and selling problem solving to deliver value, right? Because, you know,
Brett (46:08.584)
Yeah.
Spencer Gusberg (46:33.375)
The bottom line is when you go into a company to try to deliver value, doesn't matter if it's AI, machine learning, doesn't matter if it's a new piece of hardware. You have to know how to work that into how the business works, how to move people to get them to use those things, use them effectively. And I think, you know, that's an area when you think of like B2B adoption for anthropic, you know, there's there's a lot more that can be done on that front.
Brett (46:57.268)
Yeah.
Spencer Gusberg (47:03.269)
And you know, will will definitely be done.
Brett (47:05.83)
Yeah, well a lot of it comes down to explainability, right? You know, just because you built the perfect mouse trap doesn't mean it's it's catching any mice, right? Or it's solving my specific use cases. Maybe I've got a r maybe I've got a rat problem, right? Or a cockroach problem. No, but but i you know, yeah, the and I think that's a c that's been an age old problem or challenge with tech of any sort.
Spencer Gusberg (47:17.141)
Ha ha ha.
Brett (47:26.514)
Right? You know, it's the the product and engineering folks are like build it and they will come, but you know, don't underestimate the amount of work that needs to be done to train people, to to align organizations, to align incentives around using these things for the for the best outcomes possible. Because otherwise you're just not gonna win over people and people will resist. And and well, I'm certainly seeing that on the brand side from brand clients. I work with some big agencies that that service brands, and brands oftentimes
Spencer Gusberg (47:26.889)
Yeah, for sure.
Spencer Gusberg (47:51.645)
Mm, mm.
Rio Longacre (47:52.301)
Hmm.
Brett (47:56.561)
are quite a few steps behind all of our industry talk in terms of how advanced their use cases are, how comfortable they are with applying some of their day-to-day work in advanced ways via AI, right? because they don't they just don't know how to start oftentimes, right? And so I think there's a lot of education that needs to be done in the industry to to to really make this as effective as it's claiming to be for the business world, right?
Spencer Gusberg (47:59.67)
Yeah.
Spencer Gusberg (48:14.463)
Mm-hmm.
Spencer Gusberg (48:19.221)
Yeah.
Brett (48:25.18)
across the border. Are you seeing that with your clients? I mean, is that
Spencer Gusberg (48:25.503)
Absolutely.
Yeah, absolutely. I think you know, everybody is obviously sees where things are going, right? Like you said, these models are amazing. Like that's I feel like pretty indisputable at this point. The amount the amount of capability that the machine has now has just expanded so rapidly. and so all of our clients are wondering.
How should we be thinking about this? How do we approach it? What is a good competitive posture in terms of how we adopt it? You know, the only thing we know about the future is that we're gonna have to continuously innovate, right? So building systems that are robust to that and can capture model capability shifts as they happen. you know, we see a lot of interest around building really good proprietary harnesses to touch back on an earlier point.
so that you can really capture these model advances well in a way that is specific to your to your business. You know, those are all topics that are on a lot of people's minds.
Brett (49:39.369)
Yeah.
Rio Longacre (49:41.527)
So switching back, I guess, to data sovereignty and context, that's something we do want to dig into a little more. So something Brett and I have talked a lot about on this pod is how the...
like the easy access of these democratization, let's call it, of AI technology, how for 20 bucks a month, even for free, right, with the ad support, consumers can get access to incredible technology that's as good sometimes even better than what enterprises have access to with their own containerized versions, right, that are usually a couple models behind. So if the, let's say the model layer itself is increasingly commoditizing, even at the frontiers, right, is the real asset
Spencer Gusberg (50:12.596)
Yeah.
Rio Longacre (50:24.941)
than going to be that proprietary layer that harness plus context layer that you mentioned. that, I mean, is that what you're seeing ultimately? Or do think the models will still retain value and maybe open source somehow differentiates that when you factor in cost? I don't know, thoughts?
Spencer Gusberg (50:34.387)
Absolutely.
Spencer Gusberg (50:41.606)
Yeah, I I think you nailed it in that there is the models themselves are a commodity at this point, right? You know
you know, I joke that like my mom who's not particularly tech savvy has access now to the most sophisticated models in the world, right? Which, you know, now actually doesn't in the history of the world, which maybe like now actually doesn't sound that crazy, but like if you had said that like two years ago, that would be like a insane, you know, maybe two, three years ago that would be like an insane thing to wrap your head around.
Brett (51:02.044)
In the history in the history of the world.
Brett (51:13.438)
Yeah. Yeah. I mean she might have just started using a email a few years ago, right? or an iPad, right? Now she's got a g you know model access that's just greater than than than anybody's ever had.
Rio Longacre (51:19.063)
Yeah, seriously.
Spencer Gusberg (51:19.122)
Right. Right.
Spencer Gusberg (51:27.624)
Yeah, exactly, exactly. And the where the proprietary, where the the moat lies is in everything that surrounds it. You know, Rio, to your point, can be the harness, can be fine tuning of a model to on your specific data set. You know, proprietary data really is you know, one of the biggest moats, figuring out how to use that properly.
So the the framework around it is definitely a moat. and you know, business processes around it as well. Like we talked about, like Brett was pointing out with tech in general. Do you have a good way to integrate AI advances on a continuous basis? That can be a great moat, right? all of those elements are part of it, but it's a drastic change from this software age that we've been in.
where the moat is having built the software, you know, put the engineering resources in and having built the software first and being able to to utilize that. Software is not really a much of a moat anymore. You know, a a fairly junior engineer can replicate most public facing websites in a weekend if they want to.
Brett (52:44.286)
Yeah, yeah, and this and this does have this democratization of sort of access quality to it, right? You know, where where if you do let's say with you're within an organization and the entire organization has access to a certain AI you know, they may not have all access to the same data, but they've got
Access to a lot more data and insight via the company approved AI than they've ever had in the past versus specific platforms. Right? Certain teams have access to ERP, other teams have access to CRM, think of all that you know, marketing automation, all these platforms are are all you know oftentimes function specific. it seems like the opportunity here is to give to almost democratize access to data and insights, obviously with with guardrails.
Spencer Gusberg (53:20.99)
Mm-hmm.
Brett (53:29.458)
for the organization, which allows the organization to just be that much more informed and that much more proactive in terms of of getting in in in generating insights that'll benefit their specific jobs or the organization as a whole. Right? And it seems like in the past that wouldn't be the case. If you wanted insights on on your sales effectiveness in a quarter or a month
Spencer Gusberg (53:43.111)
Absolutely.
Spencer Gusberg (53:48.521)
Yeah.
Brett (53:52.915)
Right. That information was was within a specific CRM Salesforce, for example, with very specific guardrails and seats. And it was blocked off to most people within the organization. Now I'm not saying I don't know if that's gonna open up that type of data is gonna open up. And I think organizations are well behind in terms of giving that level of like data congruence across the org. But it seems like that's the opportunity. It's like how do you harness the the the intellectual capacity and ability of everybody in your organization in a way that's sort of
Spencer Gusberg (54:03.338)
Mm-hmm.
Spencer Gusberg (54:09.703)
Yeah.
Brett (54:22.494)
Bottoms up versus versus top down. It seems to me huge potential.
Spencer Gusberg (54:26.202)
Hundred percent. We huge. We we talked internally a lot about, you know, why has nobody been able to replace Microsoft Excel in an effective way? Right? Well it's it's because, you know, it's it's because of what you're talking about, Brett. It's like it allows the end user to have enough flexibility without needing to know how to code to to create custom tools for themselves.
Rio Longacre (54:37.706)
Yeah, that's a question, right?
Brett (54:38.538)
Yeah.
Brett (54:49.716)
Yeah.
Spencer Gusberg (54:54.206)
that really democratizes, you know, the power of the machine, as we say. and you know, AI tools take that to a whole nother dimension, right? And now, you know, with that technology democratized, what everyone can, you know, people who have no software engineering expertise can write up a you know a product spec and have it coded up.
Brett (54:58.644)
Yeah.
Spencer Gusberg (55:21.8)
And have a tool, you know, all to themselves now. Like obviously that carries its own risks as well on the other side of that. but it's a really empowering thing and something that you know, companies that figure out how to harness that properly will really be in a great place.
Rio Longacre (55:40.301)
You know it's funny, Spencer, second year when you talk about the democratization, And like, know, people's mothers use it. Maybe think of, I'm a big Star Trek fan, I'm a Trekkie, right? Remember the Star Trek IV, like, what was it called? The Voyage on the One with the Whales, right? Remember in that Scotty is, I mean, they're in that factory where they're making the, you know, they ended up making the transparent aluminum.
if you ever saw that one. And he sits down in front of the computer and the guys and he starts, he says, okay, I'm not gonna do the Scottish accent. Okay, computer, right? And then he's hands in the mouse and he picks up the mouse and says, okay, computer. And the guy says, no, he shows him it, right? He's like, okay. And then, but I think about it, I walk around, like I went, walked my dog yesterday, I had to compose a very lengthy email to a bunch of team members. I was able to just paste a couple of things into,
the AI chat interface on the app and talking to it for 10 minutes, back and forth, I was able to compose and send this email to a bunch of people that was very good. I got response back the next morning, what a great email. To think that that was science fiction 30 years ago, it's incredible. I don't think people realize, because it just happened,
Spencer Gusberg (56:33.329)
Mm-hmm.
Spencer Gusberg (56:51.529)
For sure.
Spencer Gusberg (56:55.858)
It is incredible.
Rio Longacre (56:57.012)
It's just happened in our lifetimes and I don't think think people are living in science fiction now with what's been produced. It's incredible. Whether this business model is sustainable or not, whether it's an AI bubble, all that aside, what's been done already is absolutely astounding in terms of the breakthroughs we've had in my opinion.
Brett (57:02.366)
Yeah.
Spencer Gusberg (57:02.803)
Yeah.
Spencer Gusberg (57:12.539)
Absolutely. Absolutely. And I have no doubt that it will okay, we may be early in the stages of it, them the technology being diffused to enterprise use cases and being used in a very robust way. We will get there though. Like that that will inevitably happen because of how powerful the technology is. and it is exciting. Like I don't know where all these you know, every now and then you see these like
articles by software engineers saying like, it's not like it used to be. I just prompt Claude to write I d you know. I find it to be way more exciting to be able to build faster, to be able to
Rio Longacre (57:56.449)
Yeah, just to code review, build a build, build fun stuff faster, test it out faster. I mean, I think it's, I mean, I say this a lot. I've having much more fun at work because the mundane things are automated and it can focus into things that actually like, like doing things like this, right? Talking to people, writing interesting things, researching more, all like the, like the kind of the nuts and bolts that goes into that, the operational tasks, those are all done much quicker now. So I think it's a, it's definitely a huge unlock, but your Excel point.
Spencer Gusberg (58:07.657)
Exactly.
Spencer Gusberg (58:11.689)
Exactly.
Rio Longacre (58:24.938)
I think this is interesting. like this, we have these incredible tools, yet people are still using Excel in companies. People are still using tools from like 20 years ago, 30 years, right? So like the pace of adoption in corporate America is painstakingly slow relative to how fast the technology is actually accelerating, fair to say.
Spencer Gusberg (58:46.013)
Yes, absolutely. I think when you think about most of these organizations, they are not built in the image of the age that we're in, right? So, you know, they may have a structure that doesn't make sense. They may not have the technical talent in house to really help them adopt tools and move forward.
Brett (58:57.384)
Yeah.
Brett (59:07.133)
Yeah. There's an organizational design component, yeah, there's a people and talent component.
Spencer Gusberg (59:11.653)
Yeah. Exactly. Exactly. and
Brett (59:14.665)
Probably misaligned incentives, right? And and and also maybe data protections in house that that don't allow everyone to have basically you're siloing information and information access throughout an organization, right? Which means you're you're you know, you're not greater than the sum of your parts. You're just a sum of parts, right, as an organization. And it seems to me the new model is that like how do you you know, th everybody keeps talking about AI transformation and digital twins and
Spencer Gusberg (59:18.057)
Hundred percent.
Spencer Gusberg (59:24.806)
Yes, yes.
Spencer Gusberg (59:34.209)
Absolutely.
Brett (59:42.482)
allowing organizations to develop a knowledge center and a knowledge graph that that c you know compounds it's almost like the Borg, speaking of Star Trek, like compound the intelligence of the entire organization and a and keep this proprietary and in-house. Yeah. a hundred percent. And and like how you know the hive mind, that's also talk that's been but but I mean how f like from what I can see brands and I think the data that Rio mentioned earlier about the the number one users of tokens
Spencer Gusberg (59:50.407)
Right.
Rio Longacre (59:56.235)
That's my favorite villain, by the way, in all the Star Trek's. The Borgs, by far my favorite.
Spencer Gusberg (59:58.825)
The boys.
Spencer Gusberg (01:00:04.136)
Mm-hmm.
Brett (01:00:12.497)
Are the startups, are these kind of on the edge, next generation startups that are, you know, the six million companies that have formed year to date in the United States. Yeah. which is interesting because y you'd expect that you'd see more adoption and heavier usage for bigger problems, bigger use cases in very large organizations, but it sounds like that's not the case. And the economics are kind of proving that out, at least right now. But there's gotta be a point where these organizations are like,
Spencer Gusberg (01:00:14.056)
Yeah.
Rio Longacre (01:00:20.534)
So far this year, we'll probably finish like eight or nine.
Spencer Gusberg (01:00:36.37)
Yeah.
Yeah, but
Brett (01:00:41.681)
in order to maintain competitive advantage, right? If as long as they can I think to your point with your paper, as long as they can figure out a mechanism that's cost efficient, right, where they can use the open weight models for for very specific use cases in in right, versus the frontier models, which they're gonna be token to death and it's gonna be potentially not looking very good on the balance sheet, right? Like figuring out that science of of how do you make this economically viable
Spencer Gusberg (01:00:46.93)
Yeah.
Spencer Gusberg (01:00:58.237)
Yes.
Brett (01:01:10.003)
to drive the type of performance improvements that we need, I think is kind of the next frontier for brands.
Spencer Gusberg (01:01:16.008)
For sure. I think you nailed it on where a lot of companies are now in terms of on old tech stacks. They don't have their data. You know, it's too siloed right now for them to use utilize it properly. You know, we work with a fair number of companies that are at that stage where they're working to get basically the infrastructure in order so they can really take advantage of these tools. And, you know, it's taken some time to get the right policies in place.
you know, with proprietary data like, okay, what what is our internal policy for whether or not we can show this to a frontier model if we have all the right protections in place? You know, what is our stance on that? This you know, these are this is all innovation, right? this is new to everybody.
Brett (01:01:56.222)
Yeah.
Brett (01:02:01.502)
Yeah.
Rio Longacre (01:02:02.316)
Well, Suspenser, I think that begs the question, though. So if that becomes increasingly important, which I'm totally convinced it will be in this conversation, certainly reinforcing that, is, but this implies that a company needs to create and maintain its own context layer, right? And like, if that's incredibly important, if the models are commodities, assuming they'll swap out, you know, between maybe frontier labs, maybe they'll go to open source, maybe they'll have...
know, switchboard that decides what they're going to use based on the complexity or sophistication of the prompt or request, right? And they can maybe toggle between models. So you have this context layer you take with you. Our companies, if companies can't even get their, can't even find alternatives to Excel, is it fair to expect these enterprises to actually maintain a context layer and all that implies?
Spencer Gusberg (01:02:37.853)
Mm-hmm.
Spencer Gusberg (01:02:49.692)
Yeah. That's a great question.
Brett (01:02:51.535)
Without hiring small armies of data scientists.
Rio Longacre (01:02:53.996)
Exactly.
Spencer Gusberg (01:02:54.47)
Right. No, it's it's it's a great question and I think one that companies need to reflect on, right? In terms of what is what is our cost, what is our benefit of doing this, right? Even with the open weight models in general, fine tuning your own model. Like the cost is not just the token costs, which may be, you know, close to nothing. You also need to, you know, obviously maintain it, understand it well enough to, you know, fix it if it's broken is
it a critical tool that if you know midnight on a Saturday it stops working, we have a mechanism for getting it back up and running quickly. You know, you do you know when the with the Frontier Labs you're paying for that in your token spend, right? Like, you know, Claude is not going to
Brett (01:03:41.235)
C help me solve this problem. Something just crack crashed or isn't working. Well, let me ask you this. On that question of of and you you were just referencing, you know, you InfoSec, legal teams internally, understanding the ramifications of both the contracts they're signing with vendors, data usage, proprietary usage of data, all this stuff. I mean, how f do you think that they're becoming more technically astute to be able to really address these things in a modern
go forward way or or is that where you s you end up having a lot of friction in this sort of approval and implementation process?
Spencer Gusberg (01:04:18.406)
you know, it's probably selection bias with the companies that seek us out. well, exactly. They they're you know, they're already forward thinking and feel strongly that like this is something competitively I need to do to to win in my market. so, you know, I think there are inevitably plenty of companies that will be left behind who are, you know, quite resistant.
Brett (01:04:23.133)
Yeah, y you're selecting companies that don't have all that friction, right?
Brett (01:04:30.396)
Yeah.
Spencer Gusberg (01:04:48.85)
to these tools. You know, I think of it the same as I think about like the personal computer, right? Like some people were like, probably, why am I using a word processor? I like hand everything handwritten. And then they realize that like, I can, you know, just copy and paste this to another. No, you know, I won't be able to keep up with, you know, the store down the street if I if I'm not using this, right? similarly, you know, it's like
there are the good and bad use cases for AI, but I think we all see what the technology can do and where it's heading. And the people who are gonna win competition for spaces will embrace this technology. You know, I laugh when I hear people say like, I'm doing this AI, you know, I'm doing you know, I'm a AI
legal firm or whatever. It's like you wouldn't say I'm a law firm that uses computers, right? You just you ha computers are available so you use it for everything that you can to help you.
Brett (01:05:54.387)
Yeah.
Brett (01:06:00.232)
Yeah. Some of that's th some of that's kind of empty positioning, right? It's you you see a lot of that in the advertising industry with agencies saying we're an operating system when when oftentimes a lot of the tech in advertising is kind of a thin AI veneer over a lot of duct tape and gum and
Spencer Gusberg (01:06:03.78)
Right, exactly.
Spencer Gusberg (01:06:15.729)
Sure.
Brett (01:06:18.681)
you know, fixing broken systems and in in legacy systems that they're trying to tie together and make sense of, right? so it sounds like with this report, you know, you guys, I mean there's a couple of kind of key takeaways that I think we should reiterate. One is sort of split I think the key one that I'm taking away is split your AI, spend by job. You know, so for sort of the broad, fast changing, judgment heavy work where you need kind of broad inputs. The Frontier models are great, but but for the kind of the high volume, repeatable
kind of measurable work, you know, just doing that specific task over and over again, you can really leverage the open weight models at a much at a fraction of the cost to run those types of processes. Is that a is that a good assessment?
Spencer Gusberg (01:07:02.343)
Yes, and that's that's exactly it. And on selecting the job you wanna make sure that you have a good data set on outcomes that you can measure the performance of your model. If you don't have that, you are gonna be lost trying to fine tune your own model and to check performance as you go.
Rio Longacre (01:07:18.367)
You also need Spencer, not only do you need some way to benchmark and assess the performance of the bottles, but I think more importantly, you need the IT chops to actually implement the model, maintain the model. And then I'm guessing that, let's say open weight models are three, four months behind. If you install an open weight model in a year, it's going to be one year and four months behind, right? So you probably want to update to new versions of open.
Brett (01:07:31.614)
Yeah.
Spencer Gusberg (01:07:33.094)
Absolutely.
Spencer Gusberg (01:07:42.768)
Yes.
Rio Longacre (01:07:44.926)
open weight models, right? So you need the ability to be able to do that periodically. So, I mean, go into a frontier model, they're spending this billions and billions of dollars on innovation, on training the next model, which they're giving you access to, right? If you want to take this in-house, you can save us one 75th the cost per token, but there are other costs that you will need to shoulder in order to actually operationalize this, fair to say.
Spencer Gusberg (01:07:50.34)
Absolutely.
Spencer Gusberg (01:08:09.915)
Yes, absolutely. And I would say even if you're renting the frontier, you for enterprise workflows, you want a process for regularly testing performance against the models like, okay, there's a new model released, it's more expensive, is it worth us actually using that? Or is our is is the delta in performance gonna be negligible? And sometimes it might even go back.
Brett (01:08:30.875)
Well, and let me ask you this, is how do you how do you consult with clients about actually setting up those frameworks, those testing measurement frameworks specific to to model performance? You know, like the A versus the B, the delta between the two, we know that this was worth that it underperformed, but we were paying half the price. What like so how do you set that up? What what what do you have what are the first steps?
Spencer Gusberg (01:08:53.221)
Yes. First step is always making sure the data is being collected, right? Like what what is good performance for this model? Right? Okay. Fraud detection, you know, that's a simple question like, is this fraud or is it not? That's clear. but you know, with like prompting, for example.
Brett (01:09:07.867)
Yeah. Yep.
Spencer Gusberg (01:09:17.125)
you know, okay, you ask a research question like, hey, what markets should I be looking at? And that that model comes back with a set of answers, well, how do you determine that those set of answers are good? Right? That's a tougher problem. but there are ways to do it. And the key is having a data set that you have vetted that is strong that you believe covers your key use cases.
And is diverse enough so that you're not missing any big area of capability that you need. and regularly testing it and regularly adding to it as well. You know, there's this term of a golden data set, which is your data set which shows what the answers what are good answers, what they should be. The golden record, exactly.
Brett (01:10:02.013)
The golden record. Yeah. Yeah. Yeah, no, and what good answers should be. Yes, that that is the model and we we've I've talked to a lot in a lot of the companies that I've worked for. It's like this is what you're you're aiming for in terms of the output for this particular problem or evaluation set, right? And and then you run the data against
Spencer Gusberg (01:10:18.767)
Tactically.
Brett (01:10:20.821)
or you run the model against the data and you see how close it is to the golden record of of what the output should look like. Right? And if it's if it matches, you know, you then you have to fine-tune it, you know, and ver and vice versa. Yeah, and then you can train one versus the other over the same core data with the same set of kind of context instructions on like what the output needs to look like. It needs to have these three things, right? And you can compare models, I'm assuming. To have a test and control? I mean, is that
Spencer Gusberg (01:10:28.379)
Exactly.
Spencer Gusberg (01:10:32.336)
Is that
Spencer Gusberg (01:10:44.784)
Yes, it
Yeah, exactly. And you wanna have an understanding of what your cost function looks like. Like let's assume that we have a model that's a fraud detection model that's eighty percent accurate. Well, is it is it even worth us putting in the resources to get it to eighty five percent to ninety percent? What does it mean for us to miss twenty percent of fraud? exactly.
Brett (01:11:06.098)
Yeah.
Brett (01:11:09.991)
Yeah, it's like the cost benefit analysis of of like how much how much incremental dollars am I spending to get this extra percent and then how does that and then you're literally running economics. You're running kind of an accounting balance sheet or a P and L, right?
Spencer Gusberg (01:11:20.955)
Hundred percent. Exactly. The business context for all these things is super important.
Rio Longacre (01:11:22.379)
Suspense?
So looking at business itself and how AI is impacting business, Patrick, I love your take on this. So there was a recent study that was published by NBER and this study said that despite 78 % of firms now using AI, annual labor growth productivity had only increased by 0.2 percentage points over the past three years. The study also found essentially no net employment effects so far due to AI.
And there is an extra, I thought I could quote that was really hard hitting it said, we keep hearing extraordinary task level productivity numbers from AI, 20%, 30%, sometimes 50%. But when economists look at the firm, these gains seem to disappear and they cite this study. So I have some thoughts on that. I'd love to get yours before I dive in.
Brett (01:12:16.017)
Yeah, me too.
Spencer Gusberg (01:12:18.246)
Yeah, so I have a somewhat weird perspective maybe being in s this a space that is very the productivity increase is not debatable, right, in software engineering. it's massive. right, from a code production perspective. And but I think that that study speaks to what we were talking about before where
Brett (01:12:34.631)
Yeah. Yeah. From a c from a code production perspective, I mean it's
Spencer Gusberg (01:12:48.164)
you know, diffusion of these tools to enterprise workflows in a really effective way is not where it it should be, needs to be, will be. you know, I think it will happen. And to the question of, you know, job displacement, I think that is inevitable. You know, I've talked with a lot of people who are like, how do we, you know
set things up so we're not taking people's jobs with AI. And I actually think that that's the wrong question because people's jobs will change. This do you use the computer analogy also, like it's inevitable. What the machine can do better than the human is different than it used to be.
Brett (01:13:23.175)
Yeah.
Brett (01:13:28.059)
Yeah. And we should we should change the definition of what jobs people should be taking versus my job is being taken out you know, redefin redefine certain jobs because those are going to change dramatically based on on, you know, some of these things being automated.
Spencer Gusberg (01:13:33.934)
Yes.
Spencer Gusberg (01:13:42.863)
Exactly. Exactly. Redesign jobs so that they can effectively capture the value of AI. you know, I people smarter than me will will work on figuring that out. You know, that doesn't mean that like it's not gonna be painful because the tech has moved so quickly that the shift like can we shift fast enough to completely couple with that? You know, probably not. but I think that, you know, those
numbers on like what the actual impact i has been to Rio's point on on that study. I think it's a story of like we haven't figured out how to translate this to boots on the ground in a lot of places. You know, it's a select few groups that have effectively done it.
Brett (01:14:30.429)
Yeah.
Yeah, and this in this sort of intelligence, super intelligence is is applied to a lot of brute intelligence use cases, right? It's brute data analysis, it's brute research, right? Third party research across the the internet, right? It's it's tasks that are super repeatable, right? Think of like the the the legal industry and the amount of documentation and sort of reproduction of documents that has to happen again and again and again. There's a lot of this rote work. To me, that's where br
Spencer Gusberg (01:14:40.544)
Mm-hmm. Yes.
Spencer Gusberg (01:15:00.41)
Yes.
Brett (01:15:02.333)
Where the brute intelligence can can serve its purpose. It's you know, and I think one of the conclusions that we've drawn you know over the last sort of six or seven episodes of Signal and Noise is that it's the human judgment, the creativity, the innovation, that that is is in the imagination that humans bring to the table that is that is completely unique, that is that is independent of of raw we'll call it wisdom, right, Rio?
Spencer Gusberg (01:15:06.565)
Right.
Brett (01:15:29.649)
Wisdom. It's independent of of of intelligence, of raw intelligence. Right? So that's oftentimes what I think is is the source a l of a lot of the lack of productivity gains is I mean I would make the argument, at least theoretically, that people are outsourcing some of their own judgment and they're they're burdening people around them with these raw intelligence outputs from AI.
Spencer Gusberg (01:15:29.734)
Mm.
Brett (01:15:53.739)
Right? Whether it's the whether it's the the email that's fourteen times longer than it should be, whether it's the document that's eighteen pages that you need to review that was literally an AI output, they're adding time to people's lives and work that that because they're because they're bypassing their own sort of strategic judgment when they're
Rio Longacre (01:15:54.187)
Yeah, well it's okay.
Spencer Gusberg (01:15:54.276)
Absolut. Absolutely.
For sure.
Rio Longacre (01:16:12.53)
It's funny, I'm gonna agree with both of you. I actually think that this study, this study may be accurate. It may be that as of now, and I think the main reason for this would be AI transformation is really org transformation. mean, like Tabaka Walda said, remember you can't change orgs, you can change people, and people are changed to slow orgs, they're really damn slow to change. So I think that is just the fundamental reality. These things are icebergs, they move very, very, or battleships, analogy you want to use, they move very, very slowly, it's gonna take a while.
Brett (01:16:24.958)
Yeah.
Yeah.
Spencer Gusberg (01:16:37.413)
For sure.
Rio Longacre (01:16:40.968)
And for those who are old enough, many listeners may not be, but turning back the clock, both of you probably remember this, there was something that was called the solo productivity paradox. And this was, I remember this when I was in high school, I remember, and it came out in the late eighties and like, this is when personal computers were starting to become more ubiquitous, right? And companies had adopted a very aggressively starting in like the late seventies, early eighties, right? And there was a study that came out that basically showed the same damn thing that this AI study showed, that basically,
No productivity gains are minimal, right? And in fact, the famous quote from this article study was, can see the computer age everywhere, but in the productivity statistics. So show the same damn thing. And for the same reason, right? It took companies a while. They had to buy the computers, people learned how to use them, then had to redesign how people worked, how offices worked, how tasks, maybe some tasks were removed altogether, but you know, it took a long time. People had to adjust, right? It took decades. It didn't take years. It took decades.
Spencer Gusberg (01:17:27.599)
Yeah.
Brett (01:17:34.473)
Yeah.
Spencer Gusberg (01:17:36.109)
For sure.
Rio Longacre (01:17:39.339)
for the actual productivity. When it came, I mean, think about it, completely transformed society. All the things, think about like even social media, Uber, ride shares, right? Like apps, like smartphones, all of that came later, right? Which completely transformed society, which you didn't see in those initial studies they performed a few years. I think we're at that age. Maybe AI's a little faster. Maybe it's not gonna take 30 years that we saw with, let's say, like personal computers.
Spencer Gusberg (01:17:51.77)
Yeah.
Spencer Gusberg (01:18:02.053)
For sure.
Rio Longacre (01:18:08.17)
and the digitization of tasks, but I think you're going to see it's going to take at least 10. And anyone who's expecting these companies will redesign themselves beyond two percentage points in a couple years is diluted and their expectations are artificially inflated. And they should just get a grip. I don't know. That's my thought on it.
Spencer Gusberg (01:18:24.345)
Yeah. No, I I think that's a fair point. I I think there's
There's a lot to be done to capture the value, but like we don't even know the best ways to capture value for these tools, right? Like I think to to Brett's point, I think you know, these models, like we have not replicated the human mind to despite what we all are like, you know, intrigued to believe. so
Brett (01:18:51.326)
Yeah.
Brett (01:18:54.875)
We're still yet to replicate the f the raw compute capacity of the you know, despite the number of petaflops, that was a term that we learned recently, are being processed per GPU or within data centers, the actual individual human brain is is
Spencer Gusberg (01:19:03.576)
Yes.
Brett (01:19:10.545)
you know, orders of magnitude more complex than a lot of the systems that we use, unless you're comparing it against, you know, kind of massive superclusters. but just we're talking about a single human brain, right? That's biologically evolved. I mean it's pretty remarkable. once we get there, I mean people are really gonna panic. Right?
Spencer Gusberg (01:19:21.413)
Right. We
Spencer Gusberg (01:19:29.167)
Well, I think I I you know, I we don't understand the human brain well enough to to replicate it. What we you know, there are ways in AI in its current state is way better than the human brain at certain things and not as competent as the human brain at certain things. So, you know, this process of like where it makes sense to have a human decision versus where it makes sense to have an AI decision, those are things that will be figured out, at least to a greater extent, in the years to come.
Brett (01:19:34.76)
Yeah.
Brett (01:19:40.67)
Yeah.
Brett (01:19:44.199)
Yeah.
Spencer Gusberg (01:19:59.012)
maybe in the months to come, that will really govern, you know, the things that Rio was talking about in terms of how we really extract the value from these models that's there.
Brett (01:20:10.535)
Yeah, yeah, and we're and we're we're you know, for those that that don't watch Mad Men or haven't seen it, it's the Harry Crane, right, and Jim Cutler problem, right? Where, you know, you had these these incredibly, you know, smart, creative
imaginative, innovative thinkers, right? the Don Drapers of the world. And then you had this machine that was taking up an entire room in the office that they felt threatened by. But I think I think that's an analogy for that machine's always going to be there and Don Draper is always going to be there. Those two those two things are completely distinct types of brain and intelligence and and we should think of them as complimentary, not as contradictory, right? Despite what we hear from some some of the AI bosses, right, that
Spencer Gusberg (01:20:33.113)
Mm-hmm.
Rio Longacre (01:20:41.076)
Yeah, I like that.
Spencer Gusberg (01:20:49.87)
Absolutely.
Rio Longacre (01:20:52.724)
Yeah. Yeah.
Brett (01:20:53.591)
You know, superintelligence, it's all about it's gonna you know, it's yeah, it will have impacts across all sciences, across all R and D, all business. But
Rio Longacre (01:21:01.258)
Yeah, I, yes it will, but I'm not convinced it'll be alive. I'm not convinced it'll, I mean, maybe it'll be, it'll be do a great job imitating it and maybe, look, maybe we'll get to like super intelligence or consciousness, but I've seen no, I mean, granted, I'm not working in one of these labs, but I've seen no evidence so far that these things do anything but copy life extremely well and predict answers extremely well at a pace and scale and intelligence that, you humans, you're right, Spencer, couldn't do, right? But they sure as hell aren't alive.
Spencer Gusberg (01:21:11.909)
Yeah.
Spencer Gusberg (01:21:30.105)
Right. R right, right. The same way a a car can travel faster than I can run. Right. The same way a traditional personal computer could, you know, copy a file to the word, to the character, to another file. You know, there are you know, it's like we said the the capabilities of the machine have changed drastically, but
We're not you know, it's not mirroring exactly what a human can do, except for what you're saying, Rio. It's like it's been trained to copy the outputs that we give.
Brett (01:22:05.757)
Yeah. And for organizations and brands out there and even the agencies that are building some of this stuff, it really does come back to the data, the context, the harness, right? The governance that you put around these systems that you're implementing and the protections from you know, data leakage, AI leakage, et cetera. Like that's you focus on that and you're building your own proprietary advantage, arguably.
that will help you not only in the short term but mid and long term as well. So
Spencer Gusberg (01:22:31.829)
Absolutely.
Spencer Gusberg (01:22:36.64)
Absolutely. Yeah. I'm a big believer that context is king. Show the models the right context at the right time and you will be ahead of the game.
Brett (01:22:47.827)
Well all right. This has been amazing. yeah, yeah, we're I we're we're we're approaching an hour and a half. Let's get to should we get to quick hits, Rio? Unless you've got Rio's always got another another question. He's always gonna
Rio Longacre (01:22:49.738)
It's been fun. It's been really fun. Yeah.
Spencer Gusberg (01:22:51.042)
Yeah, thank you guys.
Rio Longacre (01:22:55.764)
Let's do it. Let's do a quick kiss. The one I just brought out minute ago, that was my one I had in reserve. The solo paradox redux, let's call it. That was my singer.
Spencer Gusberg (01:22:58.636)
All right, love it.
Brett (01:23:02.301)
That was your final final.
Ha ha.
Spencer Gusberg (01:23:06.796)
Yeah, those are great those are those are great questions. I I didn't know that also that stat on token consumption being you know, what was it? Te you know, ninety percent of
Brett (01:23:18.471)
Eighty percent of c of consumption, right?
Rio Longacre (01:23:19.546)
1 % are using 90 % and 80 % of them are startups that are not profitable. That's a little worrying, to be honest.
Spencer Gusberg (01:23:25.346)
Yeah, wow, wow.
Brett (01:23:26.065)
Yeah, no, that is a little worrying. Speaking of a bubble, right? It it is it is that is concerning.
Rio Longacre (01:23:30.506)
there's a bubble. question is, it's not, I don't think it's the same, but people always think every bubble is the same. It's a bubble. mean, for sure, when all these startups go bust, I mean, the revenue from these companies that are then, I mean, think about lot of these VCs are getting money from their AI investments, so they're taking this money and investing in startups that use AI. This is like circular dealing now. So at some point that breaks down, but the thing is, this is like...
Spencer Gusberg (01:23:35.65)
Right. Agreed.
Rio Longacre (01:23:54.187)
Enterprise adoption is so low, Like you think about it. So, okay, if only one percent of companies using this much AI, but the reality is enterprises have barely started to use it at a scale they're going to. And even if majority of tokens are open weight, mean, crap, there's still going to be a lot of AI usage. We've barely scratched the surface of it. So I think even if things do, we do have the bubble burst, and let's say we have a year or two of spectacular losses for some investors.
Spencer Gusberg (01:24:13.55)
For sure.
Rio Longacre (01:24:22.026)
Right? I think that, you know, whatever data centers are built, they're going to be used. And we're at the beginning of AI in my
Spencer Gusberg (01:24:29.902)
I couldn't agree more with that. I think that
Brett (01:24:31.674)
Alright, so so quick hits, right? so so so tell us maybe maybe a b an example of and I know you said you're you're sort of biased towards companies that are are forward thinking, right, in terms of the types of clients that you bring on, that are a bit more well prepared maybe than the average company out there. But like can you talk about maybe a failed deployment that you learned a lot from in terms of implementing AI transformation or something comparable to it within an organization?
Spencer Gusberg (01:25:00.728)
good qu question. I I don't have I can't talk in too much detail about like a specific deployment, but what I would say is that I have yet to see you know, in my you know, 15 years of deploying data-driven tools for enterprises, I have yet to see a project fail because of lack of technical capability. It's
all the things that you guys were discussing earlier in terms of organizational change, managing the life cycle of a project, you know, like there's we're in especially now, we're in a space where this is all very new. When you talk about in an organization making an investment in something very new, you're inevitably going to get a lot of questions along with that. So understanding how you demonstrate value early, get momentum going.
build a proper culture around, you know, the initiatives you have, that's really where the secret sauce lies. And you know, like technical issues that may come up are generally pretty solvable with a few good heads, you know.
Rio Longacre (01:26:24.074)
Is the problem that companies are adding AI to old workflows instead of redesigning the workflow themselves? I think I know what you're going to say, but is that the main problem that you're seeing?
Spencer Gusberg (01:26:36.778)
In terms of like places where it's not quite working? Yeah. I
Rio Longacre (01:26:38.662)
AI adoption, yeah, you're getting value from AI.
Spencer Gusberg (01:26:43.694)
think that is definitely a part of it, right? Is like to what we talked about, you know, companies kind of being built in an image that's not synonymous with this AI age is definitely a challenge. you know that definitely can be a big impediment to getting the value you need. but they're also, you know, there's a lot of low hanging fruit to be had as well.
in existing processes often.
Brett (01:27:17.722)
Yeah, so so so if a leader changes just one AI decision they have to make after hearing this conversation, what should it be?
Spencer Gusberg (01:27:29.998)
just one decision. I I would say
Develop a framework for determining the right solution on a job basis.
If I had to if I had to distill it to one, I could also rattle off probably like five more, but
Brett (01:27:49.702)
Yeah. And then apply the technology after the fact. Apply the Yeep.
Spencer Gusberg (01:27:52.524)
Right. Apply the tech. You know, I always go back to what is our one certainty right now? It's that we're gonna have to be very good innovators, all of us. Right. So developing the framework to do that as a company.
Brett (01:28:07.302)
Yep. Yep. That's the strategic thinking. That's the human judgment that goes into this.
Spencer Gusberg (01:28:12.419)
Absolutely.
Rio Longacre (01:28:14.654)
What are the chances you can give us a percent that AI is going to kill us all?
Spencer Gusberg (01:28:18.883)
Hundred percent. No. Hundred percent and it's it's next November third. No. you know, I I'm not a big doom and gloomer. I I you know, I the whole extinction conversation, you know, I I don't know how to put a good number on that. I do think that like, you know, it's
Brett (01:28:20.978)
We're we're ending we're ending every episode with that question.
Rio Longacre (01:28:21.962)
We got you man.
Rio Longacre (01:28:38.9)
This one gets the best reactions. That's why we got to ask it.
Brett (01:28:41.388)
Yeah, totally. Totally. We know immediately if you're like a Doomer or if you're like an optimist. Depending on how you just start. You know, you can just grunt and we'll know if you're a Doomer or you're a you know. Yeah. And we've had conversations we've had conversations, you know, d names unspoken, but with doomers that honestly feel one they fe
Spencer Gusberg (01:28:47.075)
Depending on depending on whether they laugh at that question.
Rio Longacre (01:28:48.234)
You start laughing or you're...
Rio Longacre (01:28:56.584)
Your face gets really serious, right?
Rio Longacre (01:29:04.948)
We talked to a doomer and had a very uncomfortable conversation that will drop soon with a doomer.
Brett (01:29:08.4)
Yeah. You know, and there's people that fear their own children, fear raising or having children, that these children are gonna be you know I mean it's almost like they're they're imagining a Borg like world, right? Or a Battlestar Galactica like world, like in the next five to ten years. And and yeah, I get the cautioning, you gotta wear a
Spencer Gusberg (01:29:10.339)
Okay.
Rio Longacre (01:29:18.282)
Spencer Gusberg (01:29:25.729)
Yeah.
Rio Longacre (01:29:27.742)
Well, Brett, think they're going to be disappointed.
Brett (01:29:30.632)
Yeah, I'm I'm I'm thinking this is gonna have positive impact across a lot of you know, we'll figure out a way how to govern and manage and control, and in you know, we will figure out a way to do that, right, at the governmental level, the organizational level.
Spencer Gusberg (01:29:33.025)
No they won't
Rio Longacre (01:29:44.716)
I mean, know when like cars were coming out, there were people, old curmudgeons who were like, these cars are gonna destroy society. You know, every change, electricity, there are people saying how, I mean, this is the same, in my opinion. People are afraid of change, it's scary, yeah.
Spencer Gusberg (01:29:54.336)
Sharp.
Spencer Gusberg (01:29:58.852)
For sure. You know, it's it's it's unnerving, yeah, it's scary, yeah. I mean I think that, you know, these are extremely powerful tools and like will bad things happen? You know, probably. but you know, it's like i inevitable with you know, with powerful tools, with computers in general, yeah, with anything. you know, does that mean the end of civilization as we know it? I think that's a stretch, but you know.
Rio Longacre (01:30:15.199)
With anything.
Spencer Gusberg (01:30:27.939)
I don't know, maybe somebody has like a good way we get to
Brett (01:30:28.122)
Yeah. Well maybe it's an i end of civilization as we know it, meaning meaning things are going to change in terms of how we operate and and organize as a species globally and then multiplanetary applications of this. That is changing. These are th this is an evolutionary level of it from what the human species can do now going forward. Because our just our our problem solving abilities have just been multiplied by thousands, tens of thousands, right?
Spencer Gusberg (01:30:36.141)
For sure.
Spencer Gusberg (01:30:40.557)
For sure.
Spencer Gusberg (01:30:45.261)
Yep. Yeah.
Spencer Gusberg (01:30:55.349)
Ab absolutely, absolutely. And to the extent that like Yeah, exactly. And to the
Rio Longacre (01:30:56.489)
Yeah, and we haven't even seen the impact. We haven't seen the impact yet. We've only seen like, it's just a novelty at this point compared to what it will be.
Brett (01:31:04.487)
Yeah.
Spencer Gusberg (01:31:04.649)
Absolutely. And to the extent that as a society we're mirroring what we've talked about with all these companies where like we need to be able to innovate quickly and you know that's a tough thing for society. So, you know, it's like you guys said, it's you know, it can be a scary and painful process, but you know, it brings a lot of excitement and opportunity with it as well.
Rio Longacre (01:31:24.373)
People should be happy to be alive to live through this, in my opinion. Between what's happening in AI, between what's happening with space exploration, we're actually getting our shit together. We're gonna build a moon base. We're gonna go to Mars. We're building fucking robots. mean, this is an exciting time to be alive, people. I everyone's all gloomy and like, the world's gonna end. you know what, this is, I think with AI, we're gonna solve all our climate problems. We're gonna figure out how to either build...
Spencer Gusberg (01:31:27.575)
Yeah, I'm with you.
Brett (01:31:28.71)
Yeah.
Spencer Gusberg (01:31:36.375)
Yeah.
Spencer Gusberg (01:31:41.643)
Him an amazing time.
Brett (01:31:43.058)
Yeah.
Brett (01:31:50.876)
We're gonna cure cancer.
Rio Longacre (01:31:51.295)
get more clean energy, we're gonna cure, we're gonna, cancer will be cured within the next 10, 15 years, probably quicker than that, probably quicker.
Brett (01:31:56.006)
Yeah. Yeah. Yeah. And there will there will be dystopian novels, there'll be dystopian realities, right, that will surface right. Yeah, th that will surface these things, you know, sometimes they're just warnings so that you don't go down that path. But there's also going to be a lot of huge benefit for all the science fiction fans out there that have seen this stuff, whether it's a William Gibson novel or Isaac Asimov, whoever it might be, this type of stuff, seeing this actually happen in our own lifetimes is is simply amazing.
Spencer Gusberg (01:31:58.68)
Yeah.
Rio Longacre (01:32:02.709)
We need our Hollywood movies, right?
Spencer Gusberg (01:32:03.682)
For sure.
Brett (01:32:25.038)
and it is it is an incredible time to be alive and
Spencer Gusberg (01:32:25.064)
It is amazing.
Rio Longacre (01:32:26.823)
And wait until we have AI fused with robotics and what that's going to do for not only human productivity, for, mean, it's going to be, I mean, yeah, I mean, will it be, is it going to be kind of messed up when we have armies of droids fighting droids? yes, but that's going to happen. It's already happening with, in skies over Ukraine, right? I mean, imagine, you know, with drones, I mean, that's going to happen. We're going to have robots fighting each other. And there's going to be some crazy things. It's your point, Spencer. mean, like, tech comes out, you can do good things. You can...
Spencer Gusberg (01:32:30.986)
Yes, yes.
Spencer Gusberg (01:32:45.154)
Drones and yeah.
Rio Longacre (01:32:56.681)
nuclear power, can create a bomb and unfortunately incinerate tens of thousands of people, you can create nuclear power, can power spaceships with it, you can do all sorts of stuff, can split, know, so there's all sorts of positive uses to come from technologies, I mean, it's technology, just a tool to your point, I think people will realize that and, you the benefits of this tool, we've barely even begun to scratch the surface.
Spencer Gusberg (01:32:59.019)
Right. Right.
Spencer Gusberg (01:33:12.62)
So yeah.
Brett (01:33:18.758)
Yep. Well hey, this was awesome. We we've gotten into the theoretical once again. hey, thanks everybody for making it this far. DebwW dot signal noise dot AI. follow us on Apple, Spotify, YouTube, TikTok, Instagram, and subscribe to our newsletter, which comes out three days a week with just a couple pieces of content, new articles and new pod episodes. and we'll see you next time and thanks a lot, Spencer. This has been awesome.
Spencer Gusberg (01:33:20.032)
Absolutely. Yes.
Spencer Gusberg (01:33:44.192)
Yeah, thank you both. This was great. Thank you for having me.
Rio Longacre (01:33:46.475)
It's a time.





Comments