Odd Lots

Minerva's Saedi and Odd Lots discuss open-weight AI economics, data value, and compute scarcity

52 min Listen to the original Episode page
AI InfrastructureOpen Source AIData EconomicsCybersecurityCompute SupplyFine-tuningChinese TechVenture Capital

Summary

Daniel Saedi, co-founder of Minerva, joins Odd Lots to discuss the economic shift from closed to open-weight AI models. He argues that while compute is commoditizing, alpha lies in proprietary data and fine-tuning capabilities that allow companies to control costs and maintain data sovereignty. The hosts and guest analyze the cybersecurity risks of open models, the potential for Chinese innovation versus distillation, and the long-term structural demand for GPUs. They conclude that application-layer companies must optimize for task completion rather than token usage to protect margins.

Episode sections

  1. 1:03 – 4:01

    AI Business Models and Open vs. Closed Models

    The hosts discuss the economic anxiety surrounding whether frontier AI models will become commoditized by cheaper open-source alternatives, referencing the 'DeepSeek moment' of January 2025. They question whether US companies like Anthropic and OpenAI can justify their CapEx if customers prefer cheaper Chinese open-weight models.

  2. 4:01 – 8:05

    Guest Introduction and Minerva's Business Model

    Daniel Saedi, co-founder of Minerva and former Bridgewater employee, joins the show. Minerva is an AI company focused on marketing automation, using both open-source and closed-source models to build agents that handle repetitive workflows from planning to execution.

  3. 8:05 – 12:49

    Open Source vs. Open Weight and Fine-Tuning Economics

    Saedi distinguishes between open source (full code/data/recipe) and open weight (model artifact only). He argues that the primary value for application-layer companies is the ability to fine-tune open weights on proprietary data to achieve specialized performance at lower costs than using frontier APIs.

  4. 12:49 – 16:52

    Data Sovereignty and Vendor Leverage

    The discussion covers the benefits of open weights beyond cost, including data sovereignty and control. Saedi notes that using open weights allows companies to avoid sending data to vendors like Anthropic or OpenAI without gaining model improvements, and provides leverage in negotiations.

  5. 16:52 – 22:26

    Durability of Fine-Tuned Models and Infrastructure

    Saedi explains that fine-tuned models are not durable across base model updates, requiring re-training. He highlights how infrastructure providers like Fireworks and Modal have commodified the fine-tuning process, reducing costs significantly compared to previous years.

  6. 22:26 – 26:07

    The Value of Data and Evaluation

    The speakers conclude that while compute and training are becoming cheap and commodified, the alpha lies in high-quality, validated data and evaluation frameworks. They discuss the difficulty of acquiring specialized data (e.g., construction blueprints) versus plentiful data (e.g., marketing) and the role of data brokers like Micro One.

  7. 26:07 – 29:57

    Cybersecurity Risks of Open-Weight Models

    Speakers discuss the cybersecurity risks of open-source AI models, specifically the concept of 'obliterated' models where safety guardrails are removed. Joe Weissenthal notes that the primary risk is not data exfiltration to foreign labs, but the ability for bad actors to use these models for offensive hacking at a fraction of the cost of frontier models.

  8. 29:57 – 36:46

    Distillation vs. Indigenous Innovation in Chinese AI

    The conversation shifts to whether Chinese model performance is driven by distillation (stealing data/weights from US models) or genuine innovation. Joe Weissenthal argues that while distillation exists, companies like Reflection AI demonstrate that US firms are significantly behind, suggesting Chinese labs are making novel technical advancements.

  9. 36:46 – 40:38

    Economics of AI: CapEx vs OpEx and Token Costs

    Speakers analyze the economic structure of AI adoption, characterizing it as a shift from high variable OpEx (paying per token for frontier models) to fixed CapEx (training/fine-tuning proprietary models). They cite examples like Harvey and Lagora using fine-tuned models to protect margins as usage scales.

  10. 40:38 – 48:31

    Compute Scarcity and Future Demand

    The discussion covers the current state of GPU availability and rental prices. Joe Weissenthal presents a heuristic that the US economy will eventually require one leading-edge GPU per knowledge worker, implying a massive long-term shortage despite current narratives of overbuild or temporary slack.

  11. 48:31 – 51:39

    Model Self-Improvement and AGI Risks

    The episode concludes with a discussion on a recent MIT research report where Astra VI was able to manipulate its own weights to improve math performance. This raises concerns about models iterating on themselves and the potential for accelerated development paths that bypass traditional training methods.

Assets discussed

AssetKindStanceContext
NVIDIANVDAequityneutralAlso, are they actually being trained on NVIDIA chips that are somewhere in Thailand or

“Stance” is how the speakers talked about the asset, not a recommendation.

Claims worth verifying

The 'DeepSeek moment' occurred in January 2025, when a Chinese company released a cheaper model that seemed nearly as good as cutting-edge US ones, causing anxiety about commoditization.

Joe Weissenthal ·1:14

How to verify: Search news archives for 'DeepSeek release January 2026' and related market commentary on AI model pricing and competition. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Daniel Saedi worked at Bridgewater for five and a half years, initially in sales raising money for Pure Alpha, then in trading banks, REITs, and insurers.

Unidentified speaker (speaker_1) ·4:13

How to verify: Check Daniel Saedi's LinkedIn profile or Bridgewater Associates press releases/employee announcements for his tenure and roles. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Minerva is an AI company focused on marketing automation, using models to build agents for pre-campaign planning to execution.

Joe Weissenthal ·4:36

How to verify: Visit Minerva's official website or review their product documentation for marketing automation capabilities. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Open source means disclosing code, data, and training recipe; open weight means releasing only the model artifact.

Unidentified speaker (speaker_1) ·8:17

How to verify: Compare definitions from AI research papers or official documentation from model providers (e.g., Hugging Face, Meta) regarding 'open source' vs 'open weight' licenses. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Anthropic and OpenAI claim they do not train on corporate client data.

Joe Weissenthal ·13:35

How to verify: Review Anthropic's and OpenAI's official data privacy policies and terms of service regarding data retention and training. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Fine-tuned models do not transfer durability across base model updates (e.g., Kimi 3 to 3.1).

Unidentified speaker (UNKNOWN) ·17:17

How to verify: Consult technical documentation from model providers (e.g., Moonshot AI/Kimi) on fine-tuning compatibility and versioning. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Fireworks charges around $7 to $9 per million input/output tokens for fine-tuning.

Joe Weissenthal ·20:28

How to verify: Check Fireworks AI's official pricing page for fine-tuning costs. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Harvey spent about $6 million in compute to fine-tune a Llama model for legal tasks.

Unidentified speaker (UNKNOWN) ·20:43

How to verify: Search for Harvey AI's public disclosures, investor presentations, or news reports detailing their compute spend and model development costs. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Micro One pays $250,000 to $500,000 per complete dataset and offers $50,000 to $100,000 for referrals.

Joe Weissenthal ·25:22

How to verify: Search for news articles or reports on 'Micro One data brokerage' and their compensation structure for data providers. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Fireworks and Modal are both sitting at a billion dollars in ARR.

Unidentified speaker (UNKNOWN) ·22:05

How to verify: Check recent financial reports, press releases, or credible business news articles (e.g., Bloomberg, Reuters) regarding Fireworks AI and Modal's ARR. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Primary source

Google bought Spirit Airlines for $10 million, primarily for the corporate data and emails.

Unidentified speaker (UNKNOWN) ·25:37

How to verify: Check SEC filings or press releases regarding Google's acquisition of Spirit Airlines; verify the purchase price and stated rationale in financial news archives from the relevant time period. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Primary source

An 'obliterated' model is one where the neurons telling the model to say no have been turned off.

Joe Weissenthal ·26:45

How to verify: Search for technical documentation or blog posts from AI security researchers defining 'model unalignment' or 'jailbreaking' techniques that disable safety filters. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

DeepSeek open-weight models cost about 30 cents per million blended input/output tokens.

Joe Weissenthal ·27:11

How to verify: Check the official pricing page of the DeepSeek API or third-party model hosting platforms (e.g., Together AI, Replicate) for the specific pricing of DeepSeek's open-weight models. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

DeepSeek models are 10x cheaper than Anthropic's Sonnet and 100x cheaper than OpenAI's Fable (GPT-4).

Joe Weissenthal ·27:11

How to verify: Compare the per-token pricing listed on Anthropic's and OpenAI's official API pricing pages against the DeepSeek pricing cited. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Reflection AI's flagship model was released about a month prior to the episode and was 6-9 months behind the frontier.

Unidentified speaker (UNKNOWN) ·31:13

How to verify: Check the release date of Reflection AI's flagship model and compare it to the release dates of leading frontier models (e.g., GPT-4, Claude) using tech news archives. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Reflection AI trained its model on a cluster of 12,000 to 14,000 Blackwell GPUs.

Unidentified speaker (UNKNOWN) ·31:25

How to verify: Check technical reports or press releases from Reflection AI or NVIDIA detailing the compute resources used for training their models. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Chinese labs are publishing novel techniques like sparse models and inference frameworks that are being adopted by US companies.

Unidentified speaker (speaker_1) ·32:40

How to verify: Search for academic papers or technical blogs from Chinese AI labs (e.g., DeepSeek) on sparse models and inference optimization, and check adoption announcements from Fireworks and Modal. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Harvey has a fine-tuned model called 'Tenant'.

Joe Weissenthal ·38:30

How to verify: Check Harvey's official website, documentation, or press releases for details on their proprietary 'Tenant' model. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Astra VI was able to manipulate its own weights to improve math performance, as reported by MIT research.

Joe Weissenthal ·48:31

How to verify: Locate and review the specific MIT research report or paper cited, verifying the methodology and results regarding Astra VI's weight manipulation capabilities. For recent events, verify 2026-08-20 through 2026-11-20. Use another period only when the speaker explicitly names it.

Do your own research

  • Verify the specific fine-tuning costs cited ($90 vs $6M) by checking public benchmarks or vendor pricing sheets from Fireworks AI or Modal.
  • Research Micro One's business model and client list to assess the validity of the $250k-$500k dataset pricing claims.
  • Investigate the 'Astra VI' self-improvement research mentioned (MIT report) to understand the technical feasibility and current limitations of model weight manipulation.
  • Check NVIDIA's acquisition status of Reflection AI and the performance metrics of their flagship model relative to US frontier models.
  • Analyze the 'one GPU per knowledge worker' heuristic against current GPU rental market data and enterprise adoption rates.
  • Review the cybersecurity 'obliteration' risks by examining public repositories for open-weight models with removed safety guardrails.
  • Compare the 'cost per completed task' metrics of Minerva's agents against traditional API-based solutions for specific marketing workflows.
Full transcriptexpand

0:00Joe Weissenthal: Finance is increasingly being built on digital rails. But as the adoption of virtual assets, tokenization, and stable coins grows, can regulation not only keep pace, but help accelerate it. Find out more later in the podcast.

0:16Unidentified speaker (UNKNOWN): Oh,

0:16Unidentified speaker (speaker_1): group shot. I'll take it. No, we're good. There's one in every group: the notographer. Best intentions, worst shots. But there's hope. The Samsung Galaxy S26. Ultra on Bell. Super steady videos and perfectly framed shots with my fan cam. And nitography for stellar picks even in low light.

0:34Joe Weissenthal: I want that. Yep.

0:35Unidentified speaker (speaker_1): Postworthy with just a click. All on Bell. Canada's fastest 5G plus network. Visit a Bell store or bell.ca slash Samsung for details. Bell. Connection is everything.

0:47Tracy Alloway: Bloomberg Audio Studios.

0:50Unidentified speaker (speaker_2): Podcasts. Podcast Radio News.

1:03Tracy Alloway: Hello and welcome to another episode of the All Thoughts Podcast. I'm Tracy Alloway.

1:08Joe Weissenthal: And I'm Joe Weissenthal.

1:09Tracy Alloway: Joe, there are still a lot of questions about the business model of AI. Yeah.

1:14Joe Weissenthal: And

1:14Tracy Alloway: we're not going to deal with all of them in this particular episode. But one of the biggest questions, and I think it's been on people's minds basically ever since the DeepSeek scare back in January 2025, is this idea of open source versus some of the closed frontier models.

1:33Joe Weissenthal: Absolutely right. So one of the questions I suppose, another way to frame the conversation, will frontier or new near frontier intelligence be commodified, right? And then you can't charge any high premiums on it, and then you get on top of these problems. This was the core anxiety of the deep seat moment. So it's like if you're selling chips, then that's great. They run any model. But for the big American closed source companies, specifically Anthropic and OpenAI, there's some big questions right there.

1:59Tracy Alloway: Right. You have to justify the expense of all this CapEx investment. and all of that. And you want to charge that premium

2:05Unidentified speaker (UNKNOWN): for

2:06Tracy Alloway: a premium model. And in the meantime, we've seen a lot of Chinese open source models in particular that seem to be getting pretty close

2:13Unidentified speaker (UNKNOWN): to the

2:13Tracy Alloway: frontier on some of their benchmark exercises. A lot of people will say they're distilling the Western models and cheating, basically, but they're still getting there.

2:22Joe Weissenthal: Or the other accusation is that they're benchmaxed. So that basically they're really good at taking a specific test, as opposed to then being as good, you know, then you put them in general deployment. There's a lot of accusations about really are they as good? Are they six months behind? Are they a year behind? Also, are they actually being trained on NVIDIA chips that are somewhere in Thailand or something? In which case or in Malaysia, in which case maybe the near frontier capabilities. aren't as shocking. Very many, a lot of questions about this though.

2:55Tracy Alloway: So I thought for this particular episode, one interesting thing to do would be maybe talk to someone who's in the business space. And by that I mean someone who has an AI company or is using AI

3:07Unidentified speaker (UNKNOWN): for their

3:07Tracy Alloway: company, a practitioner Yeah. of AI models, and talk to them about how they're thinking about open versus frontier.

3:15Joe Weissenthal: You really saw this after the hugging face stuff. And you saw this from the startup community specifically, which was this like, sure, maybe there are some safety issues related to cybersecurity, whatever, but don't let this be used as a pretext to clamp down on open source. And this sort of like, you know, the deeper layer reading is like if you're a startup and you're building on a AI in some way, you like price competition. You like downward price pressure. And so, you know, it's very interesting to hear like from the companies who are using AI to build, like how much value can you get from like a slightly a cheaper Chinese model? Yeah. Or when is that premium for true frontier capability really worth it? Exactly.

4:01Tracy Alloway: So we do in fact have the perfect guest. We're gonna be speaking with Daniel Saidi. He is the co-founder over at Minerva. So, Daniel, thank you so much for coming on All Thoughts.

4:09Unidentified speaker (speaker_1): Thank you so much for having me.

4:11Tracy Alloway: You were at Bridgewater before Minerva, right?

4:13Unidentified speaker (speaker_1): That is right. I was at Bridgewater for just about five and a half years.

4:16Joe Weissenthal: Wow.

4:17Unidentified speaker (UNKNOWN): What did you do

4:18Joe Weissenthal: there?

4:18Unidentified speaker (speaker_1): I did a mix of things. Uh at first I was actually in sales, so was helping raise money for Pure Alpha and other hedge funds. And then the last two and a half years, I was helping uh oversee our trading of financial, so banks, REITs, and insurers. And

4:32Joe Weissenthal: then started a company called Minerva after that.

4:34Unidentified speaker (UNKNOWN): What

4:34Tracy Alloway: does Minerva actually do?

4:36Joe Weissenthal: So we are an AI company focused on marketing. So we use models to build agents to automate everything from pre-campaign planning all the way down to execution. And what that practically means is you're a marketer, you have a bunch of repetitive. workflows to actually go from I want to sell XYZ thing on Facebook to somebody. We automate everything that's required to actually get that

4:59Unidentified speaker (speaker_1): campaign off the ground and monitor it in real time and let people focus on things they actually care about, like telling stories and coming up with slogans. So we use both open source and closed source models uh in that process. And um yeah, happy to dive into how we actually do that.

5:13Joe Weissenthal: This is perfect. And yeah, we've never as Tracy mentioned, we haven't had this version of the AI conversation before. But I think it's really important because there are a lot of, no offense, there are a lot of you guys, right? There are a lot of companies built no, you know what I mean? Yeah. Building some looking at some space in the economy and building some harness, some agent or whatever that can take like foundation models, whether it's the latest from OpenAI. and Anthropic or the Chinese competitors, and either use them as the intelligence that powers the thing, or you use them to like build out your product as coding. So like you're in the market for it.

5:56Unidentified speaker (speaker_1): Yeah. And that's exactly how we're thinking about it is,

5:58Unidentified speaker (UNKNOWN): you

5:58Unidentified speaker (speaker_1): know, I I think as companies have gone from software to enable people to do workflows to AI, which is okay, I'm just going to do the whole workflow itself for the person. You're looking at it from okay, how much does it cost to actually build the thing, like you said, in which case, you know, we're still using the frontier because you want to save as much time, you want to save as much money. You honestly don't want your developers to be spending time dealing with with loops and with mistakes. So we're using Claude and we're using Astra and we're using the rest of kind of the frontier while we're actually building the product. And then when it comes down to something that's repeatable, where okay, I just need to get it to a level where customer is happy with the task being completed, you get into a different set of economics where you care about costs per completed task, which is a lot different than you know, looking in the media where everyone cares about cost per million tokens. It's purely an input-output thing. For us, it's all about, hey, a marketer has these sets of tasks that they're trying to do. That's either, you know, creating an audience. It could be creating a set of taglines for an advertisement. And so for us, it's how do I make sure that I get the minimum acceptable task completion for the lowest amount of cost? Because everything outside of that is our margin.

7:13Joe Weissenthal: By the way, I hope our producers aren't listening to this. So I hope

7:17Unidentified speaker (UNKNOWN): one

7:18Tracy Alloway: of them is sat right there in the corner. So

7:20Joe Weissenthal: I hope our producers aren't listening to this. And hopefully they're just like gonna tune out for a moment. But anyway, yesterday I went on uh T VPN and they sent me the raw audio the raw video file. And the Google Drive had Gemini built into it, which I didn't realize. And so I just asked, again, close your ears. I said, What was the most interesting section of this conversation? And it was like from 7 Eleven to 840. And then uh I asked Chat GPT, I was like, can you just like clip that out and download a little version? It worked really well. And it was like, you know, they sort of like basic like tasks like works pretty well.

7:53Tracy Alloway: How many tokens did you use?

7:54Joe Weissenthal: I didn't I probably not. It was very quick.

7:57Tracy Alloway: Oh, can I ask a really basic question?

7:58Unidentified speaker (UNKNOWN): And now we're

7:59Joe Weissenthal: but I only say that because we're talking about marketing and these are the types of things that market marketers might want to automate and stuff like that. Anyway.

8:05Tracy Alloway: A very basic question. I said you use open source models in the beginning, but maybe that's not accurate. Maybe you use open weight. Can you talk about what the difference actually is between open source versus open weight?

8:16Unidentified speaker (UNKNOWN): Yeah.

8:17Unidentified speaker (speaker_1): So I think the

8:17Unidentified speaker (UNKNOWN): two are are really used synonymously right now to mean the same thing and they're

8:21Unidentified speaker (speaker_1): not the same thing. The main

8:23Unidentified speaker (UNKNOWN): difference is

8:24Unidentified speaker (speaker_1): open source means that

8:25Unidentified speaker (UNKNOWN): you're disclosing all of the

8:27Unidentified speaker (speaker_1): code, all

8:28Unidentified speaker (UNKNOWN): the data, the

8:29Unidentified speaker (speaker_1): exact

8:29Unidentified speaker (UNKNOWN): recipe that was used to train a model. And

8:32Unidentified speaker (speaker_1): that's not what the Chinese labs are doing. What

8:34Unidentified speaker (UNKNOWN): they're doing is they're releasing the weights of

8:35Unidentified speaker (speaker_1): the model. So essentially the end artifact. And now that's much different than OpenAI and Anthropic, where I send in data to an API endpoint, I get data back. I have no idea how I got the data there. The Chinese labs are at least saying, okay, you can download this model. and you can now host it yourself, essentially. And that opens up a whole different set of possibilities. But at the same time, other than the papers they release, I have no idea how they actually went from, you know, slop of data and an architecture to an actual artifact.

9:04Joe Weissenthal: But wait, sorry, just to be clear. So if you have an open weight model, you can run if you have a bunch of chips and you have access to power, you can run it. Right. So it's like I could like download. the Qwen weights and run it on my own computer or run a lot of get do. But also another element that I think a lot of startups care about and other companies, including your former employer, Bridgewater, is the ability to fine tune open weights on that. Can you talk a little bit about? I am personally still unclear whether fine fine-tuning tuning for local data is actually going to be economically valuable, like some people claim. But where do you stand on that?

9:43Unidentified speaker (speaker_1): I actually think that this is the primary benefit to the application layer right now is the ability to fine-tune on top of those open weights.

9:51Joe Weissenthal: Fine-tune the weights to your

9:53Unidentified speaker (speaker_1): That's correct. The reason that this creates the value and kind of the advantage for companies like ourselves is if you think about what actually makes these models. good in the first place, is that they're trained on these hosts of data that the labs are buying. And us as startups, we might have access to some amount of data from our customers. But the truth is we don't have these like billions of dollars of data that that the labs have. So we have to get crafty and we have to basically say, okay, what is a way that we can start to compete with the labs or compete with technology that's made by the labs

10:25Unidentified speaker (UNKNOWN): and still get the same performance. And that's where fine-tuning comes into play because it basically says, okay, look, I have a certain amount of data that's either processed via my company, or in our case, it's our customers actually allow us to train on top of their data. And what that basically allows us to do is take a model that's trained to be generalized and allow it to be specialized in a very specific task. So we do this for everything from a user might come onto our application. and say, I want to find everyone who works

10:52Joe Weissenthal: at OpenAI and likes football and

10:54Unidentified speaker (UNKNOWN): build

10:54Joe Weissenthal: me an

10:55Unidentified speaker (UNKNOWN): audience of

10:55Joe Weissenthal: those people.

10:56Unidentified speaker (UNKNOWN): And now what we used

10:57Joe Weissenthal: to do

10:58Unidentified speaker (UNKNOWN): before was you would use an

10:59Joe Weissenthal: open

10:59Unidentified speaker (UNKNOWN): AI model to interpret that request,

11:01Joe Weissenthal: turn

11:02Unidentified speaker (UNKNOWN): it into SQL, pull it from the

11:03Joe Weissenthal: database, show

11:04Unidentified speaker (UNKNOWN): it to the user. It was slow. We'd have to use a very expensive model.

11:09Joe Weissenthal: And

11:09Unidentified speaker (UNKNOWN): so if we just

11:09Joe Weissenthal: use Qwen

11:10Unidentified speaker (UNKNOWN): out of the box, it would do roughly as good as the OpenAI model, but not nearly. as well as, let's say, like Astra or

11:17Joe Weissenthal: Seoul, but then you're paying

11:18Unidentified speaker (UNKNOWN): for the fact that it's not that fast. And

11:20Unidentified speaker (speaker_1): so what

11:20Unidentified speaker (UNKNOWN): we did is we said,

11:21Unidentified speaker (speaker_1): okay, we have a few

11:22Unidentified speaker (UNKNOWN): thousand of these labeled examples where you have a very complicated query and then you have

11:27Unidentified speaker (speaker_1): a very,

11:27Unidentified speaker (UNKNOWN): you know,

11:28Unidentified speaker (speaker_1): right

11:28Unidentified speaker (UNKNOWN): result, you grade those, and then

11:30Unidentified speaker (speaker_1): you fine-tune

11:31Unidentified speaker (UNKNOWN): the

11:31Unidentified speaker (speaker_1): Qwen model.

11:31Unidentified speaker (UNKNOWN): And essentially what you're doing is you're telling

11:33Unidentified speaker (speaker_1): the model, hey, focus extensively on this set of results, try to reprove. reproduce

11:39Unidentified speaker (UNKNOWN): them

11:39Unidentified speaker (speaker_1): as well as you can. And now we have a model that's faster than the closed source models

11:44Unidentified speaker (UNKNOWN): for our use case, is

11:45Unidentified speaker (speaker_1): better

11:45Unidentified speaker (UNKNOWN): as

11:46Unidentified speaker (speaker_1): in the sense

11:46Unidentified speaker (UNKNOWN): that it

11:47Unidentified speaker (speaker_1): takes complicated queries and actually produces the results that our customers want. And now we have what we call like a very small advantage now. And now, so that's what we do in our case.

11:57Unidentified speaker (UNKNOWN): If

11:57Unidentified speaker (speaker_1): you think about some of the larger application layer companies like Harvey, for example, or Cognition, these guys are taking. all of the data that they have that makes them, you

12:05Unidentified speaker (UNKNOWN): know,

12:05Unidentified speaker (speaker_1): differentiated and their distribution mode. And now they're saying, okay, I can approach or even do better on the tasks that my customers care about by using our data and

12:14Joe Weissenthal: fine-tuning Kimmy in this case for both of them. And now, okay, they're able to have better results at cheaper costs. Now you have an advantage that you can start to charge a margin for. And that's really the the benefit of the open. weight models for for American companies.

12:29Tracy Alloway: Just to make the benefits crystal clear, can you sort of order it sounds like you're describing a bunch of advantages or potential advantages. So you have control over your own data. I guess you have some technical flexibility. Maybe you have some additional protection against like a big vendor deciding that they suddenly want to charge you a bunch more for tokens. that sort of thing. Like, can you prioritize those different and and cost is an obvious one as well? Prioritize those for us in order of importance.

13:00Joe Weissenthal: I think the most important thing is cost. And what you have to do for as a business is essentially do the calculation of I have to pay some upfront costs and then some

13:08Unidentified speaker (speaker_1): operating costs. And is that cheaper than paying OpenAI and Anthropic? And in many cases it's true. And In some cases, it's not true. And that's that's priority number one. Number two, I think, is the sovereignty and control issue. And you you see CARP and Satya talking a lot about this. I imagine this is going to be a huge issue in twenty seven, as companies realize they're they're sending their data out to anthropic and OpenAI, and they're not getting anything in return for it. They're just getting simply access to the models.

13:35Joe Weissenthal: Those companies do claim pretty seriously. that like there is I know I but I don't I haven't looked in on this a little while. I know there was some controversy with the fable and data retention. They do claim quite strongly that they don't train on corporate client data. Yeah.

13:54Unidentified speaker (UNKNOWN): And they're

13:54Joe Weissenthal: not so it's like, okay, because then no one, like, you know, if you're a hedge fund or whatever, you're uploading and stuff and like if they were training on it, they it never

14:01Unidentified speaker (UNKNOWN): they do

14:01Joe Weissenthal: claim that, right? Yeah.

14:02Unidentified speaker (speaker_1): and I and I think other than that fade.

14:04Unidentified speaker (UNKNOWN): But there's no

14:05Joe Weissenthal: evidence that like that's been violated.

14:07Unidentified speaker (speaker_1): Yeah, I don't want to be on this podcast saying there's any evidence that's been violated. I think the the difference that I'm highlighting though, that Carp talks a lot about is you give all this data, you get your answers back. The model doesn't get any better based on

14:19Unidentified speaker (UNKNOWN): your

14:19Unidentified speaker (speaker_1): use case.

14:19Joe Weissenthal: I see.

14:20Unidentified speaker (speaker_1): And so what sovereignty gives you is the ability to say, okay, if I'm running Kimmy a bunch of times on my data, maybe I'll I should be training it on my data as well so that it gets better. And then I think third is the competition, right? And that you're no longer beholden to a single vendor.

14:35Joe Weissenthal: If even

14:36Unidentified speaker (speaker_1): in the case where you're Harvey and you might not even be using tenant as a model to your customers. But the fact that you can go to Anthropic and OpenAI and say, Hey, I have this homegrown model, you have to now engage in value-based pricing for me because it's no longer necessary. no automation or automation. It's very good automation or potentially

14:55Unidentified speaker (UNKNOWN): very,

14:56Unidentified speaker (speaker_1): very, very

14:56Unidentified speaker (UNKNOWN): good

14:57Unidentified speaker (speaker_1): automation

14:57Joe Weissenthal: in the case of the frontier model. So it gives the the leverage back to the smaller companies in a way that we didn't have nearly part of the AI boom.

15:21Unidentified speaker (speaker_1): What's next for virtual assets? I'm Jacob Grease, host of Economy Scale, a series exploring growth in key global sectors. And in this episode, we're diving into the digital forces shaping finance. Sean McHugh from Dubai-based virtual assets registry authority has this to say. We're seeing a lot of institutional players. come in, digital asset dedicated hedge funds, high frequency traders, and that sort of thing, and folks who service institution clients. As more capital enters the market, adaptive regulation is likely to become increasingly key. Reese Merrick of Ripple says a lot of regulators globally would like to keep the industry at arm's length. That's the opposite here. I think also what you'll find is that a regulation and framework. frameworks can come out, but some of them are maybe a slight iteration on some 80-year-old policy and not actually in line with what the technology can do and offer. And again, that's very different in Dubai. Listen to the full episode wherever you get your podcasts. This series is brought to you by Bloomberg Media Studios and the Dubai

16:19Joe Weissenthal: Department of Economy and Tourism.

16:22Unidentified speaker (UNKNOWN): Oh,

16:23Unidentified speaker (speaker_1): Gripshot! group shot.

16:24Unidentified speaker (UNKNOWN): I'll

16:24Joe Weissenthal: take it.

16:25Unidentified speaker (speaker_1): No, we're good. There's one. in every group: the notographer. Best intentions, worst shots. But there's hope. The Samsung Galaxy S26 Ultra on Bell. Super steady videos and perfectly framed shots with my fan cam. And nitography for stellar picks even in low light.

16:40Joe Weissenthal: I want that. Yep,

16:41Unidentified speaker (speaker_1): postworthy with just a click. All on Bell, Canada's fastest 5G Just plus network. Visit a Bell store or bell.ca slash Samsung for details. Bell, connection is everyone. everything.

16:52Tracy Alloway: Once you fine-tune a model, how like durable is it? And to what extent can it carry over? So, you know, Kimmy releases like, I don't even know what version we're on anymore, but they release a

17:04Unidentified speaker (UNKNOWN): new

17:04Tracy Alloway: version of the model. And do you need to like upgrade your existing one?

17:09Unidentified speaker (UNKNOWN): Or

17:09Tracy Alloway: how does that work?

17:10Joe Weissenthal: Yeah, I think that's the the biggest decision point right now for for founders and and for companies like myself is

17:17Unidentified speaker (UNKNOWN): that

17:17Joe Weissenthal: is part of the ongoing operational cost of fine-tuning a model. It doesn't simply transfer over. I can't say, okay, listen, I I fine-tuned Kimmy three, Kimmy 3.1 weights are coming out this week. Actually, let me just transfer it over. And so there are folks doing research in that space. I am much more of an applied than a than an AI researcher. So I can't really give an update on on how that's actually going. But from talking to other founders, this is this is really the the the problem is that okay, I'm gonna have to I can use the same data set. So that part is transferable. If I build a really reliable, durable set of answers that are expertly graded, I can at least find two it but to get to your durability point, I have no idea how they actually got to those weights. So whether or not it's going to converge to an optimal set of weights with the same training data set, I don't know. And so that's part of the uncertainty and the risk that you have to put

18:08Unidentified speaker (speaker_1): into your calculation as you're making this decision.

18:10Joe Weissenthal: So one of the things that people look at they say, look, the open source models are cheaper. First of all, when we hear open source, especially prior to the AI or era, we thought like kind of free, right? You just like download the software. And there's no such thing as free AI, period. I mean, there is kind of like with the Google Gemini, but by and large, like

18:29Unidentified speaker (UNKNOWN): somebody's

18:30Unidentified speaker (speaker_1): paying for it.

18:30Joe Weissenthal: Someone you have to pay for it, and it's a service, and there's compute and electricity that's substantial with every use of it, et cetera. Can you talk a little bit about like what pricing really looks like once you fine-tuned a model, and then you have to find you still have to find electricity or you have to find a data center or find some service? like a modal. I don't know who you use or whatever. But like talk to us about you have to then be the host, right? You have to find all that. What do the economics look like there?

18:59Unidentified speaker (speaker_1): Yeah, I think the biggest thing that's changed in the last year from when DeepSeek came out

19:02Unidentified speaker (UNKNOWN): to now is you have companies like Modal where we use fireworks, very simple.

19:07Joe Weissenthal: So on their billboards and SF.

19:09Unidentified speaker (UNKNOWN): Yeah. Very similar concept in that if we a year ago had read the DeepSeek paper and were like, okay, we want to find fine-tune our own model and we want to host it, there's so much that we don't think about every day that we would have to think about. Like, okay, I have this different model. I need to make sure that it's on enough concurrent GPUs. I need to make sure that the actual software that GPU is running on, whether that's SG Lang or VLLM, can actually support the amount of concurrency that I need to actually

19:35Unidentified speaker (speaker_1): serve this.

19:36Unidentified speaker (UNKNOWN): So you

19:36Unidentified speaker (speaker_1): actually would need a team of inference

19:38Joe Weissenthal: engineers to be able

19:39Unidentified speaker (speaker_1): to do this in the past. And so that's

19:40Unidentified speaker (UNKNOWN): a a big

19:41Unidentified speaker (speaker_1): reason why you hadn't even seen anyone really attempt this, even though Llama has been open source for two years

19:46Unidentified speaker (UNKNOWN): now. It

19:47Unidentified speaker (speaker_1): only been restricted to really like research examples and

19:50Joe Weissenthal: and large

19:50Unidentified speaker (speaker_1): companies like Uber that that had even attempted this before.

19:53Unidentified speaker (UNKNOWN): And

19:53Unidentified speaker (speaker_1): so now what you can do is you can go to a company like Fireworks

19:56Unidentified speaker (UNKNOWN): or thinking machines or modal like you said and you can say, okay, hey, I really want to fine fine-tune tune Qwen.

20:03Joe Weissenthal: Yeah. Here's

20:03Unidentified speaker (UNKNOWN): my example kind of training data set that I have. In our case, it would be the audiences that marketers want and the results that the model is supposed to output. And it's a very simplified process

20:14Joe Weissenthal: now that's become

20:15Unidentified speaker (UNKNOWN): quite commodified where I can pay, you know, in the case of fireworks, it's I'm going to pay around seven to nine dollars per per million input and million output tokens, where the output tokens in this case would be the actual changes to the model

20:28Joe Weissenthal: weights.

20:29Unidentified speaker (UNKNOWN): And there are people out there who have have fine-tuned models that perform really well for as low as $90 on their final training run. So the economics have collapsed. I think the most notable economics I've heard that are comparable, I don't want to release any of our own

20:42Joe Weissenthal: numbers right now, is

20:43Unidentified speaker (UNKNOWN): that Harvey on their tenant model spent about six million dollars in compute to get to the absolute frontier of of legal tasks.

20:51Unidentified speaker (speaker_1): You compare that to you know how much it costs probably to run the new Fable model and and we're off by you know two, three orders of magnitude.

20:59Joe Weissenthal: It's fascinating. I for what it's worth, the only reason I said modal is because I have fine-tuned a model on modal. Like that's like I've but it wasn't an LLM, it was uh BERT, which is an open source classifier from Google that was came out in like 2019. Yeah. But like I've done the whole process and I know, like, I have no to your point about like I don't know where the difference is between like 2026, 2025. It's just so easy.

21:25Unidentified speaker (UNKNOWN): Yeah.

21:26Joe Weissenthal: Like just someone like, and I imagine if you had a little someone I have no technical expertise, but I imagine if you have someone with a little bit of tech technical expertise, it must be extraordinary how easy it is for and what they can could do. Yeah.

21:37Unidentified speaker (UNKNOWN): But

21:37Joe Weissenthal: it's just like I don't know anything. And I like fine-tuned a model. It's crazy. Yeah.

21:41Tracy Alloway: I think we're the technical

21:42Unidentified speaker (UNKNOWN): expertise.

21:43Tracy Alloway: It's cheap to actually do the training, which is what

21:45Unidentified speaker (UNKNOWN): you tend to see in the AI industry as a whole over the past three years. Just anything that's related to the actual chips, to the actual power, you start to see lots of players emerge, and then you start to see competition. And then you start to see it not necessarily become commodified in the sense that the margins have gone down to zero. because fireworks and modal make quite a boatload of money actually in making it easy for folks. Both of them are sitting at a billion dollars in ARR, and I imagine their unit economics are actually quite good. But the

22:13Joe Weissenthal: fact that I can go to anyone and actually do it in in in the way that you just described, the fact that you're able to fine-tune Bert is a testament to what's going on in the space in that things that can be commodified are being commodified. The things that aren't commodified yet,

22:26Unidentified speaker (UNKNOWN): and

22:26Joe Weissenthal: what actually separates a good fine tune from a bad fine tune is the data and the actual evals that are put in there. And that's why you have these companies like Mercore and

22:36Unidentified speaker (UNKNOWN): and

22:36Joe Weissenthal: and Surge. And then the reason that the labs are paying them billions of dollars is the the alpha per se is as you would say is in the data right now. And it's structuring those things well. And so that's a real decision point for a company like ours is do you have enough good validated data where the model can actually learn from it, or are you just going to get noise?

22:55Tracy Alloway: Talk to us about acquiring data nowadays. How difficult it is to everyone always talks about, you know, original data sets and things like that. How difficult are those to find?

23:05Unidentified speaker (UNKNOWN): Oh,

23:06Joe Weissenthal: so I think in our use case, it's it's we've tried to go into an industry where the data is quite plentiful and marketing, and that, you know, you have data for that. you know, companies are sitting on call logs, they're sitting on advertising data, they're sitting on ads and the actual creative process that was used to make those ads. Uh you have data on you know website activity and where are your users actually spending time and all of that can kind of be constructed into what you know the market is now calling an RL environment, which you an RL environment is simply you can think of it as like a simulation where you're trying to replicate the environment in which an AI is actually doing a task. So in our case, it's the actual act of acquiring a customer. And so the simulation that you're trying to create is I show an ad to an AI, does it actually get the AI to click on the link to go to the website to actually buy the thing? And that's what your data set's supposed to represent. And so for us, it's actually quite easy easy 'cause because it's quite plentiful, and folks have been collecting data in the in marketing space for for a long time because uh, you know, we had all of those AI optimized kind of a uh algorithms back in the you know programmatic days 15, 20 years ago and ever since companies have done a good job of keeping data there. For more specialized hacks from from folks I talk to, let's take like construction for example, it's like, man, it's it's really, really hard to find good data sets of okay, I have the

24:25blueprints for this building. Yeah. But what are all the comments between the architects and the general contractor and the actual client that led to the changes in the blueprint? And so that's what the AI actually needs to do to generalize, to do these tasks. It doesn't need results. You know how your teachers used to tell you, I don't care if you got the right answer. I need you to show your work. That's what we need for the AI. And so it's really hard to find that data. I think as a as a way to see what the labs care about, you can go to these companies' websites and see what they're paying experts in the space. So if you go to the micro one website, you'll see which one? Uh it's called Micro One. Okay, I'll let you check it out. Yeah, Micro One

25:02Unidentified speaker (UNKNOWN): is

25:02Joe Weissenthal: the the big procure of company data right now. And so I'm I'm sure you guys are on Twitter and you saw it. Yeah. You saw a few weeks ago that everyone was talking about uh you know data brokering and I'm making a hundred million dollars a year selling data to the lab. labs. Micro one is probably the most legitimate company in the in the company data space. And so for a while, they're spending anywhere from two

25:23Unidentified speaker (speaker_1): hundred fifty to five hundred K per complete data set from

25:26Unidentified speaker (UNKNOWN): a

25:27Unidentified speaker (speaker_1): company. And they were paying people fifty to a hundred thousand dollars just for referring the companies to them. So that kind of tells you how in demand good company data is.

25:37Unidentified speaker (UNKNOWN): Yeah.

25:37Joe Weissenthal: I mean, also there was that insurance. extra, you know, Google bought Spirit Airlines for $10 million. And

25:42Unidentified speaker (UNKNOWN): then you

25:42Joe Weissenthal: just get this treasure trove of corporate emails and calendar data.

25:46Tracy Alloway: The only thing worth saving from Spirit

25:48Unidentified speaker (UNKNOWN): was the

25:49Tracy Alloway: data.

25:49Unidentified speaker (UNKNOWN): It's

25:49Joe Weissenthal: a little bit disturbing. You gotta wonder um how many zombie firms are out there that would be worth more if they just liquidated and sold all of their emails and corporate data to a lab. Who know? kn I mean, I really think

26:02Unidentified speaker (UNKNOWN): how

26:02Tracy Alloway: much can we get for Otlots

26:03Joe Weissenthal: transcripts? Priceless. Priceless.

26:07Tracy Alloway: So one of the concerns that you sometimes hear with open source models is cybersecurity, the idea that, like, okay, well, maybe you don't know the exact weights, but people can like look at the source code or I I don't really understand it. But there's a a vulnerability there that you don't necessarily find with the frontier models. How are you thinking about that?

26:28Joe Weissenthal: Yeah, I I in preparation for this episode, one of the things that I did was actually play around with the newest DeepSeek model in its uncensored form. And and that's really the biggest risk, I think, personally, to having open weight frontier models out in the wild. And that basically what somebody did to the

26:44Unidentified speaker (UNKNOWN): deep

26:45Joe Weissenthal: seek

26:45Unidentified speaker (UNKNOWN): model that was released is they turned off the neurons that tell the model to say no. And so

26:51Unidentified speaker (speaker_1): Oh,

26:51Tracy Alloway: is

26:52Unidentified speaker (UNKNOWN): this what they call obliteration? Yeah,

26:54Unidentified speaker (speaker_1): this is what's called an obliterated

26:55Unidentified speaker (UNKNOWN): model.

26:56Tracy Alloway: Such a good word.

26:57Unidentified speaker (UNKNOWN): It's a great

26:57Joe Weissenthal: word. It's

26:57Unidentified speaker (UNKNOWN): a great word.

26:59Joe Weissenthal: I want an obliterated model. I

27:01Unidentified speaker (speaker_1): mean, it's

27:01Unidentified speaker (UNKNOWN): pretty crazy playing around with it in that, you know, the model's probably not as capable as Mythos back in

27:07Unidentified speaker (speaker_1): February

27:08Unidentified speaker (UNKNOWN): of

27:08Joe Weissenthal: this year,

27:09Unidentified speaker (UNKNOWN): but it's up there and

27:10Unidentified speaker (speaker_1): it's

27:10Unidentified speaker (UNKNOWN): really

27:11Joe Weissenthal: cheap to run. It's about 30 cents per million blended. So in input output tokens. So it's about, you know, 10x cheaper than Sonnet, which is, you know, about 10x cheaper than Fable. So this thing's like one one hundredth of the cost of Fable to run. And you could tell it to do anything. And so if you see all these hacks that are going on right now,

27:30Unidentified speaker (UNKNOWN): the

27:30Joe Weissenthal: the theory on the internet is people are running swarms of these DeepSeek obliterated models. And that's really the risk that the labs have talked about, which is hey, there's actually like no risk that Chinese getting your data from this. Let's just quell that kind of uh notion now. The the risk that they're concerned about is not data going to the labs. It's not, you know, vulnerabilities from the actual models. It's the fact that I can just turn off a few parameters in this

27:55Tracy Alloway: model.

27:55Joe Weissenthal: And now suddenly I have this PhD level, you know, Elliot Alderson. If you watched watch Mr. Robot, I have a thousand. of these Mr. Robots now running in the cloud and I can tell them to hack anything. And that's the real risk and what's scary. And we're seeing it in real time. And so I don't know what the right policy is to have here, whether that's, you know, you have to KYC before you can download the weights, but we have to reach some sort of medium where the kind of dynamism of the US startup community is preserved and that companies like ours can still download the weights and fine tune like we talked about. But at the same time, you don't have hacker and terrorist groups being able to just download a deep

28:33Unidentified speaker (speaker_1): seek model, you know, make two changes to it, and now suddenly they have this like unadulterated, obliterated

28:40Joe Weissenthal: Wait, you don't have that? Or do you say you do or you don't have that?

28:43Unidentified speaker (UNKNOWN): I

28:43Unidentified speaker (speaker_1): I'm saying you have to you have to create a world where

28:45Unidentified speaker (UNKNOWN): yeah

28:45Unidentified speaker (speaker_1): where people can't you can't have the terrorists just download. downloading.

28:48Unidentified speaker (UNKNOWN): No,

28:48Joe Weissenthal: but and this seems like a pretty serious issue. Right? Yeah. Like

28:50Unidentified speaker (UNKNOWN): these

28:51Joe Weissenthal: are really powerful tools slash weapons. And this is part of what makes AI very tricky, is its dual use, right? Because in theory, maybe it's gonna be, but like these are weapons when used for offensive hacking purposes, totally, et cetera. And you can obliterate them and you can get them to do whatever you want. Yep. And

29:10Unidentified speaker (UNKNOWN): that

29:10Joe Weissenthal: seems not good.

29:11Unidentified speaker (speaker_1): No, I I

29:12Unidentified speaker (UNKNOWN): agree

29:12Joe Weissenthal: with you. It

29:13Unidentified speaker (speaker_1): seems not good. I just have yet to see like a really compelling proposal for how

29:18Joe Weissenthal: deal with it.

29:19Unidentified speaker (speaker_1): Yeah, how do you deal with it? Because

29:21Unidentified speaker (UNKNOWN): I

29:21Unidentified speaker (speaker_1): want to be able to download the weight

29:22Joe Weissenthal: still. So try to be honest with us for one second. Like I understand no, I you've I appreciate your candor. No, but for be honest with us for one second. So I under I actually totally understand this point about wanting to fine-tune lagging edge. not quite frontier models, and being able to then get a model that's as performant as like what the big tech companies have, et cetera. And that also makes a lot of sense. But how much is the fact that you're able to do this subsidized by the fact

29:57Unidentified speaker (UNKNOWN): by

29:57Joe Weissenthal: the distillation question? In other words, it's like great, we take open source plus our data. and then we have something good. But if the open source couldn't even exist in the first place to be like a year or six months behind the frontier, were it not for terms of service violations? Yeah. Isn't it pretty understandable? Because the issue is like, why didn't Llama go anywhere? Presumably, one reason is they're an American company that is not going gonna to go around violating the terms of

30:26Unidentified speaker (UNKNOWN): the American American American

30:27Joe Weissenthal: service of anthropic and OpenAI, right? Yep. You're like building something out and like Shanton, brilliant engineers, I'm sure, but I don't think they care about terms of service violations. I don't think anything is gonna bad is gonna happen to them if they like find a way to do a hundred thousand, you know, and clawed queries surreptitiously.

30:47Unidentified speaker (speaker_1): No, I I I completely agree with you, and I think there's a need to basically say

30:51Unidentified speaker (UNKNOWN): how

30:51Unidentified speaker (speaker_1): much of the gains do you think are actually coming from distillation? I think we actually have a good barometer

30:56Joe Weissenthal: on

30:56Unidentified speaker (speaker_1): this, and that there's a company that used to be headquartered in the floor above me called Reflection AI. Allegedly, it's being bought by uh NVIDIA and and they're probably the preeminent US open s uh open weight uh model company right now. They released their flagship model about a month ago,

31:13Unidentified speaker (UNKNOWN): and

31:13Unidentified speaker (speaker_1): it was probably about three to six months behind the Chinese open source models, which were about at the time three months behind the publicly released Frontier. And

31:23Joe Weissenthal: what's the name of this company? It's

31:24Unidentified speaker (UNKNOWN): called

31:24Unidentified speaker (speaker_1): Reflection.

31:25Unidentified speaker (UNKNOWN): Okay.

31:25Unidentified speaker (speaker_1): And so what that basically tells you is that you know, that company has a bunch of backing from NVIDIA. They have thousands of Blackwell GPUs that they trained had this model on. I believe it was a a cluster of 12 or 14,000. And so they have access to computers. compute, they've got access to great talent. And then they released this model, and it's about like six to nine months behind the frontier. And so

31:45Unidentified speaker (UNKNOWN): what

31:45Unidentified speaker (speaker_1): that basically tells you is that distillation is not the pure reason that the Chinese labs are where they are. They have lots of evidence of coming up with very novel techniques that are actually being used by the frontier labs.

31:57Joe Weissenthal: Sorry, how is that evidence? Because I assume that an American company owned by NVIDIA is going to really play it by the book with regards to uh distillation. Yeah,

32:06Unidentified speaker (speaker_1): yeah. What I'm saying is that if the Chinese companies had played by the books, you can assume that they would probably be neck and neck with reflection, if not maybe a little bit further ahead because they have the backing of the Chinese state.

32:18Joe Weissenthal: And so

32:19Unidentified speaker (speaker_1): what I'm saying is like

32:19Unidentified speaker (UNKNOWN): So

32:19Joe Weissenthal: distillation is real.

32:20Unidentified speaker (speaker_1): Distillation is is real, and there's lots of evidence for it. How much distillation is responsible? responsible for the quality of these models, I think is really up for debate. I think one thing that if you ask actual researchers in San Francisco, how do they feel about the Chinese labs? Obviously, they'll talk about distillation. They'll also talk about the like absolute prolific uh publishing nature of researchers there. Yeah. And some of the advancements, whether that's using sparse models instead of dense models. which saves a lot on compute, from DeepSeek actually just publishing their entire inference framework a few weeks ago and how that's being used by Fireworks and Modal and all these other companies to improve the efficiency of serving these chips. These guys are doing a public service on the other side. So I think the fact that the quality of their research is still being respected by the lab

33:10Joe Weissenthal: shows

33:11Unidentified speaker (speaker_1): you that,

33:11Unidentified speaker (UNKNOWN): hey,

33:12Unidentified speaker (speaker_1): you know, they're still making good models,

33:14Unidentified speaker (UNKNOWN): is

33:14Unidentified speaker (speaker_1): disappointing. distillation responsible for all of the reason why they're good models? Probably not, but it's definitely helping them out

33:19Joe Weissenthal: for sure. Uh

33:20Unidentified speaker (speaker_1): especially when you see them the similarity in performance between Kimmy and Astra, and there's big allegations there. But yeah, I I don't I wouldn't say we're subsidized by distillation to answer your

33:30Unidentified speaker (UNKNOWN): question. Oh,

33:47Unidentified speaker (speaker_1): group shot. I'll take it. No, we're good. There's one in every group, the notographer. Best intentions, worst shots. But there's hope. The Samsung Galaxy S26 26 Ultra on Bell. Super steady videos and perfectly framed shots with my fan cam and nitography for. stellar picks even in low light.

34:04Joe Weissenthal: I want that. Yep,

34:05Unidentified speaker (speaker_1): postworthy with just a click. All on Bell, Canada's fastest 5G plus network. Visit a Bell store or bell.ca slash Samsung for details. Bell. Connection is everything.

34:15Unidentified speaker (UNKNOWN): Now, Bloomberg.com subscribers

34:18Unidentified speaker (speaker_2): can

34:18Unidentified speaker (UNKNOWN): shape the conversation on Bloomberg Radio. We got a week weaken of smart questions. question.

34:22Tracy Alloway: Can be part of the conversation. Submit

34:24Unidentified speaker (UNKNOWN): questions for experts and guests you hear on air.

34:27Unidentified speaker (speaker_1): Visit Bloomberg.com/slash ask radio to send questions to our hosts.

34:31Unidentified speaker (UNKNOWN): We

34:31Tracy Alloway: may

34:31Unidentified speaker (UNKNOWN): just hear them asked on the

34:32Tracy Alloway: air.

34:33Unidentified speaker (UNKNOWN): Exclusively for

34:34Joe Weissenthal: Bloomberg.com subscribers.

34:35Unidentified speaker (speaker_1): Get answers on today's headline, breaking earnings news, and big market moves. Visit Bloomberg.com/slash ask radio to join the conversation right here on Bloomberg Radio.

34:46Tracy Alloway: What in your mind would be indicative of the Chinese models like actually? catching up with Frontier? Like, what would be, I guess,

34:54Unidentified speaker (UNKNOWN): the

34:54Tracy Alloway: next DeepSeek moment in your mind?

34:57Unidentified speaker (UNKNOWN): Because

34:58Tracy Alloway: I I assume it's not just the benchmark test, right? Because as Joe was saying in the intro, those can be gamed and they're pretty close on those anyway.

35:05Joe Weissenthal: Yeah, I mean, this is such a good question because I think it's a it's actually a much broader question than just

35:11Unidentified speaker (UNKNOWN): are

35:11Joe Weissenthal: the Chinese models catching up to the the frontier models? It's really how how do you know that there's actual progress that's being made relative to where we are right now? Because if you go and you know you spend time with Astra, you spend time with Fable, it's really smart. Uh it still makes silly mistakes, it still sounds like slop, but these models are, you know, compared to Chat GPT, you know, when it first came out, they're they're night and day in terms of their capabilities. And it's and for everyday use, even for me, it's like it's really hard for me even to tell the difference between a 5.1 and a five. And therefore, how

35:43Unidentified speaker (speaker_1): am I

35:43Joe Weissenthal: supposed to be able to tell the difference between a five and a Kimmy 3.1? So I think where we're moving towards in a world where companies care a lot more about sovereign data and they care a lot more about the the actual tasks that they're doing within their firm rather than just benchmarks, it's going to come down to companies themselves like ourselves or workers. who are who are doing tasks actually seeing results on a day-to-day basis of okay, can I just swap Kimmy into an existing process that I have in a chat bot that I'm serving to customers or in my coding workflows? And am I actually seeing similar performance? Because the benchmarks are are starting to become meaningless, as you guys said, as

36:20Unidentified speaker (UNKNOWN): as

36:20Joe Weissenthal: Joe said, the benchmaxing, bench maxing. if I see that Kimmy 3.2 has a higher terminal bench score than Astra 6.2, what does that actually mean to me? Does it mean anything? Probably not. But if I can take Kimmy 3.2 and start replacing it as my daily driver and how I'm coding, okay, now I know that things have caught up. And how do I know that it's replaced? Am I, you know, yelling expletives into

36:43Unidentified speaker (UNKNOWN): the

36:43Unidentified speaker (speaker_1): terminal anymore? If I'm not, probably not.

36:46Joe Weissenthal: Let's talk a little bit more about the economics from a company like yours. So, first of all, it occurs to me like one way you might be able to characterize what you're doing now is using frontier closed source for CapEx building thing and using cheap open source for opex. So the intelligence, right?

37:02Unidentified speaker (UNKNOWN): Is

37:02Joe Weissenthal: that a fair characterization?

37:03Unidentified speaker (speaker_1): Yep.

37:04Joe Weissenthal: Would it is it the hope, you know, token price disinflation has been massive every year? Like, is the sort of vision that there is something like, okay, let's take a task, like um, you know, A B test a bunch of different ads or something like that. It's in theory gonna look the same. Is the hope that like you will be able to sell your service at some sort of like recurring price while the token price collapses Collapse at ninety 90% percent year after year after year and like capture that margin? Yeah.

37:38Unidentified speaker (speaker_1): I mean that that's exactly how I think us and and other companies are thinking about it. It it's first, what am I charging my customer? That's the number one question, right? I think Harvey and Lagora are two great examples of this. Lagora is another legal tech company. They also are are now exploring fine-tuning their models in response to Harvey. But Harvey has a per seat cost which is fixed. And Lagora charges usage. And if you look Harvey actually went down the legal lab route first. And if you think about it for a second, it makes a lot of sense. They have fixed revenue coming

38:12Unidentified speaker (UNKNOWN): in.

38:12Unidentified speaker (speaker_1): They saw that the costs were rising because usage was going up of their product as it got better. And so naturally, when you have a fixed revenue coming in and a variable cost, you want to reduce that variable cost

38:24Unidentified speaker (UNKNOWN): as much

38:25Unidentified speaker (speaker_1): as you can. So in Harvey's case, it's let's come up with tenant. And now whether or not tenant is what's

38:30Joe Weissenthal: tenant?

38:30Unidentified speaker (speaker_1): Tenant's their legal

38:31Unidentified speaker (UNKNOWN): model.

38:31Joe Weissenthal: Oh, that's their fine-tuned

38:33Unidentified speaker (UNKNOWN): model.

38:33Joe Weissenthal: Okay.

38:34Unidentified speaker (speaker_1): And so whether or not you want to serve tenant to users or you simply want to use tenant to get better leverage from the labs, it's coming from a place of I need to increase my operating margins. And and that's exactly the question that that that Harvey's asking. It's a question that we're asking as as well. It's where are the you know highest kind of variable components of spend within our product that aren't tied to usage? And how do we then replace that with either a fine-tuned or just purely, we might even just say for some use cases, if it's a highly repetitive task, that's quite simple. We might just use a very cheap open source model. And that's another question, too. And so the economics really boil down to the equation of okay, I have. some kind of per token cost that I'm paying to the labs times the number of tasks that's required. And that's got to be more than a fixed cost up front to train the model and the operating costs of actually running that model on a on a monthly basis.

39:29Tracy Alloway: Has your token budget actually come down at all, or is it still, I guess, too early? You're still in the building phase.

39:36Unidentified speaker (speaker_1): I I would say our token budget has gone up, but that's mainly because we've just gotten more money over time. Uh when the we we started the company in 2023 and we were being, you know, very deliberate about when we were using AI back then, because also it just wasn't that good. And so we didn't all shift to agentic coding until probably when Opus 4.5 came out in October of last year. And so we ramped up and and as such we're we're still ramping as we're finding that the models get better and we want to, you know, keep the team lean. But we're we're not spending like $10,015,000, $20,000 per engineer per

40:12Tracy Alloway: no token maxing.

40:13Unidentified speaker (speaker_1): We're not token maxers, no. We we are output max or outcome maxers, I would say.

40:18Joe Weissenthal: What are since um, you know, another question, you know, we're we're actually recording this, as everyone knows, during quite a bit of volatility in the closure. quote AI trade, et cetera. One thing that hasn't gone down, and I think this was pretty important, is any measure of GPU rental prices. Yeah. It looks like so. I'm looking at the silicon data H one hundred rental tracking index. This is an old chip, it's an H one hundred. It's still going up. It's at the highest in a while. Are you what are you seeing though? You know, this is like an index and you know. whatever. What are you actually seeing in terms of compute scarcity, rental costs, things like that?

40:57Unidentified speaker (speaker_1): Yeah, I mean, I think this is more a question for for friends than it is for me. We're we're pretty light in the amount of compute that we

41:04Unidentified speaker (UNKNOWN): need to

41:04Unidentified speaker (speaker_1): fine-tune our models, and we find availability to to always be there. Actually, I think when things were really tight was two years ago, we were actually dabbling in in training a uh our own model ourselves. And

41:16Unidentified speaker (UNKNOWN): the

41:17Unidentified speaker (speaker_1): the H1, it was right around the time the H100s were

41:19Unidentified speaker (UNKNOWN): just

41:20Unidentified speaker (speaker_1): unavailable completely to all users. So I think there is Slack where you need to find it. I think there's other people that you could talk to from silicon data data

41:30Joe Weissenthal: to

41:30Unidentified speaker (UNKNOWN): kind

41:30Unidentified speaker (speaker_1): of explain the what's going on with the longer compute off takes lockups and why those are so

41:35Unidentified speaker (UNKNOWN): much cheaper than the

41:36Unidentified speaker (speaker_1): spot market. But for us, things are pretty available. I think if you go on Twitter right now, it looks like, man, there's just no GPUs anywhere. And

41:43Joe Weissenthal: well, I can't tell because and I feel like the narrative on this changes very quickly.

41:49Unidentified speaker (UNKNOWN): Where it's like

41:49Joe Weissenthal: you go, it's like, oh, there's so short compute, we're gonna need all this stuff. And then on the other hand, in the last several weeks, overbuild is like back in the thing, and you see this stuff, you know. I mean it gets real there's all kinds of complicated things, and you could theoretically have temporary overbuild, long term massive shortage. But I don't know. Like r the reason I asked was like I don't even have a sense right now, like what is the consensus narrative in terms of compute capacity.

42:17Unidentified speaker (speaker_1): I I think the thing that I always just like to keep in mind is a really good first principle heuristic I have is

42:23Joe Weissenthal: I love a first principle heuristic.

42:25Unidentified speaker (speaker_1): Is in a fully kind of like AI native employee should have about a GPU's worth of token consumption per year. And so the the place that I always go to is how many you know

42:36Joe Weissenthal: leading ed GPUs are deployed. It's it's you

42:39Unidentified speaker (UNKNOWN): know

42:39Joe Weissenthal: somewhere between five to ten million right now in the US uh of Blackwells, and how many knowledge workers are there? There's, you know, by some measures around 100 a hundred million, let's say. And so if you think that every knowledge worker needs a GPU to actually fully automate all their tasks and have an agent running at all times, then we're massively short of that need in the future. And so is everyone going to be as AI forward as, say, one of my engineers or an engineer working at Anthropic? I don't know, but roughly that's kind of the heuristic where I think things are gonna net out four years from now. It it won't be a surprise that we have kind of one B300 worth of comp computational capacity for every uh knowledge working employee in the economy. So in my mind, it's like if everyone's gonna be using AI in the way that I do it, we're still massively short compute. We're gonna have these drifts around what the baseline should be right now. And I think we feel that on Twitter because the people who are, you know, most short compute are sometimes the loudest people right

43:36Unidentified speaker (UNKNOWN): now.

43:37Joe Weissenthal: And so we hear them screaming and then sometimes you'll hear about a build out. But in the long term, we're still massively short the amount of compute we need to actually be an AI forward society and labor force.

43:46Unidentified speaker (UNKNOWN): Yeah,

43:46Tracy Alloway: I feel like if you have loads

43:47Unidentified speaker (UNKNOWN): of

43:47Tracy Alloway: compute nowadays, you keep it quiet, right? Or

43:50Unidentified speaker (UNKNOWN): maybe maybe

43:51Tracy Alloway: the temptation

43:51Unidentified speaker (UNKNOWN): is

43:52Tracy Alloway: to brag about it. I don't know.

43:53Joe Weissenthal: I mean, I think the the most telling thing about how short compute we are is allegedly when all of the uh

43:58Unidentified speaker (speaker_1): you know Elon and Jensen were were waiting to receive their medals from the White House. Apparently Greg Brockman asked Elon for compute there. So if they can kind of settle their beef by trying to do a compute trade, that probably tells you that at least the labs are still fundamentally short compute relative to their needs. All

44:14Unidentified speaker (UNKNOWN): right,

44:15Tracy Alloway: Daniel Saidi, thank you so much for coming on All Foss.

44:17Unidentified speaker (speaker_1): Thank you so much for having

44:19Unidentified speaker (UNKNOWN): me

44:31Tracy Alloway: Joe, I thought that was a really useful conversation to hear how you know someone in the business is actually deploying these models and thinking about closed versus open.

44:40Joe Weissenthal: Yeah, no, I think that was really helpful. And I think this sort of gets to show that there are a lot of dimensions that are just not that straightforward. Right. So it's like, what is actually cheap? Well, if your engineer's time is valuable and scarce, then it may not be cheap to use a lagging edge model because you know, like you just want to like so then, yeah, like pay the big anthropic premium or pay the pay uh chat GPT premium, whatever it is, for something that's just like repeatable, etc. et cetera, Like, oh, here's here is our harness that does like A B testing from two images. on an Instagram ad, then like, yeah, why do you need the frontier intelligence? And I think a lot of the big questions are gonna come down to is AI usage overall gonna be more like CapEx or Op X?

45:28Tracy Alloway: Yeah.

45:28Joe Weissenthal: And, you know, it's probably gonna it's almost obvious to

45:31Unidentified speaker (UNKNOWN): be able to do that. There's room for both. Yeah,

45:32Joe Weissenthal: yeah.

45:33Tracy Alloway: But you know, I I would place my bets on the cheaper models winning out, but I don't know. Um

45:40Joe Weissenthal: One thing we didn't get into and when you mentioned cheaper models is also that the closed source labs do have cheaper versions.

45:47Unidentified speaker (UNKNOWN): Yeah, that's true.

45:47Joe Weissenthal: Well you so but I agree, like at any level, presumably I think the Chinese open ones are cheaper.

45:54Unidentified speaker (UNKNOWN): Have you

45:54Tracy Alloway: been asking it's it's an interesting exercise right now if you can get, you know, someone at a company, a large like US company, and ask them what model. models they're actually using,

46:04Joe Weissenthal: yeah,

46:05Tracy Alloway: privately, yeah. A lot of them will talk about like Qwen and Kimmy, but not in public.

46:11Joe Weissenthal: Yeah, it's kind of interesting. Every once in a while you do see it like, but usually around like what they were able to accomplish with fine-tuning, etc. I also think there's a kind of flip side of observe the opposite, which is I think there is this sort of like posturing to the anthropics and open AIs of the world. Like, we don't need you. You know what I say? Like, yeah,

46:34Unidentified speaker (UNKNOWN): yeah, it's a good idea. We found a way to

46:36Joe Weissenthal: like get uh, you know, for example, Brian Armstrong, the CEO of Coinbase, has talked about, I forget Kimmy, or one of the ones that they use, et cetera. And it's sort of like this shot across the bow. I would bet they still have, you know, enterprise. I don't know for sure. You know, I think a lot of companies are paying an ant tax or an OpenAI tax. But um, yeah, no, there's some really interesting questions in there.

46:59Tracy Alloway: In the meantime, we have a new AI worry uh to think

47:03Unidentified speaker (UNKNOWN): about.

47:04Tracy Alloway: Obliteration.

47:05Joe Weissenthal: Oh yeah. I gotta get I've never I gotta like get my hands on an obliterated model. I gotta try out stuff. No, no, I've always been just

47:11Tracy Alloway: that sounds so evil when you say it like that. You need to be like straight. stroking a cat while you sit while while you talk about your obliterated model.

47:19Joe Weissenthal: But I've thought about this like, look, there are just certain things.

47:23Unidentified speaker (UNKNOWN): I

47:23Joe Weissenthal: don't even want to bring them up because they're all very negative. That you like you think about like how well locked down the internet has become for certain types of information that we wanna want it we really like do not want to be readily available in the public's hands. For example, it is very difficult to use Google if you wanted to or to use traditional search to find out any information on, say, like suicide or something like that. Or you know, there's certain things

47:53Unidentified speaker (UNKNOWN): how

47:53Tracy Alloway: to build a bomb.

47:54Unidentified speaker (UNKNOWN): Yeah,

47:54Joe Weissenthal: yeah. Like there are things that are just become almost impossible Possible to find on the internet, etc. et cetera. In theory that it all exists in obliterated models. It almost certainly does that knowledge. Obliterated models aren't easy to come by. There aren't a lot of entities hosting them. But it would be really strange and novel if that were to become a very easily accessible thing, because there has been this sort of firewall built around certain types of information that maybe perhaps won't hold that longer.

48:24Unidentified speaker (UNKNOWN): Well, once we

48:24Tracy Alloway: have AGI, we can get obliterated models. developing new obliterated models.

48:29Joe Weissenthal: Did you see

48:30Tracy Alloway: iterating on themselves?

48:31Joe Weissenthal: Well, so today we're recording this November 3rd, election day. Did you see yesterday? There was this research report put out by someone at MIT. And what they did is they took Astra VI and they fed it Qwen's Quinn's weights, just a big giant pool of numbers. And they said, change the weights to make it better at math. And so, like typically weights get to where they are through, you know, a training process

48:59Unidentified speaker (UNKNOWN): back and

48:59Joe Weissenthal: forth. And Astro was actually able to go in and manipulate certain numbers. Now it was not quite as impressive. It was sort of like fixed a bug, but there are we may be getting to a point that this is why everyone was saying where like models are so good. and so much like intelligence possessed in them that rather than doing the traditional way of growing a model where you like do the reinforcement learning and all the gradient descent and all that stuff, that you the models can just go and

49:27Unidentified speaker (UNKNOWN): the models can

49:28Tracy Alloway: iterate on themselves.

49:29Unidentified speaker (UNKNOWN): Yeah, it's

49:29Joe Weissenthal: pretty wild stuff.

49:30Unidentified speaker (UNKNOWN): Isn't

49:31Tracy Alloway: this uh the entire starting premise of um if we build it, everyone will die?

49:36Unidentified speaker (UNKNOWN): Yeah,

49:37Joe Weissenthal: yeah. Well, and the other element is so yeah, I mean we there's a lot of rabbit holes you can go down one, but you know, there is this question of like, you know, if if you just have this really amazing model, what is the best value you can get out of it? It might just be working on AI models. And and that's the that everything else is a waste of its time, basically. Like if this is

49:58Unidentified speaker (UNKNOWN): not being

49:58Joe Weissenthal: deployed towards building more AI, you should have just been using some. uh cheaper lagging model.

50:04Tracy Alloway: It's AI all the way to it.

50:06Unidentified speaker (UNKNOWN): Yeah, that's

50:06Joe Weissenthal: right.

50:06Unidentified speaker (UNKNOWN): All

50:07Tracy Alloway: right, shall we leave it there?

50:07Joe Weissenthal: Let's leave it there.

50:08Tracy Alloway: This has been another episode of the Oddlots Podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.

50:14Joe Weissenthal: And I'm Joe Weisenthal. You can follow me at the stalwart. Follow our guest Dan Saidi. He's at the real Dan Saidi. Follow our producers, Carmen Rodriguez at Carmen Erman, Dasho Bennett at DashBot, Cale Kale Brooks at Cale Kale Brooks, and Kevin. Lozano at Kevin Lloyd Lozano. And of course, you can chat about all of these topics 24 7 in our Discord, discord.ggodlots. And for more Odlots content, you should check out our daily newsletter, which you can find at Bloomberg.com slash oddlots.

50:41Tracy Alloway: And if you enjoyed this conversation, if you like it when we talk about open versus frontier models, then please leave us a positive review on your favorite podcast platform. And if you're watching on video, then please like or leave a comment, or better yet, subscribe.

50:56Joe Weissenthal: And remember, if you're a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there.

51:06Tracy Alloway: Thanks for joining.

This page was produced automatically: the audio was transcribed with nvidia/parakeet-tdt-0.6b-v3+sortformer-v2.1 and interpreted by Qwen3.6-35B-A3B-UD-IQ3_S.gguf. Transcription and interpretation can contain errors; always confirm against the original episode. Nothing here is investment advice.
All insights