The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

Jev Creator argues AI reliability and data quality trump compute, predicting a shift from front-end apps to background automation.

1h 14m Listen to the original Episode page
AI ReliabilityData QualityVenture CapitalSaaSOpen Source AIModel Scaling

Summary

Jev Creator (Diogo Almeida) argues that the primary bottleneck for AI automation is not compute or speed, but the reliability of intelligence, defined as the consistency of decision-making rather than system uptime. He posits a 'bitterer lesson' where task definition and data quality are more critical than algorithms or compute, which he claims have plateaued in effectiveness since GPT-4.5. Creator criticizes current market hype, benchmark-maxing, and the 'AI coworker' model, predicting a 'SaaS Palooza' where incumbents win by integrating AI into workflows rather than replacing them. He also discusses TypeSafe AI's fundraising strategy, hiring philosophy, and the geopolitical landscape of open-source models.

Episode sections

  1. 0:00 – 5:08

    Reliability as the Bottleneck and Data Over Compute

    Speaker_0 argues that reliability of intelligence, not uptime, is the primary bottleneck for AI automation, citing the inconsistency of models (Einstein vs. Mr. Bean). He asserts that data quality and task definition matter more than compute, which has plateaued in effectiveness since GPT-4.5.

  2. 5:08 – 8:12

    Jev's Value Proposition and the Failure of Automation

    Diogo Almeida explains that Jev aims to bridge the gap between AI's high intelligence and its poor reliability in automated software tasks. He notes that despite AI's capability, it fails to automate jobs like accounting or customer service due to high variance in decision-making.

  3. 8:12 – 12:22

    The Bitterer Lesson: Task, Data, Compute, Algorithms

    Almeida proposes a 'bitterer lesson' to the traditional 'bitter lesson' of AI: the order of importance is doing the right task, then data, then compute, then algorithms. He argues that previous scaling efforts failed because they optimized for human preference (RLHF) or benchmarks (RLVR) rather than reliable decision-making.

  4. 12:22 – 17:36

    Data Quality and Synthetic Data Utility

    The speaker distinguishes between raw corporate data and the specific, task-oriented data needed for model improvement. He suggests that synthetic data is a tool dependent on the team's ability to define the right task, rather than a standalone solution, and that data vendors must focus on effectiveness rather than just serving data.

  5. 17:36 – 19:48

    Market Hype, Waste, and Investment Justification

    Almeida expresses concern over the wastefulness of current AI investment, particularly in hiring ex-lab personnel and burning GPUs without clear utility. He believes the current spending is not justified based on expected value calculations, though he acknowledges the models are getting smarter.

  6. 19:48 – 22:00

    Competitive Advantage via Reliability, Not Speed or Cost

    Almeida states that Jev's competitive advantage is the reliability of intelligence, not speed or cost. He criticizes copycat models that focus on benchmarks and hype, arguing that true competition requires a deep focus on making models robust for background automation.

  7. 22:00 – 25:01

    Big Tech's Lack of Focus on Reliability

    The speaker claims that major providers like OpenAI do not currently care enough about reliability because they optimize for metrics like Monthly Active Users (MAU) rather than decision quality. He asserts that Anthropic models are superior in practice despite benchmark matches from Chinese models, indicating a disconnect between benchmarks and real-world utility.

  8. 25:01 – 29:56

    Mega Jevons Paradox and Future Software Landscape

    Almeida predicts a 'mega Jevons paradox' where AI usage will dwarf current LLM token consumption, leading to AI running in the background. He identifies the biggest bottlenecks as the need for 'software researchers' to push boundaries and the inefficiency of current infrastructure, comparing AI to the invention of the blue LED which enabled modern screens.

  9. 29:56 – 32:46

    Reliability as the Bottleneck and the SaaS Apocalypse

    Speaker_0 argues that LLMs are currently unreliable for high-stakes corporate decisions, leading to a state where AI is attached to the side of apps rather than deep in logic. This results in a 'SaaS Apocalypse' being a 'SaaS Palooza,' where incumbents win not due to distribution but because they are closest to the workflow problems.

  10. 32:46 – 35:04

    AI Coworkers vs. Automation and Proprietary Intelligence

    Speaker_0 dismisses the 'AI coworker' model as a 'horseless carriage' analogy, arguing that true automation requires narrow, specialized tools rather than general intelligence interfaces. He contends that companies seeking 'proprietary intelligence' by training their own models are misguided; real moats come from software engineering around models, not the models themselves.

  11. 35:04 – 42:45

    Benchmark Misleading Nature and Diminishing Returns

    Speaker_0 claims benchmarks are misleading due to perverse incentives and that raw intelligence per token is barely improving. He suggests that recent gains in specific areas like coding are due to data distribution rather than intrinsic model improvements, indicating diminishing returns in general capabilities.

  12. 42:45 – 49:52

    Open Source Models and Geopolitical Threats

    Speaker_0 asserts that American open-source models are benchmark-maxing to a 'consolation prize' level and are not truly competitive with Chinese models in raw intelligence. He rejects the premise that Chinese models pose an immediate security threat if used with proper guardrails and bias mitigation, suggesting the focus should be on reliability rather than model origin.

  13. 49:52 – 56:15

    AI Bubble, Demand Shock, and Jevon's Paradox

    Speaker_0 believes the AI market is in a bubble driven by FOMO and overpromising. He argues that a demand-side shock will occur when companies realize AI's current capabilities are insufficient for their expectations. He applies Jevon's Paradox, suggesting that as AI becomes cheaper and more reliable, it will unlock new use cases rather than simply reducing costs.

  14. 56:15 – 1:13:39

    Fundraising Strategy and VC Signals

    Speaker_0 describes raising $750M without a deck by focusing on inbound interest and validating product usage with VCs. He prioritized finding allies and smart money over maximizing valuation or using complex tranches, noting that the company was already profitable and did not need to raise for survival.

Assets discussed

AssetKindStanceContext
Anthropicprivate companyneutralAnthropic models are just the best, even if the Chinese models match it in benchmarks. It feels like there's no real competition. right now.
OpenAIprivate companyneutralWhy do you not hire from OpenAI?

“Stance” is how the speakers talked about the asset, not a recommendation.

Claims worth verifying

Jev went from zero to $100M in revenue in seven days.

Unidentified speaker (speaker_1) ·1:15

How to verify: Check TypeSafe AI's official press releases or investor updates from late 2026; verify via Sequoia Capital's portfolio announcements or Pat Grady's public statements. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

TypeSafe AI was valued at $7.5BN in a round led by Martin Casado at Andreessen Horowitz, with Sequoia also investing.

Unidentified speaker (speaker_1) ·1:15

How to verify: Check SEC Form D filings for TypeSafe AI's latest equity raise; verify via Andreessen Horowitz (a16z) portfolio page or press releases from November 2026. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

GPT-4.5 broke the narrative of scaling compute because it was the largest model by token cost but not commercially successful.

Unidentified speaker (speaker_0) ·11:00

How to verify: Check OpenAI's pricing pages and usage data for GPT-4.5; verify via industry reports on model adoption and commercial success metrics from 2026. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

RLVR's success is due to latent chain of thought, not just verifiable rewards.

Unidentified speaker (UNKNOWN) ·13:41

How to verify: Review research papers on RLVR (Reinforcement Learning from Verified Rewards) from 2026-2026; verify via arXiv or conference proceedings (e.g., NeurIPS, ICML) discussing the role of hidden reasoning in RL. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

ChatGPT optimized for MAU, leading to psychophantic behavior.

Unidentified speaker (speaker_0) ·23:58

How to verify: Review OpenAI's internal metrics or public statements regarding MAU vs. reliability; verify via industry analysis of ChatGPT's product evolution and user engagement strategies. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

AI will become a fourth fundamental logic gate, expanding software capabilities by hundreds of thousands of times.

Unidentified speaker (speaker_0) ·29:15

How to verify: This is a speculative claim; verify via future software engineering adoption rates and productivity metrics post-2026; currently uncheckable as a historical fact. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

Data matters more than compute, and the 'bitter lesson' is now 'task matters more than data.'

Unidentified speaker (speaker_0) ·8:32

How to verify: Review academic literature on AI scaling laws from 2026-2026; verify via papers from OpenAI, DeepMind, or Anthropic discussing the diminishing returns of compute and the importance of data quality/task definition. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

TypeSafe AI went from zero to $100M in revenue in seven days.

Unidentified speaker (UNKNOWN) ·29:56

How to verify: Check TypeSafe AI's official press releases or financial disclosures; verify via Sequoia Capital's portfolio announcements or Pat Grady's public statements. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

TypeSafe's latest round valued the company at $7.5BN.

Unidentified speaker (UNKNOWN) ·29:56

How to verify: Check SEC Form D filings for TypeSafe AI or press releases from Andreessen Horowitz and Sequoia Capital regarding the investment. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

Martin Casado is at Andreessen Horowitz and led the TypeSafe round.

Unidentified speaker (UNKNOWN) ·29:56

How to verify: Check Andreessen Horowitz's website or press releases for the TypeSafe investment announcement. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Diogo Almeida was a researcher at OpenAI and helped invent ChatGPT.

Unidentified speaker (UNKNOWN) ·29:56

How to verify: Check Diogo Almeida's LinkedIn profile or OpenAI's historical research publications and team announcements. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Martin Casado coined the term 'SaaS Palooza'.

Unidentified speaker (speaker_0) ·31:16

How to verify: Search for Martin Casado's tweets or blog posts containing the term 'SaaS Palooza'. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Harvey pivoted from automating large projects to a copilot for lawyers.

Unidentified speaker (speaker_0) ·38:34

How to verify: Check Harvey's official website, product updates, or press releases regarding their product evolution. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Llama 4 was benchmark maxed by Meta.

Unidentified speaker (speaker_0) ·44:03

How to verify: Check Meta's Llama 4 release notes, internal memos if leaked, or third-party analyses of benchmark performance vs. reported metrics. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

DeepSeek 4.1 Flash was not trained using the technique associated with the GPU export ban.

Unidentified speaker (speaker_0) ·49:03

How to verify: Check Will DePew's research results or independent technical analyses of DeepSeek's training infrastructure and compliance. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Coding agent market players include Cognition, Codeium, Cursor, and Factory.

Unidentified speaker (UNKNOWN) ·54:43

How to verify: Check company websites and market reports on coding assistants to verify their market positioning and revenue claims. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

TypeSafe raised $750M in a round without a deck.

Unidentified speaker (speaker_1) ·56:15

How to verify: Check SEC Form D filings for TypeSafe AI to verify the raise amount and timing. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Primary source

Andreessen Horowitz is philosophically aligned with TypeSafe AI, with partners Martin Casado and Ben Horwitz acting as strategic thought partners.

Unidentified speaker (speaker_0) ·59:49

How to verify: Check Andreessen Horowitz's public statements or press releases regarding their investment in TypeSafe AI and their stated philosophy on founder alignment. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Sonia Wang was the primary reason for Sequoia Capital's involvement in TypeSafe AI's funding round.

Unidentified speaker (speaker_0) ·1:00:28

How to verify: Review TypeSafe AI's funding announcements or press releases detailing the lead investors and specific partner involvement. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

TypeSafe AI's core value is to 'make great decisions slowly.'

Unidentified speaker (speaker_0) ·1:01:18

How to verify: Check TypeSafe AI's official website, career pages, or internal documentation for stated company values. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Anthropic pays more than any other AI lab, leading to a market for fake culture interview preparation.

Unidentified speaker (speaker_0) ·1:04:11

How to verify: Search for industry reports on AI compensation packages or news articles discussing Anthropic's hiring practices and interview coaching services. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

OpenAI researchers generally undervalue data and prefer algorithms and scaling.

Unidentified speaker (speaker_0) ·1:07:27

How to verify: Review OpenAI's published research papers to assess the proportion of work focused on data engineering versus algorithmic scaling. For recent events, verify 2026-08-16 through 2026-11-16. Use another period only when the speaker explicitly names it.

Do your own research

  • Verify the claim that GPT-4.5 marked a plateau in compute effectiveness by reviewing independent benchmark comparisons between GPT-4.5 and subsequent models.
  • Check TypeSafe AI's (Jev) actual product metrics for reliability and decision accuracy, contrasting them with the benchmark scores of competitors like OpenAI and Anthropic.
  • Review the hiring practices and employee retention rates at Anthropic and other high-paying AI labs to assess the 'mercenaries' claim.
  • Analyze the 'latent chain of thought' architecture claims by examining technical papers or documentation from labs using RLVR.
  • Investigate the 'mega Jevons paradox' prediction by analyzing current token consumption trends versus projected background automation use cases.
  • Look for evidence of 'benchmark maxing' by comparing public benchmark scores with independent, out-of-distribution test results for open-source models.
  • Check the financials of companies hiring ex-lab personnel to see if their valuation is justified by revenue or merely by acquisition potential.
  • Review the definition of 'reliability' in the context of AI automation to understand how it differs from traditional uptime metrics.
Full transcriptexpand
0:00
Unidentified speaker (speaker_0): Almost everyone who's paying attention to AI knows reliability is the absolute largest bottleneck. You get Einstein, like maybe 95% of the time, and Mr. Bean the other 5%. Data matters a lot more than compute. I don't want to be called a neolab because, from my sense, most neolabs are incredibly, incredibly wasteful.
0:20
Unidentified speaker (speaker_0): Anthropic models are just the best, even if the Chinese models match it in benchmarks. It feels like there's no real competition. right now. I truly believe that we're gonna like mega Jevons paradox. AI is so clearly awesome, yet software itself hasn't changed. Everyone has just been in this like mono model like horse blinders
0:40
Unidentified speaker (speaker_0): for so fucking long. I've believed that AI has been a bubble for a while. So I would like to keep planning for an eventuality of bubble burst. What we need is allies. to fight the largest companies in the world. We don't need the money or the valuation. This
0:55
Unidentified speaker (speaker_1): is 20 VC with me, Harry Stavings. Now joining me in the hot seat state is one of the best founders of our time, one of the hottest companies, Diogo Almeida. Diogo is the founder of TypeSafe, the maker of Jev. Jev is used by a third of the Fortune 500. Pretty incredible. Yep, it gets better. Sequoia's Pat Grady says Jev Jeff went. from zero to
1:15
Unidentified speaker (speaker_1): a hundred million in revenue in just seven days. And when you're that fast and that good, it comes with some benefits. They've been valued at $7.5 billion, and they had their round led by Martin Casado at Andreessen and Sequoia also invested. Today we get into the future of models with Diogo. This is an incredible discussion. But before we
1:35
Unidentified speaker (speaker_1): dive into the show today, Blitzy, the autonomous software. development platform that's built for enterprise scale, is proud to sponsor today's show. Enterprise code bases have grown beyond human comprehension. Tens of millions of lines, decades of tech debt, complexity existing tools cannot fathom. Blitzy starts by understanding. Thousands of
1:55
Unidentified speaker (speaker_1): agents reverse engineer your code base into a proprietary knowledge graph that maps architecture, dependencies, and business logic. That graph serves surfaces every CVE and zero day hiding in your code before anyone hands you a ticket. Grounded in that context, Blitzy then executes entire epics autonomously, modernize off legacy
2:15
Unidentified speaker (speaker_1): systems, ship new features 5x faster, or batch remediate CVEs at scale, surfaced by the platform. Blitzy not only generates code, it validates and end end-to-end to end tests every single line. And that's why Fortune 500 enterprises are shipping enterprise grade code five times faster. Go to
2:34
Unidentified speaker (speaker_1): blitzy.com forward slash 20vC to learn more. While Blitzy helps you build faster, Flex helps you run your finances smarter. Business owners run their whole financial life on Flex. One platform from business revenue to their personal spend. Float every purchase for 60 days.
2:53
Unidentified speaker (speaker_1): Tap capital that grows with your revenue and pay vendors in 170 countries across 32 currencies. Plus the whole back office, bills, expenses, accounting, all in one place. So you spend less time reconciling and more time growing. That's why thousands of owners use Flex, named one of
3:13
Unidentified speaker (speaker_1): Fast Company's most innovative. companies of 2026. Visit Flex.one, that's F-L-E-X.on e and use the code 20VC. While Flex runs your finances, Vanta keeps your compliance covered. When you're building a company, you learn that trust is what closes deals. You
3:33
Unidentified speaker (speaker_1): may have the best product, but no buyers will sign these days without proof of your security. Here's what happens. if you're not prepared: a prospect asks for proof of compliance. The deal stalls when you scramble, your engineer gets pulled off the roadmap to audit prep. Every enterprise conversation turns into this horrible fire drill. That's where Vanta comes in. Vanta is the leading
3:53
Unidentified speaker (speaker_1): agentic trust platform that not only gets you compliant fast with frameworks like SOC 2, ISO 27001, HIPAA, and GDPR, but keeps you compliant by continuously monitoring your controls. So your deals keep moving and your engineers really keep building. Now access the Vanta agent everywhere you work, even if your team lives in Claude
4:13
Unidentified speaker (speaker_1): or Cursor. And that's why Vanta's trusted by more than 16,000 companies like Ramp, Harvey, and Writer. So prove you're ready for business and get $1,000 off Vanta when you go to vanta.com forward slash 20 VC. That's vanta.com/slash 20 VC.
4:30
Unidentified speaker (speaker_2): You have now arrived at your destination.
4:32
Unidentified speaker (speaker_1): Diogo, I'm so looking forward to this. We we had a brilliant exchange following Eric's show from Modal, who I actually spoke to last night about you. And so I'm very excited for this. Welcome to the show.
4:44
Unidentified speaker (speaker_0): Thank you. Thank you. Yeah, a brilliant, it's a very small exchange, but I'm I'm glad you liked it. Um yeah, it was it was uh really fun, and I didn't realize it would become this.
4:55
Unidentified speaker (speaker_1): Well, dude, we we said that it would be a spicy discussion. So before we get into
4:58
Unidentified speaker (UNKNOWN): the
4:58
Unidentified speaker (speaker_1): spice itself, I just want to lay some frameworks and like foundations. 60 seconds. So people who don't know what TypeSafe and JEV is, can you just give us the 60-second intro?
5:08
Unidentified speaker (speaker_0): I would argue that JEV is not a specialized model, but my pitch for the people who are not like purely software developers is actually more about the state of AI. Like, how can AI be so smart at so many things? You know, like the Millennium Prize problems in math, yet not transform software basically at all. Like all the SaaS we use is basically
5:28
Unidentified speaker (speaker_0): the same as it was a decade ago. And we are not automating like almost any Like entire job. You know, like drive-throughs we keep failing, customer service, like there is AI customer service. but it's actually can't be trusted for the big decisions. How can that be the case? Because it seems so good at some stuff and so bad at other stuff. And to me, what we are doing and what Jev
5:48
Unidentified speaker (speaker_0): is is trying to bridge that gap, make AI too good to be true for automation instead of just human in the loop assistance.
5:56
Unidentified speaker (UNKNOWN): I
5:56
Unidentified speaker (speaker_1): spoke to Sonali at Sequoia, and speaking of kind of full and to and automation, she said, why is reliability the biggest obstacle? to automation?
6:04
Unidentified speaker (speaker_0): Ooh. Well, uh, I will give a caveat first about reliability, which is that I don't mean like uptime. I don't mean like, oh, it's like, you know, of you know, available and you get lots of throughput. I mean reliability of intelligence, which I'm still trying to find a good word for, but like this non-jaggedness Enness of intelligence. And I would argue
6:24
Unidentified speaker (speaker_0): almost everyone who's paying attention to AI knows. reliability is the absolute largest bottleneck. So here's the example. How good are you at math? How good am I at math? There's not a chance we could even help the Millennium Prize solution. I I I believe. I haven't read it, but I would I would wager that. And yet that these same models cannot be even used for
6:43
Unidentified speaker (speaker_0): accounting. You know, it's much simpler stuff. And the question is why? And it's not about these models are extremely high variance. variants. It's that you get uh d people love this this saying that I've coined uh b so but it's like you get Einstein like maybe 95% of the time, and Mr. Bean the other five percent. And the problem is that in
7:03
Unidentified speaker (speaker_0): expected value it just doesn't work out. Therefore companies quite wisely learn that you can't actually automate stuff. The number of companies I've seen over like the multiple AI hype cycles over the last like four years. I've seen so many try to automate accounting. They keep failing. And it's because you know the math doesn't work out. It's not a diffusion problem in the slightest. Reliability is what
7:23
Unidentified speaker (speaker_0): gets you, you know, maybe you can trade off like some level of Einstein for just getting it more commonly and more often and getting the same functional intelligence. That to me is Jeb Jeb's superpower. Aaron Ross
7:35
Unidentified speaker (UNKNOWN): Powell
7:35
Unidentified speaker (speaker_1): So how do we get there? What is the solution then to removing Mr. Bean? as much as I love Mr. Bean?
7:40
Unidentified speaker (UNKNOWN): I
7:40
Unidentified speaker (speaker_0): love Mr. Bean too. I and I actually, you know, I love when agents are like insane, you know, bad vibes forever. It's it's it's it's very fun. It's a very fun story. And also, that is not what I want in my software that runs in the background. Like
7:52
Unidentified speaker (UNKNOWN): as
7:52
Unidentified speaker (speaker_0): an engineer, I want to like write a thing. I want to spend a lot of effort up front. Like maybe I don't realize it'll be a lot of effort, but it always is. And I want wanted it to run in the background and solve the problem forever. And you just need like nines of intelligent reliability to do that. And I would argue that the way to do that, oh man, I wish I could like bring up my slides here, but I will I'll I'll try to use my
8:12
Unidentified speaker (speaker_0): heads, I guess. So I call it AI's bitterest lesson. The Sutton coined a bitter lesson, which is is roughly that compute matters more than algorithms. This is all of like the scaling pilled piled era we had for twenty 2019 nineteen to twenty 2022 twenty two. to 2023 was a scaling build era. You know, like just scale and you the models get better.
8:32
Unidentified speaker (speaker_0): This kind of plateaued at GPT 4.5, and then we kind of stopped scaling as a field because it stopped having a payoff. I would argue that there's a bitterer lesson, which is that um data matters a lot more than compute. This is very obvious to anyone who's done any applied ML, but for you know, the the people in the peanut gallery don't seem to realize this, but like data. is obviously more
8:51
Unidentified speaker (speaker_0): valuable than compute. And I would argue the most important thing by far is what is your North Star, what do you define as your right task? And the thing is that pre-GEV, the two right tasks that existed in LM land were RLHF, which is reinforcement learning from human feedback, which means please humans, and RLVR, which
9:11
Unidentified speaker (speaker_0): means but like score high on benchmarks. And neither of those actually translate to like run in the backward background software. And I would argue that this is like the root of all of the problems. It is not that the AI is not capable, because the AI is clearly super capable. The problem is that, and you know, I you are uh
9:31
Unidentified speaker (speaker_0): experienced in the real world, uh everyone who's experienced in the real world knows this. You get what you optimize for, right? And we just happen to optimize for the human in the loop.
9:40
Unidentified speaker (speaker_1): Yeah, uh thank you so much for uh bluntly giving me so many for
9:43
Unidentified speaker (UNKNOWN): the things. Can I say one thing?
9:44
Unidentified speaker (speaker_0): Can
9:44
Unidentified speaker (UNKNOWN): I say one
9:44
Unidentified speaker (speaker_0): thing about this too? Um uh on the note of reliability, chat to GPT never fails at being chat to GPT. Like, when is the last time you got like random characters, right? Like it's kind of odd that it or or even if you ask for a polite email, it's not gonna start saying, um I I I probably shouldn't create cursing a podcast, but it's not gonna start saying slurs, right? Um
10:04
Unidentified speaker (speaker_0): but humans might make that error sometimes. But the problem is not that it it's not impolite or psychophantic. The problem is it makes wrong decisions within the text. And it's interesting that it can be reliable at some stuff, but not others. And if you optimize for the important stuff to be reliable on, which is decision-making decision making and judgment, that is
10:24
Unidentified speaker (speaker_0): how you get real reliability. You
10:26
Unidentified speaker (speaker_1): said that about there components. about kind of the plateauing of compute in terms of the benefits that accrue. First, have we got to that stage? Are you serious? I thought the whole point was
10:34
Unidentified speaker (speaker_1): purely on pre-training, sure. But like more and more compute still leads to better and better output, no?
10:40
Unidentified speaker (speaker_0): Hmm. Yes, but like it's it you need a lot more data than people once upon a time thought. So, you know, like maybe people have like kind of warped this to seem like they were right the whole time. But the narrative back in the day was just scale to the moon and you will get AGI. And GPT-4.5 is what broke
11:00
Unidentified speaker (speaker_0): that narrative.
11:05
Unidentified speaker (speaker_0): But if we believe that the labs were pricing roughly based on inference compute, then GPT GPD 4.5 is still the largest model ever. By token, cost. And it was not commercially successful. And I think that this is obviously the case because you can't just scale compute without really thinking
11:25
Unidentified speaker (speaker_0): about it. What people are doing people are throwing a lot of compute at training, but it's a you need a lot of data with it. This is why you see so many like get rich quick by selling data, RL data to lab type stuff. And you can't really do very much without that data. And if you actually saw oh man, I believe might have
11:45
Unidentified speaker (speaker_0): been Dwarkesh. He had a recent recent Twitter post that showed what matters more data or compute or algorithms. And data was something like many, many times more effective. It was something this week. And this is something that everyone has known for like since the beginning of ML. This is not even specific to AI. It's just that the compute. scaling error was a like a weird outlier in this.
12:05
Unidentified speaker (speaker_1): When we say like the value of data, how it's so much more primary over compute, is that synthetic data? Is it like proprietary enterprise data that everyone has in their companies? Is it AI systems generating their own experiences? Is it certain types of data? Is it just broad data? Can you help me understand that?
12:22
Unidentified speaker (UNKNOWN): I
12:22
Unidentified speaker (speaker_0): wish I had my visuals here. So uh let me work backwards again from my bitterest. lesson, which is in order of importance, there's doing the right task, then data, then compute, then algorithms. This is what we like extremely, extremely learned when we made RLHF. You know, like the quality of your uh underlying algorithm matters very little. As long as you do the right
12:42
Unidentified speaker (speaker_0): task. Um, a GPT-2-sized model could beat something like GPT-9, which is you know, and GPT nine 9 doesn't even exist. In order to have data, you actually need to know the right task. And actually, the data for the task is normally when you create a new task, the data doesn't even exist. When we made RLHF, we uh like like the
13:02
Unidentified speaker (speaker_0): human preference data of like comparing LLM outputs did not exist anywhere in the internet. It was not in company database, it was not on the internet. We had to like define it, understand what are the instructions, get human labelers, iterate. and figure out what that data should look like. And that is what unlock like doing the right task is what unlocks the data in the first place.
13:22
Unidentified speaker (speaker_0): And the problem is bootstrapping a new task is hard because by construction the data doesn't exist. It's a similar thing for us, but it's also similar for RLVR. Before RLVR existed, uh and I actually want wanna there there's actually a tricky thing with what the right task of RLVR is um reinforcement learning from verified rewards. Sorry if I'm like going a little bit too advanced
13:41
Unidentified speaker (UNKNOWN): here.
13:41
Unidentified speaker (speaker_0): Is that RLVR is not at all about the verifiable rewards, but the big part of RLVR is this like latent chain of thought, like hidden reasoning, so the models can think in private and then answer. Because even in like 2021, there were verifiable rewards. People tried to do RL, but like
14:01
Unidentified speaker (speaker_0): PPO without reasoning on unit tests, and it kept failing. So it was actually not at all about the verifiable rewards. It's about the structure of the task. And it just turns out that once you're doing RLVR, the models are a lot better at this. And what even good data means is now a function of how good the models are. This is why people need,
14:21
Unidentified speaker (speaker_0): you know, the people who make data need access to the cutting-edge cutting edge models, because what actually makes the data good is whether or not it improves the cutting edge models. So they need to be at like exactly the right level of difficulty in order to like push the models forward. And there's a lot of like nuance and unpoliciness of data. But get back to the original question: like,
14:41
Unidentified speaker (speaker_0): is data is the right data stored in corporate, you know, whatever data lakes? I would argue no, because it's probably not shaped the right way. People are definitely trying to do this, so a lot of data vendors are trying to like yoink this data and then sell it. Maybe they can do some massaging to make it work. But the most important thing is focusing on how do
15:01
Unidentified speaker (speaker_0): we use this thing to push our task forward. And if you forget that, you are good hearting a local objective.
15:08
Unidentified speaker (speaker_1): How do we use this thing to push our task tasks forward? Are the data providing companies, Surge, McConnell? McCor, are they good businesses? Are they services businesses now, which have to go end to end where they really go into implementation and effectiveness of model, not just data serving like they did in the early days? Are they good businesses?
15:29
Unidentified speaker (speaker_0): That's more of a you question than a me question. Do you think they're good businesses?
15:33
Unidentified speaker (speaker_1): I do personally, because I think the market is just so large for the amount of money
15:37
Unidentified speaker (UNKNOWN): that
15:37
Unidentified speaker (speaker_1): these LLMs will spend on data. And then I think if you think that we're gonna move into a world where companies own their own intelligence, their data will always be needing supplementary data. And so then actually the TAM becomes just moving from LLMs to everyone.
15:50
Unidentified speaker (speaker_0): Interesting. Interesting. That might be a fun debate on whether or not companies will own their own intelligence. That might be a fun one. I will actually argue that we are still in hype wave territory and then there's always like a trough trow of disillusionment. and then like the reality thing. I think we've not hitten the reality part of it yet. I believe that in the
16:10
Unidentified speaker (speaker_0): future, both RLHF and RLVR are immensely valuable. I don't know if people will invest as much as they are right now, but I think that each of them result in like their own, like even if RLVR never existed, ChatGPT would be like an incredibly valuable consumer brand. And you will need data for that. Data will always be
16:30
Unidentified speaker (speaker_0): necessary, but how much that company will be willing to spend on data when they actually need to be a real sustainable business, I don't know what that number is. Very hard to ballpark that.
16:40
Unidentified speaker (speaker_1): What value does synthetic data hold in model improvement?
16:43
Unidentified speaker (speaker_0): I don't think that there's like one one form of synthetic data. I think that it synthetic data is kind of like a tool more than it is. like a idol to pray to. And it's dependent on like what the the hands of the team. Like if you read the research, you'd think that synthetic data is mostly overhyped because it's only used to make small models
17:03
Unidentified speaker (speaker_0): catch up to large models, but it can't be used to improve large models or or large state-of-the-art models. So it's really dependent on how you use it. We use a lot of synthetic data, but like we do not use it in the way others. other people do. We think that the more interesting part is not that it's synthetic, but like how we wield this tool with an extreme laser focus on the
17:23
Unidentified speaker (speaker_0): right task, which is like the reliability of intelligence. Aaron
17:26
Unidentified speaker (UNKNOWN): Ross Powell
17:26
Unidentified speaker (speaker_1): You said about the hype and the trough, my friend. There is a lot
17:29
Unidentified speaker (UNKNOWN): of trough.
17:30
Unidentified speaker (speaker_0): That's good to know.
17:31
Unidentified speaker (speaker_1): I tried to do it elegantly. Okay, I'm British.
17:36
Unidentified speaker (speaker_1): Are you concerned by the amount of money going? in, or do you think it is justified?
17:41
Unidentified speaker (speaker_0): I am concerned for the world, I think. I think that throwing money into the furnace is a bad thing for the world. Oh man, I really don't want to be like too negative, but I don't want to be called a neo lab because from my sense, most neolabs are incredibly, incredibly wasteful. It feels like their business model is hire a bunch of, you know, ex ex-lab lab
18:01
Unidentified speaker (speaker_0): people, get a bunch of GPUs. have fun, and then eventually get acquired. That seems like a waste of resources to me. I am not a fan of that, on one hand. On the other hand, like people are throwing a lot of money, for example, into RL. And I actually think that this is kind of cool. There are diminishing returns massively, but the models are getting smarter. It's kind of good for the world. I think it's very
18:21
Unidentified speaker (speaker_0): cool. Um, would I make that investment with my money? No, obviously not, because you know, we have money to make bets and we're obviously not gonna do that. So I I'm not a fan of that in the world either, but maybe, you know, like the VC math works out where it's like a one percent chance of moonshot it actually like has positive expected value. And wait, you had a question, like am I concerned and
18:41
Unidentified speaker (speaker_0): there was like
18:42
Unidentified speaker (UNKNOWN): Are you
18:42
Unidentified speaker (speaker_1): concerned or is it justified?
18:44
Unidentified speaker (speaker_0): Oh. Um I would guess based on my expected value calculations, it is not justified. but would be very happy to be proven wrong. Or at least right now.
18:55
Unidentified speaker (speaker_1): Did you expect it to work as quickly as it did?
18:59
Unidentified speaker (speaker_0): So I would say that for years now, even before TypeSafe existed, I thought that I must be missing something about the field. Obviously, I I convinced myself I If was not missing something, but it seemed so weird where this direction to me was unbelievable. unbelievably obvious. So freaking obvious that I gave
19:19
Unidentified speaker (speaker_0): up when I first came to the idea because I thought surely anthropic is already like 80% done. It's like the most obvious thing I've ever heard of. So to me, it's really, really obvious that this would be valuable in the world. I still think people don't get it, even though the launch was obviously phenomenal. The part I didn't expect is the ChatGPT Chat GPT for developers moment, because even
19:38
Unidentified speaker (speaker_0): non-developers were talking about it. That is that that I could not have predicted. That was super weird.
19:43
Unidentified speaker (speaker_1): What do people not get? What do people get most wrong about Jav and you?
19:48
Unidentified speaker (speaker_0): So my team hates when I talk about this, but my rule is on a podcast I can be let off the rails. I believe it is a competitive disadvantage for me to talk about what really Jeb Jev is doing, because people keep missing the mark. And if we just didn't tell anyone, then we would just forever be king of the hill. And I just care too much about the field to not just yell in
20:08
Unidentified speaker (speaker_0): in like painful insanity, what's the fuck's going on? And the thing that people really don't get is that what we is special about the model is not the cost or the speed. It's again the reliability of intelligence. And people are still thinking in LM land, like like so many people I think are burning their goodwill. There should be copycat
20:27
Unidentified speaker (speaker_0): models in the future, but they should give a lot of fucks about reliability, in my opinion. And I think that the people who are shipping copycats right now do not. They are trying to like do the same thing in LMs, which is like benchmark max, get hype, try to show like, you know, like the the little things like, you know, n numbers that are not relevant, like speed and
20:47
Unidentified speaker (speaker_0): cost, when the real thing they need to do is like hunker. up in the bunker and make the models really, really smart. And people don't realize that that is what we are all about. When we see benchmarks, even if we are like unbelievably in the lead, we are still pained when our model is dumb, and that is the thing we focus on.
21:07
Unidentified speaker (speaker_0): And I think that's what everyone should be focusing on in the field. Like, hey, why is it 97% accurate? What was the missing piece? And to be clear, it's not always a model thing. It's very oftentimes a prompting thing. But that is how it is to build a developer tool.
21:27
Unidentified speaker (speaker_0): Is there some nuance in this language that could help people make more robust prompts? Or do we need to make the model smarter? And this is the thing that people really, really miss. And I think development. developers get it, and they got it way faster than I thought, because they already don't give a shit about like the benchmarks, and they're just like, your model is clearly the best at new stuff.
21:47
Unidentified speaker (speaker_0): But the same people who are like, oh, this is really boring, it's just a classifier, who ended up making the copycats who don't understand that there's a reason why you can like put so much effort to make the model even more and more and more robust.
22:00
Unidentified speaker (speaker_1): You said about copycats there. Can I be blunt? OpenAI and
22:04
Unidentified speaker (UNKNOWN): others
22:05
Unidentified speaker (speaker_1): have moved very quickly in trying to create their own decision model products after you launched. Why don't the big model providers just copy you and you get competed out?
22:15
Unidentified speaker (speaker_0): I don't think that that's a blood quest at all. I think that that's a like a totally fair, reasonable one. I think that if they one day started caring a lot about reliability, then that would become like an interesting battle.
22:29
Unidentified speaker (speaker_1): How do they not care about reliability? They want enterprises.
22:32
Unidentified speaker (UNKNOWN): You have
22:32
Unidentified speaker (speaker_0): no fucking clue how little they care. Aaron
22:34
Unidentified speaker (UNKNOWN): Ross Powell
22:35
Unidentified speaker (speaker_1): What? How how? They want enterprises, they want revenue. How are you gonna get a CIO to sign up if you don't have reliability?
22:41
Unidentified speaker (speaker_0): I think people have a hard time understanding intelligence as a product, which is why people try to use benchmarks as band-aids. You know, like benchmarks are kind of a proxy for understanding the intelligence of the underlying thing. But as far as I can tell, the string field, you know, over you know, like maybe like the last nine months have learned that benchmarks don't mean shit. Enthropic models are just the
23:01
Unidentified speaker (speaker_0): best, even if the Chinese models match it in benchmarks. And like even when you like measure benchmark maxing, you still think like the anthropic models are better than this, uh better than the competition, even though they're not on the Pareto curve of pretty much any benchmark. Basically, it looks like the subfield of like System One or decision models learned this in like I think there was like competition maybe like a starting a week
23:21
Unidentified speaker (speaker_0): after. It felt like they people learned it
23:22
Unidentified speaker (UNKNOWN): in
23:22
Unidentified speaker (speaker_0): like two, three weeks, where we're like, oh yeah, the other stuff is just benchmark max. It's total crap. It's very easy to see that it's different. I think that it takes a different type of organization to, again, really, really care about reliability.
23:35
Unidentified speaker (speaker_1): If they don't care about reliability, what do they care about then, Diogo?
23:38
Unidentified speaker (UNKNOWN): Well it
23:38
Unidentified speaker (speaker_0): really doesn't. depends on like their business objectives, right? Back in the chat GPT is everything days, this is before Claude Code, they cared about like monthly active users. That's actually a very different metric to optimize for than the best chatbot experience. This is how psychophanty emerged, and it kept going in a cycle of like, oh, it's more psychophantic, it lies
23:58
Unidentified speaker (speaker_0): sometimes, but hey, that is what people want, so we just keep doing it in some pockets of space. So you get what you optimize for. That was like what ChatGPT optimized for. I think now people are optimizing for like the MAU equivalent of coding agents. And if they're not training a dedicated model for this and really putting like a lot of like intense thought, I don't think that
24:18
Unidentified speaker (speaker_0): they will be able to make a comparable model anytime soon. I hope that all of them do, because I think it would be very good for the world. It's obviously going to be bad for the business. but right now we are in the place where it feels like there's no real competition right now, and that surely will change eventually, right? Like I hope that we get competition and we just show that we are the best because we care
24:38
Unidentified speaker (speaker_0): the most and we have like the smartest people who are working like the absolute most laser focused on the North Star. But the g again, like this is not yet competition. Aaron
24:46
Unidentified speaker (UNKNOWN): Powell
24:47
Unidentified speaker (speaker_1): When you look at the decision model product landscape for mod for models that you're
24:50
Unidentified speaker (UNKNOWN): going to be.
24:51
Unidentified speaker (speaker_1): in, what does that look like in three years? Is it like an Uber Lyft market? Is it like a GCP AWS Azure market? Like what does the market composition look like in three years' time? Aaron
25:01
Unidentified speaker (UNKNOWN): Powell
25:01
Unidentified speaker (speaker_0): I'm a little bit of a I technological idealist, I think. Like so I actually don't really think of what the compet competitive landscape is gonna be like. I care more about the technological one. Our strength is always going to be technological shock and awe. So we are going to have to keep shipping awesome stuff. And the way I see that
25:21
Unidentified speaker (speaker_0): landscape is I'm going to ignore competition. I think that in terms of like number of tokens, I I truly believe that we're gonna like mega Jevons paradox. It's not going to
25:31
Unidentified speaker (UNKNOWN): be
25:32
Unidentified speaker (speaker_0): like a subtle Jevins, it's not going to be a fake Jevins, Jevons like LLMs, it's going to be like super duper. There's going to
25:38
Unidentified speaker (UNKNOWN): be
25:38
Unidentified speaker (speaker_0): like tons and tons. and tons of usage. And I think that it will completely dwarf at least the total token consumption of string LLMs. I think that these things are going to be used everywhere and we're gonna finally have AI run in the background. And I think we're gonna start to even have like a shift in how people feel about AI from being like this thing that like slops crap at them to like, oh,
25:58
Unidentified speaker (speaker_0): it actually makes life better, because people truly like it when technology improves their lives.
26:02
Unidentified speaker (speaker_1): When you say mega Jevons Paradox, do we have the energy and infrastructure to power the demand of a mega Jevons paradox that you foresee? And what are the bottlenecks to your tech utopia?
26:16
Unidentified speaker (speaker_0): Our models are incredibly efficient in comparison. We're not doing the standard subsidized playbook. We actually can serve these things incredibly, incredibly efficiently. And I think that there's like many boundaries to push on that efficiency angle. I think of course we're gonna have to do some sick research, but like that's the thing I trust us to do. But we'll have to push
26:35
Unidentified speaker (speaker_0): efficiency quite a bit in order to have that mega Jevonsing occurring. I think that the largest bottlenecks are going to be it feels a little bit single player right now, where it's on us for pushing the boundaries of the technology from an AI point of view. And from a field point of view, we're going to need to interact a lot with developers. We're going to
26:55
Unidentified speaker (speaker_0): need to help them push the f the boundaries. This is what I've been talking about, like a role I dream of. I'm so sick of like the status games, especially in SF, where ML researchers are like a higher tier than ML engineers, which are Or higher a tier than software engineers. I think that that's completely messed up. What my hope is, is we can create a new type of role that
27:15
Unidentified speaker (speaker_0): I call my my current favorite name is software researcher. Someone who actually pushes the boundaries of software. Like if you knew that you were in the Cambrian explosion of the internet or mobile, you'd think that there's like so many boundaries to push, and there's so much like low-hanging fruit that you need like trailblazers to go out there and build
27:35
Unidentified speaker (speaker_0): like, you know, like these like uh you know, lighthouses of like hey, there's value here. Let's like have a gold rush there. And I'm so excited about encouraging people to push those boundaries. I wish I had the time to do it myself. And there's so many crazy exciting things to do. So I think that those two are going to be like the biggest bottlenecks to get to like you know, system one
27:55
Unidentified speaker (speaker_0): models everywhere, AI actually automating shit. And if I could say one more thing, I don't know if you're a software engineer, I guarantee every single person. who's ever been a software engineer has had a conception in their head of I can automate this or I can't automate this. Like you need to know, right? Like if you wanted to like automate like some thing that moves files around, you can automate this.
28:16
Unidentified speaker (speaker_0): If you want to make like a voice interface for your laptop so you don't need to like type with your hands, you can't automate this. And my hope is that AI will actually completely remove the can't automate part. It's only going to be a matter of effort. I have like this entire soapbox about the future of software. I will cook on this undercooked idea, but this is something that I find to be like such a
28:35
Unidentified speaker (speaker_0): beautiful analogy for AI, where AI is so clearly awesome, yet software itself hasn't changed. Writing software has very much changed, but the software is the same, and that's tragic. And I love the analogy of the blue LED. You know, like when the red and green LED. existed, they were very narrow uses. Like they could barely be used for anything, just indicator lights,
28:56
Unidentified speaker (speaker_0): because they're very efficient. But when you add the blue LED, it didn't just expand the market cap of LEDs by 50%. It made all sorts of things possible. Like we couldn't have the screens we have today. We couldn't have like white light LEDs that are so much more efficient without this missing composable piece. And my hope is that AI ends up becoming like a
29:15
Unidentified speaker (speaker_0): fourth fundamental logic gate. You know, uh like with there's like three fundamental logic gates that all software is built on top of. They are so dumb, but it's actually the brilliance of software engineers that enables you to use these bitwise comparisons to create everything today in software. And what if we could add a fourth one? And I think that that should expand what we can do by way
29:35
Unidentified speaker (speaker_0): more than 33%. It should expand what we can do by like hundreds of thousands of X. And that is why I'm excited about a camping explosion specifically of software. Aaron
29:44
Unidentified speaker (UNKNOWN): Powell
29:44
Unidentified speaker (speaker_1): Why do you think the current state of software is kind of a tragedy in the state that it's in? And you know, if you look at enterprise software today, it it's enterprise software with a sprinkling of AI at best. Can you talk to me? How do you feel about that?
29:56
Unidentified speaker (UNKNOWN): I
29:56
Unidentified speaker (speaker_0): think it's actually extremely logical. Um, you know, like I don't think it's a distribution problem. I think enterprises have so much insane. incentive to adopt AI, and the people who have adopted it the hardest were the ones that were burnt the most. The argument I always make is that LLMs are unreliable, so they cannot be trusted for decisions with stakes to the company, but you can always
30:16
Unidentified speaker (speaker_0): make them make decisions with stakes to the user, because companies tend to not give a shit about their users, tragic, but that is what happens. So what ends up happening is you don't have AI deep in the logic. of your m uh apps. What you end up having is like the same B2B SaaS with AI attached to the side where it does not it cannot do like
30:36
Unidentified speaker (speaker_0): high-stakes things, but it can like shoot slop at the user because people kind of like it because it's responsive, and they make sure that all of the burden of checking like is this right or not is on the user's plate. This is actually extremely logical for the state of reliability. So I think that it is tragic that people wasted so many resources, like go doing the roundabout way to get
30:56
Unidentified speaker (speaker_0): to where we're at right now. And also I think that this is the right spot, you know, given how smart and reliable the models are. Aaron
31:02
Unidentified speaker (UNKNOWN): Powell
31:02
Unidentified speaker (speaker_2): Do the biggest benefits accrue
31:03
Unidentified speaker (speaker_1): to the incumbents who have distribution? When you look at a sales force of the world, and then you kind of reflect on what you just said about the tragedy of current state of software, do the benefits fall to those who have distribution or is there a new generation? who benefits accrue to?
31:16
Unidentified speaker (speaker_0): Number one, SAS Pocalypse is obviously not happening. Martin Casado coined it as a SASA palooza, and I love that. I don't necessarily think it means that big SAS wins. I think that the people who are best positioned to win are the people who care the most.
31:33
Unidentified speaker (speaker_0): AI companies in that they're actually familiar with the workflows. They have like the scale in order to like spend the investment up front to do the engineering, because it takes real engineering to automate stuff and think about all of the stuff. And the in order to like actually solve people's tasks. And that is the real valuable thing here. I don't
31:52
Unidentified speaker (speaker_0): necessarily think big SaaS will always be the winner, but I think that they are best positioned, not at all due to their distribution, but because they are the ones closest to the problems. They understand, hey, if we are at uh I don't know anyone about at Intuit, you know, like if we're in TurboTax, where could we change the interface to make it smoother? And there's not a
32:12
Unidentified speaker (speaker_0): chance anyone at OpenAR Anthropic knows how to do that, right? Like they are still living in their ivory tower and thinking that like software is just not going to exist as a concept. So they could be the big winners. You
32:22
Unidentified speaker (UNKNOWN): said
32:22
Unidentified speaker (speaker_1): they're about kind of knowing the processes, knowing the workflows and the importance of that. I've had Lynn on the show from Fireworks, I had Eric on the show from Modal, and they both said about the importance of owning your own intelligence as a company and the benefits of having this incredible proprietary data and these specialized models on a per company basis. Diogo, if the importance is in the process and
32:42
Unidentified speaker (speaker_1): the workflow, surely this is the holy grail, no?
32:46
Unidentified speaker (speaker_0): Oh man, how do I say this elegantly and Britishly? You know, like that's a very that's an opinion, I think, that goes in the current rails of how the current world works. The current world thinks about AI, where AI is trying to be posed as a coworker. So AI is being like presented as a coworker. This is like the direction Anthropic is really
33:06
Unidentified speaker (speaker_0): pushing towards, where it's kind of like the same interface as is like a human being. And I believe that this is going to be like the horseless carriages of the AI era. Just like you know, the in the industrial era, people were like, let's just make it look like a carriage, but like without the horse. I believe that the AI coworkers are a form of that. It just
33:26
Unidentified speaker (speaker_0): makes sense that if you're doing automation, automation will have a different strength profile than a human would be, and you'd want to lean into the strength of automation. Kind of like in the industrial revolution, uh, you don't need to wait for a whole coworker like a human. humanoid robot to do tasks. You just find the extremely narrow thing, like twist bottle cap on bottle, and then you automate it and it just
33:45
Unidentified speaker (speaker_0): gets solved forever. So I think that's how work will generally go. I think that if you assume that AI coworkers are the way to go, which I absolutely do not, and you assume that most companies have the willingness, capabilities, and desire to train them, because they need to be trained in distribution. for your company, then I think that owning your own
34:05
Unidentified speaker (speaker_0): intelligence direction can kind of make sense. I do believe that I would think that the labs' approach of like AI coworker, but general intelligence makes a lot more sense to me, because pretty much any company, if you gave them infinite interns that cost like a thousand X less, you could probably find a lot of productive use for
34:25
Unidentified speaker (speaker_0): them. And the f like we as a field should answer the question, why are we not getting that effect right now? And the people who say you need to own your intelligence, their excuse is, oh, you need something like continual learning or you need something more narrow. And I just don't think that that jives with reality. Aaron
34:41
Unidentified speaker (UNKNOWN): Ross Powell
34:42
Unidentified speaker (speaker_1): So Fireworks have helped several very top-tier brand And name firms in the valley. I think your Harvey's in and your ramps, and I think it's cognition. I might have got something wrong there. With building their own models, are you saying that is a mistake to pursue that route and horizontal or alternative models will be as good and it's basically
35:01
Unidentified speaker (speaker_1): a futile exercise?
35:04
Unidentified speaker (speaker_0): Oof. Oof, oof. It's complicated. I would 100% believe narrow models would so number one, benchmarks are incredibly misleading. And the people who are making these have like a lot of perverse incentive to look like their models are better internally and also from an external narrative perspective to not seem like they will be completely boned by the
35:23
Unidentified speaker (speaker_0): large models. I don't know how good they are. I would imagine that they could get cheaper for sure, and they could get more narrow. I think that this is kind of a sign of like maturity when like we're plateauing at a certain capability level. And I think that AI tends to
35:39
Unidentified speaker (speaker_0): increase in like more discrete steps. And these discrete steps occur when you start getting data in a particular distribution. Like I think it'd be hard to deny that the largest jump in coding agent performance happened around like November-ish last year. I can't remember which Opus that was. And that was because that is when they put it in distribution.
35:59
Unidentified speaker (speaker_0): And everything since then has been uh, you know, like we could talk about like how we love Fable 5.2 or Opus 5.5. It seems like it has been diminishing returns since then. And it's when you're in the diminishing returns that you can start doing like the micro optimizations like cost per token. My view of the world is that these are all like symptoms of
36:19
Unidentified speaker (speaker_0): not reliably solving problems. Like software engineers that I know, and I'm a software engineer too, we hate updating dependencies. We hate going into the latest and greatest stuff. The only reason we would do updates is because it didn't work sufficiently before. Otherwise, you would just like lock it away in the basement and only update it when it gets like way cheaper. And
36:39
Unidentified speaker (speaker_0): I believe that we have a lot of insufficient automation right now, and that is like what I am biased, of course, but that is what I think the data most suggests. And my view of the world is that because of JEV and JEV-like models, we will start like locking in real automation, and people will not want to update that.
36:59
Unidentified speaker (speaker_0): And the way companies get their own proprietary intelligence is not from having their own trained models, because it's hard to do that, but by having software around them. And the reason why it's so important for the narrative of companies like Harvey and Cognition to say that. they own their intelligence is because otherwise they're a very light wrapper on top of the LMs and
37:19
Unidentified speaker (speaker_0): it looks like they have no moat. But software and engineering is even if it wasn't a moat, it's like a phenomenal thing. You know, investing upfront to understand real problems, to automate them so they never have to be solved again, and humans don't have to do this roadwork. I think that that's just so good for the world. And I think that's a much more plausible way that the companies
37:39
Unidentified speaker (speaker_0): of the future progress. And like their proprietary intelligence happens at the code level. Kind of like how like there's no maybe there's some moat in like which database you use, but like the real meat of it is how you use those databases.
37:53
Unidentified speaker (speaker_1): Diego, I'm fascinated. I know the Harvey Legor market pretty well. Do you think they are a thin wrapper? I I I to me, it is a very specific device. domain that requires deep product functionality that can only be done in a vertical fashion. And that is why they exist and they're very successful. I'd push back strongly that they're a
38:13
Unidentified speaker (speaker_1): thin wrapper.
38:14
Unidentified speaker (speaker_0): Oof. So uh thin is all relative. Um they are much thinner than I uh I doubt uh that that that that sounds politically correct right now. Um it it's like it depends on how deep I would want it to be. And my understanding, last I checked with Harvey, I obviously can't even afford a license, so I don't fully know. Back when
38:34
Unidentified speaker (speaker_0): Harvey was working with OpenAI, they had this dream of like, oh, we want to automate larger and larger projects autonomously. That kind of stuff didn't end up working. They ended up having to pivoting into copilot for lawyers, which my understanding is they still are. And yes, you can do. a lot of domain-specific things as a copilot for
38:54
Unidentified speaker (speaker_0): lawyers, but like there's only so much engineering you can do around a copilot. Is it like the domain domain-specific MCP tools to pull up a rag that is based on like real law and all of that? And the reality is people want to automate like large chunks of work and not just make people like double digit percent more efficient, which I think is tends
39:14
Unidentified speaker (speaker_0): to be where a lot of AI uh lands. So, yes, like I think it's awesome they're doing a lot of engineering around it. I think that there's just going to be a lot, lot more engineering. Like, even system messages are a form of like really ghetto engineering where you're programming the LLM to do behaviors, but you have like no guarantees, no
39:33
Unidentified speaker (speaker_0): confidence, and like no real way to test how effective it is, other than like very weird evals. that people do these days. So I would like more engineering absolutely everywhere, especially things that are like AI adjacent. And I would actually hope that we can one day move past the copilots and just automate work that takes weeks to do in the background,
39:53
Unidentified speaker (speaker_0): and maybe that'll be very differently shaped than what work that current uh humans would do, because automation tends to require like an ad adaptation. on what is available technologically. Aaron
40:05
Unidentified speaker (UNKNOWN): Powell
40:05
Unidentified speaker (speaker_1): you said that uh earlier about the cost of tokens literally in at the beginning of kind of a two to three minute talk there. You said about how
40:13
Unidentified speaker (UNKNOWN): I let that oh
40:14
Unidentified speaker (speaker_3): shit.
40:15
Unidentified speaker (speaker_1): No, I love it. It's fantastic But the trouble is for me I have to remember parts of what you say. And my question really was going to be when you look at raw intelligence today, is raw intelligence per token actually improving? Like which hold it is?
40:29
Unidentified speaker (speaker_0): Ooh. Cam Kusan, your take on that. I actually want to write a blog post on this.
40:33
Unidentified speaker (speaker_1): No, I don't think anymore. I think it was dramatically. And I think raw and I think it it is, but like m minutely, like almost to the point where you wouldn't know. It's kind of like, you know, you meet a 180 IQ person than a 190, you're not gonna going to tell the difference. So yes, but not significantly.
40:48
Unidentified speaker (speaker_0): I actually think even the 180 to 190 IQ. person is an extremely generous analogy to what LMs are doing right now, because I wouldn't call them 180 IQ people or 190. I would call them occasionally, maybe it's 180, I don't really know, but like then sometimes you get like an 80, uh, or even worse. And my
41:08
Unidentified speaker (speaker_0): take is that it's actually not a single dimension. So number one, the benchmarks don't matter, so you we should toss them out. If you try to look at bench like dollar per benchmark over time, that's complete bullshit. Number two, I think that there's like a general notion of intelligence, which is kind of like pre-trained intelligence per dollar of compute, which to my understanding
41:28
Unidentified speaker (speaker_0): is barely moving right now. I do think that as you get more data and as you focus on different subsets of tasks, you can make them more in distribution. Back to the bitterest lesson. and you get way better performance within that data distribution. Like, are we, from an you know, intelligence per dollar perspective,
41:47
Unidentified speaker (speaker_0): massively moving the needle on coding? Super duper, but not because the base models intrinsically are more general, but because we're putting coding more in distribution, so it's more reliable on it. But like, has writing improved in the last nine months? I I don't think so. Maybe I'm writing with them wrong. But like there's many other capabilities where there's
42:07
Unidentified speaker (speaker_0): absolutely no improvement at all, and it becomes it's you know, because if you keep scaling up data in coding agent land, it's not going to improve lots of the other forms of data, right? There's diminishing returns to it. So I think it's a it's a complicated and nuanced.
42:23
Unidentified speaker (speaker_1): You said there about like the variance in terms of model capabilities, sometimes 180, sometimes 80. Can I ask you when we look at the different models, we see open source now on many different levels be as close as possible to a lot of frontier in in in a lot of respects. How do you see the open model landscape? And what
42:43
Unidentified speaker (UNKNOWN): do
42:43
Unidentified speaker (speaker_1): not many people know that they should know?
42:45
Unidentified speaker (UNKNOWN): Oh,
42:46
Unidentified speaker (speaker_0): that last question's a hard one. Um oh boy. You
42:49
Unidentified speaker (speaker_1): can say anything, I'm there. it's just me and you.
42:51
Unidentified speaker (speaker_0): Yeah. So I I think the thing that people should know is that benchmarks are total bullshit. All benchmarks are bad. There's perverse incentives. And ironically the people it hurts most are the people who do not have time to actually play with the models, because they're the ones who get manipulated by the benchmarks themselves with higher stakes. Including for
43:11
Unidentified speaker (speaker_0): everything open weight and open research. There are ways of measuring intelligence. It is very hard. Labs don't publicly talk about it. They have somewhat proxies internally, of course. It's a very unfortunate public state of affairs that there's so much misinformation out there, and people are not incentivized for actually
43:31
Unidentified speaker (speaker_0): releasing their own best measures of intelligence, which I think is tragic.
43:35
Unidentified speaker (speaker_2): Diego,
43:36
Unidentified speaker (speaker_1): who who bullshits the most?
43:38
Unidentified speaker (UNKNOWN): Who
43:38
Unidentified speaker (speaker_0): bullshit? bullshits the most?
43:40
Unidentified speaker (speaker_1): You said spice on ice. I'll give you spice.
43:43
Unidentified speaker (speaker_0): The top-tier labs are incentivized to be truthful. So I believe that OpenAI and Anthropic are mostly truthful. The second-tier labs are incentivized to lie to themselves. They want to think that they can become a first-tier lab. They have lots of internal pressure to benchmark Max, but they cannot. admit it to
44:03
Unidentified speaker (speaker_0): themselves. Otherwise, you know, like their org chart will go ballistic. This is what happened with Llama 4, where they were actually too on the nose about benchmark maxing, even though there's so much internal incentive to do that. Same thing for Google right now, and same thing for a lot of the other Chinese labs. Like I think that they're trying to aim for the top, and therefore
44:23
Unidentified speaker (speaker_0): they are not trying to explicitly benchmark Max. I think a lot of the people fighting for like consolation prizes are just doing completely insane levels of benchmark maxing. So one domain, if I were to be like a little spicy here, would be the rough domain of American open source leader. It's enough of a like a side uh
44:43
Unidentified speaker (speaker_0): consolation prize that they're not aiming for top tier. And that is pure bullshit and in optics because no one actually don't actually use those models. So who's gonna call them out? And the decision model landscape is so much worse, in my opinion, too, because no one even understands
45:03
Unidentified speaker (speaker_0): what good performance is. So they are all like benchmark maxing to the absolute gills. And it's kind of uh really good for us because no one will believe those benchmarks. It gives a lot of power to our. us over the benchmarks itself, which I think is the optimal for the field. Like in the th in the ideal world, no one ever cares about benchmarks on any machine learning model,
45:23
Unidentified speaker (speaker_0): string or system one. But there's so much of that right now.
45:27
Unidentified speaker (speaker_1): Let's unpack this, dude. Top tier one labs have uh a duty to be truthful or it's in that impact- No,
45:33
Unidentified speaker (UNKNOWN): sorry, it's it's not a
45:34
Unidentified speaker (speaker_0): duty. They have a they do not need to cheat in order to be high up there. And they want to truly believe in like morale is very important. They truly want to actually be top tier, and they can be. So they don't need to cheat, is the answer.
45:57
Unidentified speaker (speaker_1): This company forward, Whether it's reflection or or poolside or thinking machines. Do you feel that we're we do need to propel an American open source model?
46:06
Unidentified speaker (speaker_0): Who is we and what is propel?
46:08
Unidentified speaker (speaker_1): I think there's a consensus that Chinese open source models are very efficient. And unless we have an alternative to them, Chinese open source models will be considerably used and threaten some elements of American security.
46:23
Unidentified speaker (speaker_0): So it's it's actually interesting to dig into the threat model. There's definitely a threat if you run them vanilla as how they've been post-trained. But if you're actually using um I can't remember what you've how you described it, like you know, like the private intelligence of companies. In theory, if you're training the models, you can remove a lot of bias to different extents. But
46:43
Unidentified speaker (speaker_0): like in theory, this type of thing is totally doable. So it actually depends on like is the threat model like literally plug and playing these models? Aaron
46:49
Unidentified speaker (UNKNOWN): Powell
46:50
Unidentified speaker (speaker_0): So
46:50
Unidentified speaker (UNKNOWN): you don't think that
46:51
Unidentified speaker (speaker_1): threats from from Chinese open source models in terms of dangerous prompt injections that you don't know about, backdoors?
46:58
Unidentified speaker (speaker_0): I would not use a Chinese model in my coding agent vanilla, just to be clear. But like that again is one threat model. If you live in a world where coding agent is the only shape of AI, then sure, that is very much a threat. But the alternative we need to like take in into account is how good are the American open source
47:18
Unidentified speaker (speaker_0): models and can they catch up to the Chinese models? Because my understanding is they hold themselves up to a higher ethical and legal baseline. And it might be very hard to catch up to those models because of that. And if the models are actually not useful, like it really depends on like how much of a difference it makes to have a slightly smarter model, which is going to be hard
47:38
Unidentified speaker (speaker_0): to forecast. especially as things are plateauing. But it might be a fool's errand if they cannot be competitive. That is going to have to be like an open question for the field if you just assume the extremely narrow view of plug them into coding agents literally as is. If you imagine a world where there's many other alternatives, like hey, we can measure
47:58
Unidentified speaker (speaker_0): bias, we can have threat models for bias, we can train the bias out, we can put guardrails on it, you know, like this is actually a huge use case for us right now, which is JEV classifying if the tool calls are safe or are they phoning home to China or whatever else. Like, there's so many other ways to get security that I think that everyone has just
48:18
Unidentified speaker (speaker_0): been in this like mono model like horse blinders for so fucking long. And like I I just reject that premise originally. Aaron
48:26
Unidentified speaker (UNKNOWN): Powell
48:26
Unidentified speaker (speaker_1): Do you think American open source models will catch up? Aaron
48:29
Unidentified speaker (UNKNOWN): Powell
48:29
Unidentified speaker (speaker_0): In terms of true intelligence, my guess is no, unless China decides to stop trying, not due to capabilities, but due to ethics. But it might not make a difference. Like if the field plateaus hard enough, like maybe like that X percent better doesn't matter. Aaron
48:45
Unidentified speaker (UNKNOWN): Powell Do
48:45
Unidentified speaker (speaker_1): you think we're right to have the export ban with China? Aaron
48:47
Unidentified speaker (UNKNOWN): Powell
48:48
Unidentified speaker (speaker_0): Oh boy. My understanding is like there's different parts of the uh I don't know if you mean like the GPU expert ban? GPU
48:53
Unidentified speaker (UNKNOWN): export banana.
48:55
Unidentified speaker (speaker_1): Because they're showing on some of the latest deep seat models and some of the latest Chinese models, they were in fact actually changed with the latest black wells.
49:03
Unidentified speaker (speaker_0): Oh yes, that was uh Will DePew's result. I've been talking about that with my friends. Though there's some spicy stuff online. What I heard online was that using the same technique, you could tell DPS. DeepSeek 4.1 Flash wasn't trained in that way. And maybe that is consistent with the field. I don't know. My job is to navigate all of the
49:23
Unidentified speaker (speaker_0): insane hardware bubble neo cloud craziness in a way that we end up getting the future we want. And because we are so efficient, we can be in like the the sidelines and not be like a main player in the GPU stuff and hopefully ride up. out whatever crazy stuff is happening right now. Aaron
49:40
Unidentified speaker (UNKNOWN): Ross Powell
49:41
Unidentified speaker (speaker_1): The crazy stuff that's happening, you mentioned that and neo clouds. Dude, it's a wild space in the neo cloud layer. What happens in the next twelve months in neo clouds that people are not contemplating or seeing?
49:52
Unidentified speaker (speaker_0): Well, I've believed that AI has been a bubble for a while. So I would like to keep planning for an eventuality of bubble burst.
50:00
Unidentified speaker (speaker_1): What will happen in the bubble burst? Because to me, it's not going to be a supply side, it's just gonna be a demand side. And we're seeing no demand side shock. And so what will cause a bubble burst?
50:12
Unidentified speaker (speaker_0): Well, oh man. Okay, uh we we can get into that. That's a spicy one. Coding agents have definitely changed the game for AI, but this argument has been made for many years now. And actually, I think it's very interesting when you combined many different exponential variants type things. I believe that some amount of demand is caused by
50:32
Unidentified speaker (speaker_0): FOMO. FOMO is caused by people's thinking that they have good results and thinking that they have to like go all in on this type of thing. I do believe demand can burst quite quickly when people start facing the realities of what AI can do. My experience of a lot of the world of like people spending money on AI is that they do not have a full
50:51
Unidentified speaker (speaker_0): understanding of AI's capabilities and they're quite miscalibrated and actually quite overestimating of what AI can do. I think that's the kind of thing that can cause a demand shock. So I think that there's just been so much overpromise under deliver in the AI space.
51:06
Unidentified speaker (speaker_1): That goes against everything that people say, which is that if model progression did not improve for a significant amount of time, there is still so much value left for humans to extract. that they haven't extracted yet. That kind of goes against what you're saying.
51:19
Unidentified speaker (speaker_0): I believe that the people who say that are the people who try to extract value ab of out of existing models. So there's a little bit of bias into that. I don't disagree with the spirit of what that said, that says, which is, oh my God, the models are so fucking smart. But again, we need to look at I agree that they're so fucking smart. I just think that like their interface.
51:39
Unidentified speaker (speaker_0): and like how that smart is unearthed is suboptimal. And again, we need to look at the flip side of the people who went way too all in into AI and got burned by it. There's a lot of companies who are like, let's fire a lot of our workforce because AI is solving all of these problems. And they need to like backtrack on that. And that is like a lot of long-term ramifications
51:59
Unidentified speaker (speaker_0): to like the future of their companies. And those people try tried to not just extract the maximum or overplay their hand. They tried to like actually just bet logically with the reality that AI can do a lot more. So I actually think that that reality is very nuanced. Or that story is very nuanced on whether or not people have
52:19
Unidentified speaker (speaker_0): the capability of extracting a lot of that value, even if the capabilities pause.
52:23
Unidentified speaker (speaker_1): I think a lot of people look at the margin structures of a lot of these companies and say that it's bubble-like. Do you have good margins?
52:29
Unidentified speaker (speaker_0): We do, yeah.
52:30
Unidentified speaker (speaker_1): How do you get to scaled revenue, dude? Like you're you're so cheap, which is brilliant. It cost is a huge benefit of using Jev. But the the scale required to get to a $10 billion revenue business is just enormous. Yep,
52:45
Unidentified speaker (speaker_0): yep.
52:46
Unidentified speaker (speaker_1): Yep. How do you help me get that? there?
52:48
Unidentified speaker (speaker_0): Uh Devon's Jev's paradox, right? Like, and and the way Jevon's paradox tends to work, it's not like just magic, you make it cheaper and it gets better. That's a like an insane way the world would work. The way it should work is, hey, I have this budget for AI. Oh, Jev makes it that much cheaper. Now I can make this project
53:08
Unidentified speaker (speaker_0): work. And ideally it works so reliably because Jev's also smarter on the axes that matter for automation, that now I can do these other projects.
53:15
Unidentified speaker (speaker_0): or I can throw intelligence at things I would never throw intelligence at. And even then, if you're starting to throw intelligence at things you haven't thrown thrown intelligence at before, that would not make you increase your budget. And Jevon's paradox means increase the budget. What you would need to see is so much value from those projects finally succeeding, which AI tends to fail at,
53:35
Unidentified speaker (speaker_0): that you're like, hey, this actually works. Let's keep betting until the budget matches what it was before and even increases beyond that. So this is why reliability of intelligence is so important. And what I like to say is in the grand metrics that truly matter, which is real automation, Jev, Fable, and Astra are all tied at roughly 0%.
53:55
Unidentified speaker (speaker_0): Like, yes, there's You
53:57
Unidentified speaker (UNKNOWN): know,
53:57
Unidentified speaker (speaker_0): a like couple of little bits here and there, but like we have so freaking much automation to do that I think
54:02
Unidentified speaker (UNKNOWN): that's a good thing.
54:03
Unidentified speaker (speaker_0): that uh like l like you know, like we're trying to make an industry that currently doesn't exist, which is run in the background AI. And it would be clearly user than the whole user. Is that uh bigger? Um
54:16
Unidentified speaker (UNKNOWN): than
54:16
Unidentified speaker (speaker_0): larger, thicker than uh coding agent, uh assisting Existing software engineers, uh making software like twenty percent more efficient to write or something like that.
54:26
Unidentified speaker (speaker_1): How do you analyze the coding agent market today? You see cognition, fourth place so far, be a one point six billion dollar revenue business. The depth of this market is so large. How do you analyze this market today? Um what does that look like in three years' time?
54:42
Unidentified speaker (speaker_0): Who's third place?
54:43
Unidentified speaker (UNKNOWN): Well,
54:43
Unidentified speaker (speaker_1): you've got claw code, you've got codecs, Codacs, you've you got cursor, you
54:46
Unidentified speaker (UNKNOWN): got
54:47
Unidentified speaker (UNKNOWN): cognition. Yeah,
54:48
Unidentified speaker (speaker_1): you've You got factory.
54:49
Unidentified speaker (speaker_0): If I will be totally free. frank, I don't analyze the coding agent market. I actually personally want Jev to be used for coding agents, and I think there's very cool sci-fi that could be done. I actually think that the further away we stay from the coding agent market, the more that they keep doing their little fight and we can keep focusing on automation. So from a
55:09
Unidentified speaker (speaker_0): company perspective, it might even be better in short term if Jev Jeff had absolutely no coding agent. uses in order to make it a little bit easier. It's a little bit too late for that, but I can't tell. But I don't analyze the size of that market. I think that it will be a very large market. Will the market be much larger than it is today? I don't know because
55:29
Unidentified speaker (speaker_0): FOMO really moves the needle on these things. I've been around in AI long enough, and surely you have too. You must have seen, like, AI looks like this exponential upward, but it's actually been many. up down, up, down, up, down, up, downs. And each FOMO of the previous up adds to the next up thing. But like a lot of them had just like crashed down. And I think it would be
55:49
Unidentified speaker (speaker_0): kind of arrogant to assume this is the one thing that will just be up, up, up, up.
55:53
Unidentified speaker (speaker_1): I so remember like 2015, 2016, there was like this real like hype for about a year and there was like Bloomsbury AI, message.ai,
56:03
Unidentified speaker (speaker_1): and like a load of these companies. There was exactly that. So yes, I completely get you there. We said about the bubble light moment. With the greatest of respects, you raised a shit ton of money.
56:14
Unidentified speaker (speaker_0): Sure. Um
56:15
Unidentified speaker (speaker_1): tell me about that round. Was that just like, you know, stop off at one VC, collect term sheet, stop off Up at another VC, collect term sheet? sheets. Ah, I've got 28 twenty-eight term sheets. That's that's how did that round go?
56:28
Unidentified speaker (speaker_0): It was a very, very confusing time for us. It was kind of an insane time. Oh my god, did we learn so much about VCs themselves?
56:37
Unidentified speaker (UNKNOWN): What
56:37
Unidentified speaker (speaker_1): did you learn?
56:38
Unidentified speaker (speaker_0): How technologically advanced the VC's portfolio companies were. Basically it was all inbound. We did not have a deck. We we had to tell people like, hey, we don't have time for this deck, and it doesn't seem anyone else needs this deck. You tell us about the market signals you're seeing because everything's on fire right now and we don't really have the time to pull these things up for you.
56:58
Unidentified speaker (speaker_0): And it was so sick to see how it evolved for different VCs over the course of the first days and weeks. Like how their excitement changes, how they are telling us, how their portfolio companies are using us. We don't snoop on their data, so we actually don't know what they're using it for. And it's so cool to hear them about like, holy shit, you guys are the real deal, you're crucial,
57:18
Unidentified speaker (speaker_0): this is the future. That was like a really fascinating additional s I honestly I miss that signal on like the state of the field. And it was so fucking cool to hear about who's using us and why and how. And that is what enabled the round to work out.
57:34
Unidentified speaker (speaker_1): Did you start with 750 as the ask at seven and a half, or did it move up?
57:38
Unidentified speaker (speaker_0): We it it it moved around a lot. When we even started the conversations, we did not know what we were really looking for in the round. What we ended up deciding on is like, hey, we don't actu it this is maybe like kind of crazy because we did raise a lot of money, but we realized that like we don't need to valuation max. We don't need to do like a cock tranche or anything
57:58
Unidentified speaker (speaker_0): like that. We don't care about that. We don't need we actually don't need a fuck ton of money either. What we need is allies to fight the largest companies in the world. Let's find the best people to like be part of the team. And that's what Rhe really optimized for. The rest to me was like more in the micro optimization territory versus discovering
58:18
Unidentified speaker (speaker_0): like, hey, we want like the smartest possible money, the best allies, the people who can help us navigate the rocky waters ahead, the the rough waters ahead. And
58:28
Unidentified speaker (UNKNOWN): it
58:28
Unidentified speaker (speaker_0): was I think we're optimizing from very different things from other companies because like we are profitable. We are, you know, money in the bank is going up right now. Quite frankly, a company that's raised as little as we'd raised before should not have a plan to spend, you know, whatever we raised right away.
58:44
Unidentified speaker (speaker_1): I can't believe cuck tranche traunch is actually a word these days, but let's roll with it. Um Wait,
58:49
Unidentified speaker (UNKNOWN): wait, wait.
58:52
Unidentified speaker (UNKNOWN): I
58:52
Unidentified speaker (speaker_1): do. It's completely fucked, and it makes
58:53
Unidentified speaker (UNKNOWN): me just
58:54
Unidentified speaker (speaker_1): Yeah, I absolutely
58:58
Unidentified speaker (UNKNOWN): that came
58:58
Unidentified speaker (speaker_0): up for us. We had so much demand for that. Um and people were like even saying, like, if if you're gonna be raising at 3x the valuation, count me in. And the thing is, we didn't believe that.
59:09
Unidentified speaker (speaker_1): That's a bubble. That's bullshit. Like, I'm sorry, I lo I do I really like like a tw someone who's like YOLO, Twitch. 21 billion. I don't care. You know what? You just need to get some self-esteem, because that is like sitting and watching.
59:22
Unidentified speaker (speaker_0): We don't need the money or the valuation.
59:24
Unidentified speaker (speaker_1): Why did you choose Sukura and Andreessen? I love them, but like why did you choose them?
59:29
Unidentified speaker (speaker_0): Oh man. Oh man. It's gonna s seem like I'm sucking up, but I love them a lot. And Dreesen is I am like kind of insane about technology and techno optimist. I think A6DZ is just like the embodiment of that. You know, like they are as philosophically aligned as we can be. I feel like most VCs are they are, from
59:49
Unidentified speaker (speaker_0): a strategic perspective, lovers of technology, but they are financial institutions that care a lot about how to play their financial cards. While Andreessen, And Driesson deep into the firm, like they care so, so, so much. Like that podcast I did with like Martin Casado and Ben Horwitz, I had so much fun. It felt like we were like old friends talking
1:00:08
Unidentified speaker (speaker_0): about from our hearts. I mean, maybe they're so good that they can convince me and they've sold me like on like some fake ass thing. But I don't believe so. I think that they just are so philosophically aligned. And I think like what led us to where we're at right now, and I hope we can keep being this way, is like wearing our heart on our sleeves about what we care about, and just blazing our own path in the AI
1:00:28
Unidentified speaker (speaker_0): field. And I would love to act in accordance with that and like have them on board. Obviously, Bartite is brilliant. And actually, to me, Sequoia, if I'm like gonna be totally frank, oh man, how do I say this? I didn't want their money, I wanted Sonia Wang. That is why Sequoia is around. I did not get to know Sequoia super well as a fund, but I love
1:00:48
Unidentified speaker (speaker_0): Sonia as well. And I just love that.
1:00:51
Unidentified speaker (speaker_1): Help me help me understand. I I want to be the best ambassador. What did you love so much about Sonia that made you choose her over everyone else?
1:00:58
Unidentified speaker (speaker_0): Oh man. I would say that she is not just brilliant, but a very unique type of brilliant that I think we extremely would want on our side. So it's that particular shape of brilliance that I find that no matter how many advice. advisors I would talk to, if I have a conversation with Sonia, like
1:01:18
Unidentified speaker (speaker_0): I would get a different, very interesting perspective that should be part of our strategic cards. I care a lot about strategic thinking for myself. One of my spicy type safe values, you know, is make d great decisions slowly, very non non-startu, startup, but I think that that is just like how it should be done. And
1:01:39
Unidentified speaker (speaker_0): she is an incredibly valuable thought partner there.
1:01:42
Unidentified speaker (speaker_1): Make great decisions slowly. What bad decision did you make fast?
1:01:46
Unidentified speaker (speaker_0): I would say I probably make decisions slower than most other people. My sense is a lot of the business world is on autopilot. I hate working with traditional business people. Like they just do things and I'm okay with doing things if it doesn't matter how well they are done. But like for important things, I actually care a lot about the why.
1:02:06
Unidentified speaker (speaker_0): What's the philosophy? What's the framework? How is this done? And whenever I work with traditional business people, I feel like I'm being driven crazy because they never do things so well that I don't need to worry about it. And when I do worry about it, I need to understand the why. And I'm like, oh, this person's person just doing random shit. I I think having a plan actually matters quite a bit. There's so many
1:02:26
Unidentified speaker (speaker_0): things. I would spend way more time thinking on. I wish I could spend more time thinking about like our data and our data roadmap. I want to think more about product. I want to more think more about like, you know, how to be more trustworthy. I want to think about more about like every single role we hire for to make sure that we get like the best possible people for these roles. I think that I should acknowledge
1:02:46
Unidentified speaker (speaker_0): that I don't have time for all of these things and that it's so fucking important to like think, think, think.
1:02:50
Unidentified speaker (speaker_1): How hard is the hiring market today? You have to get the best people on your team. How hard is it actually?
1:02:56
Unidentified speaker (speaker_0): Oh man. I will say that the hardest part is hiring for our data demon team. You know, back to Bitter's lesson. When you have a new task, you have new type of data. You know, you need a new shape of data in order to make it work. We basically have our own little subfield of like it's like kind of like imagine like a, you
1:03:14
Unidentified speaker (UNKNOWN): know,
1:03:14
Unidentified speaker (speaker_0): the best. lab in the world, but only internal to a company to make data the way we make data. And I would think that this should be the easiest thing in the world to hire for because it's so unbelievably cool. And also we have such a hard time hiring for it because no one knows how to get into it. You know, we keep getting the wrong type of people applying. That to me
1:03:34
Unidentified speaker (speaker_0): is the thing that keeps me up at night these days: of like, how can we convince like these brilliant people? who don't even think they can do like they can push the boundaries of ML because they're not an ML researcher to work at this role where we're literally have a subfield where you'll get tutored by the experts in the world at this. And it's gonna be so valuable because you actually make the model smarter.
1:03:54
Unidentified speaker (speaker_0): And you know, like I don't know how to communicate that to the world. And that to me sucks. Everything else is like
1:04:00
Unidentified speaker (UNKNOWN): It's the
1:04:00
Unidentified speaker (speaker_1): comp wild. We we had Eric at Model say this. that he has people who are he has to fight $20 million packages for some grads who are utterly brilliant. Are the comp packages just unprecedented?
1:04:11
Unidentified speaker (speaker_0): Not for us. So happy that we got clarity on this. Like we got real clarity that we do not want mercenaries. We want to be like good, but not like, you know, like the insane lab ones. I think that that feels great. Anthropic, I think, is suffering a little bit from. their over mercenaries lately, because like they want missionaries,
1:04:31
Unidentified speaker (speaker_0): obviously, but also they pay more than anyone else. So like there's a lot of incentive for not just mercenaries to join, but like fake their culture interviews, which there's like an entire market now to prepare people for anthropic culture interviews. I am so glad that we decided to not do that. We want to pay people like good amounts. That is not the highest thing ever. And we
1:04:51
Unidentified speaker (speaker_0): are just super happy where we're at, algorithm-wise, of like this is the kind of thing that takes care of people and also filters out the the hired guns.
1:05:00
Unidentified speaker (speaker_1): Final one for you. If you could bake one character trait into a potential hire, what would it be? For my company, it would be obsessive. Like I
1:05:09
Unidentified speaker (UNKNOWN): need
1:05:09
Unidentified speaker (speaker_1): someone to be obsessive about their craft, addicted to making it better. They can't sleep at night. It actually compels them in every way. The ability to become obsessed with something.
1:05:20
Unidentified speaker (speaker_0): I don't know if that's the same for every role, but if I had to pick, I don't know a better word for this, uh, and it sounds a little sexual, but I we don't mean it in the sexual way. Technophilia. Now non-sexual, just to be clear, non-sexual. Okay, relax. But like we truly love technology. That's the kind of thing that's like so beautiful. in a bubble, and it's the kind of thing I want in
1:05:40
Unidentified speaker (speaker_0): every single role. So we can talk about technology at lunchtime or when we're hanging. I want people around me that I like, and I want us to all be like this one big team working towards like constant, constantly pushing the boundaries of sci-fi, because that is how we we actually ship technological shock and awe. Again, non-sexual, non sexual. I don't know a better word.
1:06:00
Unidentified speaker (speaker_0): Technophile, techno yeah, yeah. Yeah, just to just to be clear. uh if if you guys edit this out, that'll be very embarrassing. But uh yeah.
1:06:07
Unidentified speaker (UNKNOWN): Well
1:06:07
Unidentified speaker (speaker_1): don't worry, we'll we'll keep it and no no sexual. We're gonna do a we're gonna do a quick fire around this. So I say a short statement, you give me your immediate thoughts. Okay,
1:06:15
Unidentified speaker (UNKNOWN): I
1:06:15
Unidentified speaker (speaker_0): I tend to not do quick fire as a warning. I'll try.
1:06:18
Unidentified speaker (speaker_1): What have you changed your mind on most in the last twelve months? How
1:06:21
Unidentified speaker (speaker_0): much hype matters.
1:06:23
Unidentified speaker (speaker_1): How much hype matters? Yes.
1:06:25
Unidentified speaker (speaker_0): I I think it matters much less than I thought. Like when Jev launched, it's like, holy shit, hype matters a lot. And then other stuff launched, and they were like benchmark maxing to the gills. And when we talk to real people, everyone is like, yeah, we tried them. They suck. No one cares. We can tell already that you are the best and you care the most. And I was like, wow, this is so convenient. We can just do what we
1:06:45
Unidentified speaker (speaker_0): love most, which is under underpromise, promise, over deliver. This is the best world ever. I would not have expected this at launch time. and this is so awesome for us.
1:06:52
Unidentified speaker (speaker_1): What is the biggest belief you hold that the smartest people at Frontier Labs would disagree with you on?
1:06:58
Unidentified speaker (speaker_0): How much AI can do today and how real important reliability is.
1:07:04
Unidentified speaker (speaker_1): Why do you not hire from OpenAI?
1:07:07
Unidentified speaker (speaker_0): My sense is that But there's a certain selection bias you need to do to get into open OpenAI AI that is kind of like an anti-signal. where I feel like you need to be kind of in a particular Kool-Aid, number one. And number two, when I say this, I mostly say I don't hire researchers from OpenAI. Because the researchers at OpenAI, by
1:07:27
Unidentified speaker (speaker_0): construction, love to do algorithms and then scaling. They tend to hate data. Data is a very low status thing at most labs. And obviously, they don't think about the right task because that is a thing that is the like quite rare. There's maybe like three people at OpenAI who think about the right task, and I wouldn't hire them either, not because they aren't fantastic, but
1:07:47
Unidentified speaker (speaker_0): because if you're obsessed with the right task, why would you want to do my right task? Like my right task is machine-native AI. If you're obsessed with like making text to image a thing, which Aditya did do at OpenAI, like holy shit, he invented an entire field. That is so fucking cool. He is so unbelievably good. And also like you you can't like
1:08:06
Unidentified speaker (speaker_0): transplant him into another task. Like he should just be cooking on his sci-fi too, because that is what you do when you're like so good at that. So by construction, the all that's left of the ML research that could be hireable are the ones who obsessed with algorithms. That to me is the kind of thing to do when you want to throw money in the fire fur furnace.
1:08:24
Unidentified speaker (speaker_1): What is the most overhyped category today?
1:08:27
Unidentified speaker (speaker_0): My instincts tell me robotics, not because robotics is not cool, but I just think that if you look at the grand scheme of reliability, they're just very far away from it. If you imagine what are the things that need to be nailed first, there's like tons of prereqs that still haven't been nailed. And I think robotics is the furthest away from that. Aaron
1:08:44
Unidentified speaker (UNKNOWN): Powell
1:08:44
Unidentified speaker (speaker_1): What is the It's most underhyped?
1:08:46
Unidentified speaker (speaker_0): This direction of software research. People playing with AI plus software to make new things, especially new interfaces. I think that that's so unbelievably underhyped. It's like tragic.
1:08:58
Unidentified speaker (speaker_1): Final one for you. What do you believe today that looks crazy? Okay, looks crazy, but will be incredibly common in five years' time.
1:09:08
Unidentified speaker (speaker_0): I think that principal principle agent problems are abound in the world. You know, like when you have a principle, like another an electorate being represented by an agent that is sometimes misaligned with it, like a representative. And why can't the electorate directly represent their desires and have it behave more like a true democracy? This is just like one contrived example, but it
1:09:28
Unidentified speaker (speaker_0): happens all over the place. People reporting to you, stockholders to companies. There's so many principal principle agent problems abound. My view is that we will find way better solutions to all of these. And it's going to be crazy that we allowed the level of misalignment that we have today. That will be such a such a sweet world where we can start enforcing
1:09:48
Unidentified speaker (speaker_0): the letter of the law instead of the spirit of the law instead of the letter. And AI is basically that. Like the the law is kind of like writing software. You know, it's like writing like a detailed algorithm to follow in order to make it fair and reproducible. And it was just missing the primitives to specify the spirit. And I think that this will really unite society.
1:10:08
Unidentified speaker (speaker_0): Because I think even for our most controversial topics, I think that a lot of people can unite on the spirit of things. It the problem is that the spirit can't be enforced, so you need to put the letter, and the letter then has a ton of disagreements.
1:10:21
Unidentified speaker (speaker_1): Do you know my problem with you, Diogo? It's that you just don't have much energy.
1:10:24
Unidentified speaker (speaker_0): My my bad, my bad. There's a reason that they call me Cocaine Laboo Boo on Discord.
1:10:29
Unidentified speaker (speaker_1): Dude, this has been so much fun. Thank you
1:10:31
Unidentified speaker (UNKNOWN): so
1:10:31
Unidentified speaker (speaker_1): much for bringing the energy and for being so willing to be so open.
1:10:35
Unidentified speaker (UNKNOWN): Hell
1:10:35
Unidentified speaker (speaker_0): yeah. It it was tons and tons of fun. It's uh it's like we we should stay in touch. It's also it's definitely was very spicy. Spicier than I intended.
1:10:44
Unidentified speaker (speaker_1): But before we leave you today, Blitzy, the autonomous software development platform that's built for enterprise scale. is proud to sponsor today's show. Enterprise codebases have grown beyond human comprehension. Tens of millions of lines, decades of tech debt, complexity existing tools cannot fathom. Blitzy starts by understanding. Thousands
1:11:04
Unidentified speaker (speaker_1): of agents reverse engineer your codebase into a proprietary knowledge graph that maps architecture, dependencies, and business logic. That graph surfaces every CVE and zero-day hiding. in your code before anyone hands you a ticket. Grounded in that context, Blitzy then executes entire epics autonomously, modernize off-legacy
1:11:24
Unidentified speaker (speaker_1): systems, ship new features 5x faster, or batch remediate CVEs at scale, surfaced by the platform. Blitzy not only generates code, it validates and an end-to-end tests every single line. And that's why Fortune 500 Enterprises. are shipping enterprise grade code five times faster. Go to
1:11:43
Unidentified speaker (speaker_1): blitzy.com forward slash 20vC to learn more. While Blitzy helps you build faster, Flex helps you run your finances smarter. Business owners run their whole financial life on Flex. One platform from business revenue to their personal spend. Float every purchase for 60 days.
1:12:03
Unidentified speaker (speaker_1): Tap capital that grows with your revenue and pay vendors in 170 countries across 32 currencies. Plus the whole back office. Bills, expenses, accounting, all in one place. So you spend less time reconciling and more time growing. That's why thousands of owners use Flex, named one of
1:12:23
Unidentified speaker (speaker_1): Fast Company's most innovative companies of 2026. Visit Flex. Flex.1, that's F L E X.on e and use the code 20VC. While Flex runs your finances, Vanta keeps your compliance covered. When you're building a company, you learn that trust is what closes deals. You
1:12:42
Unidentified speaker (speaker_1): may have the best product, but no buyers will sign these days without proof of your security. Here's what happens if you're not prepared a prospect asks for proof of compliance. The deal stalls when you scramble, your engineer gets pulled off the roadmap to audit prep, every enterprise conversation turns into this horrible fire drill. That's where Vanta comes in. Vanta is the leading
1:13:02
Unidentified speaker (speaker_1): agentic trust platform that not only gets you compliant fast with frameworks like SOC 2, ISO 27001, HIPAA, and GDPR, but keeps you compliant by continuously monitoring your controls. So your deals can keep moving and your engineers really keep building. Now access the Vanta agent everywhere you work, even if your team lives in Claude
1:13:22
Unidentified speaker (speaker_1): or Cursor. And that's why Vanta's trusted by more than 16,000 companies like Ramp, Harvey, and Writer. So prove you're ready for business and get $1,000 off Vanta. When you go to Vanta.com forward slash 20 two zero VC, that's vanta.com slash 20vc. twenty vc.
This page was produced automatically: the audio was transcribed with nvidia/parakeet-tdt-0.6b-v3+sortformer-v2.1 and interpreted by Qwen3.6-35B-A3B-UD-IQ3_S.gguf. Transcription and interpretation can contain errors; always confirm against the original episode. Nothing here is investment advice.

More from The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

20VC: Taking on NVIDIA: Fractile CEO on the Future of Chips | What Everyone Gets Wrong About Capex Spending | The Winners and Losers in Neo Clouds with Walter Goodwin

Episode discussion: Walter Goodwin on Fractile's 3D-stacked memory chips, AI infrastructure bottlenecks, and NVIDIA's future valuation

Speakers’ claims: Walter Goodwin, CEO of Fractile, argues that frontier AI intelligence is not yet commoditized, distinguishing it from general cloud compute. He positions Fractile's 3D-stacked memory architecture as a solution to the memory bandwidth bottleneck that limits LLM inference speeds. Goodwin contends that land and power, rather than silicon, are now the primary constraints for AI infrastructure, favoring energy-efficient designs. He predicts NVIDIA will reach a $10 trillion valuation within five years, driven by the finite nature of data center power and the need for chip replacement cycles.

SemiconductorsAI InfrastructureVenture Capital
1h 16m3 assets25 claims to verify
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

20VC: Jensen Huang Investigation: What Happens Now | Sequoia Leave London, Close Scout Program and End Evergreen Fund | NASDAQ Loses $2.5TRN in 24 hours | Anthropic targets $1.4TRN IPO valuation

Episode discussion: Sequoia winds down Evergreen fund, NVIDIA faces inquiry, Anthropic targets $1.4T IPO

Speakers’ claims: Speakers analyze Sequoia's strategic shifts, including exiting London and ending its Evergreen fund structure due to LP liquidity conflicts and operational complexity. They discuss the NVIDIA board's investigation into CEO Jensen Huang regarding compliance and personal conduct, while noting the Nasdaq's recent $2.5T daily loss as reactive rather than fundamental. The episode highlights Anthropic's $1.4T IPO target and OpenAI's revenue growth, contrasting public market advantages with private capital constraints. Additional segments cover AI security breaches, M&A trends, and the potential obsolescence of seat-based SaaS models like Figma in favor of agentic design.

Venture CapitalNVIDIAAnthropic
1h 7m5 assets22 claims to verify
All insights