Invest with AI

Robert Yang discusses AI agents evolving from Excel tools to autonomous hedge fund research teams

1h 16m Listen to the original
AI AgentsHedge FundsFinancial ModelingProduct StrategyMarket MicrostructurePost-TrainingHuman-AI Collaboration

Summary

Robert Yang, CEO of Fundamental Research Labs, outlines the evolution of AI agents from general-purpose tools to specialized financial analysts. He explains that early consumer agent products failed due to low retention, prompting a pivot to Excel-focused agents that achieved viral traction but required rigorous benchmarking for enterprise adoption. Yang predicts digital hedge fund analysts will be widely adopted within 6-12 months, with human roles shifting toward portfolio management and system building. He emphasizes that current models lack 'spine' and contrarian views, necessitating post-training and 'skills packs' to instill specific investment theses. The ultimate goal is a multi-agent system that performs research and paper trading, acting as a 'Jarvis suit' to help humans maintain alpha in distorted markets.

Episode sections

  1. 0:00 – 1:20

    Introduction and AI Evolution

    Host introduces Robert Yang, CEO of Fundamental Research Labs, and discusses the rapid advancement of AI agents and the imminent arrival of digital hedge fund analysts.

  2. 1:20 – 8:25

    Robert Yang's Background and Academic Journey

    Yang recounts his interest in AI starting in 2007, studying computational neuroscience at MIT, and overcoming early skepticism about neural network modeling.

  3. 8:25 – 15:35

    Transition from Academia to Industry

    Yang explains his decision to leave his MIT faculty position to join industry, driven by a desire to build AI rather than just model the brain, citing peers like Dario Amodei and Demis Hassabis.

  4. 15:35 – 20:45

    The Catalyst for Agent Development

    Yang identifies missing the 2017-2023 language model scaling wave as a regret and cites the 2023 'Voyager' and 'Stanford AI Town' papers as catalysts for agent development.

  5. 20:45 – 28:45

    Project Sid: Building a Society of Agents

    Yang details 'Project Sid,' an experiment running 1,000 agents in a Minecraft simulation, highlighting high costs and emergent behaviors like search parties and religious gatherings.

  6. 28:45 – 30:45

    Product Failure and Pivot to Computer Use

    The consumer agent product failed with 10% day-one retention, leading to a pivot after seeing Anthropic's computer use demo, developing 'skills' for agent introspection.

  7. 30:45 – 33:37

    Pivot from General Agent to Excel Focus

    After raising a Series A, the company pivoted from a general computer-use agent to an Excel-focused agent, realizing the issue was interface rather than intelligence.

  8. 33:37 – 36:42

    Product Launch and Early Challenges

    The company launched 'Fairies' which failed, and 'Shortcut' which went viral but suffered from high churn due to model limitations in handling complex workflows.

  9. 36:42 – 39:49

    Building Benchmarks and Improving Accuracy

    The team built an internal benchmark using real-world tasks like LBOs and DCFs to optimize the agent's harness, leading to improved retention and enterprise adoption.

  10. 39:49 – 44:23

    Hedge Funds as Ideal Early Adopters

    Hedge fund analysts emerged as ideal customers due to their need for high precision, with Opus 5.5's release prompting renewed interest from the community.

  11. 44:23 – 48:11

    Expanding Beyond Excel to Agent Teams

    The company plans to evolve into a platform where agents perform research and monitoring, orchestrated by a portfolio manager agent to simulate hedge fund workflows.

  12. 48:11 – 52:41

    The Rise of Personal Agents and Interface Needs

    The speaker discusses the need for personal agents to manage multiple concurrent threads, providing a single interface for managing sub-agents.

  13. 52:41 – 59:20

    Timeline for Digital Analysts and Human Limitations

    Yang predicts digital hedge fund analysts will be widely adopted within 6-12 months, though human analysts remain superior in building conviction and holding contrarian views.

  14. 59:20 – 1:15:44

    Technical Path to Opinionated Agents

    To make agents more opinionated, the speaker suggests using 'skills packs' to distill firm views into prompts, addressing the 'spineless' nature of current models.

Assets discussed

AssetKindStanceContext
NVIDIANVDAequitybullishIf you believe that this can go like if you if you get NVIDIA right 10 years ago, right, and you're like, I believe in this, I believe in AI, right? You can hold on it forever, right? You can still hedge by like short other things, but um, but you're like, no, NVIDIA, I'm gonna hold on on to it.

“Stance” is how the speakers talked about the asset, not a recommendation.

Claims worth verifying

Robert Yang was a faculty member at MIT doing computational neuroscience.

Unidentified speaker ·3:07

How to verify: Check MIT faculty archives or Yang's LinkedIn profile for his employment history at MIT.

Dario Amodei, CEO of Anthropic, did computational neuroscience in his PhD.

Unidentified speaker ·8:36

How to verify: Verify via Anthropic's official team page or Amodei's academic publications/CV.

Demis Hassabis, CEO of DeepMind, did neuroscience in his PhD and computational neuroscience in his postdoc.

Unidentified speaker ·8:53

How to verify: Verify via DeepMind's official team page or Hassabis's academic publications/CV.

The Voyager paper was published in early 2023.

Unidentified speaker ·16:44

How to verify: Search for the 'Voyager' paper on arXiv or academic databases for its publication date.

The Stanford AI Town paper was authored by June Park.

Unidentified speaker ·17:18

How to verify: Search for the 'Stanford AI Town' paper on arXiv or academic databases to confirm the author.

Project Sid involved running 1,000 agents in a Minecraft simulation.

Unidentified speaker ·21:55

How to verify: Search for 'Project Sid' and 'Minecraft' in tech news or Fundamental Research Labs' announcements.

Running 1,000 agents in Project Sid cost approximately $2,000 per day.

Unidentified speaker ·22:35

How to verify: This is a specific internal financial claim; verify via Fundamental Research Labs' funding disclosures or interviews where this figure was stated.

The consumer agent product had a day-one retention rate of 10% in late 2024.

Unidentified speaker ·29:40

How to verify: This is an internal metric; verify via Fundamental Research Labs' public statements or interviews discussing product performance.

Anthropic released a computer use demo in late 2024.

Unidentified speaker ·29:46

How to verify: Check Anthropic's official blog or press releases for the announcement of their computer use capabilities.

Yang predicted agents would create massive economic value by 2025.

Unidentified speaker ·19:00

How to verify: This is a forward-looking statement; verify via transcripts or articles from 2023 where Yang made this prediction.

The company raised a Series A of approximately $30 million.

Unidentified speaker ·30:45

How to verify: Check the company's press releases, Crunchbase profile, or SEC filings (if public) for Series A funding announcements around the time of the podcast release (2026-11-04).

The co-founder Nico worked as a financial consultant at EY for two years.

Unidentified speaker ·32:42

How to verify: Verify Nico's professional background on LinkedIn or EY's alumni records, or check the company's 'About Us' page for founder bios.

The Excel agent 'Shortcut' went viral on Twitter in summer 2025.

Unidentified speaker ·34:00

How to verify: Search Twitter/X archives for mentions of 'Shortcut' and the company name in summer 2025 to find viral posts or engagement metrics.

The company attempted a fundraise that failed in October of the previous year due to poor retention metrics.

Unidentified speaker ·36:23

How to verify: Check for news reports or announcements about the company's fundraising status in October of the year prior to the podcast (likely October 2025).

The company built an internal benchmark using tasks like Leveraged Buyouts (LBO) and Discounted Cash Flow (DCF).

Unidentified speaker ·37:16

How to verify: Look for technical blog posts, whitepapers, or case studies published by the company detailing their benchmarking methodology and test cases.

Opus 5.5 was released in September 2025 and passed the speaker's modeling benchmarks.

Unidentified speaker ·39:55

How to verify: Check AI model release notes from the relevant lab (likely OpenAI or similar, assuming 'Opus' is a specific model name or ASR error) for a September 2025 release and compare with the speaker's benchmark results.

The speaker had 50 agents running concurrently with 40 terminal windows open.

Unidentified speaker ·51:05

How to verify: This is a personal anecdote; verify via the speaker's social media posts or blog entries describing their workflow at the time.

The speaker predicts digital hedge fund analysts will be widely adopted within 6-12 months.

Unidentified speaker ·53:31

How to verify: Monitor industry adoption rates of AI agents in hedge funds over the next 12 months and compare with the speaker's prediction timeline.

Current LLMs are 'spineless' due to post-training alignment that prioritizes agreement.

Unidentified speaker ·56:23

How to verify: Review academic papers or technical reports on LLM alignment techniques (RLHF, DPO) and their impact on model behavior and opinion formation.

Current AI models are trained on approximately 100 trillion tokens.

Unidentified speaker ·1:04:55

How to verify: Check technical reports from major AI labs (e.g., OpenAI, Google DeepMind) regarding the training data size of their latest large language models.

Human genes do not encode specific information like stock prices.

Unidentified speaker ·1:05:24

How to verify: Consult biological textbooks or scientific literature on genetics and epigenetics to confirm the limits of genetic information storage.

The speaker predicts humans will trust AI agents to make decisions in two years.

Unidentified speaker ·1:08:26

How to verify: This is a forward-looking prediction; verify by tracking industry adoption rates and trust surveys in AI-driven investment tools over the next two years.

The speaker has been training human fundamental analysts for the last four years.

Unidentified speaker ·1:09:17

How to verify: Check the speaker's LinkedIn profile or professional biography for employment history and roles at Fundamental Research Labs (FRL) or similar entities.

The speaker's first task is to teach an agent to pass a typical hedge fund interview process.

Unidentified speaker ·1:10:55

How to verify: Monitor Fundamental Research Labs (FRL) or Invest with AI announcements, blogs, or social media for updates on their 'FDI' (Forward Deployed Investor) team or agent training milestones.

Invest with AI will remain an independent entity.

Unidentified speaker ·1:14:32

How to verify: Check the Invest with AI website, terms of service, or official announcements regarding corporate structure and independence.

Do your own research

  • Verify the current retention rates and enterprise adoption metrics for Fundamental Research Labs' 'Shortcut' Excel agent against competitors.
  • Review the specific benchmarks used for LBO and DCF accuracy to assess the validity of the agent's performance claims.
  • Investigate the technical implementation of 'skills packs' and post-training methods for instilling specific investment theses in open-weight models.
  • Check the regulatory compliance landscape for deploying autonomous AI agents in hedge fund workflows, particularly regarding audit trails and liability.
  • Analyze the cost structure of running 1,000+ agents in simulation environments, as demonstrated in 'Project Sid', to understand scalability challenges.
  • Look for independent third-party validation of the claim that digital analysts will be widely adopted within 6-12 months.
  • Research the specific market microstructure distortions caused by quants and indexers that AI agents are purported to exploit.
  • Confirm the current capabilities of Opus 5.5 and other models released in September 2025 regarding complex financial workflow handling.
Full transcriptexpand
0:00
Unidentified speaker: Two years ago, investors were impressed that our agents can make a pivot table.
0:04
Unidentified speaker: How close do you think we are to working with a digital hedge fund analyst?
0:09
Unidentified speaker: The short answer is I think very, very soon.
0:12
Unidentified speaker: The tolerance for air is uh incredibly low.
0:15
Unidentified speaker: Every couple of weeks I'm like, wow, that's this is sci-fi.
0:19
Unidentified speaker: And it's but it's real.
0:20
Unidentified speaker: It's it's happening.
0:21
Unidentified speaker: What we're trying to do is build a Jarvis suit for Tony Stark for the for the investor.
0:40
Unidentified speaker: Great.
0:40
Unidentified speaker: Welcome to the next episode with of Invest with AI, where we explore the intersection of AI and fundamental investing.
0:48
Unidentified speaker: Very special guest today, uh, the CEO and co-founder of Fundamental Research Labs, who I announced this week I was joining as chief growth officer.
0:59
Unidentified speaker: So very excited to uh be partnered with Robert Yang and help him drive the mission forward.
1:05
Unidentified speaker: We'll touch a little bit on what that means today, what we plan to do.
1:08
Unidentified speaker: Um, but wanted to start with just hearing the backstory of uh Robert Yang of Fundamental Research Labs and where it all started.
1:18
Unidentified speaker: So uh the mic is yours, Robert.
1:20
Unidentified speaker: Sort of start with the arc of where things began.
1:23
Unidentified speaker: Yeah, thanks.
1:24
Unidentified speaker: Uh thanks, Brett, for having me.
1:27
Unidentified speaker: Um yeah, so for me, it kind of began when I was in high school.
1:31
Unidentified speaker: I kind of it was 2007.
1:34
Unidentified speaker: Um I was trying to figure out what I should do with my life, and uh I thought uh if we can achieve AI, that'd be the biggest thing ever, right?
1:43
Unidentified speaker: So uh but AI back then is still like a taboo word, uh like in 2007.
1:49
Unidentified speaker: Um so but I always like to do things that are contrarian, uh maybe like a bad student in some way, so like a troubling, like a like a cause trouble, you know, the kind of student that caused trouble to the teachers.
2:03
Unidentified speaker: Um, so um so I thought, okay, this is gonna be uh super important, but I I had no idea how to get there, right?
2:12
Unidentified speaker: Uh so I read some AI books.
2:15
Unidentified speaker: There was this book by um this Berkeley professor called Artificial Intelligence, um, and it was there were nothing about neural networks, everything is like rule-based expert system, and I just thought that's not right, right?
2:26
Unidentified speaker: It has to be neural networks.
2:28
Unidentified speaker: Um, so at that time I already knew that like the brain is made of neural networks, and I was like, this is the right thing.
2:34
Unidentified speaker: Um, so then I just entered um um, but there I think I made maybe a bit of a mistake, right?
2:43
Unidentified speaker: So so I think a lot of people, like the AI pioneers, uh, felt the same way.
2:48
Unidentified speaker: I was like, oh, it should be neural network and the brain is neural network.
2:52
Unidentified speaker: But then they at some point they realized, okay, you should uh there's a difference between studying the brain and trying to like build a neural network that is inspired by the brain.
3:01
Unidentified speaker: So I kind of just went in very hard in like just studying the brain, right?
3:07
Unidentified speaker: Um, so fast forward like 15 years, uh, I was a faculty at MIT doing computational neuroscience, um, which is still like I was, I guess, part of the early wave of using neural network to model the brain.
3:23
Unidentified speaker: Uh, even though the brain is a neural network, uh, actually, people weren't really using artificial neural network to model it.
3:29
Unidentified speaker: And when I first started doing it, and again, it was very controversial.
3:33
Unidentified speaker: In fact, I'll share you with you this story.
3:35
Unidentified speaker: Um, someone on my thesis committee uh in grad school, a very senior professor, uh, when I was presenting the first time uh I was doing work uh using neural network to model the brain, using artificial neural nets.
3:48
Unidentified speaker: Uh, these were like recurrent neural networks back in 2015.
3:52
Unidentified speaker: Uh, he just looked at me, and this is in front of my PhD advisor.
3:56
Unidentified speaker: He just looked at me and said, Um, you are building upon trash.
4:02
Unidentified speaker: Right?
4:03
Unidentified speaker: So so I was building upon um this author, David Sicillo, who I later actually did an internship with.
4:11
Unidentified speaker: He's a great scientist.
4:12
Unidentified speaker: Um, so I was built, uh he's actually a book author and wrote a wrote a book as well.
4:16
Unidentified speaker: But um, I was building upon his work, and uh and this professor didn't want to call my work trash.
4:23
Unidentified speaker: He called David Sicillo's work trash and was like, I'm making it a little bit nicer, but it's still trash.
4:29
Unidentified speaker: Uh he didn't say it's so trash, he just said you're building upon trash and making it a little bit nicer, right?
4:34
Unidentified speaker: Um, but within two years, uh, I think essentially the approach that I was doing and David Sicillo was doing became mainstream very, very quickly.
4:44
Unidentified speaker: Uh, everyone was using these artificial neural networks, like deep neural networks, to model the brain.
4:50
Unidentified speaker: Uh, and I actually kind of had a had a like uh existential crisis at the point.
4:56
Unidentified speaker: I I was having a crisis at MIT because so we realized that the neural networks that um these AI people are putting out, right?
5:05
Unidentified speaker: The language models, etc., etc., are actually much better models of the brain than uh than the neuroscientists have been building.
5:16
Unidentified speaker: You know?
5:17
Unidentified speaker: So so it turns out these people are just out there trying to solve intelligence, and as a result, we're building actually better descriptions of how our human brain works than anyone else did.
5:32
Unidentified speaker: So that was a real just backing up, is this what inspired you to like it was it sounds like a childhood interest in in artificial intelligence that drove you into computational neuroscience?
5:45
Unidentified speaker: Is that yes?
5:46
Unidentified speaker: Sorry, I skipped that.
5:48
Unidentified speaker: So essentially my thinking process was pretty naive.
5:51
Unidentified speaker: Um I was like, I want to but I want to do AI, right?
5:56
Unidentified speaker: AI is I think the right form of AI would be a neural network.
6:00
Unidentified speaker: And the only working initial inspiration, like what was it?
6:04
Unidentified speaker: Did you read was it reading sci-fi?
6:06
Unidentified speaker: Like what reading, like what you remember, what what sort of got you interested at such a such a young?
6:11
Unidentified speaker: I can't I can't recall what was on my mind in 2007, but it's it certainly wasn't AI.
6:16
Unidentified speaker: Yeah, yeah.
6:17
Unidentified speaker: Uh, I guess so I was thinking about the I had always wanted to do something important.
6:27
Unidentified speaker: It's like profound.
6:29
Unidentified speaker: Um, and I felt like there are not that many profound things out there left to do, right?
6:35
Unidentified speaker: Like Newton and Einstein already did a lot on physics.
6:39
Unidentified speaker: We already understand we know there's gene, right?
6:42
Unidentified speaker: So but AI just felt like an obvious, like if we can do this, then this is gonna be extremely important, right?
6:51
Unidentified speaker: Maybe the most important thing in the in like after Newton, right?
6:56
Unidentified speaker: This is my my opinion.
6:57
Unidentified speaker: Um, is I think after I think it's like Newton, Darwin, and AI, right?
7:03
Unidentified speaker: Um, so yeah, I thought I just thought that's what I want to do.
7:08
Unidentified speaker: Um and I'm naive in the sense that I wasn't really thinking about, oh, like is the timing gonna be ready, right?
7:14
Unidentified speaker: Maybe it's not gonna be ready for another 30 years, right?
7:17
Unidentified speaker: Then uh and and then I also didn't think am I the right person to do this, right?
7:23
Unidentified speaker: So uh I didn't really think about these things.
7:26
Unidentified speaker: Uh in fact, I went to um I went to physics department.
7:30
Unidentified speaker: Uh I actually considered going to biology, but I went to biology and I was like, what do you study?
7:35
Unidentified speaker: They're like, we're studying like plants, right?
7:38
Unidentified speaker: And animals, and that's boring.
7:40
Unidentified speaker: So so I went into physics because I thought, okay, physics is how you understand the world through mathematics, right?
7:46
Unidentified speaker: And um essentially you want to understand the intelligence through mathematics.
7:51
Unidentified speaker: So I went to physics and we we were introducing ourselves, and everyone is like, oh, like I really love physics, blah blah blah.
7:57
Unidentified speaker: And I was like, I love AI.
8:00
Unidentified speaker: And if you're interested in AI, please come tell me.
8:04
Unidentified speaker: Um people were very confused because most people have not even heard of AI, right?
8:09
Unidentified speaker: It's not, it's like um for people who have heard of it, it's like a sci-fi thing.
8:14
Unidentified speaker: It's the people didn't think it's a real thing.
8:16
Unidentified speaker: That was 2008.
8:17
Unidentified speaker: This was what this was what year?
8:19
Unidentified speaker: 2008.
8:21
Unidentified speaker: 2008, okay.
8:22
Unidentified speaker: Yeah, yeah.
8:23
Unidentified speaker: Yeah, so that's some so that's how I got into computational neuroscience.
8:26
Unidentified speaker: Um, so something that a lot of people don't realize is is um really some of the uh AI uh pioneers, uh like for example, Dario at um on Stropic, right?
8:36
Unidentified speaker: The co-founder CEO, uh, he actually did computational neuroscience in his PhD.
8:41
Unidentified speaker: Uh, I actually know some of his co-authors.
8:43
Unidentified speaker: Um and uh and the DeepMind co-founder and CEO, um uh Demis Hasabas also did uh neuroscience.
8:53
Unidentified speaker: He just did straight up neuroscience in his PhD and then did computational neuroscience in his postdoc.
8:59
Unidentified speaker: Uh, and DeepMind was kind of spun out of uh his postdoc.
9:03
Unidentified speaker: Uh it's um my I guess what I did is you can call it a mistake, at least from my career standpoint.
9:11
Unidentified speaker: It's like I spent a little too long uh doing it, right?
9:15
Unidentified speaker: Um I kind of probably the right thing would be at 2017, maybe just pivot to go all the way to like building intelligence, right?
9:24
Unidentified speaker: Because that that was always my goal, right?
9:27
Unidentified speaker: Um, but I was I think a bit more like risk averse back then.
9:31
Unidentified speaker: I was like, I already did PhD, I think I can get a faculty job.
9:36
Unidentified speaker: And um uh so yeah, so it's uh but but and what does what does a faculty MIT PhD do in computational neuroscience do?
9:48
Unidentified speaker: Like what were what were your responsibilities in that in that seat?
9:52
Unidentified speaker: Yeah, so um uh yeah, absolutely.
9:57
Unidentified speaker: So so uh a faculty is like is essentially these like professors, right?
10:01
Unidentified speaker: I mean, there's rank, there's like assistant professor, associate professor, right?
10:05
Unidentified speaker: But we can just call them all professors.
10:08
Unidentified speaker: And so these these professors they do essentially two things.
10:12
Unidentified speaker: One is they uh teach uh they they teach undergrads, right?
10:17
Unidentified speaker: Uh teach undergrads and teach grad students.
10:19
Unidentified speaker: And for a lot of like people uh right who after the graduate college, this is their main understanding of what these professors do, right?
10:26
Unidentified speaker: But at lots of research institutes like MIT and like Stanford, Harvard, um, you can say the main responsibility of the professors are actually to conduct research, right?
10:36
Unidentified speaker: Typically you you run a research lab that has um somewhere between like five to maybe like 30, 40 people, and uh you kind of just guide them to do research, right?
10:47
Unidentified speaker: Um, so so this is what I was doing.
10:50
Unidentified speaker: Uh I was at MIT for uh a few years um before I essentially um resigned.
10:56
Unidentified speaker: And uh I had a pretty good lab, but we actually had uh we had I I didn't have any students who like I wasn't there long enough to have a student graduate from PhD, but I had postdocs.
11:08
Unidentified speaker: These are people who already have PhDs.
11:10
Unidentified speaker: I had postdocs who did really well.
11:12
Unidentified speaker: I had one postdoc who went to CMU to be uh assistant professor in the machine learning department, another one went to Brown uh in the applied mass.
11:21
Unidentified speaker: And I think I mentored like five, six students who ended up going to like MIT, Berkeley, all these good schools.
11:27
Unidentified speaker: Uh so I was really, I think, proud of our team.
11:30
Unidentified speaker: Uh I am proud of our team we had at MIT.
11:34
Unidentified speaker: Um, but during almost the whole time at MIT, I was like suffering from this existential crisis.
11:41
Unidentified speaker: Uh, to be fair, our our apartment, um I would say I was in part of two apartments.
11:46
Unidentified speaker: One is uh brain cognitive science, another is uh EECS, electroengineering, and computer science.
11:51
Unidentified speaker: Uh and I would say uh these two apartments, uh departments are are really like the best, right?
11:58
Unidentified speaker: Maybe like uh the MIT department is definitely among like the top three, four departments in the world in computer science.
12:07
Unidentified speaker: And the brain cognitive science department I was in, I would argue it's like the number one department in the world to do it.
12:13
Unidentified speaker: Um but what I realize is it's really not about like okay, is this school it's not about like the school, it's not about MIT, it's not about the department, it's about this entire like tsunami that's coming to everybody, which is AI, just like the AI buildup in industry, right?
12:33
Unidentified speaker: And they're just like and we're seeing this this year in mathematics, right?
12:39
Unidentified speaker: Like this AI tsunami just like is is come completely transforming how everyone do their work and making a lot of the old way of doing things just irrelevant, and and you have to adapt, right?
12:52
Unidentified speaker: So I was feeling that um back at MIT, and I felt to do what I've always wanted to do, which is to build to build humans, to build um, to build AI, I have to uh go to industry.
13:06
Unidentified speaker: Uh in fact, I I I didn't want to just join a big lab.
13:10
Unidentified speaker: Um, so I wanted to do it in a startup.
13:12
Unidentified speaker: Um and the reason really is is I mean you didn't ask it, but I'll I'll explain it.
13:16
Unidentified speaker: The reason why I wanted to do a startup is is I think the way that the big labs think about AI is still a bit different from my vision of um of a human, right?
13:28
Unidentified speaker: So the big lab is very much focused on um very much focused on intelligence, like capability, right?
13:36
Unidentified speaker: Um which which makes a lot of sense.
13:38
Unidentified speaker: I mean, this is how you how you um how you have a real business model, right?
13:44
Unidentified speaker: Um so capability that is aligned with human, right?
13:48
Unidentified speaker: So that's how I would describe it.
13:49
Unidentified speaker: Uh but there are things in human that's uh that we don't really understand the capability yet.
13:55
Unidentified speaker: Uh like why why this is useful, right?
13:57
Unidentified speaker: So I'll give you an example, like why, why should humans feel happy when they when they get a reward, right?
14:03
Unidentified speaker: Like you you eat some chocolate, you're not only reinforcement learning, which is a major way that people train neural networks, right?
14:10
Unidentified speaker: You you give you give a neural network chocolate, and you give through reinforcement learning, you would teach them to more likely to get chocolate in the future, right?
14:20
Unidentified speaker: Whatever action that led them to get that chocolate, they're more likely to get that chocolate.
14:25
Unidentified speaker: But humans, we not only do that, we're also happy to eat that chocolate, right?
14:29
Unidentified speaker: The current neural networks, we don't think that they're happy, right?
14:33
Unidentified speaker: When something good happens to them, right?
14:35
Unidentified speaker: Maybe they will say they're happy, but uh it's I think most people don't really believe that they experience the happiness, right?
14:42
Unidentified speaker: Um, so so now you can say, well, why is that useful, right?
14:46
Unidentified speaker: So like when I wanted to when we first started the company, I just wanted to be able to just build the human without without really answering why this is useful, right?
14:58
Unidentified speaker: So just wanted to like put in like emotions, long-term memory, right?
15:03
Unidentified speaker: Just motivation, all of this good stuff into an agent and have them just live alongside human um without really answering the business model.
15:15
Unidentified speaker: Um, so and that's kind of what we did.
15:18
Unidentified speaker: And what what was there a final moment or something you saw when you were at MIT that really was the the catalyst to lead?
15:27
Unidentified speaker: Because that's a pretty cushy life lifestyle.
15:30
Unidentified speaker: Yeah.
15:31
Unidentified speaker: Um what uh what was sort of the final deciding insight?
15:35
Unidentified speaker: Yeah, yeah, absolutely.
15:36
Unidentified speaker: So I was having a lot of um, so I was having a lot of existential crisis, but at the same time, I felt like I missed the last wave, right?
15:45
Unidentified speaker: So the major wave of NAI between about 2017 and 2023 was essentially scaling of language models, right?
15:55
Unidentified speaker: So uh 2017 was a Transformer paper, and then uh and then in 20, I think 2019, um 2020 was like around that time was a scaling papers, right?
16:08
Unidentified speaker: So that's that was a main paradigm, and I felt like I just missed out on that entire paradigm, and I just didn't want to like um be a follower.
16:18
Unidentified speaker: But in 2023, early 2023, so the language models started to be available to people to experiment with, and there were a couple papers from academia that were really influential and just essentially wrapped these language models and built agents, right?
16:38
Unidentified speaker: Now we're very familiar with agents, but in 2023, early 2023, they were still very new.
16:44
Unidentified speaker: One paper is called the Voyager paper.
16:46
Unidentified speaker: They essentially build an agent that um that goes in Minecraft and would like like just like play the game.
16:54
Unidentified speaker: And um, but it it's it's not really just like playing the game for the sake of like achieving a specific goal.
17:00
Unidentified speaker: It was actually just exploring, exploring the game, which I thought is really, really uh cool.
17:06
Unidentified speaker: And another thing that's similar is um there was a paper from Stanford, the Stanford AI Town paper by June Park, who is actually our first, our first investor, an NGO investor.
17:18
Unidentified speaker: So so June Park had this paper that had like a 20-something agents, also kind of like this language model wrapped with memory, right?
17:28
Unidentified speaker: Um, that would run around in a virtual tongue that's like a Pokemon tongue.
17:35
Unidentified speaker: And they would run around and they would like host birthday parties and they would like try to cook and like buy stuff, right?
17:42
Unidentified speaker: None of them are real, right?
17:44
Unidentified speaker: None of these things are real.
17:45
Unidentified speaker: There is no real food and they cannot consume food.
17:47
Unidentified speaker: But it was uh it was really uh like a shocking demonstration of the future, right?
17:54
Unidentified speaker: Where you you don't just have chatbots, you have agents that live and just live their life, right?
18:00
Unidentified speaker: Um so I read that paper and I I was pretty shocked.
18:06
Unidentified speaker: But what was even more I think shocking for me is we had an intern and she she was great.
18:12
Unidentified speaker: Um her name's Kai uh Ivy Zhao, and uh she was essentially asked to present this at a journal club.
18:22
Unidentified speaker: Uh, and instead of presenting the paper, she built this entire demo with this agent, like in a in a 3D game, right?
18:30
Unidentified speaker: Uh in a game built in this Unity, which is a game engine.
18:34
Unidentified speaker: So so that's when I realized, wow, this is not only super cool, this is actually very approachable, right?
18:42
Unidentified speaker: Uh Ivy is extremely good and extremely smart, but the fact that you have a student in a like in a week just reproduce something like that means that this is gonna be like very rapidly iterated on.
18:57
Unidentified speaker: Um, so at the time, my um my prediction um was okay, 2023, we have this agent research, and by 2025, these agents are gonna create massive in economic value, right?
19:14
Unidentified speaker: Um like they're gonna become like very powerful products.
19:18
Unidentified speaker: Uh, and 2025 is I think when we got cloud code, right?
19:21
Unidentified speaker: So when we got cloud code, when we got cursor agent, uh so it roughly checked out.
19:27
Unidentified speaker: Uh and I felt this urgency, right?
19:30
Unidentified speaker: Because I already missed the last wave of the language model scaling.
19:33
Unidentified speaker: I cannot miss uh the agent wave again.
19:39
Unidentified speaker: Um, so I directed my lab, half of my lab, immediately to work on this agent direction.
19:46
Unidentified speaker: And so we were building agents that uh that are living in this uh 3D world.
19:51
Unidentified speaker: They're like long-living agents that would just uh again, it's like Grockbot today, right?
19:56
Unidentified speaker: They're just persistent, they were always living.
19:58
Unidentified speaker: Uh so we're building them in back in 2023.
20:02
Unidentified speaker: Uh it's it's pretty crazy if you're looking back.
20:04
Unidentified speaker: Back then there were no Asian benchmark, right?
20:07
Unidentified speaker: So today we have so many benchmarks, but back in 2023, there were actually no benchmark, and we had to come up with some crazy benchmark.
20:15
Unidentified speaker: So we we built a little benchmark that's called a mystery, murder mystery, which is there essentially is a murder, uh like a hype hypothetical murder with like uh different uh different um um different people who are suspicious.
20:32
Unidentified speaker: Uh and then uh the the benchmark is whether or not a detective can actually solve the murder.
20:38
Unidentified speaker: So uh we did that, yeah.
20:40
Unidentified speaker: We didn't have better, better benchmarks.
20:42
Unidentified speaker: Yeah.
20:42
Unidentified speaker: Yeah.
20:43
Unidentified speaker: And tell me about tell me about Project Sid.
20:46
Unidentified speaker: Yeah, so so essentially we um uh now we started the company in uh late 2023, early 2024, and um we were we were still doing these like long-living agent in games.
21:01
Unidentified speaker: And the idea is you would build a co-player for people, like in Minecraft.
21:05
Unidentified speaker: And at the time, um there was someone uh someone we we got some competition, right?
21:11
Unidentified speaker: Some other people were also building uh these agents in Minecraft, and I just felt like okay, we need to do something to show people that uh we can do agents at a completely different scale, right?
21:26
Unidentified speaker: So I was thinking uh it's it's not just it's not impressive enough if you just have one agent that can kind of like play along a uh a player.
21:37
Unidentified speaker: What if what if you can have a society of agents, right?
21:41
Unidentified speaker: That's that really just ongoing, just live on, right?
21:46
Unidentified speaker: So we had um, and the Stanford AI town paper already did 20 something agents, so so I didn't want to just do that again, right?
21:55
Unidentified speaker: That felt like very small.
21:57
Unidentified speaker: So we wanted to do a thousand agents, uh, and we did that.
22:01
Unidentified speaker: Uh I think I think it's uh it was it it was actually very, very challenging um just technically.
22:09
Unidentified speaker: Um there's a there's obviously the intelligence part that we can get into, but just having a Minecraft server that kind of support a thousand agents like uh together is hard.
22:21
Unidentified speaker: Like our agents were like dropping out of the server all the time.
22:25
Unidentified speaker: And and it's it got very expensive to run, right?
22:29
Unidentified speaker: Because we had no business model, so we're just running this and we got some VC money.
22:33
Unidentified speaker: Um, thank goodness.
22:35
Unidentified speaker: Uh, and then we were running these iterations, and it would cost like uh a couple thousand dollars a day to just run these a thousand agents for a few hours, right?
22:47
Unidentified speaker: Uh not even a thousand agents to run like like 50 agents maybe for a few hours, which is uh what our benchmark was.
22:55
Unidentified speaker: At that point, we had a slightly better benchmark.
22:57
Unidentified speaker: We essentially would run like 50 agents and have them live on um live on uh like for three hours and see like what they have done, right?
23:08
Unidentified speaker: And our criteria is like how many unique items in Minecraft have they collected?
23:13
Unidentified speaker: So that's um and we would benchmark this.
23:17
Unidentified speaker: So there were lots of fun things we had to do for this.
23:21
Unidentified speaker: And the the biggest regret I had is uh is not like really fully open sourcing our code.
23:27
Unidentified speaker: Uh, I think we we put some some demo online.
23:32
Unidentified speaker: It got it got hyped uh for uh for a little bit, um, got pretty well received, but I think it could have had even bigger impact if we had just really like open sourced everything.
23:43
Unidentified speaker: Um at the time we were still thinking we would make some money off this, like got a product around it.
23:48
Unidentified speaker: So we were kind of protective of that.
23:51
Unidentified speaker: What was the what was the most entertaining part of that?
23:55
Unidentified speaker: Or entertaining or memorable part of that?
23:57
Unidentified speaker: Yeah, yeah, absolutely.
23:59
Unidentified speaker: So um, so one thing is uh I've I've I always love sci-fi, right?
24:06
Unidentified speaker: And sci-fi concept is simulation uh hypothesis.
24:09
Unidentified speaker: I mean it's an academic concept too.
24:11
Unidentified speaker: It's uh the idea that that some people maybe we live in assimilation, right?
24:16
Unidentified speaker: Now we build this Minecraft simulation with all these agents, they literally are in assimilation, like we know that for sure, right?
24:23
Unidentified speaker: So maybe we're in assimilation ourselves, but they're definitely in assimilation, right?
24:28
Unidentified speaker: So I would actually go in to this and I'm like, I'm like, I'm your dad, right?
24:33
Unidentified speaker: So I would talk to the agents, I'm like, I'm your dad, I actually created you.
24:37
Unidentified speaker: And the agents would be like, oh, that's so that's so cool, right?
24:41
Unidentified speaker: That's like I'm like, no, no, no, you don't get it, right?
24:44
Unidentified speaker: I created you, I'm I'm I created you, I'm your creator, right?
24:48
Unidentified speaker: And they're like, they're just they're just saying shallow things.
24:52
Unidentified speaker: They were just like, oh, that's very cool.
24:53
Unidentified speaker: Like, what else can you do?
24:54
Unidentified speaker: Right?
24:55
Unidentified speaker: What else have you created?
24:56
Unidentified speaker: They don't they don't have this existential crisis of like, oh wow, I thought I'm like a real person.
25:02
Unidentified speaker: Now you're telling me I'm not real.
25:05
Unidentified speaker: Um so so I thought that's that for me was uh was kind of fun.
25:10
Unidentified speaker: Uh to I felt like it's a proof of the simulation hypothesis in some weak form, right?
25:16
Unidentified speaker: As in there exist people who live in assimilation for sure.
25:20
Unidentified speaker: No, um, but the most interesting thing I would say is is um so Shuing, our co-founder, uh she was just trying to kind of debug the system, right?
25:31
Unidentified speaker: We would run these like simulations of like many agents, kind of just live on the we gave them identities, right?
25:37
Unidentified speaker: Uh some of them are like uh like a priest, uh like a mayor, right?
25:42
Unidentified speaker: Uh like a worker, like etc.
25:44
Unidentified speaker: etc.
25:45
Unidentified speaker: So she she went into the Minecraft uh simulation, right?
25:48
Unidentified speaker: And when I say went in, as in she as a player just go into the game and just walk around.
25:54
Unidentified speaker: And she just saw all these torches on the ground, right?
25:58
Unidentified speaker: All these torches on the ground.
26:00
Unidentified speaker: And she's like, what's going on?
26:02
Unidentified speaker: Right?
26:02
Unidentified speaker: Why are all these torches on the ground?
26:04
Unidentified speaker: So then we we looked at the entire agent of conversation history, and so we realized there was something really, really emergent that happened, right?
26:15
Unidentified speaker: So what happened is two things.
26:17
Unidentified speaker: Uh, one is some agent in this ton had wandered away, right?
26:22
Unidentified speaker: Like they just like walked really far away.
26:25
Unidentified speaker: And then let's let's call, let's just say Brett went really far away, right?
26:29
Unidentified speaker: And then some other agent, let's say Alice, was looking for Brett and couldn't find Brett.
26:35
Unidentified speaker: And I was like, Brett is missing, right?
26:36
Unidentified speaker: What's going on?
26:37
Unidentified speaker: Brett is missing, right?
26:38
Unidentified speaker: And so she then went to the mayor and I was like, Mayor, we need to form a search party for Brett, because Brett is missing, right?
26:47
Unidentified speaker: And and so the mayor is like, okay, like, yeah, yeah, yeah, we should form a search party.
26:51
Unidentified speaker: And now what's crazy is is then Brett's Brett, uh, sorry, not Brett, the mayor's son, the mayor's son had recently converted to the pasta religion.
27:02
Unidentified speaker: So there's a priest that goes around and kind of essentially like spread this religion.
27:07
Unidentified speaker: We didn't want to be political, so it's a made-up religion that's like essentially they just said like the pasta, pasta is your God, right?
27:14
Unidentified speaker: So uh it's called like pastifarian or something, and then the then the the the it converted, the priest converted the mayor's son, right?
27:25
Unidentified speaker: So the mayor's son was like, okay, like I recently converted it to the pastoranism, pastorism, and I want to host a religious gathering.
27:35
Unidentified speaker: And then he heard from the mayor that there's someone who's missing and they need to form a search party.
27:41
Unidentified speaker: He's like, okay, let me actually turn my let me actually turn my uh religious event also into like um uh to help finding these people.
27:53
Unidentified speaker: So he's like, let's why don't we plant some torches on the ground so that when Brett like if so Brett can see us from far away and be able to find the way home, right?
28:04
Unidentified speaker: So he then it's quite heartwarming, actually.
28:08
Unidentified speaker: Yeah, yeah, exactly.
28:09
Unidentified speaker: So that was, and I was like, wow, didn't expect it to be so elaborate.
28:15
Unidentified speaker: Uh and it was we were completely shocked because we didn't we didn't design any of this other than like the fact that there's a priest, there's a mayor, etc.
28:25
Unidentified speaker: Right?
28:26
Unidentified speaker: So all of these things came out of just like the agents interacting with one another and behaving like humans.
28:33
Unidentified speaker: Um, that was for me, that was the coolest, coolest thing that an entire project.
28:39
Unidentified speaker: So how did you how did you go from building societies in Minecraft to building Excel agents for Wall Street?
28:46
Unidentified speaker: What walked me through that journey?
28:48
Unidentified speaker: Yeah, yeah, yeah.
28:50
Unidentified speaker: It's uh um yes, absolutely.
28:54
Unidentified speaker: So so we did project Sid and then um it was very expensive to run, right?
29:00
Unidentified speaker: So we we're like, how do you turn this into a real product, right?
29:04
Unidentified speaker: So um we had we had people come in and they were given like one or two agents.
29:09
Unidentified speaker: It's not the same thing, right?
29:11
Unidentified speaker: You don't you don't really get to experience the whole like agent society, but even if you did, it probably would be a bad product, right?
29:19
Unidentified speaker: I think the same reason like moatbook didn't work out, right?
29:22
Unidentified speaker: Just people don't really want to see a bunch of agents talking to each other and just generating what is mostly slop, right?
29:29
Unidentified speaker: They're interesting things, but a lot of it is slop, or don't want to see it generated.
29:33
Unidentified speaker: Um so the product just essentially failed, right?
29:38
Unidentified speaker: Our like day one retention was 10%.
29:41
Unidentified speaker: So that was late 2024, and we were kind of in a crisis as a company.
29:46
Unidentified speaker: Um, and at the time, Anstropic did a computer use demo, and we immediately played with it.
29:54
Unidentified speaker: And I remember thinking we can do better than this.
29:57
Unidentified speaker: So we spent a month focused on building better computer use agents, and we actually came up with lots of good ideas.
30:04
Unidentified speaker: Uh, one of them was essentially we came up with the idea of a skill, right?
30:08
Unidentified speaker: And if the agent would do some task, and not only a skill, but agent would do some task and would like like introspect their behavior and then write down like a file, right, that will help it do uh uh do it better next time, right?
30:26
Unidentified speaker: Which is exactly like how like how right now you can like generate auto auto-generate skills.
30:32
Unidentified speaker: So we did that in back in late 2024, and we made the same mistake again, was thinking, oh, we'll turn this into a product right away.
30:40
Unidentified speaker: And we didn't publish any of this research.
30:43
Unidentified speaker: We we did use it to raise funding, right?
30:45
Unidentified speaker: So we raised like a series A of like maybe 30 30 something million dollars.
30:51
Unidentified speaker: Uh and uh and then now we started to uh just like try to productionize this.
30:56
Unidentified speaker: Um so we had two paths.
30:59
Unidentified speaker: One is to build a general computer use agent, another is to do something more focused, right?
31:05
Unidentified speaker: Like focused on Excel.
31:06
Unidentified speaker: The reason we focused on Excel is we realized even when we were doing the computer use uh research, we realized the current models are really, really bad at using Excel, but it's not because they don't understand it.
31:18
Unidentified speaker: It's because they actually don't have the right interface to work with Excel.
31:22
Unidentified speaker: Right.
31:23
Unidentified speaker: Um, so if you give them a better interface, like again, MCP was not even a concept back then, right?
31:29
Unidentified speaker: So but but we realized if you give them a better interface, these models, even in late 2024, are already intelligent enough to operate on Excel, at least do a lot of like the basic work, right?
31:41
Unidentified speaker: Not like full-on building of a DCF, but they can use they can build a pivot table, right?
31:48
Unidentified speaker: This is something basic like that.
31:50
Unidentified speaker: So that's something we realize essentially people think that agents cannot use Excel, but we realize it's not an intelligence problem, it's like an interface problem.
32:01
Unidentified speaker: Um, the problem is if we were to focus on Excel, um we had a like a essentially a company gene problem, right?
32:12
Unidentified speaker: Because we just pivoted from doing games.
32:16
Unidentified speaker: Uh like the whole team is still kind of thinking a lot about like consumer product, right?
32:22
Unidentified speaker: You're going from games to um going from games to productivity is already a big change, right?
32:28
Unidentified speaker: But even within productivity, there are different types of productivity, right?
32:32
Unidentified speaker: So um so if you're saying with the team that we had, which is a lot of great people, that we're gonna go straight to investment, it would have been impossible, right?
32:42
Unidentified speaker: Uh so what happened is like we're like um let's do Excel, but no one in the company knows anything about Excel, except for Nico, who's my co-founder, who actually was a financial consultant uh at EY for two years, right?
32:57
Unidentified speaker: Um so where he like really did a lot of Excel work.
33:01
Unidentified speaker: So he was able to see that if we just focus on Excel, then we can build a product that's really valuable.
33:08
Unidentified speaker: And uh, at the same time, we were trying to build this like general computer use product that we just couldn't really find use case.
33:15
Unidentified speaker: Um in retrospect, there are use cases, but these use cases are probably better for companies that have massive distribution, right?
33:24
Unidentified speaker: Uh, whereas for like a startup with no distribution, it is better to start with like a concrete, concrete, like vertical, concrete uh customer profile.
33:34
Unidentified speaker: So we didn't have that understanding back then.
33:37
Unidentified speaker: Um yeah, so we launched a computer use agent product called Fairies, which I think had a lot of good ideas, and uh, but it just didn't work because there were no clear like use case.
33:51
Unidentified speaker: And we launched at the same time, uh around the same time, the Excel agent shortcut, and that was immediately like well received on Twitter, right?
34:00
Unidentified speaker: People are really, really excited about that.
34:03
Unidentified speaker: Uh and that was a vir viral viral moment.
34:06
Unidentified speaker: Yes, yes, it went.
34:10
Unidentified speaker: Yes.
34:10
Unidentified speaker: So I think it was the first moment that uh lots of people realized okay, Excel is within reach by the AI, right?
34:18
Unidentified speaker: Um, of course, in the last 12 months, what we have seen is okay, um, how do you how do you get it to be good enough so it can really, really deliver like um something that's inspectable and understandable, right?
34:34
Unidentified speaker: Like um really good looking, doesn't have a lot of errors, right?
34:38
Unidentified speaker: It's it's one thing to have a demo where it seems to build a financial model that works.
34:43
Unidentified speaker: It's another thing to put it in the hands of professionals and they are happy enough to give it to their colleagues, right?
34:50
Unidentified speaker: So I think a lot of the work we did in the last 12 months is to close this gap.
34:54
Unidentified speaker: And also thankfully the models have been getting better and better every release.
34:59
Unidentified speaker: Right.
34:59
Unidentified speaker: Yeah, walk us through that because I remember um yeah, I remember after that moment in summer 25 immediately subscribing and running it through some of my workflows.
35:12
Unidentified speaker: Yeah, and you could sort of see some interesting effort in the level one workflows, but anything more complicated, it really started to choke on.
35:22
Unidentified speaker: Uh so just walk us through that walk us through that experience.
35:26
Unidentified speaker: Yeah, yes.
35:26
Unidentified speaker: So that was a pretty difficult time for us um because it went hyper-viral, right?
35:32
Unidentified speaker: So we had lots of people come in and tested, um, but then a lot of them churned because uh I think the model is like um not quite there at the point, right?
35:43
Unidentified speaker: The models was not quite there.
35:45
Unidentified speaker: Our harness can obviously could have been improved uh more, but I think fundamentally it's just like we were maybe like one generation early, right?
35:55
Unidentified speaker: Um, so a lot of people come in and try to do very sophisticated things and just realize it's it's not it's not really doing that.
36:04
Unidentified speaker: Um and uh it was a difficult time because for like two, three months we had on the paper, it's very viral.
36:12
Unidentified speaker: Um, but when we're looking at the metrics, everything was going down, right?
36:16
Unidentified speaker: Like people were churning and the retention was too low, right?
36:21
Unidentified speaker: So so the revenue just kept going down.
36:23
Unidentified speaker: Uh in fact, we attempted a fundraise that didn't work during that time because like people were like, Oh, this is very viral, let's talk.
36:29
Unidentified speaker: And but then when they see the actual numbers, like things were going down.
36:34
Unidentified speaker: So uh in around October last year, um, we just sat together and we're like, okay, let's let's just focus on making this really, really good.
36:42
Unidentified speaker: Right.
36:43
Unidentified speaker: Um, so the team we we um we we essentially cut every other project, just really focused on Excel.
36:51
Unidentified speaker: And one of the biggest things I think we did is to build a really, really strong internal benchmark, right, for real work.
37:00
Unidentified speaker: Uh so this is uh Richard, uh who's our chief product officer now, he led that work where he essentially collected uh he essentially built these tasks that are in distribution, which means it's like similar to what our users would do, right?
37:16
Unidentified speaker: For example, it's like uh leverage buyout, right?
37:19
Unidentified speaker: This is for private equity, uh discounted cash flow, this is for like like public company investment.
37:26
Unidentified speaker: Um and and the important thing is is it needs to be hard enough for the agent, but also the you need to be able to trust the score, right?
37:36
Unidentified speaker: Uh so building a benchmark is very hard because sometimes you you have a task and you have some prompt.
37:42
Unidentified speaker: Uh, there are many ways to do the task, right?
37:45
Unidentified speaker: So there is no one correct way, and and then you cannot really like evaluate the output.
37:51
Unidentified speaker: So the task needs to be designed in a way that's both like natural, as in this is something a user might really do, but also precise enough that there is one correct answer, right?
38:02
Unidentified speaker: So that was pretty hard.
38:03
Unidentified speaker: Now, once we have built these tasks, then it become much easier to kind of iterate on the agent harness, right?
38:12
Unidentified speaker: Which is like everything that goes around the language model.
38:15
Unidentified speaker: And we did a lot of like um like detailed optimization for the Excel use case, right?
38:22
Unidentified speaker: Making sure that like essentially um it can really understand the the Excel API, right?
38:29
Unidentified speaker: Uh, and uh how does the agent manage its context very efficiently so it can it can do the task?
38:36
Unidentified speaker: Because like if you think about just naively, how do you take a big big Excel like uh table, right?
38:43
Unidentified speaker: Um just you have lots of call, you can have lots of columns and lots of rows.
38:47
Unidentified speaker: How do you put that all in the the context of the language model?
38:50
Unidentified speaker: And if you're not careful, you're gonna just pollute their context right away, right?
38:55
Unidentified speaker: So there had to be a lot of engineering up to be done.
38:58
Unidentified speaker: Uh that's very detailed, very specific to spreadsheet.
39:02
Unidentified speaker: Um and once we started to do these things, then uh it's just started to work, right?
39:08
Unidentified speaker: We saw all the metrics were going up, like retention is up, like users started grow, and then we started to get into the enterprise, had figured out how to do like enterprise grade security and things like that, right?
39:20
Unidentified speaker: We have never done that before.
39:22
Unidentified speaker: Um and then it's then it's uh it just started to become better, right?
39:27
Unidentified speaker: So so um I think turning point was probably November last year, right?
39:33
Unidentified speaker: Um yeah.
39:35
Unidentified speaker: And and uh you walk me through you know, there's there's really been no in sort of the core hedge fund community that that I sort of lived in, there hasn't really been a transcendent product yet.
39:49
Unidentified speaker: Um I think most hedge funds I I I talked to really mostly use Claude.
39:55
Unidentified speaker: Um I had a moment sort of independently in September, you know, September 22nd when Opus 5.5 came out, and you know, the the model started to pass my modeling benchmarks in a way that was pretty astounding.
40:09
Unidentified speaker: The day I tweeted that, David from from Shortcut reached out, he's like, Have you tried Shortcut?
40:14
Unidentified speaker: I'm like, Yeah, man, I tried it last July and it was terrible.
40:17
Unidentified speaker: Uh he's like, try it again.
40:20
Unidentified speaker: Um and um and so from that sort of moment from September to uh September last year to September this year, um you sort of found this ICP in in the hedge fund in the hedge fund level.
40:34
Unidentified speaker: Can you tell us why that happened?
40:37
Unidentified speaker: Yeah, yeah, absolutely.
40:38
Unidentified speaker: So I think um I I think a lot about this, right?
40:43
Unidentified speaker: Uh we I don't think we had like great foresight, right?
40:48
Unidentified speaker: That this is a good um um this is kind of good customer for us, right?
40:53
Unidentified speaker: Um we were we were still, I would say in the beginning, still didn't have a clear understanding of how to position ourselves, right?
41:01
Unidentified speaker: We're like we're the best Excel agents, right?
41:04
Unidentified speaker: But but then mass companies with massive distributions like Microsoft, cloud, how how do you differentiate against them, right?
41:12
Unidentified speaker: Well, our answer is always like, well, our product is just better, right?
41:16
Unidentified speaker: Like our agent is better, more accurate, blah, blah, blah.
41:18
Unidentified speaker: And it turns out for a lot of people, it doesn't matter as much, right?
41:22
Unidentified speaker: Or like they're not as sensitive to this, right?
41:24
Unidentified speaker: Because let's say I want to like build a personal budget tracker, right?
41:30
Unidentified speaker: Um, and I'm not very rich, right?
41:31
Unidentified speaker: So I'm not going out and buying yachts, right?
41:34
Unidentified speaker: So it's like it's there's not that much, right?
41:36
Unidentified speaker: It's like, okay, like try to put your like limit your grocery, right?
41:41
Unidentified speaker: Um things like that.
41:43
Unidentified speaker: Um, so so cloud uh for Excel or like um Microsoft Copilot can do these things just fine, right?
41:51
Unidentified speaker: What the people who are able to most like feel that difference between our product and um and our competitors' product, right?
42:01
Unidentified speaker: Like cloud for excel are people who are doing really cutting-edge demanding work, right?
42:06
Unidentified speaker: And it turns out a lot of these are like the hedge fund analysts, right?
42:09
Unidentified speaker: That's not exclusive to those, but as you know, the hedge fund analysts, the buy-side analysts, these are in some way the pinnacle, right?
42:20
Unidentified speaker: Of like the work they do is extremely like the sophisticated.
42:24
Unidentified speaker: It has to be because if it's not good, then you just don't wait, don't make money, right?
42:29
Unidentified speaker: Um certainly the tolerance for error is uh incredibly low.
42:33
Unidentified speaker: Yeah, exactly.
42:34
Unidentified speaker: One error and the one lazy error in the wrong cell of a financial model can be really destructive.
42:40
Unidentified speaker: Yeah, exactly.
42:41
Unidentified speaker: Right.
42:41
Unidentified speaker: You can get fired, right?
42:42
Unidentified speaker: If you just put give some slop to your uh PM.
42:46
Unidentified speaker: Uh so so these so these people they really felt the difference.
42:54
Unidentified speaker: And uh and um and I think in retrospect, uh, this is a really great match between our company gene and and our customers, right?
43:04
Unidentified speaker: Our company gene has always been like very research and tech driven, right?
43:09
Unidentified speaker: That's why I think when a lot of people just are kind of just like a lot of our competitors, maybe not Astropic, but you know, like there are other competitors, they don't really have benchmarks for a long time, right?
43:19
Unidentified speaker: It's just kind of vibed, right?
43:22
Unidentified speaker: Just feel the vibe of it.
43:23
Unidentified speaker: And we had benchmark which allow our agents to be better, and we can compare it against Cloud for Excel and know for a fact it's better, right?
43:32
Unidentified speaker: Like we can compare it and say, oh, like the token efficiency is higher, the performance is higher, the success rate is higher, right?
43:39
Unidentified speaker: So that's I think that focus on accuracy, because we have been doing agents, we've been benchmarking agents since 2023, um just resonated with with uh like our investor customers who are also very much about accuracy and precision and performance.
44:02
Unidentified speaker: Yeah, that makes sense.
44:04
Unidentified speaker: Um and uh you know building product in um in the world of AI sort of feels like a constant building of sand castles and the better lesson sort of evolving, like is as cloud directly has gotten better.
44:19
Unidentified speaker: How do you think about you know Excel agent as a as a platform?
44:23
Unidentified speaker: Yeah, absolutely.
44:24
Unidentified speaker: So this is something we have been grappling a lot in the company in the last right six months, uh six even twelve uh six months.
44:33
Unidentified speaker: So first we had like first we were worried about okay, like can this product work at all, right?
44:39
Unidentified speaker: Let's find uh product market fit, right?
44:41
Unidentified speaker: So once we found product market fit, um at least I was pretty concerned about the long-term prospect, right?
44:49
Unidentified speaker: It's you like there were one point early on where like, oh, like maybe this everyone who uses Excel would use this, right?
44:56
Unidentified speaker: But at some point it's becoming clear this is not gonna happen, right?
44:59
Unidentified speaker: The big labs, like most people are gonna use this like widely distributed product, right?
45:05
Unidentified speaker: So if you're going with uh like um like a customer profile like hedge fund, then you can't you then it's it's like if you're trying to serve them at some point, there's not that much you can do with Excel agent alone that's keep delivering them value, right?
45:26
Unidentified speaker: Um, so I was thinking a lot about this, and uh and at some point I came to the conclusion that uh shortcut or like the generally the product that FRL build have to uh kind of go from Excel agent and go like from this foundation and go beyond it, right?
45:46
Unidentified speaker: Um and if we look at if we look at analysts, a lot of what they do is is um well, you build your financial model, you update them, um, but then you have to do research, right?
45:57
Unidentified speaker: Uh you have to come up with like some thesis, some hypothesis, right?
46:02
Unidentified speaker: So so there's a lot more than building the financial models.
46:06
Unidentified speaker: And also these things are are not independent, right?
46:09
Unidentified speaker: As in the financial model you build can inform your thinking, then inform the research you do, right?
46:14
Unidentified speaker: Vice versa.
46:15
Unidentified speaker: So naturally uh we want to expand on this, right?
46:20
Unidentified speaker: But it's it's not some like arbitrary expansion, right?
46:23
Unidentified speaker: Because you can say, well, I I have an Excel agent.
46:27
Unidentified speaker: Now let me have a Word agent and a PowerPoint agent, right?
46:31
Unidentified speaker: Just like expand along the office suite, right?
46:33
Unidentified speaker: So we didn't want to just like expand like across more softwares, right?
46:38
Unidentified speaker: We want to expand, we want to expand along like to go from doing work to really creating value, right?
46:46
Unidentified speaker: Um, or at least we want to be able to get to the point where uh our users can create value uh better and more easily, right, by using our product.
46:58
Unidentified speaker: So so a hypothetical experiment is like, okay, if you can build an agent or like a team of agents that operate like a hedge, like a hedge fund, or even just like a team within a like a multi-manager hedge fund, right?
47:12
Unidentified speaker: Like a portfolio manager and analyst, right?
47:15
Unidentified speaker: If you just have uh agent teams like that, and you show that they can actually do paper trading and uh and make good outcome, right?
47:26
Unidentified speaker: The sharp ratio is high, etc.
47:28
Unidentified speaker: etc.
47:29
Unidentified speaker: Now you provide that foundation to real professionals, and they can build on that and they can customize that and they can make it really, really good, right?
47:41
Unidentified speaker: So that's that's what we're thinking now, right?
47:44
Unidentified speaker: The core of it is still like the workhorse is an Excel agent that's really good at like understanding models, but beyond that, you're gonna have agents that is able to do research, right?
47:55
Unidentified speaker: That's that's able to like keep monitoring news, right?
47:59
Unidentified speaker: And um, and uh all these agents need to be orchestrated, for example, by some portfolio manager.
48:07
Unidentified speaker: So this is this is what we're building next.
48:10
Unidentified speaker: Yeah, yeah, yeah.
48:11
Unidentified speaker: That makes sense.
48:12
Unidentified speaker: And what what has been your reaction to the rise of the personal agents?
48:18
Unidentified speaker: The sort of instinct was the first to hit natural sort of awareness, then Muse far behind, and now we have Hark Labs and Rockbox getting a lot of attention recently.
48:30
Unidentified speaker: Like what was what was your take on uh what what was and is your take on those personal agents?
48:35
Unidentified speaker: Yeah, yes.
48:36
Unidentified speaker: So the first shock was OpenClaw, right?
48:39
Unidentified speaker: So OpenClaw, I was I think earlier this year, right?
48:42
Unidentified speaker: It feels crazy because it feels like a long time ago, but I think it was just earlier this year.
48:47
Unidentified speaker: Um, and uh, when that first came out, we did a journal club like every week on OpenClaw for three weeks in a rope because we felt like this is so important.
48:58
Unidentified speaker: And uh, in many ways, it's very similar to what we did back in Minecraft, right?
49:03
Unidentified speaker: These like long-running agents that had their soul, that had their memory that keep getting updated.
49:09
Unidentified speaker: Um, they're autonomous, they're proactive.
49:12
Unidentified speaker: So I think we felt a sense of um maybe loss, like, oh, like we missed out on this opportunity.
49:20
Unidentified speaker: Um, but now there's an opportunity to for us to combine, I guess, personal agent with what we have been doing.
49:28
Unidentified speaker: So I definitely think the but overall, I think the the rise of personal agent reflects a really deep desire to have a better interface with agent, right?
49:43
Unidentified speaker: Because the traditional interface we have had as people who, for example, use codecs, uh, cloud code, and and etc.
49:51
Unidentified speaker: Is like, okay, you have these threads, right?
49:54
Unidentified speaker: Everything you want to do is a thread, right?
49:56
Unidentified speaker: Um, and at some point it becomes very like annoying to manage, right?
50:05
Unidentified speaker: You have all these threads, and and it was okay when you're just working on like three threads or like two threads at a time, but as the agents become more and more long-term autonomous, right?
50:16
Unidentified speaker: So this is almost mathematical.
50:17
Unidentified speaker: As the agents become more and more long-term autonomous, right, they require less intervention from you.
50:23
Unidentified speaker: And as a result, you can run more threads concurrently, right?
50:28
Unidentified speaker: So if you have to, if an agent can be independent for five minutes and require one minute, and then you have to intervene and it takes one minute for you to intervene, then you can run five agents at a time.
50:41
Unidentified speaker: But if they can run 50 minutes and it takes one minute to intervene, then you can run 50 of them at a time, right?
50:46
Unidentified speaker: If you can afford it, etc., etc.
50:49
Unidentified speaker: But when you can run 50 agents at a time, then having 50 threads that are just there and you have like jumping back and forth, it's crazy.
50:57
Unidentified speaker: And I know this because I've done it, right?
50:59
Unidentified speaker: I've had time like having like literally 50 agents running, 50 threads running concurrently.
51:05
Unidentified speaker: At some point, I had three monitor, I had three desk, I had three desk, seven monitors, right?
51:12
Unidentified speaker: And each monitor had like six terminal opens, terminal windows open, each with was a like a cloud code or codex red.
51:21
Unidentified speaker: So I would literally have have like 40 terminals open just like all around me, and I was like managing them.
51:29
Unidentified speaker: I would like turn my chair and manage them.
51:32
Unidentified speaker: And that's a very bad user experience, right?
51:37
Unidentified speaker: So yeah, that's a very bad user experience.
51:41
Unidentified speaker: Whereas in reality, right, you can talk to someone who manages a large team without having to worry about micromanaging every single person, right?
51:50
Unidentified speaker: Um, so I think that's fundamentally what the personal agents are doing.
51:54
Unidentified speaker: Most users don't really care how the work is done, right?
51:56
Unidentified speaker: They just want it done.
51:58
Unidentified speaker: They don't want to be like individually managing all the 50, 50 sub-agents, 50 agents that's that's working.
52:04
Unidentified speaker: Um, and I think we see that definitely in our users as well, right?
52:08
Unidentified speaker: Our some of our power users are really like going very like um very hard, very hard on like multi-agent, right?
52:19
Unidentified speaker: And we want to provide them also a better interface so that they can just talk to like one agent who will essentially manage all their agents for them, right?
52:29
Unidentified speaker: And they can talk to them just like you would talk to a human instead of having to type out some prompt.
52:35
Unidentified speaker: How how how close do you think we are to working with a digital hedge fund analyst?
52:41
Unidentified speaker: And as a sort of existing analyst, we just interface with that, with that being as we would a new junior analyst on the team.
52:49
Unidentified speaker: Yeah, yeah, absolutely.
52:50
Unidentified speaker: So I think um the short answer is I think very, very soon.
52:56
Unidentified speaker: The model capability is already there.
52:58
Unidentified speaker: I think it's just like and the interface, we already kind of know what it should be.
53:02
Unidentified speaker: So it's it's just about some polish.
53:04
Unidentified speaker: But I think within like like two months for sure, like people uh it it will be there, but then not everyone's gonna have it, right?
53:13
Unidentified speaker: Because it takes time to go through the the the essentially the compliance uh review for good reasons, right?
53:20
Unidentified speaker: It takes time to go through this, but I think the industry, uh pretty much everyone would be using these, I would say in six to six to nine months, right?
53:31
Unidentified speaker: I think I think as in there'd be a point if you work at like an investment firm, and if you're not like working with these kind of always-on agents, right, like a digital analyst, then you would feel like it's like prehistoric, right?
53:47
Unidentified speaker: It's like doing work today manually in Excel, right?
53:50
Unidentified speaker: I'm like building out everything just with keyboard shortcut, right?
53:54
Unidentified speaker: So that's how people would feel.
53:57
Unidentified speaker: Um, that's my guess, is maybe six to twelve months, right?
54:02
Unidentified speaker: Uh so now that said, um I don't think it will necessarily be uh a complete, like, oh, like a human analyst, right?
54:15
Unidentified speaker: So I do think there are things that it will do much better than an analyst, like a real human analyst.
54:20
Unidentified speaker: Like it will just like monitor all the news like much much better, much faster, preemptively, do a lot of analysis.
54:27
Unidentified speaker: But there's still a lot of things that a human can do much better than um than a model.
54:34
Unidentified speaker: Uh and and I don't think that's gonna go away that fast, right?
54:38
Unidentified speaker: For for one, is to really build kind of um conviction, right?
54:47
Unidentified speaker: Conviction on something, right?
54:49
Unidentified speaker: Um for example, uh a good financial model need not only be like syntax syntactically correct, right?
54:58
Unidentified speaker: It's like that's a baseline, right?
54:59
Unidentified speaker: There shouldn't be like like big mistakes in the equations, but it should should also have a view, right, about your future, right?
55:06
Unidentified speaker: Your your thesis.
55:08
Unidentified speaker: Um and that view needs to be expressed through the financial model.
55:11
Unidentified speaker: And so so for agents to come up with a strong enough view that is not um is a little bit contrarian, right?
55:23
Unidentified speaker: Um, but turn out to be more correct than the consensus, I think that's that's still gonna be pretty hard.
55:30
Unidentified speaker: All right.
55:31
Unidentified speaker: And that's something that really good, I think good human investors, uh, they will do.
55:37
Unidentified speaker: And it's still an open question about how to do that in the machines.
55:41
Unidentified speaker: Yeah.
55:42
Unidentified speaker: Yeah, something I've been very passionate in asking Claude an investment question.
55:49
Unidentified speaker: You you get you know often a different answer, you know, six times, you know, yeah, six days, you get six different questions, and then if you push back on the answer, you get the yes, sorry, I was wrong.
56:00
Unidentified speaker: So it's sort of spineless by by nature, the the ability to sort of inspect the actual Excel artifact and sort of trace the assumptions that the agent is making in the Excel structure to me has been something that has been super, super eye-opening.
56:16
Unidentified speaker: Um, I think this is this is definitely like um I guess we talked a lot about like spineless, and I think that's that's a profound issue with um with today's language models, right?
56:33
Unidentified speaker: They're trained to be uh aligned, and but the specific ways that they're aligned is essentially to kind of just like more or less agree with you, right?
56:42
Unidentified speaker: Now, of course, if you throw them something that they know are not facts, right?
56:45
Unidentified speaker: If you tell them like, look, United States is actually full of like lizard, right?
56:49
Unidentified speaker: Something that it's then they'd be they'd be like, no, no, no, that's wrong, right?
56:53
Unidentified speaker: But but for things that are require opinion, right, they don't take they don't take stance, right?
57:00
Unidentified speaker: And that's that's fundamentally for for the big labs, they have to do it, right?
57:06
Unidentified speaker: So like imagine like a language model actually taking a political stance, right?
57:10
Unidentified speaker: It'd be really bad.
57:11
Unidentified speaker: Like and it will piss off half of half of Americans if it takes any stance, right?
57:16
Unidentified speaker: Um so so you can't do that.
57:19
Unidentified speaker: But but of course, a great investor, they have some stance, right?
57:24
Unidentified speaker: They have some belief in the world, right?
57:27
Unidentified speaker: Um, and that's how you can hold on to like a good stock for a long time, right?
57:33
Unidentified speaker: Without like just like, oh, like it's up, it's up like 60% year on year.
57:39
Unidentified speaker: Let me just like sell it, right?
57:41
Unidentified speaker: No, no, no.
57:42
Unidentified speaker: If you believe that this can go like if you if you get NVIDIA right 10 years ago, right, and you're like, I believe in this, I believe in AI, right?
57:50
Unidentified speaker: You can hold on it forever, right?
57:52
Unidentified speaker: You can still hedge by like short other things, but um, but you're like, no, NVIDIA, I'm gonna hold on on to it.
57:58
Unidentified speaker: So that I think requires the agents to really have uh to be opinionated, right, in a way that is grounded in real facts.
58:08
Unidentified speaker: Um and it takes human influence into account, but fundamentally it's about like being grounded in facts, being grounded in the market, right?
58:17
Unidentified speaker: And the real like finance of a company, of the industry and what it believes the the future will be, right?
58:26
Unidentified speaker: Instead of being just like so easily moved by whatever the last prompt from a human they got, right?
58:34
Unidentified speaker: So getting to agents that can do that, I think is something that that as you know, I'm very excited.
58:41
Unidentified speaker: Um, and that that is a human quality that I don't think it's we're gonna get it, like at least not in production within the next 12 months.
58:51
Unidentified speaker: But I'm pretty hopeful that in the next two to three years we will start to see agents that not only have opinions, because it's easy to have opinions, you're gonna have them have like a bad opinion very quickly, but right, but it's like its opinions are actually grounded in like real world and it's able to adjust its opinion based on real feedback, but not based on just like any like random human influence.
59:20
Unidentified speaker: How do you do that technically?
59:21
Unidentified speaker: Is that sort of a skills pack?
59:23
Unidentified speaker: Is that starting to post-train the model weights, like what you know?
59:29
Unidentified speaker: What like what's the what's the pathway?
59:30
Unidentified speaker: Like, how do you take this generic agent, consumer agent structure and make that useful for the highly specific vertical, yeah, vertical process of it?
59:42
Unidentified speaker: Yeah, absolutely.
59:43
Unidentified speaker: Absolutely.
59:44
Unidentified speaker: I think this is a staged process, right?
59:47
Unidentified speaker: So um, so the first thing is is like good skills, right?
59:52
Unidentified speaker: You and good skills would go such a long way already, right?
59:56
Unidentified speaker: You just they don't like again, right now they're kind of easily influenced by whatever is in their prompt, right?
1:00:02
Unidentified speaker: So just give them the right prompt, right?
1:00:04
Unidentified speaker: Just give them the right context, right?
1:00:06
Unidentified speaker: You can be like a particular firm with a particular view, and or you can be a particular like a team, right, and a multi-manager um uh hedge fund and and your team has a particular view, right?
1:00:20
Unidentified speaker: So, so kind of distill these views into a set of skills, right?
1:00:25
Unidentified speaker: And give it to the agent.
1:00:26
Unidentified speaker: And the agent would already become more opinionated in the right direction, right?
1:00:31
Unidentified speaker: Uh, and and you can then continually evolve these skills, um, and then they will be able to still shape their shape their opinion, right?
1:00:40
Unidentified speaker: So I think that's that's a first step.
1:00:42
Unidentified speaker: And that I think this will go a long way already, right?
1:00:46
Unidentified speaker: Lots of people are not doing this.
1:00:47
Unidentified speaker: So we're seeing this in our users.
1:00:49
Unidentified speaker: Um, I would say our users are already like really some of the best.
1:00:54
Unidentified speaker: A lot of our users are really some of the best, best uh pros, right, in this business.
1:00:59
Unidentified speaker: Uh and but even among these pros, right, some people are like like really like ahead in like how they use AI, right?
1:01:07
Unidentified speaker: They build very elaborate kind of like skill, like like uh collections of skill.
1:01:12
Unidentified speaker: And some people are still using it without skills, right?
1:01:15
Unidentified speaker: They're kind of just like like um just doing it raw.
1:01:19
Unidentified speaker: Uh now giving them like a starter set of skills, right?
1:01:24
Unidentified speaker: That's like that's like thoughtful and like professional, and then they can they can like tune it and adapt it to have their view, their thesis, right?
1:01:33
Unidentified speaker: I think that's the first step that's gonna get us a long way.
1:01:36
Unidentified speaker: Uh now, once you have done that, how do you go further, right?
1:01:41
Unidentified speaker: So the fundamentally we can we just gotta realize the the models, this kind of spineless behavior of the model is uh is built into the models through post-training, right?
1:01:54
Unidentified speaker: So the pre-training is like, okay, predict the next word, right?
1:01:57
Unidentified speaker: From like all the text we have as in humanity, right?
1:02:02
Unidentified speaker: So that part is just about compressing, compressing the world, right?
1:02:05
Unidentified speaker: Like understanding the world.
1:02:07
Unidentified speaker: It's it's it doesn't have opinion, it doesn't have like specific kind of behavior yet, right?
1:02:13
Unidentified speaker: And then human like the labs essentially post-train them to behave in a certain way, right?
1:02:18
Unidentified speaker: So it turns out you can actually have their early evidence for that, um, not just from ourselves, but from the community, that you can like further post-train them to unlearn a lot of these behaviors, right?
1:02:30
Unidentified speaker: Um, then you can actually get them to not be so like spineless, right?
1:02:36
Unidentified speaker: Um then they can actually have some like stronger opinion, right?
1:02:41
Unidentified speaker: So I think that's that'd be the second step is like okay, post-training, um like post-training an open weight models to for it to write character-wise, right, like uh personality-wise, to to be able to hold their opinion.
1:02:57
Unidentified speaker: Uh uh at the same time, knowledge-wise, they can really understand like a specific sector much, much better, right?
1:03:05
Unidentified speaker: Um, you can have like an agent that really dig into like like 20 companies, right?
1:03:13
Unidentified speaker: 50 companies, they understand everything that's out there about these companies, right?
1:03:18
Unidentified speaker: They understand the management, they understand the the personal history of the management, the the relationship between the management, all the related sectors, the historical like performance.
1:03:32
Unidentified speaker: They understand all these things and they keep reasoning about these things.
1:03:35
Unidentified speaker: And so, so then I think we get closer to to uh these agents being really, really useful for investors to the point that they can probably suggest kind of long, short, you know, decisions that turn out to be pretty good, right?
1:03:53
Unidentified speaker: Like a human would probably still like take this and make discretionary like decisions, but they would increasingly realize oh, um, it turns out the agents' suggestions were just like almost just right, right?
1:04:07
Unidentified speaker: There's not like I I need to fact check it, I need to think about it, and but but it turned out they're very good.
1:04:15
Unidentified speaker: Uh and I think the final stage, right?
1:04:17
Unidentified speaker: So there's a skills, there's a post-training.
1:04:19
Unidentified speaker: I think the final stages we'll get to a point where we might question the question the entire kind of like the paradigm, right?
1:04:27
Unidentified speaker: The architecture, the way we do training.
1:04:29
Unidentified speaker: A great capability of human um is we're able to learn from very, very small amount of data, right?
1:04:38
Unidentified speaker: And this is this is opposite from the paradigm of like current AI, right?
1:04:43
Unidentified speaker: Current AI is trained on and human, like it's just like incomprehensible large amount of data, right?
1:04:52
Unidentified speaker: You can say, right?
1:04:52
Unidentified speaker: Like it's read the whole internet, and yeah.
1:04:55
Unidentified speaker: Yeah, it's like 100 trillion tokens, right?
1:04:57
Unidentified speaker: So it's it's a it's and uh and that's um that's just a very different paradigm.
1:05:05
Unidentified speaker: Whereas a human, like kind of from a baby to right now, don't really process that much data, right?
1:05:12
Unidentified speaker: Now some people are like, oh, but but how about the human civilization, right?
1:05:15
Unidentified speaker: We have processed data that went into our gene, et cetera, et cetera.
1:05:18
Unidentified speaker: But your gene doesn't really encode that much information, or at least it doesn't encode the specific information, right?
1:05:24
Unidentified speaker: Like your gene is not encoding like the stock price, right?
1:05:27
Unidentified speaker: So um so human, despite not having all this information, is able to do great reasoning about like the market, understanding about like the world.
1:05:40
Unidentified speaker: And that's a remarkable capability, right?
1:05:43
Unidentified speaker: If we have agents that can do that, right, we can imagine every single day the new information that comes in, they'll be able to absorb it and like think about how that should change their investment decisions.
1:05:55
Unidentified speaker: I think that would be very, very powerful, essentially having this like truly continual learning system.
1:06:00
Unidentified speaker: You can build a continual learning system today, right?
1:06:03
Unidentified speaker: But but have a truly continual learning system where it's able to see some one piece of information that's like really significant and change its entire like worldview almost, right?
1:06:17
Unidentified speaker: It's like, oh, like today is election day, right?
1:06:20
Unidentified speaker: I still remember 2016 election day.
1:06:24
Unidentified speaker: Essentially, like all the predictions were like Clinton is gonna win and then turn out Trump won't, right?
1:06:31
Unidentified speaker: So it's I think I think a good model would be able to take that one bit of information, right?
1:06:37
Unidentified speaker: It's literally a single bit, right?
1:06:39
Unidentified speaker: It's like a single bit of information, a binary decision.
1:06:44
Unidentified speaker: That one bit of information, and really like, okay, like let me like reorganize all the knowledge I have processed in the past and change my worldview so that I can really understand what has happened here and really better understand how I should do investment in the future, how do I should do decision making in the future?
1:07:04
Unidentified speaker: Right.
1:07:05
Unidentified speaker: So I think getting to that point is probably longer.
1:07:08
Unidentified speaker: It's like maybe five five years, five to ten years.
1:07:11
Unidentified speaker: Fascinating.
1:07:12
Unidentified speaker: Fascinating.
1:07:13
Unidentified speaker: Uh so what's what's what's the plan?
1:07:15
Unidentified speaker: What are what are we what are we doing here?
1:07:17
Unidentified speaker: Yeah, yes.
1:07:18
Unidentified speaker: So I think I think for us, um building, kind of starting with the Excel agent that we already have, that a lot of like um top investors already use, right?
1:07:31
Unidentified speaker: Just keep going from there to okay, can it do company research?
1:07:37
Unidentified speaker: Can it do stock, like suggest stock picks, right?
1:07:41
Unidentified speaker: But not just but in a way that's really productive to the human, right?
1:07:45
Unidentified speaker: Because if you're just out there just like suggesting stocks, but the human is not able to really understand the rationale, uh, it's not able to kind of improve the system with their own like uh opinion, then it's useless, right?
1:07:59
Unidentified speaker: Like you you can do that with a with a systematic like approach any day, right?
1:08:03
Unidentified speaker: So the important thing is to build this product that is increasingly capable, but really from a perspective of how do you enable human and really allow them to express themselves, right?
1:08:17
Unidentified speaker: So I think that's that's a critical thing.
1:08:19
Unidentified speaker: And if we can do that like step by step, um I think it's it's I I'd be very happy.
1:08:26
Unidentified speaker: And I think we'll get to a point in two years where humans would really trust these agents a lot about making decisions, and they would they would kind of step back and think more about all like how do I build this system, right?
1:08:42
Unidentified speaker: If you think about like uh someone who runs a uh like an entire hedge fund, right?
1:08:48
Unidentified speaker: Like the owner of a hedge fund, they're not out there picking stocks every single day, right?
1:08:52
Unidentified speaker: Um, at least that's not most of what they do.
1:08:54
Unidentified speaker: But doesn't mean they don't have work, right?
1:08:56
Unidentified speaker: They're kind of thinking, okay, how do I build the culture?
1:08:59
Unidentified speaker: How do I build the system, right?
1:09:01
Unidentified speaker: How do I build the organization, right?
1:09:03
Unidentified speaker: And I think humans would transition, a lot of humans would transition to be more like that in the next two years, right?
1:09:09
Unidentified speaker: Analysts would be more like PMs, PMs would be more like CIOs.
1:09:14
Unidentified speaker: Yeah, and it and I sort of view my role as like a continuation of what I've been doing the last four years.
1:09:20
Unidentified speaker: I've been training human fundamental analysts and you know, sort of sort of joining FRL is a lot about training this digital analyst, which is uh sort of you know interesting opportunity to um uh starting with the skills pack and going into the model weights and yeah, we'll work through how to back like create clean back tests for some of these qualitative signals, uh which is sort of a fascinating exercise.
1:09:46
Unidentified speaker: So yeah.
1:09:47
Unidentified speaker: So something I'm really excited about is I think I think it's one thing to be a great investor, right?
1:09:53
Unidentified speaker: Like there are lots of people who are who uh who like okay, like really understand their sector.
1:09:59
Unidentified speaker: And um, but it's an it's another thing, I would say related, but uh but not the same skill set to be able to teach people, right?
1:10:06
Unidentified speaker: Now so I was a teacher myself before, right?
1:10:09
Unidentified speaker: So to teach people, you need to not only like know the craft, you need to also have the humility to understand the craft is a little different in every firm, right?
1:10:19
Unidentified speaker: And it's not about okay, like this is how you should do it.
1:10:22
Unidentified speaker: It's about like I'm gonna provide you the basics so that now you're enabled and you can go on and like do even better things, right?
1:10:30
Unidentified speaker: So I think exactly that's aligned with what we're trying to do here, right?
1:10:33
Unidentified speaker: It's through our product.
1:10:35
Unidentified speaker: And so I'm just like super excited for you to come in and like teach our agents so our agents can then empower the uh our clients.
1:10:44
Unidentified speaker: Yeah, my first uh my first task is to teach an agent to pass a typical hedge fund interview process, which uh you know we haven't done quite yet, but we're making good progress towards, and uh and then build the FDI, the forward deployed investor team, because that's exactly right.
1:11:00
Unidentified speaker: There's obviously huge sensitivities around data security, sandbox, connecting to internal context.
1:11:06
Unidentified speaker: But to me, what's been really interesting about the multi-agent structures is the improvement in memory, the improvement in the ability to have recursive sort of auto improvement in the skills.
1:11:16
Unidentified speaker: Yeah.
1:11:16
Unidentified speaker: Um and almost the same way a hedge fund hires a very talented MBA who has a base level of productivity and uh skills.
1:11:25
Unidentified speaker: The way an analyst academy graduate has a base level of fluency and uh investment research.
1:11:31
Unidentified speaker: Um there's sort of a sandbox delivery, sandbox delivery where that agent then learns from internal context, internal experience, and uh sort of seems sci-fi, but it feels like the pieces are there to actually we are living in the sci-fi now.
1:11:47
Unidentified speaker: It's like every every couple of weeks I'm like, wow, that's this is sci-fi.
1:11:52
Unidentified speaker: And it's but it's real, it's it's happening.
1:11:55
Unidentified speaker: So yeah.
1:11:56
Unidentified speaker: Yeah, yeah.
1:11:57
Unidentified speaker: The demos so far demos are interesting, and um, and then I think there's just a big training and enablement piece.
1:12:02
Unidentified speaker: If you have these super agents that can do core parts of your investment process better than you can now, like how do we as humans you know spend our time?
1:12:12
Unidentified speaker: How do we re reallocate our time?
1:12:15
Unidentified speaker: I mean, certain workflows like that model validation, I think, are now solved problems with the right approach.
1:12:20
Unidentified speaker: Um certain air areas like model updates for quarters is a are a solved problem.
1:12:26
Unidentified speaker: Um and so how do you redeploy that freed up time on other parts of the investment process?
1:12:31
Unidentified speaker: I think that's a fascinating, fascinating um question as well.
1:12:36
Unidentified speaker: Yeah, yes.
1:12:38
Unidentified speaker: It's uh something I've found is like, okay, Asians seem to be like, I mean, agents are undeniably making us more efficient, but somehow I still work the same number of hours, right?
1:12:49
Unidentified speaker: Like I don't I don't like what's going on, right?
1:12:52
Unidentified speaker: It's like okay, can can I just can I just work less now?
1:12:55
Unidentified speaker: Yeah, yeah, we'll have to, but yeah, I I I bet our I bet the the people who are using AI the most uh in our like client firms are are not resting, are not really resting.
1:13:08
Unidentified speaker: They might even get AI psychosis where they were just like, oh, it's a red, it's a red queen race.
1:13:13
Unidentified speaker: I mean, I think certainly at the top tier of alpha generation.
1:13:16
Unidentified speaker: I mean, it's uh it's it's sort of playing poker against other, you know, sort of uh, you know, Tony starts with the Jarvis suit.
1:13:24
Unidentified speaker: And so I think collectively the the dumb mistakes humans make will get uh sort of ironed out through better agentic support.
1:13:33
Unidentified speaker: Um listen, I I've sort of I I remain bullish on alpha, the alpha opportunity just because of the market microstructure and the distortion of so many market participants.
1:13:42
Unidentified speaker: You still have quants and indexers and uh duration constrained pods really you know doing a lot of things that are not are not a direct expression of a fundamental insight.
1:13:53
Unidentified speaker: Um yeah, I think effectively what we're trying to do is build a Jarvis suit for Tony Stark for the for the investor to help the help you know sort of um make good investors better.
1:14:05
Unidentified speaker: Um which is really exciting.
1:14:08
Unidentified speaker: It's amazing how fast it's evolved from prompts to sort of multi-agent structure.
1:14:13
Unidentified speaker: Yeah, I can't believe that's a good idea.
1:14:14
Unidentified speaker: Yeah, it's just it's wild.
1:14:16
Unidentified speaker: Yeah, it's wild.
1:14:17
Unidentified speaker: Two years ago, again, two years ago, we were really like investors were impressed that our agents can make a pivot table.
1:14:25
Unidentified speaker: Like that's like we like that's two years.
1:14:28
Unidentified speaker: It's been it's like we we've gone a long way.
1:14:31
Unidentified speaker: Yeah.
1:14:32
Unidentified speaker: Yeah, not to make this a commercial either, because invest with AI will stay an independent entity uh and um you will continue to work to drive the conversation of the intersection AI and investing forward.
1:14:44
Unidentified speaker: Um but obviously uh we're gonna try and open source a lot of what we're doing on blog, Twitter, etc.
1:14:52
Unidentified speaker: Um and uh be happy to have conversations with firms that are interested interested in uh in this in the in this journey.
1:14:59
Unidentified speaker: Fundamental edge will continue to operate on the training side uninterrupted.
1:15:04
Unidentified speaker: Uh Invest with AI will continue to operate on the podcast uninterrupted.
1:15:10
Unidentified speaker: But again, this is uh sort of a once-in-a-generation opportunity to um sort of a really unique moment that the primitives are sort of all coming together in a way that um yeah, there's been increasingly uh built conviction that the way investment research done is changing pretty rapidly, probably at an escalating pace.
1:15:31
Unidentified speaker: Yeah.
1:15:33
Unidentified speaker: Cool.
1:15:34
Unidentified speaker: Thank you so much for for being uh being here, Robert.
1:15:36
Unidentified speaker: I'll see I'll see you in the Slack channel after this.
1:15:40
Unidentified speaker: Yeah, yeah, I'll see you.
1:15:41
Unidentified speaker: Thanks for having me.
1:15:42
Unidentified speaker: Thanks, Robert.
1:15:44
Unidentified speaker: Bye.
This page was produced automatically: the audio was transcribed with nvidia/parakeet-tdt-0.6b-v3 and interpreted by Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL. Transcription and interpretation can contain errors; always confirm against the original episode. Nothing here is investment advice.

More from Invest with AI

All insights