Street Signals

Michael McNair argues AI-driven cyber risks are systemic, underpriced, and trapped in a prisoner's dilemma preventing regulation.

1h 1m Listen to the original Episode page
CybersecurityArtificial IntelligenceSystemic RiskInvestment PhilosophyGeopoliticsRegulationTail RiskMarket Sentiment

Summary

Michael McNair, an analyst at the Retirement Systems of Alabama, argues that AI-driven cyber risk is a systemic, economy-wide threat that markets have not yet priced in. He contrasts modern AI attacks with past threats like Y2K, noting that AI combines high skill with massive scale, creating irreversible damage through shared dependencies and phishing. McNair frames AI development as an 'AI Prisoner's Dilemma,' where competitive incentives and the inability to verify compliance force nations and companies to accelerate development despite the risks. He suggests that tail risks are underpriced and that a major cyber event could drive demand for gated security SaaS models.

Episode sections

  1. 0:03 – 4:05

    Introduction and Guest Background

    Host Tim Graff introduces guest Michael McNair, an analyst and portfolio manager at the Retirement Systems of Alabama, a $60-70 billion internally managed public pension fund. McNair discusses his background, noting he started during the financial crisis and focuses on US domestic stocks, particularly TMT, communications, materials, and utilities.

  2. 4:05 – 12:15

    Investment Philosophy and Mental Models

    McNair describes his investment process as similar to George Soros, focusing on marginal investor sentiment rather than intrinsic value alone. He cites 'Jesse Livermore' as a key influence for his first-principles approach and emphasizes writing detailed theses to ensure logical soundness and avoid bias.

  3. 12:15 – 17:54

    Cyber Risk as an Underappreciated Systemic Threat

    McNair argues that cyber risk is a systemic, economy-wide threat that the market has not yet priced in, comparing the current market complacency to the pre-pandemic dismissal of epidemiological risks. He believes the market is 'whistling past the graveyard' regarding the severity of AI-driven cyber threats.

  4. 17:54 – 21:32

    Paradigm Shift: Skill and Scale in Cyber Attacks

    McNair contrasts modern AI-driven cyber attacks with Y2K and 2000s worms, arguing that while past attacks lacked skill or scale, AI now combines both. This allows for cheap, scalable, zero-day exploits that can target individuals with the precision previously reserved for nation-states, creating irreversible damage.

  5. 21:32 – 24:45

    Open-Weight Models and Offensive AI Capabilities

    The discussion highlights that open-weight models from Chinese labs (like DeepSeek) can have safety guardrails removed, enabling superhuman cyber capabilities. McNair notes that models capable of defense are also capable of offense, creating a security dilemma where releasing defensive tools releases offensive power.

  6. 24:45 – 30:30

    Systemic Risks: Shared Dependencies and Supply Chains

    McNair identifies two main systemic risks: shared dependencies (like Linux kernels or cloud providers) where a single flaw can compromise thousands of systems, and the 'death by a thousand cuts' via AI-enhanced phishing against individuals, which serves as the weak point for corporate security.

  7. 30:30 – 32:24

    Political Backlash and the AI Prisoner's Dilemma

    McNair discusses the political backlash against AI, citing referendums on data centers in Ohio and bipartisan anger. He frames AI development as an 'AI Prisoner's Dilemma,' where competitive incentives force actors to continue accelerating development despite the risks, similar to nuclear proliferation.

  8. 32:24 – 34:08

    AI vs Nuclear Prisoner's Dilemma

    Speaker_2 argues that unlike nuclear weapons, AI development cannot be paused via international coordination because there is no physical way to verify that other nations are not secretly cheating. This inability to verify compliance forces all nations to continue developing AI to avoid falling behind, trapping them in a prisoner's dilemma.

  9. 34:08 – 37:25

    Mutually Assured Destruction (MAD) and AI

    The speaker explains that MAD does not apply to AI because the technology used to attack is the same as that used to defend. Unlike nuclear weapons where defense could theoretically be distinct, AI defense requires superior AI, creating a race where any catastrophe accelerates development rather than pausing it.

  10. 37:25 – 42:02

    Political Risks and the Nirvana Fallacy

    Speaker_2 discusses the 'Nirvana fallacy' in politics, where voters compare real-world constraints to utopian solutions. He argues that unilateral AI regulation by democracies like the US or Europe would make them less competitive and less safe, while non-democratic rivals like China would continue developing without pause.

  11. 42:02 – 48:44

    Development vs. Deployment and Distillation

    The speaker distinguishes between developing models and deploying them commercially. He notes that companies like Anthropic and OpenAI are gating access to frontier models to prevent distillation (copying), while China may allow unrestricted release to force US labs to compete on price or risk losing market share.

  12. 48:44 – 54:16

    Liability and Regulatory Pressure

    Speaker_2 highlights bipartisan political angst against AI, particularly regarding data centers and liability. He argues that US AI labs desperately need federal legislation to limit state-level liability (similar to Section 230 or the Gun Manufacturers Protection Act) to remain competitive against foreign rivals.

  13. 54:16 – 58:16

    Market Reaction and 'Black October'

    The speaker references 'Black October' as a period of exponential cyber risk growth. He suggests that a major systemic cyber event would create insatiable demand for gated security SaaS, potentially making the gated access model more profitable than the commoditized model of selling raw model capabilities.

  14. 58:16 – 1:01:24

    Tail Risk Hedging

    Speaker_2 advises that tail risks from AI cyber attacks are currently underpriced by markets. He suggests that hedging against these risks is cheap, similar to the period before the COVID pandemic, and that investors should consider the low cost of insurance against a correlated market drawdown.

Assets discussed

AssetKindStanceContext
Anthropicprivate companyneutralSo the companies like Anthropic and OpenAI are gating them their like most frontier models.
OpenAIprivate companyneutralSo the companies like Anthropic and OpenAI are gating them their like most frontier models.

“Stance” is how the speakers talked about the asset, not a recommendation.

Claims worth verifying

The Retirement Systems of Alabama is a $60-70 billion self-managed public pension fund for nearly 400,000 members.

Tim Graff ·0:52

How to verify: Retirement Systems of Alabama investor relations website or latest 13F/annual report. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Michael McNair runs a mid-cap fund focused on TMT, communications, materials, and utilities across different funds at the Retirement Systems of Alabama.

Unidentified speaker (speaker_2) ·2:58

How to verify: Retirement Systems of Alabama organizational structure disclosures or McNair's Substack/X profile. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

George Druckenmiller is an investor whose process McNair considers most similar to his own, focusing on macro and marginal investor sentiment.

Unidentified speaker (speaker_2) ·4:05

How to verify: Michael McNair's Substack archives or public interviews. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

The NASDAQ was off 4 percent at the time of recording, a couple of days after the US midterm elections.

Tim Graff ·11:27

How to verify: NASDAQ composite index historical data for November 2026. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

CDS spreads for Oracle were widening out, indicating financial risk concerns.

Tim Graff ·11:47

How to verify: Bloomberg or Refinitiv CDS data for Oracle (ORCL). For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

In early February 2020, epidemiologists universally predicted a global pandemic, but the market treated it as a minor headline until it crashed 35% in a month.

Unidentified speaker (speaker_2) ·13:18

How to verify: Historical news archives from February 2026 and S&P 500/NASDAQ performance data. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

The UK, France, Japan, and Australia have advised citizens to keep three days' worth of food, water, and generators.

Unidentified speaker (UNKNOWN) ·17:45

How to verify: Government emergency preparedness guidelines from the UK Home Office, French government, Japanese government, and Australian government. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

SQL Slammer, I Love You, and Code Red were examples of 2000s worms that scaled but lacked skill.

Unidentified speaker (speaker_2) ·19:12

How to verify: Historical cybersecurity reports from Symantec, Microsoft, or CISA regarding these specific malware events. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

A user with a $20 OpenAI account found a root-level zero-day in Ubuntu in approximately two hours.

Unidentified speaker (speaker_2) ·28:36

How to verify: Ubuntu security advisories or cybersecurity news reports from late 2026 regarding the specific Ubuntu vulnerability. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

GMO Internet Group in Japan was hacked, with ransomware threats issued to customers' machines.

Unidentified speaker (UNKNOWN) ·29:03

How to verify: GMO Internet Group press releases or cybersecurity news reports from November 2026. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Nuclear weapons had multiple physical signatures that allowed verification of compliance, whereas AI training in data centers is physically indistinguishable from other compute.

Unidentified speaker (speaker_2) ·33:03

How to verify: Primary source: Michael McNair's published article or research paper on 'Cyber Risk and the AI Prisoner’s Dilemma' referenced in the episode description. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Hugging Face's CEO stated that they were attacked with AI but also defended against AI, and since they cannot stop others, they must push AI capabilities further.

Unidentified speaker (speaker_2) ·36:42

How to verify: Primary source: Public statements or interviews by Hugging Face CEO (e.g., Clément Delangue) around the time of the podcast release (Nov 2026). For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Europe regulated its AI, which made it less safe because it disconnected from frontier defensive capabilities while remaining connected to global internet risks.

Unidentified speaker (speaker_2) ·39:06

How to verify: Primary source: EU AI Act implementation reports and cybersecurity analyses comparing EU defensive capabilities to US/China during the verification period (2026-08-12 to 2026-11-12). For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

China uses distillation to copy Anthropic's models at a fraction of the cost to stay competitive with the US.

Unidentified speaker (speaker_2) ·42:45

How to verify: Primary source: Technical reports on AI distillation techniques and Chinese AI development strategies from 2026, or public disclosures by Anthropic/OpenAI regarding model leakage. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Anthropic launched a security SaaS business model offering gated access to verified users for hardening code.

Unidentified speaker (speaker_2) ·47:34

How to verify: Primary source: Anthropic's official product announcements or press releases from late 2026 regarding their security offerings. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

The industry has named the recent period of exponential cyber risk growth 'Black October'.

Unidentified speaker (speaker_3) ·54:16

How to verify: Primary source: Cybersecurity industry reports or news articles from October 2026 referencing 'Black October' and associated cyber incidents. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

US AI labs need federal legislation similar to Section 230 or the Lawful Commerce in Arms Act to limit state-level liability.

Unidentified speaker (speaker_2) ·50:45

How to verify: Primary source: US legislative records or public statements by AI industry lobbyists in late 2026 regarding proposed liability protection bills. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Markets typically do not crash on new risks without historical precedent, as seen with the initial market reaction to COVID.

Unidentified speaker (speaker_3) ·57:16

How to verify: Primary source: Historical market data analysis comparing S&P 500 performance during the early COVID-19 period (Jan-Mar 2026) versus current market behavior. For recent events, verify 2026-08-12 through 2026-11-12. Use another period only when the speaker explicitly names it.

Do your own research

  • Verify the claim that cybersecurity experts universally view AI as a systemic risk by reviewing statements from major cybersecurity firms (e.g., CrowdStrike, Palo Alto Networks) and government agencies (CISA).
  • Check for recent legislative actions or referendums on AI data centers in Ohio and other states to assess political backlash trends.
  • Analyze the competitive landscape between US and Chinese AI labs, specifically looking for evidence of 'distillation' techniques and open-weight model releases from Chinese entities like DeepSeek.
  • Review financial filings of major cloud providers and security SaaS companies to identify exposure to 'gated access' business models versus commoditized model sales.
  • Investigate the regulatory environment for AI liability in the US, specifically looking for discussions around federal preemption of state-level liability similar to Section 230.
  • Compare current market pricing of cyber insurance and tail risk hedges against historical data from pre-COVID periods to assess if risks are indeed underpriced.
  • Look for primary sources on the 'alignment problem' in AI defense, such as technical papers from Anthropic or OpenAI regarding the dual-use nature of defensive AI models.
  • Verify the claim that Y2K remediation costs were in the tens of billions by reviewing historical IT spending reports from the late 1990s.
Full transcriptexpand
0:03
Unidentified speaker (UNKNOWN): This
0:03
Tim Graff: is Street Signals, a weekly conversation about markets and macro brought to you by State Street Markets. I'm your host, Tim Graff, Head of Macro Strategy for Europe. Each week we talk about the latest insights from our award-winning research, as well as the current thinking from our strategists, traders, business leaders, clients, and other experts from financial markets.
0:23
Unidentified speaker (UNKNOWN): If
0:23
Tim Graff: you listen to us and like what you're hearing, please subscribe, leave us a good review. get in touch, it all helps us to improve what we offer. With that, here's what's on our minds this week.
0:32
Unidentified speaker (UNKNOWN): There
0:32
Tim Graff: are so many wrinkles to the market's fascination with investment in artificial intelligence, and we've done our fair share of episodes on that. But this week we have what I think is one of the more unique takes from a portfolio manager who's actively allocating capital around this space, but who also sees risks that he thinks are dramatically underappreciated and on. underpriced by markets. And
0:52
Tim Graff: that's quite a thing to say after the last few weeks, especially when you consider the volatility we saw in AI stocks following the U.S. midterm elections. But as you're about to hear, these sobering aspects of the AI trade are critical to get in front of people and to think about. My guest is Michael McNair. He manages an equity portfolio for the retirement systems of Alabama, which is a $60 sixty
1:12
Tim Graff: to $70 seventy billion dollar self self-managed managed public. pension fund for nearly 400,000 active and retired members. He also shares his thoughts on markets on his own Substack, which is how I came across him. Michael McNair, it's so great to meet you and to get a chance to talk to you. I've been reading your stuff for about the last year, and we got a lot of cool stuff to talk about with respect to AI and the geopolitics of it
1:32
Tim Graff: and the domestic politics of it now. So first and foremost, thanks so much for joining the podcast this week.
1:38
Unidentified speaker (UNKNOWN): Well,
1:38
Unidentified speaker (speaker_2): thanks for having me. on. I I I really appreciate that uh you found the work interesting and I think a lot of my stuff has kind of run a gamut where we were talking about a lot of trade and capital flows and more recently have been focused on AI and cybersecurity rests, which we'll get into. But
1:56
Tim Graff: yeah, no well uh it's funny because I think I found your work probably as I say about a year ago. It was the picture of a chart from actually one of our presentations. It must have been like a CFA society thing or something that you got it from. I'm I'm not even sure, but I was curious actually just to start, uh you know, especially for guests who haven't been on before, I'm curious to hear kind of your background, where you're from, what you do really, and and kind of
2:16
Tim Graff: how you got to where you are right now, just as a starting point.
2:19
Unidentified speaker (speaker_2): So yes, I'm from Alabama, live in Alabama now, and you know, we're a sixty, seventy billion dollar pension fund. which depending on kind of where the pl private placement marks go, we're a little unique, or I think very unique in the fact that we are a hundred percent internally managed. So all the private equity, the public equity is
2:39
Unidentified speaker (speaker_2): all it's all internally staffed. No outside managers, no consultants, anything like that. And it's a phenomenal place to work. I uh we have you know our my boss is you know just doesn't believe in any bureaucracy. So, you know, I've run a ran a mid cap fund for a long time, now on kind of
2:58
Unidentified speaker (speaker_2): TMT communications to materials and utilities book where I run that across our different funds. It's awesome. We don't you know, there's no committees, it's very few meetings, everything's just kind of open door. You know, like I said, it's it it's it's uh unique in the fact that, you know, everything is
3:19
Unidentified speaker (speaker_2): is done internally here. So, you know, I actually started here uh during the financial crisis. Such an interesting time to you know, but but such a great experience to ha having seen that. And, you know, I have kind of been, Then you know, on the public equity space, just focused on US domestic
3:39
Unidentified speaker (speaker_2): stocks.
3:40
Unidentified speaker (UNKNOWN): Yep.
3:40
Tim Graff: Yeah. I mean, uh it's it's quite clear from reading your stuff. And by the way, I'll mention it a few times, but Michael has a substack that I encourage people to check out. Everything so far I think is free. It's pretty clear reading your stuff that you're very, very well read and and especially well read in the investment universe. And I'm curious if there's a thinker or someone in markets who you've kind of looked at as said,
4:00
Tim Graff: a like who I'm really keen to follow. Their process really makes a lot of sense to me.
4:05
Unidentified speaker (speaker_2): I mean, I think that if you're talking about an just a an investor, I mean, Drunkenmiller is kind of the guy that that I probably that people would know that that I think the process is most similar to where, you know, it is kind of macro focused. I use this example of where to
4:25
Unidentified speaker (speaker_2): a lot of you know our anal young animals. analysts where my mental model uh is kind of like a shopkeeper where it's what matters is you want to have inventory that your customers want to buy. And you know, your real opin opinion of it doesn't matter. It's the marginal investors' opinion. So it's
4:45
Unidentified speaker (speaker_2): understanding where the narrative's gonna go and that narrative drives price and you know not getting stuck with stuff stale inventory. It's not that valuation doesn't matter to me. It's it's that it matters because it matters to other investors, you know, and that's a dynamic cost that can matter more or less at different times. It's understanding who your customer is, right? Who is it
5:05
Unidentified speaker (speaker_2): is a that it's gonna be buying these stocks? Is it a you know an in ESG funds, you know, getting a bunch of fund flows in Europe or is it, you know, pension funds, is it you know, it it's it's understanding, you know, who the customer is and understanding my overall opinion on something doesn't matter. It's the
5:24
Unidentified speaker (speaker_2): marginal investor who you know you're selling to and buying from that matters. I think he probably has you know a similar kind of I guess philosophy. But in terms of people that I love to read, there's you know someone on, I guess, r ran a a blog for a long time. You guys by the pseudonym Jesse Jesse
5:44
Unidentified speaker (speaker_2): Livermore. And I I love the way he kind of breaks things down from this, you know, first principles perspective. And he lays out the entire logic and the mental models that he needs to be able to come to this eventual conclusion. And he puts everything out there. And so with my process, I write constantly. I I feel
6:04
Unidentified speaker (speaker_2): like that's the only way I can really make sure my logic is sound in at every single step. I mean, I think Jeff Bezos, who you know ex DJ guy, when he got to Amazon, he instituted I think it was like the thirteen page memo where he outlawed the use of PowerPoint. And he said
6:24
Unidentified speaker (speaker_2): that that PowerPoint allows you to gloss over a lot of the little interconnected details that you need to form your logic. And writing forces you to see where the gaps are. And so I am constantly writing and thinking about, okay, let me put this down. That also helps me when
6:44
Unidentified speaker (speaker_2): I go back and I'm like, okay, did I get if I got this stock right, did I really get it right for the right reasons? And are the things that in my thesis are are we seeing the things that should have we should have seen if this thesis was correct or not? And so it keeps you honest. It allows you to go back and evaluate your trades. And you know a lot of the stuff
7:04
Unidentified speaker (speaker_2): I can't really write a lot of stock specific stuff
7:09
Unidentified speaker (speaker_2): on my Substack or on X. So
7:13
Unidentified speaker (speaker_2): the substance of that is more bigger picture, kind of macro, different themes where I'm not having to deal with compliance on on on specific stocks. But you know, the majority of my writing internally is that's exactly what it is. It's it's about, you know, different industries
7:28
Unidentified speaker (UNKNOWN): and
7:29
Unidentified speaker (speaker_2): you know companies that I'm trying to trying to work out the thesis for. So I think Jesse Livermore has kind of been somebody that I have really enjoyed. reading and probably the the archetype of kind of what I've tried to do with a lot of my writing.
7:45
Tim Graff: Yeah. Yeah. Well, I mean, it's great because it is, as I say, it is how I first caught notice of you and really grateful to to get your time. And we're going to talk a lot about some of the the current theses you have. But one of the things you said about Druckenmiller really struck me because, you know, as as an institution, we are very focused on institutional. behavior and flows and consensus
8:05
Tim Graff: risks and things like that. The narratives, especially, are things we really try and track. But of course, there's a flip side to all of this behavioral type of work, behavioral finance type of stuff like the Akerloff and Schiller books that I think most people listening to this will have probably at least have a familiarity with. But with the positives of behavioral finance, you also have the negatives.
8:25
Tim Graff: And I'm curious, and I ask this of everybody. every buy-side client who I have on, is we all have biases. And I'm curious to hear about, you know, sort of the self-reflection you've done as an investor. What are yours? What do you have to kind of work on to overcome when it comes to investing?
8:39
Unidentified speaker (speaker_2): I have, I think inherently I have a contrarian streak. And that can be dangerous because, like I said, you know, what matters is what the marginal buyer and the seller. seller think. That leads me to oftentimes be too early. I start that that's been a process of understanding, okay, when is the narrative going to change?
8:59
Unidentified speaker (speaker_2): What is the event or the realization? And sometimes I I want to try to get to the conclusion before I think the market in general has come to that conclusion. So being contrarian can be difficult because you, by definition, the majority of
9:19
Unidentified speaker (speaker_2): the market does not see what you're seeing yet. It's the point of having the thesis. Let me write this out and then say, okay, what am I looking for when other people will start to notice this? And there are contrarian trades you can make where you say, All right, I want to step in now and it's an
9:39
Unidentified speaker (speaker_2): asymmetric fit. But then there's trades where you're just early and it just the market's not going to recognize that for, you know, a year or more. And so it's been a learning process of just pattern recognition and certain steps of of like I said, the writing the thesis down. Yeah.
9:58
Unidentified speaker (speaker_1): And
9:58
Unidentified speaker (speaker_2): be like, okay, here it is. Come back to this. Here are the steps and now I can kind of ha have a process where I can, you know, use some AI tools to help me, you know, evaluate, like, all right, come back to this and remembers everything that I wrote and talked to about it
10:18
Unidentified speaker (speaker_2): and and can help me remember all right, this is what you talked about six months ago. Like here are all the signposts you were looking for. And I think that is relevant for what we're gonna talk about because the cybersecurity stuff is something I've been extremely focused on for the past five months,
10:38
Unidentified speaker (speaker_2): spent an enormous amount of time on it, but have haven't really taken big swings at it, other than like focused focus that on on like the cybersecurity stocks, which I've ripped for the past few months, but
10:53
Unidentified speaker (UNKNOWN): all
10:53
Unidentified speaker (speaker_2): although have have pulled back here in the last couple of weeks. But the big thing for me is just trying to get timing right with
11:01
Unidentified speaker (UNKNOWN): that
11:01
Unidentified speaker (speaker_2): personality and not try to just swim against the the grain too much.
11:07
Unidentified speaker (UNKNOWN): Yeah,
11:07
Tim Graff: I mean that that's the big topic that we are going to get into is is cyber risk, but it's all of course part of the broader AI narrative. And I've been doing a bunch, I think any any podcaster focused on financial markets is talking about AI or has done a lot of episodes about it. But today I think we're going to take a bit of a unique slant, but it's it's a really interesting day, literal day to record it, in that NASDAQ's off four
11:27
Tim Graff: percent. We are a couple of days after the US midterm elections, which we're also gonna talk a little bit about and how that might impact AI and cybersecurity. But of course, this is a narrative that has started to get a lot of red flags thrown at it through financing channels and some of the struggles that the neo-cloud companies have been having and CDS spreads for
11:47
Tim Graff: companies like Oracle widening out. All of these things are becoming threats to what has up till recently been a mostly bottom-left to top right trend in AI-related stocks. And markets are starting to become more wary of. these risks, financial risks, cyber risks, negative externalities around
12:07
Tim Graff: AI, politics. Which of these is the one that really stands out to you, just as kind of the segue to the topic? What's the big one
12:14
Unidentified speaker (UNKNOWN): for
12:14
Unidentified speaker (speaker_1): you?
12:15
Unidentified speaker (speaker_2): The the financing, the questions over profitability. I've kind of spent a lot of time on that. That argument's out there. There's a question of, again, like when does the market start worrying about that? Now, like for me, it is finding the topics and the the things that are going to become relevant that
12:35
Unidentified speaker (speaker_2): people aren't thinking about yet. That's not priced.
12:37
Unidentified speaker (speaker_1): Yeah.
12:38
Unidentified speaker (speaker_2): And to me, what has happened on the cyber risk is is something that is is very obvious to the experts, but the market has just not digested any of this yet. I I I hate making like this like Doomerism. I I that is not my personality. I I don't make these
12:58
Unidentified speaker (speaker_2): big bearish, you know, calls. I've literally made one time before have I made a public bearish call before. Unfortunately, that situation has a lot of parallels with what I see right now. And that was in early February of twenty twenty. before the pandemic. And we had, you know,
13:18
Unidentified speaker (speaker_2): access to a lot of experts and they were these epidemiology, they were they were I mean, universally saying this is going to be a global pandemic. And the market was treating it like another Ebola headline. And when you really dug down, you know, and I was telling people warning people to this is coming and and everybody was just told
13:38
Unidentified speaker (speaker_2): me we can't have a global pandemic in this day and age. And so you saw that the market was conditioned for one thing, but the evidence was overwhelming that we were going to have a global pandemic. And the market didn't wake up to this risk. You know, it basically was went from all-time highs to crashing 35% in a month.
13:58
Unidentified speaker (speaker_2): And so we had to digest that entire, you know, global pandemic. in a matter of months. That's similar to the risk that I see now on the cybersecurity stuff, where I have spent the last five months just finding every expert I can talk to. And they are universally saying what is
14:18
Unidentified speaker (speaker_2): coming with the cyber risk is a systemic
14:23
Unidentified speaker (speaker_2): economy-wide risk. And the market is just whistling past the graveyard.
14:31
Unidentified speaker (UNKNOWN): I
14:31
Unidentified speaker (speaker_2): mean, it was they aren't even they're treating it like why is this cyber risk different than we we went through the 2000s and people used to have bugs all the time. You know, you go on and download something from the appster and you get a virus, right? Yeah. So people are like, well, what is different about this? And I and I think what has happened is like if you go
14:51
Unidentified speaker (speaker_2): to the forums and and just part of the internet where cybersecurity acts were talking about, it's like people are absolutely terrified.
14:59
Unidentified speaker (UNKNOWN): Yeah.
14:59
Unidentified speaker (speaker_2): And increasingly so within the finance and market people, it that hasn't started percolating over in the understanding. And so there hasn't been anybody I've seen that really okay, let me explain why this is so bad. Like and so it's hard for people that don't feel focus that aren't experts on this to understand and say, okay, this is why it's different.
15:19
Unidentified speaker (speaker_2): Yeah. So that's what I was hoping to do today is just kind of explain to people, this is why it is a significant risk.
15:27
Unidentified speaker (speaker_1): Yeah, definitely. Definitely.
15:28
Unidentified speaker (UNKNOWN): And and
15:29
Unidentified speaker (speaker_2): and and I I should say that like there are, you know, in terms of for the market, you have, you know, just business disruption risk, economy disruption. risk. But the first thing that's going to hit the market and is is going to be, and you don't even need for there to be a you know severe economic disruption from
15:49
Unidentified speaker (speaker_2): it. It's the second-order political blowback that comes from AI catastrophes and just, you know, a change in just the paradigm of everybody just constantly getting attacked is going to be intense regulation. You know, I have
16:04
Unidentified speaker (speaker_2): very close with, you know, who I think is one of the best political analysts on the street. He made it clear to me. He's like, I've been in the center for since the mid 90s. And the only time I've ever seen a bigger just bipartisan backlash more than what people have on data centers right now was
16:23
Unidentified speaker (speaker_2): privatizing Social Security. And he's like, you can go to the most red state no in Oklahoma, uh, you
16:27
Unidentified speaker (UNKNOWN): know,
16:27
Unidentified speaker (speaker_2): Red County in Oklahoma, and they are just, I mean, he's like, you can't even talk to people about it sensibly about it. I mean, people are across the board just very anti data center. And I think it's a manifestation of people's AI risks, just their their
16:47
Unidentified speaker (speaker_2): nervousness about what AI is. And so this is like a policy thing that they actually have in front of them right now that they can have an opinion on and made that felt. But I don't think it's going to stop at data centers. The point being it's like there's an economic risk and and people need to understand what's different. And then there's like what happens to the market because of that. So I
17:07
Unidentified speaker (speaker_2): think what everybody's going to be talking about to talk about like, okay, what's where's the narrative going to change? I think how everybody had to become a you know an amateur epidemiologist in 2020s. Everybody came, you know, straight up for moose supply chain expert. People are gonna be cybersecurity experts in at least six months. So
17:26
Tim Graff: yeah. Let's go with that cybersecurity risk because as
17:29
Unidentified speaker (UNKNOWN): you
17:29
Tim Graff: mentioned, you know, this is something I we we've lived with. We had Y2K, we had very similar response to what you know. It's interesting, you get governments, the UK government has started to talk about people. needing three days worth of food, water, generators, things like that. And it's it's you you start to wonder what do they
17:45
Unidentified speaker (UNKNOWN): know?
17:45
Tim Graff: And I wanted to start to talk about what is different. What why is this the paradigm shift in in cyber risk versus some of the things you mentioned or Y2K?
17:54
Unidentified speaker (UNKNOWN): Yeah.
17:54
Unidentified speaker (speaker_2): First of all, it's not just the UK. France,
17:57
Tim Graff: Yep.
17:57
Unidentified speaker (speaker_2): Japan, Australia. That tells you it is cyber that to they're worry worried about. about. Because you know, you could say it's Russia if you're in your but Australia tells its citizens to have a weak birth of food and supplies. The number of of of these warnings that are coming out, it's it's kind of hard to deny right now. I I think it's clearly
18:17
Unidentified speaker (speaker_2): the cyber arrest that they're worried about. You're right. So so when people talk about you know Y2K, I mean that that was something everybody thinks about, and they're like, uh, that was like the boy who cried wolf. Like that that it didn't actually end up causing any big catastrophe. And that's that's actually not true. The reason why it didn't cause a problem is because we spent tens of billions of
18:37
Unidentified speaker (speaker_2): dollars rewriting all the code to be able to handle that'cause we knew when this would hit, you know, and it it we knew exactly what we needed to fix. The problem is we do not have that time right now. So we have
18:51
Unidentified speaker (speaker_2): been
18:52
Unidentified speaker (speaker_2): put in a massive paradigm shift, and we have a very unsecure system that is about to get attacked in ways that we just do not have time to prepare for. Yeah, like in the 2000s, there were worms and they weren't small. They they scale. You had, you know, SQL Slammer
19:12
Unidentified speaker (speaker_2): and, you know, I love you and Code Red. and all these. What they showed is that attacks can scale. But the important point is that it wasn't skilled. So a worm's dumb. It it ran one fixed payload against everyone. It plagued zero attention to any individual victim. It couldn't read your email or map your network or figure out who you are and find
19:32
Unidentified speaker (speaker_2): the one fact that ruins you and be able to act on it. So the damage was really limited, right? So you your computer stopped working. So you called like the Geek Squad or you were reinstalled. And that was the extent of the damage. But that is not what is going to happen now because before the security never rested on it being really
19:52
Unidentified speaker (speaker_2): secure. It rested on the fact that the skilled attacks didn't scale. Skill and scale were mutually exclusive, where you could attack a bunch of people very unskilled, right? Or you could have like a, you know, very, you know, a nation state attacking one individual and you could do all kinds of stuff, right? Now what
20:12
Unidentified speaker (speaker_2): is just broken is that we have skill and scale together. Now you you can have, you know, millions of of agents that are hacking and looking at what, you know, and and then they're able to find you know, zero days that, you know, that it used to be worth millions of dollars. And actually the price
20:32
Unidentified speaker (speaker_2): of these things collapsed because the supply is so ready. And they're finding new zero days constantly, right?
20:39
Tim Graff: Yeah. I should just quick interject here. A zero day is when you get a hack that
20:45
Tim Graff: the target doesn't know about yet. It they It's called zero day. They've had zero days to to address or fix the problem. So sorry, just
20:52
Unidentified speaker (speaker_2): And so the damage they can do with that could be irreversible, exfiltrating data and and act like a skilled hacker would do, right? I mean, i if a skilled hacker wanted to hack you, you could. But we were safe simply because the number of skilled hackers were so tiny relative to the almost endless amount of
21:12
Unidentified speaker (speaker_2): targets. So you were safe just because no skilled hacker got around to hacking you. That is no longer the case. So those new holes went being really rare to being incredibly cheap and industrial. Right. So zero days used to take a world class human a long time. And now,
21:32
Unidentified speaker (speaker_2): you know, take 10 minutes of anybody just saying prompting a model and saying, you know, find vulnerable vulnerability. And I think we should probably also say that the models from places like Anthropic and OpenAI, you know, they have strong safety guardrails, though you could still get them to do stuff, especially from OpenAI. But the open
21:52
Unidentified speaker (speaker_2): weight models from you know the Chinese labs in particular, those can be have their safety guardrails taken on. off. And their capabilities just lag maybe really like a few months. behind the US left frontier models. And so we first saw with Mythos where it comes out and anthropics like, okay, we cannot put this out
22:12
Unidentified speaker (speaker_2): there because it has superhuman cyber capabilities. That really got me working on it because I I knew what was coming with the open weight models when that happens, because it is just around the corner. And that's exactly what we're seeing saying now. I mean deep seeks 4.1 obliterated model, which is a model that can have its safety guardrails taken off to do all
22:32
Unidentified speaker (speaker_2): kind of, you know, hacking and everything. I mean, playing around with it. And I I don't know if I'd recommend this, but just pointed it at myself. And if you ask Claude to do that, they'll be like, absolutely do not do this. And
22:43
Unidentified speaker (UNKNOWN): I
22:44
Unidentified speaker (speaker_2): don't it's that actually gets to a broader point that like it's very hard in the cyber surety people are just like, I don't know what to do. But what companies are doing is like, you know, these models that are capable enough to defend are also the models that are that do cyber offense. And you can't
23:03
Unidentified speaker (speaker_2): release a capable model to defend yourself without releasing a model that can attack the system. And so that creates a big problem, right? So, you know, with when Hugging Face Scott got hacked, it's like they couldn't use Claude because it wouldn't let 'em they had the safety guardrails on.
23:19
Unidentified speaker (speaker_1): Right.
23:19
Unidentified speaker (speaker_2): So I mean when you see like with all the alignment guardrails taken off, I I hate to cuss, but I don't know another way to to describe it. They are evil
23:30
Unidentified speaker (speaker_2): I had to stop because I I you just can't control someone. something Like what they're going to do really. And it it is it was incredibly disturbing. I mean, you know. I'm like, oh, I didn't even know my soundbar has a microphone and it has access to listening to my house. Yeah. You know, it found all these passwords I had no idea were there. And but you know, it did
23:50
Unidentified speaker (speaker_2): help me because now I'm able to patch that. And like if there's one thing I can get passed for the listener, it is to take a number of steps. You know, even go just ask your LOM like, what do I need to secure my own personal file, you know, uh life? You know, freezing your credit, getting a password manager. And there's a whole list
24:10
Unidentified speaker (speaker_2): of things, but it takes a long time. It's like five hours of work. And so most humans, you know, most individuals aren't going to do that.
24:16
Tim Graff: Right. So let's let's get a bit specific. And particularly in how this could become something more systemic. Are there examples in your mind of exploits or tactics or techniques, or maybe even things we've already seen that kind of point you to the belief
24:36
Tim Graff: that a systemic problem might arise in the coming months because of the development of these tools and capabilities?
24:45
Unidentified speaker (speaker_2): There there's a couple of different risks that we can talk about. There's the big supply chain risk of like the shared dependency. Um that that that has systemic risk, right? That you have one shared dependency, you know, a Linux kernel that everyone uses that gets hacked and now they have, you know, rude access to this Linux
25:05
Unidentified speaker (speaker_2): kernel that basically allow people to put malicious code on anybody's computer that that's running that. That's a systemic risk. But then there's a death by a thousand cuts. And this is where I don't think even the cybersecurity experts are really focusing on right now because I mean, just the nature of what they focus on is kind of the big companies, those types of hacks. And
25:25
Unidentified speaker (speaker_2): what is going to happen is that individuals are going to be able to be attacked like nation states were attacked before. You know, you might have some scammer right now, and you know, just imagine, right? And what do they do? They're calling your grandmother and they're trying. to scam around. I mean, that's really bottleneck. Now, what they're gonna be doing, they're just gonna get on DeepSeek obliterated
25:45
Unidentified speaker (speaker_2): model and say, go attack this. And they can do millions of people at massive scale, and it can be perfectly tailored to that individual, knows their context. Like it is the most skilled hacker that you can imagine, except for tireless, and it has reached reach that no other human attacker has. So it
26:05
Unidentified speaker (speaker_2): used to be where you get like a virus on your computer, like I said, you reboot. But now can you imagine individuals trying to navigate the internet where instead of just getting a virus a computer virus, you open yourself up to being completely wiped out? How are people going to use
26:25
Unidentified speaker (speaker_2): the internet in that? Because that is absolutely kunk. And we saw the individuals also the weak point for the companies. And we saw this with the Binance hack.
26:36
Unidentified speaker (UNKNOWN): Yeah.
26:37
Unidentified speaker (speaker_1): This
26:37
Unidentified speaker (speaker_2): phishing. And so what did they do? They they this person clicked to download some software that they didn't know was malicious. And you know, next thing you know, hackers are off with 7 billion in crypto and trying to. navigate the internet when these incredibly, incredibly sophisticated AI hacks like is going
26:57
Unidentified speaker (speaker_2): to be a landmine. And that is something people aren't focusing on. Uh just that aspect of it. The capability is already there. Can I actually, you know, I I'm not sure I can fully make myself safe, but what you can at least do, you know, you don't always have to outrun the bear, but if you outrun the other person, the bear is chasing, you just make yourself
27:17
Unidentified speaker (speaker_2): harder to attack, then maybe they go on to the next target. But like if you think people are anti data center and AI right now, when that starts happening, it is going to be a massive headache. And maybe you can talk briefly about the kind of the the systemic risk, US, which is, you know, companies, you know, with these big supply chain hacks where
27:37
Unidentified speaker (speaker_2): you have like a shared dependency. So, you know, we've seen a number of these. Instead of picturing the internet as kind of like a thousand separate independent systems, it really isn't. It's thousands of systems standing on a few foundations. So one system that the websites use to prove who they really are. And there's a few cloud
27:56
Unidentified speaker (speaker_2): providers and a small set of open source uh building blocks everyone reuses. And those are so those are like the shared dependencies. And they create a really specific kind of danger because you don't have to attack a million companies at once. You basically find a single flaw in the one thing they all depend on. And you can hit them all at once. And so, you know, we saw this, you know, a
28:16
Unidentified speaker (speaker_2): small version of this, you know, with France, like 700 supermarkets and the refrigerators got hacked, hack right? Coming out. I mean, it's every day I'm seeing something that is just you're like, if that came out, it would be the biggest. thing of the year. And it is every 24 hours something like this. So there was a there was a a guy that's not even you know a cybersecurity expert. He has a twenty
28:36
Unidentified speaker (speaker_2): dollar OpenAI account and you know he points it at Ubuntu which is it's an open source Y nex system. So he was able to basically find a bug a root level zero day, and I think it was like two hours. Gave them, you know,
28:56
Unidentified speaker (speaker_2): access to basically put malicious code on anybody running the system machines.
29:03
Unidentified speaker (UNKNOWN): I
29:04
Unidentified speaker (speaker_2): think, you know, we saw it with GMO internet group in Japan, uh, where it's FX trading platform, and you know, it was able to, you know, execute a bunch of FX trades and it claimed that. it put ransomware and was threatening to on everybody, all their customers' machines. And I'm not really sure what happened with that. But we've seen another. I mean, this happened on the same
29:24
Unidentified speaker (speaker_2): day that we saw the Binance attack. There was a polymarket attack. Yeah. And polymarket wasn't personally attacked. You know, they had a vendor that was attacked, and because it was a trusted vendor and their software, that allowed them to be attacked themselves. What's happening now is just it's it's like the whole trust of the system that that everything's relied on is coming under attack. What we're gonna
29:44
Unidentified speaker (speaker_2): see is that it's just gonna keep getting worse as the models and the capability get grow. That's the systemic risk that you see this where, you know, right now you see the stuff that gets caught.
29:54
Tim Graff: I mean, there's so much there to ask about. And one of the things that I was keen to to chat about was the development of these tools. And you've already kind of addressed why AI on the offensive is potentially able to overwhelm AI on the defensive. We're going to talk a little bit about what that means at a state level.
30:14
Tim Graff: But first, I actually wanted to ask the question, and this is a poignant question given the results of the midterm elections, the results of a lot of local referendum. I'm originally from Ohio, and I think there was like 25 referendums on the ballot across various localities. in Ohio about data center construction and wanting to slow it.
30:30
Unidentified speaker (UNKNOWN): But
30:31
Tim Graff: you've written a recent piece that I thought was fascinating about development and deployment of AI tools. And I wondered if you could talk a little bit about some of the things you wrote there about why is development of these tools, why can we not slow it down? First of all, can we slow it down?
30:44
Unidentified speaker (speaker_2): In my piece, what but I I really I I saw the backlash coming, and I thought that it would get worse. You know, there would be a referendum basically with the midterms. And this was going to accelerate people's calls to pause AI development. And so I decided that all right I need to put this out there
31:04
Unidentified speaker (speaker_2): in early October, you know, about a month ago put this out. And it it explained Explain the full logic because people think it's crazy. It's like there's so much risk. Why would we not regulate these companies? Why would we not pause development? What are we doing? But what ends up happening is that there's a lot of examples of this just in the
31:24
Unidentified speaker (speaker_2): world where people get stuck in a competitive dynamic and it forces you to go down a path that is not optimal, right? And so this report was explaining that we are the AI and development is a prisoner's dilemma. And it's one of the most just insiduous prisoners'
31:44
Unidentified speaker (speaker_2): dilemma that I've ever come across. And so what I wanted to do is basically walk through okay, this is the full logic of like these very specific things that are present with AI that make it just an unsolvable Solvable prisoner's dilemma. And so the main point is that continuing to accelerate. development is not
32:04
Unidentified speaker (speaker_2): the action of someone who does is naive to the risk. It is the action of somebody that is stuck in a trap where the incentives make it such that it is the only rational decision is to continue to go forward. And I compare it to nuclear weapons, right? So nuclear proliferation. So that was a similar prisoner's
32:24
Unidentified speaker (speaker_2): dilemma where the existential reason. risk and we, however imperfectly, have been able to coordinate on that. Weapon stockpiles have stayed, you know, have been reduced and the number of nuclear states stayed much lower than it could have. What made that somewhat capable of escaping that prisoner's dilemma? The sobering
32:44
Unidentified speaker (speaker_2): reality is that all of the steps that are present in nuclear prisoner's dilemma are either absolutely absent from AI or reversed. And so the first one I talk about is that if you are going to have a coordinated, you know, between nations agreement to pause development, you have to be able to
33:03
Unidentified speaker (speaker_2): verify the other countries aren't secretly developing AI. Because if you paused and then they secretly cheated and they come on the other side with this decisive advantage in and AI capabilities, you know, it could be catastrophic. And so with nuclear weapons, there were multiple signatures that kept it from being able to be hidden from the world. You know, trust but
33:23
Unidentified speaker (speaker_2): verify was possible because there was something physically to verify. But with data centers, it takes place in, you know, the training of of AI models takes place in data centers that are physically indistinguishable from any other type of compute that you're doing, whether that's cloud services or imprints imperants or. whatever. The point being is that you can never verify that
33:43
Unidentified speaker (speaker_2): other people are adhering to the agreement to pause development. And so you have to assume they're cheating, which means you have to cheat. And so that inability to verify the other one's complying, you know, I think on its own is probably enough to basically say that you cannot basically escape the prisoner's dilemma through coordination because the the agreement agreement
34:03
Unidentified speaker (speaker_2): would fall apart before the ink dry because you can't verify that the others are complying.
34:08
Tim Graff: Right. So when you talk nuclear weapons and you're talking about game theory as well, the natural thing that springs to mind is the concept of mutually assured destruction. And you you write about this in the piece. Can you elaborate on what the differences are for AI development and mad mutual? mutually assured destruction, and
34:28
Tim Graff: why it doesn't really apply in this case.
34:31
Unidentified speaker (speaker_2): MAD is one of the main reasons why we were able to coordinate on nuclear weapons, because you couldn't use your nukes without also being attacked yourself. And
34:42
Unidentified speaker (speaker_2): I I used the the thought experiment of saying, like, okay, let's imagine a hypothetical situation where nuclear you know, technology not only gave you the ability to strike, but it was also the shield. So that if you got far enough ahead in your nuclear technology, you could also defend
35:02
Unidentified speaker (speaker_2): against your rivals attack. And I asked, you know, do you think the Soviet Union, the United States, would have settled into a puke peaceful, kind of stable standoff where they agree to, you know, take their nuclear weapons, weapon stop stockpile down and all and the obvious answer is no. They would have actually raced ahead for the decisive advantage for
35:22
Unidentified speaker (speaker_2): pure fear that the other one would have gotten the decisive advantage first. And so like the fear of the technology doesn't cause you to to pull back, and it causes you to race forward. And unfortunately, with AI, that actually is the case. It's not hypothetical. The thing that allows you to attack somebody is the same thing
35:42
Unidentified speaker (speaker_2): that allows you to defend. So the weapon is the shield. And the only way to defend against AI is with better AI. That's like the relative game, having better AI, especially with cyber, it is incredibly important. So it's not just an absolute game. Like you need to be able to find the bugs that the other models can,
36:02
Unidentified speaker (speaker_2): right? So that takes just incrementally better having better AI. What would end up happening, in my opinion, if there is a say AI cyber catastrophe, I think everybody assumes that, you know, this will cause people to say, all right, we're going to pause development. That will be the initial reaction. And some of the public that don't understand this might, yeah, that
36:22
Unidentified speaker (speaker_2): they're going gonna to be for that. But when the politicians sit down and they get their national defense, you know, briefing, they're gonna be sat down and solid, like, no, the only way we can defend, we can't stop others from developing this stuff. So the only solution is to have better AI. So a catastrophe, instead of causing you
36:42
Unidentified speaker (speaker_2): to pause development, forces you to accelerate because it's the only defense you have. Hugging Face's CEO said this exactly. He's like, listen, we w we were attacked with AI, but we were also defended against AI. And since we can't stop other people from doing this, we have to push our AI capabilities further. It's not going to create this consensus to to pause
37:02
Unidentified speaker (speaker_2): because that that actually makes you worse.
37:05
Unidentified speaker (speaker_1): Yeah.
37:05
Tim Graff: So this is really interesting to me. I mean we have a very concrete example of the intersection of AI and politics this week with the midterm elections. We're going to come back to that in a second. A bigger picture, I wanted to see if you could talk through a little bit about the considerations that populations have to make, how they act, what backlash might look like, and what governments and
37:25
Tim Graff: states do about that. Aaron
37:27
Unidentified speaker (UNKNOWN): Powell
37:27
Unidentified speaker (speaker_2): There is a political risk in democracies because I mean voters often and countries do things that are or somewhat irrational. and There's an economist that, as you talked about, this Nirvana fallacy. And the Nirvana fallacy is basically where you compare your solutions or your options with some like kind of
37:46
Unidentified speaker (speaker_2): utopian view rather than the options that are actually available in the real world under the real world constraints. Stopping and pausing AI development isn't going to make you safer if your primary geopolitical rival just is able to continue on
38:06
Unidentified speaker (speaker_2): developing. It it can't achieve its goals. I made the example of Europe rev regulating carbon emissions. And I don't know, this is not a commentary on whether that was a something that we should have done. But the problem with global carbon emissions is also a president's prisoner's dilemma, but you cannot
38:26
Unidentified speaker (speaker_2): achieve it unilaterally. You know, and it it creates a real cost, right? So it makes your energy more expensive, it makes your less competitive, unless everyone agrees to do it. And Europe unilaterally, you know, lowers global carbon emissions. Carbon emissions just went up more than the rest of the world, including
38:46
Unidentified speaker (speaker_2): especially China, right? And so it fully offset. So they got the consequence. consequences of restraining them their industry less competitive without even succeeding in and securing the global result. That's exactly what will happen to the country that decides to regulate its AI, right? Europe also regulated
39:06
Unidentified speaker (speaker_2): its AI, right? It regulated itself out of the AI race. And it has actually made it less safe because they're still connected to the global internet. So they're still at risk from all the models that exist everywhere else in the world, except now they don't have the frontier, the company's producing
39:26
Unidentified speaker (speaker_2): frontier models that allow them to have the defensive capabilities to harden their digital infrastructure, which is happening now. It is being gated. So the companies like Anthropic and OpenAI are gating them their like most frontier models. And you have to have basically verified access. And and you can guess who is going to be
39:45
Unidentified speaker (speaker_2): controlling who gets access to that is the US government. It it it does help for certain things.
39:57
Unidentified speaker (speaker_2): You know, on the internet to to get. And so
39:59
Unidentified speaker (UNKNOWN): there's
39:59
Unidentified speaker (speaker_2): going to be vulnerabilities. Trevor
40:00
Unidentified speaker (UNKNOWN): Burrus,
40:00
Unidentified speaker (speaker_2): So
40:00
Unidentified speaker (UNKNOWN): Jr.:
40:00
Tim Graff: let's think a little bit about the the here and now and any response that we might get politically to what is clearly some backlash to AI from the midterm elections. I mean, this Nirvana fallacy you talk about, is that a risk we run by virtue of being a democracy where these risks are ever present?
40:18
Unidentified speaker (speaker_2): That is the risk that the United States puts itself if they decide that we're going to regulate and pause, because China does not answer to voters. Their policy is not going to be changed because of a cyber attack. In fact, they have a very perverse incentive for that, something like that to occur. So
40:37
Unidentified speaker (speaker_2): that the US does pause development. It hands them the most decisive, most important advantage in in history, the the the prospects of recursive self-improvement self improvement where machines that you know are able to improve themselves, right? There's already, you know, a lot of evidence that this is at least happening
40:57
Unidentified speaker (speaker_2): at some scale to the to the extent that that intensifies and and spreads out, like it now becomes, you know, you could get a winner take all scenario where if you have the lead, you're just going to pour pull more into the lead league because you you have this feedback loop of having better AI capability leads to more even better AI. capability. And it's almost
41:17
Unidentified speaker (speaker_2): non-catch-upable. So if you pulled yourself out of that feedback loop when you're on the verge of that, you basically put yourself behind, you know, at a disadvantage, you know, forever. Um, everything points towards development has to continue. And even if a democracy like the United States chooses to kind of pull out of that,
41:36
Unidentified speaker (speaker_2): China's not. So you're not going to make yourself any safer. And development will continue in China. And
41:42
Tim Graff: one of the really interesting things you bring out in the piece is the differences between development, which we've just gone through, and deployment, how these models actually make their way into the public sphere. And I'm wondering if you can now segue into that discussion. What are the differences between the two in terms of the calculus that nation states make and the incentives
42:02
Tim Graff: they have to pursue deployment versus not.
42:05
Unidentified speaker (speaker_2): The deployment question has kind of a different set of incentives. Development is: are we developing and training new models? And the deployment is actually, are we allowing them to be commercially available or not? That has a different set of incentives because the government has an incentive and companies as well
42:25
Unidentified speaker (speaker_2): to not put their frontier models, make them readily available, because once you put a model out there, it can be effectively copied via distillation and reverse engineering, right? And so it allows you to disseminate your your advantage. And it, you know, like China has has has As
42:45
Unidentified speaker (speaker_2): used you'd use this a lot. It's not the only engine that's driving China's AI advances, but it is certain has certain certainly helped them stay up with the United States because for a fraction of the cost, you know, that you can distill Anthropic's model and have, you know, basically very close capability model. And the US doesn't want to do
43:05
Unidentified speaker (speaker_2): that. Well, if the United States starts withholding their frontier models, right? So not allowing deployment, the strategy against that, and so where they can hopefully keep racing ahead and developing, you know, the AI capabilities that no longer can be distilled because they're, you know, held behind gated access. The strategy against that
43:25
Unidentified speaker (speaker_2): for the follower for China is to allow all releases for, you know, without any kind of guardrails on. Because to the extent that you start not allowing access to that frontier, those frontier capabilities, China will eventually put out a model within probably three months, and they'll probably catch up. And that is more
43:44
Unidentified speaker (speaker_2): capable than what's commercially available. The biggest customers for these labs are, you know, software engineers. And and the same thing that capability that allows you to check your code is free of bugs is what people can use to attack the system. And that's what people what is being gated. And
44:04
Unidentified speaker (speaker_2): so the open weight models will end up being more capable than what the models that are commercially available. And so it is a strategy to force the US to release. And if they don't, anthropic will, you know, the they will lose market share. And the United States might
44:24
Unidentified speaker (speaker_2): want the labs to withhold, but they don't fund these companies. These companies are funded by private investors on the expectation of future future profits. And so these companies can be compelled and Washington's compelled to allow these companies to to release because if their market share and their, you know, investors start
44:44
Unidentified speaker (speaker_2): waking up like, okay, we no longer have a competitive product, you know, product and we had have two trillion dollar market cap on these things, you know, there's a lot of profit x future profit expectations built into that. So if we don't release new models, then we lose. So you can be forced or compelled, and that's exactly what China is is trying to do. And
45:04
Unidentified speaker (speaker_2): by releasing it, disseminates the advantage and keeps you from getting too far ahead. And so China's not going to stop. And not to mention that a cyber catastrophe in the West, you know, it's not going to have the same political impact on every country. And so they're probably hoping that it does cre create political blowback so that the United States hands them to league. You know, there's also a
45:24
Unidentified speaker (speaker_2): haven dynamic here. If you try to stop deployment just the same way that corporations, you know, steer profits to tax havens, development will be steered to some other jurisdiction that doesn't have restrictions. Um, and so it basically creates the defection that destroys. any kind of pause that you were hoping for.
45:41
Tim Graff: Yeah, that commingling of the private and public sphere where you have these labs that are privately funded with an expectation, some future expectation of profit that you alluded to, set against all these public safety concerns and regulatory concerns, and especially in the current environment where that's clearly top of mind for voters.
46:01
Tim Graff: What does the outcome look like? for governments? How do they navigate this? I mean, obviously these companies are closely tied to the government informally. Is there room for that to become a bit more formalized?
46:14
Unidentified speaker (speaker_2): The administration needs a way to thread the needle before between protecting the edge and allowing these companies to be commercially viable, right? So Washington is actually incredibly incentivized to ensure that these labs are very profitable. And so what will happen is they're going to
46:34
Unidentified speaker (speaker_2): start treating their labs like national champions. And they will likely need to put, you know, up artificial barriers to protect their profitability, whether that's banning open weight models in the United States, whatever, you know, that that's probably coming. But like it's it's really unclear if that even works because an open weight model can be downloaded and used on a
46:54
Unidentified speaker (speaker_2): private server. So the United States also doesn't control other jurisdictions. And so then OpenAI and Anthropic have to uh eventually, if they want to compete in those markets, and again, because the valuations are so high, there's an expectation that they need to be globally dominant and not just domestically. So it it will be interesting to see what
47:14
Unidentified speaker (speaker_2): the they try to do. But there is, you know, an e uh a very obvious business model. This, you know, if you start basically limiting what can be deployed, right? It creates a new business model that it's this gated access business model. Right. So I said the most obvious one is that you basically have a security
47:34
Unidentified speaker (speaker_2): SaaS business where people pay you to basically harden your code. And at the time that I wrote that, and the day I wrote it, like Anthropic came out with that exact business. And so it it instead of just them hardening your code for it, they they have dated access with for verified users. And then you pay them a bunch of money, and
47:54
Unidentified speaker (speaker_2): now you get access to their front tier defense model. The problem is that that is only viable in a situation where there is a withholding. regime where you can't get the capability elsewhere. And so you need to have it industry wide. What will decide whether deployment gets withheld is whether China also
48:14
Unidentified speaker (speaker_2): chooses to withhold because if they don't, it undercuts their business model and no one's gonna pay for open and AI and Anthropic to harden their code when they can get the capability elsewhere.
48:24
Tim Graff: Well let's let's take two things that are going on. literally this week as we record this, which is the U.S. political process. And I wanted to get your thoughts on that because I mentioned the referendums on data centers and the bipartisanship you mentioned in terms of how how unified backlash to this is, whether that is is meaningful, whether it will
48:44
Tim Graff: possibly cost the US its advantage in development. But also, and this is a markets podcast and I want to finish on markets, kind of the market reaction is, as I mentioned earlier in the recording, you know, a significant down day for tech. And we've seen this in NeoCloud over the last couple of weeks, even. Let's start with the politics. What was your take on the midterms this week with respect to AI development and
49:04
Tim Graff: employment in the US specifically?
49:05
Unidentified speaker (speaker_2): Yeah, I mean it's bipartisan just angst against AI. And that manifests, like I said, it with data centers. That was the policy that was in front of. people to kind of get up in arms about. But I think it represents kind of everybody's underlying angst about what AI is. And you've seen now,
49:25
Unidentified speaker (speaker_2): you know, a lot of of congress congressmen and women and calling for this should be the number one thing that they the Democrats That should go after is, you know, regulating AI. It is incredibly popular. And so that puts these companies at an enormous risk because you might look at it and say, well, you know,
49:45
Unidentified speaker (speaker_2): Donald Trump would just veto any kind of regulation. The problem is that these companies don't just need to avoid regulation. They actually desperately need legislation to get passed because they need better federal legislation that that will stop. a state court, especially in California, from holding them
50:05
Unidentified speaker (speaker_2): liable, you know, for any damage done by somebody else using their models and the damage that that's caused from that. I mean, I I hate to say it, but like if and when a ruling like that goes the wrong way, it's going to make US labs uncompetitive. And then the funding goes, right? I mean, who can underwrite Right, that the type of liabilities that are potentially facing them if they are
50:25
Unidentified speaker (speaker_2): held liable for their product? products. So what do we have? You know, in and how have how have we gotten around this in other cases? Well, you know, for the internet platforms, we have section two thirty, right? The gun manufacturers had to pass a law. I think it's like the Lawful Commerce and Arms Act that indemnifies them from liabilities of people using
50:45
Unidentified speaker (speaker_2): their products. They had to get that legislation passed. And man, it does not seem like there's the political ability to get a bill like that passed right now. So you're gonna be seeing the court cases starting to come up. And and and before you get a ruling on this, like people are gonna be starting to focus on it. And I don't know how
51:05
Unidentified speaker (speaker_2): you can underwrite the companies with that type of liability. It that that to me is the big takeaway from you know the policy poly is just that I I think once you
51:15
Unidentified speaker (speaker_2): lay it out and explain that it's really hard for people to accept that AI can pose this much risk and we're doing nothing to regulate. It but the problem is that since you can't regulate the labs, you just put your own labs at a massive competitive disadvantage. You haven't even reduced the risk because the models still get built, but just
51:35
Unidentified speaker (speaker_2): by your rival. People want to judge a real world arrangement against this idolized alternative rather than the conditions that exist under real constraints, right? What are the real art alternatives actually available? And the real alternatives available is either you regulate these guys and you lose and no longer it doesn't even matter what you're
51:55
Unidentified speaker (speaker_2): doing, right? Or you just allow have to allow them to keep on and don't regulate them and keep them from being having, you know, liable. for these products because it's a competition and you cannot ignore that dynamic. It's an incredibly frustrating situation that that we're put in that. And it's difficult for people to to do that. I mean there's like they want some
52:15
Unidentified speaker (speaker_2): sort of action. If these companies are held liable and there's regulation, it's only going to be on the US labs. And that's the competitive disadvantage that causes them to lose the race. And in doing so, you hand China the biggest advantage in human history.
52:28
Tim Graff: Right. So from the US's perspective, at least, that's neither possible nor preferable. So what does that threading of the needle that you referred to look like in practical terms for the coming months?
52:40
Unidentified speaker (speaker_2): I I think that when politicians are like sat down, I think it's pretty easy to understand when when you were told this, right? I know don't we really have a lot of sound and fury about regulating AI. And I think when it comes down to it, they'll say, okay, like this policy can't actually achieve its intended goals. So we'll give the people something to look like we're really
52:59
Unidentified speaker (speaker_2): regulating and hurting them. But you know, in in fact, you know, these companies aren't at risk. The problem, like I said, is that it's not just that they need regulation, it's that they need legislation. And that seems seemed dead on arrival. And I don't know a situation around that. And anthropic, I mean, they they put out, you know, their advice like legislation and people went
53:19
Unidentified speaker (speaker_2): crazy. And people didn't know, they're like, this makes it harder for the states to actually do anything to regulate these economies. That yeah, that's exactly right. It it that whole the whole bill that they put forward is to stop a state court from making a ruling that holds them liable for their product. And good luck getting that through Life legislation Slice right
53:37
Unidentified speaker (speaker_1): now.
53:37
Tim Graff: Well
53:38
Unidentified speaker (UNKNOWN): uh
53:38
Tim Graff: it begs the question I think you yourself asked on X a couple of weeks ago about whether that is bullish or bearish for anthropic, insofar as the continued development is necessary, but it's necessary for the reasons of the prisoner's dilemma that you've highlighted
53:55
Unidentified speaker (UNKNOWN): over
53:55
Tim Graff: the last hour or so. But also, So I wanted to bring us full circle to where we started with the discussion of a potential systemic. event for markets similar to the pandemic. And as a kind of two-parter, what does that look like for you? First of all, does it make you bullish, these these labs? And what does the systemic scenario for markets
54:15
Tim Graff: look like in your view?
54:16
Unidentified speaker (speaker_3): You know, uh yeah. So the point was that if there was a global catastrophe, you know, and we're seeing this, because I want to get back, you know, w to talking about Black October, which has been kind of named. the industry ha has named for what has happened over the last month. But the fact that all these the the cyber, you know, risks are going exponential right now means that
54:36
Unidentified speaker (speaker_3): people there is absolutely insatiable demand for the gated access to these comp these labs frontier models to harden their own infrastructure. So that revenue is going to explode and that's a much better business model if you can do it than you know competing to have the best model the way they're doing right now and effectively fairly commoditized.
54:56
Unidentified speaker (speaker_3): You know, this can't be copied, right? But again, it is dependent on whether China allows full releases for all capabilities for their models. And so if they don't allow, if they do also go, you know, some sort of dated gated regime for their labs, then it would be incredibly good for the labs. I mean that is a great scenario.
55:16
Unidentified speaker (speaker_3): It's not just going to be security SaaS, it's going to be all kinds of services that you have to get gated access to use their product. And then it's no longer a commodity and it can't be copied. So that is a phenomenal scenario. But you know, China understands that. So I'm not sure that will be the scenario we get. But yes, I mean,
55:36
Unidentified speaker (speaker_3): anthropics revenue from this gated access is going to get pair. parabolic. But the initial reaction to a big, because there's coming, there's going to be a big visceral, a cyber attack that viscerally affects people on a personal basis, right? A big systemic one, right? So this the shared dependency type attack. And when that happens,
55:56
Unidentified speaker (speaker_3): that won't be the risk. That will just wake everybody up to, I mean, it will be a risk, but the risk. real is that we're in a brand new world here. And the ability for attackers to basically attack the system in a way that it was never done before, that we're in a new paradigm where, you know, that's coming.
56:16
Unidentified speaker (speaker_3): And then the individual, you know, death by a thousand cuts, where you don't want to go on the internet. So that is going to be a situation that impacts the economy. The the main point is that it's hard to know like what is going to get attacked. When I see the things that I've seen, it's like, and what
56:36
Unidentified speaker (speaker_3): security experts are saying, it's like everything gets attacked. It's just going to be one thing after another. And we're just not prepared right now. And it's it's kind of the reality I think that markets need to wake up to a little bit because you know, this is like I said, what everybody's gonna be talking about in six months. And I I feel pretty strongly about that. And just
56:56
Unidentified speaker (speaker_3): the way that, you know, I think for reasons, you know, markets typically do a really good job of if there's a signal that they've seen before, it does, you know, typically markets don't crash multi-highs. Yeah. Modern markets, they it's a rounding process where it takes in all the data over time and starts, you know, discounting it. And COVID
57:16
Unidentified speaker (speaker_3): fell from an all-time high because even though experts were sitting here saying, like, this is coming, the market just kind of ignored it because it didn't have a way. You know, if you have a quantitative model, you know, like and you aren't accounting for a certain factor, like it's not just bad at discounting the odds of it, it just doesn't even account for it. And so right
57:36
Unidentified speaker (speaker_3): now people there there's no historical evidence. to kind of look at and say, you know, this is what this could look like. And the market is also in the case, you know, it's just it's it's been lured into one cyber risk headline after another for so long that it has trouble understanding like what is actually
57:56
Unidentified speaker (speaker_3): different this time. And who knows when it is, but there will be some visceral like event that wakes markets up. and then you're gonna get politicians really going after them. And I you know, uh to the answer to the question, I I don't think Anthropic will do very well in that event, even if they get a bunch of revenue from the gated access model. But, you
58:16
Unidentified speaker (speaker_3): know, especially in a market that is this kind of levered to this one story, you know, it's it
58:21
Unidentified speaker (UNKNOWN): it
58:21
Unidentified speaker (speaker_3): could it's gonna probably have a negative reaction to this stuff. And so my point is that I made is that listen, I don't like making bearish bets or putting a bearish thesis out there. I mean, there
58:32
Unidentified speaker (UNKNOWN): are
58:32
Unidentified speaker (speaker_3): dime a dozen on the internet. The reason to say all this is that the ability to hedge against a tail risk that isn't being priced is cheap. And so if it doesn't happen, and I'm completely wrong, you're out a of small premium, right? But that was the case during COVID. Like if you were worried, like, you could hedge your
58:52
Unidentified speaker (speaker_3): risk extremely easy because it wasn't being priced. And so that's kind of the point that I wanted to make is I think this is a much bigger risk than the market has priced. And so the insurance is very cheap to protect against it.
59:05
Tim Graff: Yeah. I think that is a perfect note to finish on. It's been a sobering conversation. I mean, you said there's no historical precedent, but as a final thought, we've been in markets as you note, trading around this narrative, but there's been rotation around it. As if something gets drawn down, something gets built up. And we have realized and implied correlations in markets that
59:25
Tim Graff: are basically at the zero bound. And I think the answer is that we have a much higher probability of a correlation of one environment, similar to the pandemic, and it doesn't really matter what you're in or what you're not in.
59:37
Unidentified speaker (UNKNOWN): It's
59:37
Tim Graff: a drawdown. It's a de-grossing. We've seen some of this on the hedge fund side already, potentially a lot to see on the real money side. But Michael, there's so many questions. There's more questions on my list that I didn't ask than I did, which is a testament to how good this has been. It's been really great to get your time. We'll have to do all the stuff on trade and capital flows and currencies that you've written about
59:58
Tim Graff: another time, but this has been fantastic. Thank you so much for joining the podcast.
1:00:01
Unidentified speaker (speaker_3): Well, I really appreciate you having me on. and so it was great talking to you. And uh, you know, I would love to come on any other time. So thanks for having
1:00:07
Unidentified speaker (UNKNOWN): me.
1:00:10
Tim Graff: Thanks for listening to Street Signals. Clients can find this podcast and all of our research at our web portal, Insights. There you'll be able to find all of our latest thinking on markets, where we leverage our deep experience in research on investor behavior, inflation, media sentiment, and risk. All of which goes into building an award-winning strategy product.
1:00:30
Unidentified speaker (UNKNOWN): And
1:00:30
Tim Graff: again, if you like what you've heard, please subscribe wherever you get your podcasts and leave us a review. We'll see you next
1:00:36
Unidentified speaker (UNKNOWN): time.
1:00:45
Tim Graff: Legal or tax advice, or recommendation or solicitation to buy or sell any security or financial instrument. It does not consider any recipients in particular investment objectives, strategies, investment priorities. horizon, or tax status, and is not intended to replace the exercise of an investor's own independent review and judgment regarding any investment decision. These and opinions expressed herein
1:00:56
Unidentified speaker (UNKNOWN): are
1:00:56
Tim Graff: as of the dates specified are subject to change without notice and may not necessarily reflect the institutional views of State Street or its affiliates.
1:01:09
Tim Graff: or otherwise, for any losses, liabilities, damages, expenses, or costs, either direct, indirect, consequential, special or punitive arising from or in connection with any use of this communication and or the information herein. State Street or its affiliates may have positions in trade or have commercial relationships involving instruments or issuers discussed herein. This communication may contain information deemed to be forward looking statements and may not be redistributed without prior written consent to State Street. Past performance is no guarantee of future results.
This page was produced automatically: the audio was transcribed with nvidia/parakeet-tdt-0.6b-v3+sortformer-v2.1 and interpreted by Qwen3.6-35B-A3B-UD-IQ3_S.gguf. Transcription and interpretation can contain errors; always confirm against the original episode. Nothing here is investment advice.

More from Street Signals

All insights