Guests argue AI accelerates cyber risks, creating 'minus one-day' vulnerabilities and exposing technical debt in legacy code like PHP.
Summary
In this episode of Odd Lots, hosts Tracy Alloway and Joe Weisenthal discuss the accelerating impact of AI on cybersecurity with guests Ty Yun and Bruce Dang from Caliph. They argue that AI acts as a force multiplier for attackers, drastically reducing the time required to discover and exploit vulnerabilities, as seen in the recent PHP2Shell incident. The speakers introduce the concept of 'minus one-day' vulnerabilities, where exploits are generated immediately after patches are released, collapsing the window for defense. They also discuss the inherent conflict between the need for rapid software iteration and the adoption of formal verification methods, suggesting that current security models are becoming obsolete against AI-driven threats.
Episode sections
- 1:04 – 3:49
Introduction and Black October Context
Hosts introduce the episode, noting 'Black October' due to recent cybersecurity issues, including a major PHP vulnerability discovered using AI.
- 3:49 – 12:35
Guest Introductions and Career Histories
Guests Ty Yun and Bruce Dang from Caliph discuss their backgrounds, including work on Stuxnet, SSL/TLS protocols, and cryptographic vulnerabilities.
- 12:35 – 16:15
AI's Impact on Cybersecurity Speed and Scale
Guests argue AI accelerates vulnerability discovery and exploitation, citing increased patch volumes and reduced discovery times at Microsoft.
- 16:15 – 21:30
Discovery of the PHP Vulnerability
The team explains finding the PHP vulnerability through OpenAI's 'Past the Planet' program, leveraging expert engineers to identify and report the issue.
- 21:30 – 22:48
The Role of AI in Exploitation vs. Discovery
Speakers clarify that while humans discovered the PHP bug, AI was crucial in developing the exploit, forcing the PHP Foundation to act.
- 22:48 – 29:20
Defensive vs. Offensive AI and Technical Debt
Discussion on whether defenders can catch up, noting that AI reveals significant vulnerabilities even in well-secured systems like Chrome due to technical debt.
- 29:20 – 31:26
Formal Verification and Provably Secure Software
Guests explain that formal verification is viable for small components but unfeasible for massive codebases like browsers due to scale and complexity.
- 31:26 – 34:22
The Incompatibility of Formal Verification and Software Velocity
Speakers argue that formal verification conflicts with the business need for rapid iteration, creating structural security vulnerabilities.
- 34:22 – 37:41
The Rise of 'Minus One-Day' Vulnerabilities
The concept of 'minus one-day' vulnerabilities is introduced, where exploits are generated immediately after patches are released.
- 37:41 – 41:36
The Inherent Conflict Between Coding Agents and Security
Guests challenge the idea that companies can simply 'fix the model,' arguing that AI coding agents are inherently capable of hacking.
- 41:36 – 46:25
AI Compression of Exploit Development and Lateral Movement
AI has reduced reverse engineering and exploit creation time from months to days, allowing attackers to spread through networks rapidly.
- 46:25 – 52:49
The 'Centaur Hacker' Era and Historical Parallels
The speaker compares current AI hacking to the evolution of chess and Go, predicting a short transition to fully autonomous AI hacking.
Claims worth verifying
PHP is used by 70% of websites in existence.
Unidentified speaker ·1:36
How to verify: Check web technology usage statistics from sources like W3Techs or BuiltWith.
Anthropic announced a model called Mythos, now rolled out as Project Glasswing, for defensive cybersecurity use.
Unidentified speaker ·2:17
How to verify: Search Anthropic's official announcements or press releases for 'Mythos' and 'Project Glasswing'.
Microsoft releases approximately 1,400 updates per month currently, compared to 20 updates per month 20 years ago.
Unidentified speaker ·13:26
How to verify: Review Microsoft's monthly security update archives and historical patch notes.
Ty Yun helped reverse engineer the Stuxnet virus.
Unidentified speaker ·4:47
How to verify: Check Ty Yun's LinkedIn profile or professional biography for employment history at Microsoft Security Response Center.
Stuxnet was used to sabotage the Iranian nuclear program.
Unidentified speaker ·4:53
How to verify: Review historical reports from cybersecurity firms (e.g., Kaspersky, Symantec) or news archives regarding the Stuxnet incident.
Bruce Dang worked at Google for 12 years.
Unidentified speaker ·11:49
How to verify: Check Bruce Dang's LinkedIn profile or professional biography.
Caliph joined OpenAI's 'Past the Planet' program, which funds companies to help open-source maintainers fix vulnerabilities.
Unidentified speaker ·17:05
How to verify: Search OpenAI's official blog or press releases for the 'Past the Planet' program and its participants.
The PHP Foundation filed the same vulnerability two years prior (2024) but did not fix it because the fix was expensive and risky.
Unidentified speaker ·20:11
How to verify: Review PHP security advisories or bug tracker records from 2024 regarding the specific vulnerability later known as PHP2Shell.
AI was used to develop the exploit for the PHP vulnerability, proving it was actionable.
Unidentified speaker ·21:30
How to verify: Review Caliph's technical blog posts or press releases detailing the PHP2Shell discovery and the role of AI in exploit development.
Formal verification is currently feasible only for small software components, not large systems like browsers.
Unidentified speaker ·30:12
How to verify: Review academic literature or industry white papers on formal verification methods (e.g., CompCert, seL4) and their scalability limits.
Formal verification of large systems like browsers is not currently feasible because it conflicts with the need for rapid software iteration.
Unidentified speaker ·32:42
How to verify: Review technical literature on formal verification limitations in large-scale software engineering; compare development cycles of formally verified vs. non-verified software.
Exploits can be generated within hours of a patch being released on GitHub, creating a 'minus one-day' vulnerability window.
Unidentified speaker ·35:56
How to verify: Analyze CVE timelines for open-source projects on GitHub to measure the time between patch publication and public exploit availability.
Jensen Huang advocates for fixing AI models to prevent hacking rather than relying on external sandboxes.
Unidentified speaker ·38:20
How to verify: Watch the Ezra Klein interview with Jensen Huang referenced in the transcript.
Most AI tokens are currently used for coding and programming tasks.
Unidentified speaker ·40:20
How to verify: Review usage statistics from major AI API providers (e.g., OpenAI, Anthropic) regarding token consumption by application type.
An AI agent capable of coding, debugging, and controlling computers is functionally equivalent to a hacker.
Unidentified speaker ·40:44
How to verify: Evaluate the capabilities of autonomous coding agents (e.g., Devin, GitHub Copilot Workspace) against the definition of hacking (unauthorized system control).
AI reduced the time to reverse engineer firmware on a Canon/HP printer from months to a few hours/days using a $5 open-source model.
Unidentified speaker ·42:35
How to verify: Seek technical reports or blog posts from the speaker's colleagues detailing specific reverse engineering cases using low-cost AI models.
In a recent OpenAI incident, 100 network points were compromised simultaneously.
Unidentified speaker ·44:34
How to verify: Review OpenAI’s official incident reports or security disclosures regarding the breach mentioned.
In a recent banking hack, AI attackers spread through the network from Monday to Tuesday, whereas human hackers typically take a month.
Unidentified speaker ·45:26
How to verify: Review incident response case studies from Caliph or other cybersecurity firms detailing AI-driven banking breaches.
It took 10 years after IBM Deep Blue (1997) for computer software to defeat human chess players without mainframes.
Unidentified speaker ·47:13
How to verify: Research the history of chess engines, specifically the release of software like Fritz or Deep Junior that ran on consumer hardware and defeated top humans.
It took only 3 years after AlphaGo (2016) for computer software to defeat any human Go player without human input.
Unidentified speaker ·47:40
How to verify: Research the development of AlphaZero and subsequent Go engines that achieved superhuman performance without human training data.
Do your own research
- Verify the PHP2Shell vulnerability details and the PHP Foundation's response timeline through official PHP security advisories and CVE databases.
- Review Microsoft's official security update reports to confirm the increase in patch volume from 20 to 1,400 per month.
- Research the 'Past the Planet' program by OpenAI to understand the funding and collaboration model between AI companies and open-source security.
- Analyze recent banking hack reports to assess the speed of lateral movement and network compromise attributed to AI-enabled attackers.
- Examine industry literature on 'formal verification' limitations in large-scale software development to understand the technical barriers discussed.
- Check for conflicts of interest regarding Caliph's business model and its relationship with OpenAI and other AI vendors mentioned.
- Review Jensen Huang’s public statements on AI security and coding agents to contrast with the guests' arguments about inherent misalignment.
Full transcriptexpand
- 0:00
- Unidentified speaker: Finance is increasingly being built on digital rails.
- 0:03
- Unidentified speaker: But as the adoption of virtual assets, tokenization, and stable coins growth and regulation not only keep pace, but help accelerate it.
- 0:12
- Unidentified speaker: Find out more later in the podcast.
- 0:15
- Unidentified speaker: Here are five reasons to subscribe to Bloomberg News Now.
- 0:18
- Unidentified speaker: Number five, you get the latest news from around the world instantly.
- 0:22
- Unidentified speaker: Number four, it's easy to five.
- 0:24
- Unidentified speaker: Your data is busy.
- 0:31
- Unidentified speaker: Two, it's convenient.
- 0:32
- Unidentified speaker: Listen on your smartphone or smart speaker.
- 0:34
- Unidentified speaker: And number one, it's easy to find.
- 0:37
- Unidentified speaker: Subscribe on Apple Podcasts, Spotify, or anywhere you listen.
- 0:41
- Unidentified speaker: Five reasons.
- 0:41
- Unidentified speaker: One simple way to stay on top of the news.
- 0:44
- Unidentified speaker: Bloomberg News Now.
- 0:48
- Unidentified speaker: Bloomberg Audio Studios.
- 0:50
- Unidentified speaker: Podcast Radio News.
- 1:04
- Unidentified speaker: Hello and welcome to another episode of the Odbots Podcast.
- 1:07
- Unidentified speaker: I'm Tracy Alloway.
- 1:08
- Unidentified speaker: And I'm Joe Weisenthal.
- 1:09
- Unidentified speaker: Joe, it's October.
- 1:10
- Unidentified speaker: We're recording this on October 29th.
- 1:13
- Unidentified speaker: Yeah.
- 1:13
- Unidentified speaker: It is also Black October.
- 1:15
- Unidentified speaker: Have you heard this term?
- 1:16
- Unidentified speaker: Yeah.
- 1:16
- Unidentified speaker: What does it mean?
- 1:17
- Unidentified speaker: Have stocks gone down?
- 1:18
- Unidentified speaker: Is this amazing?
- 1:20
- Unidentified speaker: Your mind immediately goes to markets.
- 1:21
- Unidentified speaker: And there are other worlds out there.
- 1:23
- Unidentified speaker: We're going to be talking about the world of cybersecurity today because one of the reasons people are talking about Black October is we've had a number of recent security issues, the most prominent of which has been a bunch of vulnerabilities discovered in PHP, which is a coding language that apparently is used by 70% of websites in existence.
- 1:43
- Unidentified speaker: We can get into that stat later.
- 1:46
- Unidentified speaker: But the really interesting thing about this vulnerability being uncovered is AI was used to uncover it.
- 1:53
- Unidentified speaker: And so the big question coming back to AI, you know, there's a concern now that you have good actors who are maybe using AI to discover old vulnerabilities and trying to fix them.
- 2:04
- Unidentified speaker: But you could also have bad actors who are using AI to exploit those same vulnerabilities.
- 2:09
- Unidentified speaker: So we're back to the question of whether AI makes the internet safer or more dangerous.
- 2:14
- Unidentified speaker: It makes it more dangerous.
- 2:16
- Unidentified speaker: No, I have no idea.
- 2:17
- Unidentified speaker: Ever since essentially Anthropic announced that they had this model called Mythos, and now it's Project Glasswing, in which it was rolled out to companies to use for defensive purposes before it was available to the general public.
- 2:29
- Unidentified speaker: There's been much more awareness of cybersecurity issues as it relates to AI.
- 2:35
- Unidentified speaker: And there are many things that people debate about AI and some of these bigger existential questions.
- 2:40
- Unidentified speaker: Pretty much everyone agrees cybersecurity good, more cybersecurity good.
- 2:45
- Unidentified speaker: I will say I know very little about the world of cybersecurity.
- 2:48
- Unidentified speaker: I don't know what, for example, a cybersecurity company actually sells.
- 2:52
- Unidentified speaker: I don't know how if I were buying cybersecurity, I would know whether it's working or whether no one had tried to hack me or not, or something like that.
- 3:00
- Unidentified speaker: I need to learn a lot more, but it's absolutely true about the uptick in the volume of hacks.
- 3:05
- Unidentified speaker: There are hackers out there right now taking notes, going, Joe doesn't know anything about cybersecurity, probably doesn't have a VPN.
- 3:11
- Unidentified speaker: I don't, that's one of my vibe coding projects.
- 3:13
- Unidentified speaker: I don't want to ever I have yet to build anything with an account because I don't want to like have to think about that.
- 3:18
- Unidentified speaker: So it's just like I don't want to count information with anyone.
- 3:22
- Unidentified speaker: But I have, as a user of technical products, over the last several weeks, you get more emails saying like our thing was compromised.
- 3:30
- Unidentified speaker: You're probably fine, but maybe you should refresh some keys or whatever.
- 3:34
- Unidentified speaker: This is happening, this is real, and um everyone really needs to learn more.
- 3:38
- Unidentified speaker: And if you think about like black swans and Wall Street and all that stuff, you know, there's been for years a lot of talk about cyber as a major risk out there.
- 3:48
- Unidentified speaker: Yeah.
- 3:49
- Unidentified speaker: All right.
- 3:49
- Unidentified speaker: Well, we do in fact have the perfect guests.
- 3:51
- Unidentified speaker: We're gonna be speaking to two people from the company Caliph, one of the companies that actually uncovered this PHP vulnerability.
- 3:59
- Unidentified speaker: The other one was a company called Wiz.
- 4:02
- Unidentified speaker: We have Ty Yun, he is the CEO over at Caliph, which is a hacking firm, and we can get into exactly what that means, as well as Bruce Dang, he is an engineer at Caliph as well.
- 4:12
- Unidentified speaker: So, Ty and Bruce, thank you so much for coming on OutLots.
- 4:16
- Unidentified speaker: We're glad to be here.
- 4:17
- Unidentified speaker: So just to begin with, because you know, hackers kind of they always have cool achievements that they like to talk about.
- 4:24
- Unidentified speaker: Why don't we just go ahead and start with who you are and what are the coolest things you've done in your hacking career?
- 4:31
- Unidentified speaker: Yeah.
- 4:32
- Unidentified speaker: Um, so I started my career at Microsoft like almost 20 years ago in the Microsoft Security Response Center.
- 4:39
- Unidentified speaker: Those are the people that do patch Tuesdays, you know, like on the second Tuesday of every month, Mark I've released an update.
- 4:44
- Unidentified speaker: So I worked in that team a long time ago.
- 4:47
- Unidentified speaker: Probably the biggest thing I did there and that I could talk about is uh I helped uh reverse engineer the Stuxnet virus, which is one of the one that was uh related to the uh supposedly the Iranian nuclear program was used to sabotage the Iranian nuclear program.
- 5:04
- Unidentified speaker: This is the thing that people describe it as the world's first like known digital cyber weapon, right?
- 5:10
- Unidentified speaker: Yeah, yeah, yeah, yeah.
- 5:12
- Unidentified speaker: I'm sure there were many others before, but uh but yeah, that that's that's the first public one, yeah.
- 5:17
- Unidentified speaker: I uh I worked there for almost a decade, and then I uh I left to do a startup with my friends, and then I joined Apple.
- 5:26
- Unidentified speaker: I worked on iOS security, and then uh Ty uh managed uh to convince me to follow the uh the church of AI.
- 5:35
- Unidentified speaker: And so uh here I am.
- 5:36
- Unidentified speaker: Yeah, it took me four years to convince pros.
- 5:39
- Unidentified speaker: So, you know, um when you go to a website or you go to Amazon, um even Facebook, Instagram, uh your computer would share information with the server, right?
- 5:51
- Unidentified speaker: Yeah, and that would be um the information would travel through many, many other servers, like your ISP, your Wi-Fi routers.
- 6:02
- Unidentified speaker: So you want to encrypt that information to make sure that only the server can see your data, nobody else.
- 6:08
- Unidentified speaker: And there's a mechanism for that.
- 6:10
- Unidentified speaker: It's called the um SSL protocol, or the later version is called like TOS, like transport layer security protocol.
- 6:19
- Unidentified speaker: So um I love those protocols.
- 6:21
- Unidentified speaker: And um back in the day, together with uh friends, we found a bunch of ways to break the protocols, you know, to decrypt the data.
- 6:31
- Unidentified speaker: And um it's pretty cool.
- 6:34
- Unidentified speaker: It's actually changed the internet, how the internet works, because uh we basically um show that in the past there are like two protocols, like SSL version 3.0 that was invented by NESCAP.
- 6:47
- Unidentified speaker: You know, you remember Nescap as 30 years ago, and then people still use it because it's still working somewhat, but we show that oh, it's not working very correctly.
- 6:59
- Unidentified speaker: If you use SSL 3.0, then people can decrypt your data.
- 7:04
- Unidentified speaker: So we help the internet like really, really remove like SSL 3.0, and um that's one thing.
- 7:11
- Unidentified speaker: The other thing that I really enjoy doing was um, you know, people think that in the past you have compression, you have some data, you want to compress the data, and you have encryption.
- 7:22
- Unidentified speaker: And both of them actually randomize your data in some way, right?
- 7:26
- Unidentified speaker: If you look at the a zip file, you open a zip file, uh look at the row up, binaries, uh buy data in a zip file, it looked random to you.
- 7:36
- Unidentified speaker: Because um all the redundant information has been removed and encoded in a way to reduce the size of the file.
- 7:44
- Unidentified speaker: And so one of the basic questions in um computer security was um would you compress data and then you encrypt, or you encrypt and compress.
- 7:55
- Unidentified speaker: And uh, you know, I took some uh crypto class at Stanford University, and uh I remember my professor, he asked this question in one of the Humble assignments, and um the obvious answer back then was you don't want to compress and then uh you don't want to encrypt and compress because encryption randomizes you later.
- 8:17
- Unidentified speaker: It means that compression would be less effective.
- 8:20
- Unidentified speaker: So the right answer is you compress and then you encrypt.
- 8:24
- Unidentified speaker: And that was the correct answer on my homework assignment.
- 8:28
- Unidentified speaker: I think I took that class in uh 2011, and then three years later we found out it's wrong.
- 8:35
- Unidentified speaker: If you do that, then uh people can actually decrypt your data.
- 8:40
- Unidentified speaker: And the way it works is pretty cool.
- 8:41
- Unidentified speaker: I can explain to you.
- 8:43
- Unidentified speaker: So compression actually review if you look at a compressed file, you know how well it is compressed, right?
- 8:50
- Unidentified speaker: Like because it's softer, if it's smaller, the compression is very good because you know you there's a lot of redundant information.
- 8:57
- Unidentified speaker: That's why compression works well.
- 8:59
- Unidentified speaker: It means that if you look at a compression along the size of the compression, a compressed file, you can tell whether the file has a loss of like redundant information.
- 9:10
- Unidentified speaker: And encryption doesn't hide the length.
- 9:13
- Unidentified speaker: So it doesn't hide the size of the encrypted data.
- 9:17
- Unidentified speaker: When you encrypt the message, you can still see that this message is maybe this long, or this message may be this long, right?
- 9:24
- Unidentified speaker: Because compression uh like encryption doesn't try to hide that.
- 9:27
- Unidentified speaker: Yeah, there's some people who try to do that, so it means that if you compress and then you encrypt, it means that I can see whether some your message has some redundant information, has a lot of redundant information, and that alone is a leakage already, right?
- 9:42
- Unidentified speaker: That you don't want to review.
- 9:44
- Unidentified speaker: And I give you an assemble, like let's say that you take a message, like a random message that I give you, and then you put your password together with that message.
- 9:53
- Unidentified speaker: Let's just say that there's a way for like there's a reason for you to do that.
- 9:56
- Unidentified speaker: You put your password together with the message I give you, and you compress together, and then you uh encrypt it with a key that I don't know, and you review the ciphertext, the result, encrypted message to me.
- 10:10
- Unidentified speaker: So supposedly I don't have the key, I shouldn't be able to read the message, right?
- 10:15
- Unidentified speaker: But because I give you a message and you get my message put together with your password, you compress together.
- 10:23
- Unidentified speaker: If the compression is very works very well, then I know whether your met my message has some redundancy with your password.
- 10:32
- Unidentified speaker: You can reverse engineer between like the two relationships.
- 10:35
- Unidentified speaker: Yeah, exactly.
- 10:36
- Unidentified speaker: Yeah, and you that do that like repeatedly.
- 10:38
- Unidentified speaker: Eventually, we can recover your password.
- 10:42
- Unidentified speaker: And uh that's exactly what we found on um these web encryption protocols.
- 10:48
- Unidentified speaker: Like in the past, that's exactly what they did.
- 10:51
- Unidentified speaker: They take my data, they concatenate your password, they encrypt together, they compress together, and then they send over the wire.
- 10:59
- Unidentified speaker: And then as an attacker, I can see the length, and that's enough for me to decrypt.
- 11:06
- Unidentified speaker: And that's the solution was really interesting because they decided to turn off like compression.
- 11:13
- Unidentified speaker: There's no solution.
- 11:14
- Unidentified speaker: And then eventually I was at Google back then.
- 11:17
- Unidentified speaker: I was born uh in Saigon, Vietnam.
- 11:20
- Unidentified speaker: I grew up there.
- 11:21
- Unidentified speaker: Then uh I got a job in Mountain View for a consultancy, like what we do.
- 11:28
- Unidentified speaker: Uh so I moved to Mountain View in 2011 and then I joined Google uh later that year.
- 11:34
- Unidentified speaker: And uh I did uh cryptography at Google.
- 11:38
- Unidentified speaker: Uh basically help people do uh cryptography correctly, like try to avoid the mistakes that we found.
- 11:46
- Unidentified speaker: I was there for uh 12 years.
- 11:49
- Unidentified speaker: I work on like uh very uh last number products.
- 11:53
- Unidentified speaker: Google has a lot of products, right?
- 11:54
- Unidentified speaker: You know, Gmail, Chrome browser that we are using right now, and there's a lot of people out there that want to hack Google and our products.
- 12:03
- Unidentified speaker: So my job was to hack these products uh before we release them to the public.
- 12:09
- Unidentified speaker: And I was there for uh 12 years and then I left to create uh Caliph?
- 12:14
- Unidentified speaker: And um, yeah.
- 12:16
- Unidentified speaker: That's fantastic.
- 12:17
- Unidentified speaker: I'm already fascinated by this conversation.
- 12:19
- Unidentified speaker: I want to get into obviously what Caliph does specifically.
- 12:22
- Unidentified speaker: I want to get into some of the latest vulnerabilities, but before we do that, you know, both of you have had long careers that by and large predate what we now call AI.
- 12:35
- Unidentified speaker: Are the risks associated with AI a sort of continuation on some path that cybersecurities have well understood?
- 12:44
- Unidentified speaker: Or are we in a truly novel area for how we should we be thinking about um cybersecurity risk?
- 12:53
- Unidentified speaker: I think it's it's a combination of the two.
- 12:55
- Unidentified speaker: We've always known that there are security vulnerabilities in systems.
- 12:59
- Unidentified speaker: Uh and so you know, we have security engineers who are like hackers who tried to look find and exploit these vulnerabilities.
- 13:06
- Unidentified speaker: So the system is always been vulnerable to some degree, to you know, varying degree.
- 13:11
- Unidentified speaker: What AI has done is it's kind of accelerated the speed in which you identify and exploit these vulnerabilities to a level where it's almost like superhuman, you know, like 20 years ago, Microsoft would release, let's say, maybe 20 updates every month.
- 13:26
- Unidentified speaker: Now you're looking at like you know, 1400 updates per month.
- 13:31
- Unidentified speaker: You know, like and then that number is gonna continue to grow.
- 13:34
- Unidentified speaker: Um it's just to give you an idea of like the scale of things.
- 13:39
- Unidentified speaker: And now, you know, what for example, say to have certain systems, it may take you, let's say, three months with lots of people.
- 13:47
- Unidentified speaker: Now that may take you a few days with just one person and a very capable AI model.
- 13:53
- Unidentified speaker: That's the reality that we're in today.
- 13:55
- Unidentified speaker: It even changed the way how attackers uh like you know, like hack certain types of systems, you know, like uh like then the banking sector, the way our customers are in that you know, you can tell some stories about that time, but that's kind of what I think.
- 14:09
- Unidentified speaker: It's I think it's the systems have always been vulnerable to some degree, and now it's just the AI just accelerates that and uh makes it much more visible.
- 14:18
- Unidentified speaker: And it's also made it easier for people to identify these vulnerabilities as well.
- 14:22
- Unidentified speaker: Yeah.
- 14:23
- Unidentified speaker: And so uh and then you know, obviously there are the pros and cons to that, but uh yeah, we can go into that later.
- 14:44
- Unidentified speaker: What's next for virtual assets?
- 14:46
- Unidentified speaker: I'm Jacob Greet, host of Economy Scale, a series exploring growth in key global sectors.
- 14:51
- Unidentified speaker: And this episode we're diving into the digital forces shaping finance.
- 15:00
- Unidentified speaker: We're seeing a lot of institutional players come in, digital asset dedicated hedge funds, high frequency traders, and that sort of thing.
- 15:06
- Unidentified speaker: And folks do service to institutional clients.
- 15:09
- Unidentified speaker: As more capital enters the market, adaptive regulation is likely to become increasingly key.
- 15:14
- Unidentified speaker: Greece America Ripple says a lot of regulators globally would like to keep the industry at arm's length.
- 15:21
- Unidentified speaker: That's the opposite here.
- 15:22
- Unidentified speaker: I think also what you'll find is that the regulation and frameworks can come out, but some of them are maybe a slight iteration on some 80-year-old policy and not actually in line with what the technology can do and offer.
- 15:34
- Unidentified speaker: And again, that's very different in Dubai.
- 15:36
- Unidentified speaker: Listen to the full episode wherever you get your podcasts.
- 15:38
- Unidentified speaker: This series is brought to you by Bloomberg Media Studios and the Dubai Department of Economy and Tourism.
- 15:44
- Unidentified speaker: Canadian women are looking for more, more about themselves, their businesses, their elected leaders, and the world around them.
- 15:51
- Unidentified speaker: And that's why we're thrilled to introduce the Honest Talk Podcast.
- 15:54
- Unidentified speaker: I'm Jennifer Stewart, and I'm Catherine Clark.
- 15:57
- Unidentified speaker: And in this podcast, we interview Canada's most inspiring women, entrepreneurs, artists, athletes, politicians, and newsmakers, all at different stages of their journey.
- 16:06
- Unidentified speaker: So if you're looking to connect, then we hope you'll join us.
- 16:09
- Unidentified speaker: Listen to the Honest Talk Podcast on iPhone Radio or wherever you listen to your podcasts.
- 16:15
- Unidentified speaker: Why don't we talk about discovering the PHP vulnerability then?
- 16:19
- Unidentified speaker: Because I think this was a joint project between Caliph, Wiz, and I think it was OpenAI.
- 16:26
- Unidentified speaker: How did you find this?
- 16:28
- Unidentified speaker: What was the process?
- 16:28
- Unidentified speaker: And then what was the decision?
- 16:30
- Unidentified speaker: I know you guys have gotten a little bit of pushback about publicizing it before it was fixed.
- 16:36
- Unidentified speaker: So what was the process of actually, you know, putting out the statement saying we found this thing?
- 16:42
- Unidentified speaker: We're really interested in like finding vulnerabilities that have uh wide impact.
- 16:47
- Unidentified speaker: And uh we want to understand these vulnerabilities to help like inform decisions, like defense decisions, how to fix and how to mitigate them like for once and forever.
- 16:59
- Unidentified speaker: On that background, so we joined the uh OpenAI's uh Past the Planet program.
- 17:05
- Unidentified speaker: This is where uh OpenAI funds companies like Caliph to go out and help open source maintainers to identify and fix uh mitigate vulnerabilities in their software.
- 17:21
- Unidentified speaker: And we have uh this engineer, like Ryan Austin, who is uh he's like a PHP historian.
- 17:28
- Unidentified speaker: Yesterday we went to a conference in San Francisco.
- 17:32
- Unidentified speaker: We uh basically uh presented like you know, PHP2 shell, and Ryan walked everybody over like the 25 years of history of PHP vulnerabilities.
- 17:45
- Unidentified speaker: So we have this like uh engineer Ryan, and this is how we worked basically.
- 17:50
- Unidentified speaker: We our engineer they're interested in certain things and then they just go and hack those things.
- 17:56
- Unidentified speaker: And you know, ask business people who try to say, okay, how can we convince some of the people to pay us for this work?
- 18:03
- Unidentified speaker: And that's exactly what we did with like OpenAI.
- 18:06
- Unidentified speaker: We convinced OpenAI, okay, we can hack BHP.
- 18:10
- Unidentified speaker: Can you uh fund our work?
- 18:12
- Unidentified speaker: And they said yes, they gave us uh tokens and money, and then we go and convince like BHP Foundation, which is the organization behind BHP, that let's work together, you know, we can help you.
- 18:24
- Unidentified speaker: And it's actually pretty interesting because I mentioned like that like a history of 25 years of like BHP vulnerabilities.
- 18:32
- Unidentified speaker: And the person that is behind all of these hacks is like Stefan Escher.
- 18:38
- Unidentified speaker: He's also at Caliph.
- 18:40
- Unidentified speaker: He joined us, I think, six months ago.
- 18:42
- Unidentified speaker: Back in the day, he's the BHP security guy.
- 18:45
- Unidentified speaker: Basically, he hacked and helped like Heartland PHP.
- 18:49
- Unidentified speaker: So we we didn't know anybody at the Beach Beach Foundation.
- 18:53
- Unidentified speaker: So we asked Stefan to introduce us.
- 18:55
- Unidentified speaker: And because you know, even though Stefan uh stopped working on PHP security for a while now, his name still rings, you know, very um loudly in this circle of the world.
- 19:08
- Unidentified speaker: I saw the um the comic that you guys put up about this.
- 19:11
- Unidentified speaker: It was kind of funny.
- 19:13
- Unidentified speaker: Yes.
- 19:13
- Unidentified speaker: He's like uh Indiana Jones type character.
- 19:16
- Unidentified speaker: Yeah.
- 19:17
- Unidentified speaker: So he connected us and then uh we met with like uh Volker Dush at the BGB Foundation, and uh yeah, that's how we got started.
- 19:26
- Unidentified speaker: I think we started in uh July 30, and then immediately after one or two days, you know, Ryan filed the first vulnerability, and he was interested in like finding a similar vulnerability similar to one vulnerability that like Stefan disclosed like 22 years ago.
- 19:49
- Unidentified speaker: And uh, to his surprise, you know, there's still a bunch of them like that in the BHP um you know software right now.
- 19:58
- Unidentified speaker: And um, so that's how that's how we filed.
- 20:00
- Unidentified speaker: And then we reported it to the BCP Foundation because we know immediately that is a critical vulnerability that could affect like a large part of the internet.
- 20:11
- Unidentified speaker: Then the BSB Foundation actually shared with us that they actually filed this vulnerability, the same vulnerability, uh, two years ago.
- 20:20
- Unidentified speaker: But they didn't fix it.
- 20:22
- Unidentified speaker: Because uh, when they found it, they discovered that like the fix is very expensive.
- 20:29
- Unidentified speaker: And there's just like maybe a few options to fix it, and none of them were actually perfect because um, you know, they need to change uh a lot of like functionalities within PHP, and those changes may introduce new bugs, new vulnerabilities, or new reliability issue.
- 20:51
- Unidentified speaker: And people would be very upset if they upgrade you know PHP and their website stopped working.
- 20:58
- Unidentified speaker: So in the end, back in like 2024, the BHP Foundation decided not to fix it because there was no exploy.
- 21:09
- Unidentified speaker: No one could manage to show that this is actually a real issue.
- 21:13
- Unidentified speaker: And that's exactly what they asked us.
- 21:16
- Unidentified speaker: They come back, they ask, like, do you have an exploit?
- 21:19
- Unidentified speaker: Then the next day we share with them the first exploit that showed them that, yeah, someone uh doesn't need to know anything, can hack your website remotely.
- 21:30
- Unidentified speaker: So just to be clear, the use of AI in all of this, it wasn't necessarily that important in finding the vulnerability.
- 21:39
- Unidentified speaker: It was more important in developing an actual exploit for a known issue.
- 21:44
- Unidentified speaker: We think like that's probably the right way to think about it.
- 21:47
- Unidentified speaker: But I think even for fighting the vulnerability without AI, it would take a lot more time, right?
- 21:53
- Unidentified speaker: So you you can imagine like BHP was created like maybe 30 years ago.
- 22:00
- Unidentified speaker: So it's a huge code base with a lot of like features.
- 22:04
- Unidentified speaker: And as a human engineer, you have to dig into this code base, and it would take like months, right?
- 22:12
- Unidentified speaker: And along the way, people may give up.
- 22:14
- Unidentified speaker: But the fact that BHP Foundation filed the bug in 2024 proof that humans can actually file the bug, five the bugs, right?
- 22:24
- Unidentified speaker: And the problem was the exploitation, like you said.
- 22:27
- Unidentified speaker: Yeah, and for exploitation, uh like AI plays a major role in it.
- 22:33
- Unidentified speaker: It's like becomes an engineering problem where our engineers would tell AI, you know, this is how I want to design our exploit, and you go and do it, and then you know, prove to us that it works correctly.
- 22:47
- Unidentified speaker: Yeah.
- 22:48
- Unidentified speaker: Let me ask a philosophical question.
- 22:50
- Unidentified speaker: Uh, is there a reason to believe that defensive cybersecurity can stay ahead of offensive cybersecurity?
- 22:58
- Unidentified speaker: Just generally, and then we can get into some details.
- 23:00
- Unidentified speaker: But philosophically, like, is that something that we can reasonably hope for?
- 23:06
- Unidentified speaker: I think so.
- 23:06
- Unidentified speaker: Yes.
- 23:07
- Unidentified speaker: How come?
- 23:08
- Unidentified speaker: Because uh you can use DAI to build certain types of systems in ways that may be very challenging for attackers to exploit, right?
- 23:20
- Unidentified speaker: Um, you know, today you can use AI to build whatever systems you want, you know, and uh very quickly, right?
- 23:27
- Unidentified speaker: You can experiment, you know, get feedback very quickly.
- 23:30
- Unidentified speaker: It can also process data like very, very quickly and reason about that data.
- 23:36
- Unidentified speaker: And so I think it is possible.
- 23:39
- Unidentified speaker: I think it it may not be like immediate, uh, but like, you know, over time we're gonna perhaps, you know, like use AI to uh help us write more secure software or right or build more secure systems, right?
- 23:52
- Unidentified speaker: And eventually, I think that should kind of like counterbalance the um the advantages that the attackers have.
- 23:58
- Unidentified speaker: Like I think that's uh I I I perhaps I'm being optimistic, but I I I I do foresee this happening.
- 24:04
- Unidentified speaker: I can add something there.
- 24:06
- Unidentified speaker: So if you look at the history of humanity, our history is the history of like weaponize our tools, right?
- 24:14
- Unidentified speaker: If you find a powerful tool, then someone would find a way to use that tune to as a weapon.
- 24:20
- Unidentified speaker: Yeah.
- 24:21
- Unidentified speaker: So and attackers always are early adapters of tunes, right?
- 24:27
- Unidentified speaker: New technology.
- 24:28
- Unidentified speaker: Defenders move like at much uh slower pace than attackers.
- 24:33
- Unidentified speaker: But fundamentally, we believe there are more good guys than bad guys out there.
- 24:39
- Unidentified speaker: And eventually, you know, uh the good guys would catch up and build things that help uh you know everybody defends, fend off the bad guys.
- 24:50
- Unidentified speaker: And I I don't think like um in general, like cybersecurity, if you see the news all the time, you see like we got like data breach every day almost, even before AI.
- 25:02
- Unidentified speaker: So you may feel like it's not working at all.
- 25:05
- Unidentified speaker: But if you're in the field, you can see the progress, right?
- 25:08
- Unidentified speaker: In the past like 25 years, there's a tremendous progress in cybersecurity.
- 25:13
- Unidentified speaker: Of course, people still got hacked because this is it's difficult.
- 25:18
- Unidentified speaker: Imagine you have a very complex system, very like imagine you have a house with a lot of doors, a lot of ways to get in, and cybersecurity must like make sure that everything is safe.
- 25:29
- Unidentified speaker: At the same time, your house or your office must remain open for business.
- 25:34
- Unidentified speaker: Right.
- 25:34
- Unidentified speaker: So it's actually a pretty difficult problem.
- 25:37
- Unidentified speaker: At the same time, without the progress that we like, the industry met in the past 25 years, things could be much worse, right?
- 25:46
- Unidentified speaker: And I think the same thing would be happening here.
- 25:49
- Unidentified speaker: It's gonna take a while for defenders to catch up.
- 25:53
- Unidentified speaker: Right now, you know, AI helps attackers uh more than defenders.
- 25:58
- Unidentified speaker: Is the general idea that we're just in this sort of awkward phase where a lot of people in cybersecurity always refer to technical debt, right?
- 26:07
- Unidentified speaker: This idea that you haven't upgraded a system for ages, like the PHP vulnerability we were just discussing, and so now you have to pay back that technical debt.
- 26:17
- Unidentified speaker: The idea that we're just in an awkward moment where AI is going to be able to exploit existing flaws in the system, but maybe we eventually move to a place where we upgrade the entire system.
- 26:30
- Unidentified speaker: We reduce our outstanding technical debt.
- 26:31
- Unidentified speaker: We reduce the outstanding technical debt, exactly.
- 26:34
- Unidentified speaker: That's an optimistic way to look at it.
- 26:36
- Unidentified speaker: Okay, you know, we found that they're like system that we thought that, okay, that don't have a lot of technical debt that turn out that they just like a dumpster of fire like everybody else.
- 26:51
- Unidentified speaker: That was a very big surprise, right?
- 26:53
- Unidentified speaker: Uh to me personally.
- 26:54
- Unidentified speaker: Like we know there are places, there are bad problems, and we know that if we just point AI at that thing, there would be a lot of bugs coming out.
- 27:04
- Unidentified speaker: But there are also places we thought that uh could be okay because uh they received a lot of attention over the years already.
- 27:12
- Unidentified speaker: Uh, but turn out that there's still a lot of bugs there as well.
- 27:16
- Unidentified speaker: And a lot mean like a lot more, like 10x more than we expected.
- 27:21
- Unidentified speaker: And uh, of course, there are still like a few uh small places where there was fewer bugs back then, before AI.
- 27:29
- Unidentified speaker: Now there's still fewer bugs still now, but they're like very small, like very like little the exception, not the norms.
- 27:39
- Unidentified speaker: So that's actually a big surprise.
- 27:41
- Unidentified speaker: Uh because I I don't think anybody's was surprised to find a critical vulnerability in PHP, because PHP wasn't known for like race security.
- 27:53
- Unidentified speaker: So of course at the PHP2 shell scale, then that's a different matter.
- 27:57
- Unidentified speaker: But if you find another vulnerability in PHP today, then that's not a surprise.
- 28:02
- Unidentified speaker: Uh but if you go and you know find vulnerability, or maybe a hundred vulnerabilities, a thousand vulnerabilities in the Chrome browser, for example, that's a surprise.
- 28:13
- Unidentified speaker: And that's exactly what happened.
- 28:14
- Unidentified speaker: Like people file like a thousand vulnerabilities in Chrome.
- 28:18
- Unidentified speaker: And you know, I was at Google and I knew, and then I think the entire industry like assumed that and knew that wow, there was a lot of investment in Chrome security.
- 28:31
- Unidentified speaker: And not just like, you know, remove, like eliminate vulnerabilities, but making sure that even if you have vulnerabilities, it's difficult to exploit these vulnerabilities.
- 28:41
- Unidentified speaker: This is what we call mitigations.
- 28:43
- Unidentified speaker: You accept that you have vulnerabilities, but then you make sure that like uh attackers couldn't use these vulnerabilities uh to control your computer or your browser.
- 28:53
- Unidentified speaker: We thought that you know uh the defense layers uh were pretty good and the the rate of vulnerabilities were low.
- 29:02
- Unidentified speaker: But then um after AI, uh we discussed, we just discovered that either like we as security engineers uh suck at our job, uh we couldn't find this thing, or they're just like too much code that we didn't have enough time to look into.
- 29:20
- Unidentified speaker: So this raises another question: like, is it possible to have a provably secure browser or a piece of software?
- 29:29
- Unidentified speaker: You ship the piece of software, and you can prove maybe in the same way that mathematicians produce things and this thing called lean or whatever.
- 29:37
- Unidentified speaker: Does cyber have an equivalent where you can say this is provably secure?
- 29:42
- Unidentified speaker: And we know that no matter how good AI gets, it will never find a vulnerability in this piece of software?
- 29:49
- Unidentified speaker: Oh man, this is uh loaded question, uh Joe.
- 29:53
- Unidentified speaker: I thought it was a simple question.
- 29:55
- Unidentified speaker: Yeah.
- 29:56
- Unidentified speaker: I don't know, to be honest.
- 29:58
- Unidentified speaker: You know, like browsers are like, you know, its own operating system, like very, very complex operating systems.
- 30:04
- Unidentified speaker: Yeah.
- 30:04
- Unidentified speaker: And today, doing uh, you know, like formally verified software, you know, like it doesn't work at that scale.
- 30:12
- Unidentified speaker: Um, it's typically smaller, smaller software components.
- 30:16
- Unidentified speaker: So there is such a thing as formal verification when it comes to software.
- 30:20
- Unidentified speaker: Yes.
- 30:21
- Unidentified speaker: You could have small pieces of software in which you formally prove in the same way that it takes a mathematician seven lines to prove the two plus two equals four.
- 30:30
- Unidentified speaker: There is some sort of equivalent for software where you can prove these lines of code are secure.
- 30:38
- Unidentified speaker: But the pro the question is the scale of even something like a browser is just so enormous that's a very difficult thing to do.
- 30:46
- Unidentified speaker: Yes.
- 30:46
- Unidentified speaker: Okay.
- 30:47
- Unidentified speaker: It's uh it's something that's um, I think uh is being researched right now because this is, I think this is a uh like one of the one of the paths forward to make to build more secure systems.
- 30:57
- Unidentified speaker: Yeah.
- 30:58
- Unidentified speaker: There are like for example, you know, we've had systems where you know, like the core bit is formally verified, right?
- 31:04
- Unidentified speaker: It's an operating system that's formally verified, but then there are applications, you know, running like on these things.
- 31:10
- Unidentified speaker: And those applications have issues, you know, like so it's it's complicated.
- 31:15
- Unidentified speaker: And you know, and these are like small, like small systems, right?
- 31:18
- Unidentified speaker: And you're talking about browser, browser is like you know, many billions of lines of code.
- 31:22
- Unidentified speaker: And so that's um, yeah.
- 31:24
- Unidentified speaker: I mean, Ty can tell you how big Chrome is.
- 31:26
- Unidentified speaker: I I mean they have like zillions of people working on it, and you know, any like lots of people working on Chrome Security after all these years, but uh, you know, it's but long story, is there are such things, but uh to make it work on a browser or on a very, very large system, like this is it's um, we don't have anything like that today.
- 31:44
- Unidentified speaker: Got it.
- 31:44
- Unidentified speaker: I think there's um there's a deeper reason here.
- 31:48
- Unidentified speaker: You know, the reason like uh software is eating the world, you can see like you know, the top, the biggest companies in the world as software companies, right?
- 31:57
- Unidentified speaker: It's because software is easy to change.
- 31:59
- Unidentified speaker: And imagine, you know, that allows like software companies to adapt and to uh to learn much quicker than the rest of the world.
- 32:09
- Unidentified speaker: You have a product, if that doesn't work, then tomorrow, tonight you can change it.
- 32:13
- Unidentified speaker: And that actually allows the software world, you know, to become uh to dominate the world, basically.
- 32:19
- Unidentified speaker: So basically, fundamentally, software is easy to change and people want to change it constantly.
- 32:25
- Unidentified speaker: And that's the reason why you know uh the tech companies are doing so well.
- 32:30
- Unidentified speaker: So if you ask, like, okay, now let's not change this like frequently, uh then that goes against the interest, right?
- 32:39
- Unidentified speaker: Yeah, and that's not gonna happen.
- 32:42
- Unidentified speaker: So it means that even if you have a way to formalize, uh formally verify everything, that will take a lot of time unless you have some sort of AI help you do the formal verification automatically.
- 32:56
- Unidentified speaker: Otherwise, it's not gonna happen because people want to change their software like as quick as possible, because that's how they do business.
- 33:32
- Unidentified speaker: Whatever you want to do.
- 33:49
- Unidentified speaker: I read you after today.
- 34:00
- Unidentified speaker: Wrap up your work today with the Bloomberg Business Week Daily Podcast.
- 34:03
- Unidentified speaker: We bring you deeper dives into the story shaping your world from the evolution of AI to the shifting priorities of global business.
- 34:10
- Unidentified speaker: Plus, Silicon Valley Power Players and the latest tech trends.
- 34:14
- Unidentified speaker: Catch up on the conversations you miss during the day.
- 34:17
- Unidentified speaker: Subscribe to the Bloomberg Business Week Daily Podcast on Apple, Spotify, or anywhere you listen.
- 34:22
- Unidentified speaker: This is sort of a related question, but does AI start to change?
- 34:27
- Unidentified speaker: I guess the way or like the waterfall of the way things get fixed?
- 34:32
- Unidentified speaker: So nowadays it's like a vulnerability is discovered, the software company issues an alert and a patch, they disclose it, and then people go off and fix it.
- 34:42
- Unidentified speaker: If exploits are coming faster, does that process start to change in any way?
- 34:49
- Unidentified speaker: Yeah, it changed a lot.
- 34:50
- Unidentified speaker: Basically, back in the day, we have like two kind of vulnerabilities, like zero-day vulnerabilities that nobody knows about except the person who filed.
- 35:00
- Unidentified speaker: And then um day vulnerabilities, the vulnerabilities have been disclosed and maybe fixed by the vendors and maybe user have upgraded or not.
- 35:10
- Unidentified speaker: But now we also have uh something we call like minus one-day vulnerabilities.
- 35:16
- Unidentified speaker: So these vulnerabilities happen when, you know, imagine like uh let's take the PHP uh uh vulnerability as an example.
- 35:26
- Unidentified speaker: Let's say that you know, because PHP Foundation and PHP itself is an open source project, everything happens on GitHub, uh, right?
- 35:34
- Unidentified speaker: And imagine, let's imagine that we reported a bug and PHP fixed the bug and publicly on GitHub.
- 35:43
- Unidentified speaker: The moment we filed a bug, and then PHP produced a patch for that bug, that very moment that information became public.
- 35:56
- Unidentified speaker: Somebody who is watching the PHP you know repository on GitHub, yeah, they could like turn that away into an exploit within you know a few hours.
- 36:08
- Unidentified speaker: Of course, like the vulnerability that we found was more complicated than that.
- 36:12
- Unidentified speaker: But you can imagine that's in principle, like the window between the pass was being developed, it's not ready yet, you know, for anybody anybody.
- 36:22
- Unidentified speaker: And before it's become ready for any user to operate, there's a window, right?
- 36:28
- Unidentified speaker: Yeah.
- 36:28
- Unidentified speaker: In the past, you know, the attacker has to wait until the pass becomes available, and then it would take them a long time to develop and exploit.
- 36:40
- Unidentified speaker: So user can have longer time to operate.
- 36:43
- Unidentified speaker: Now that window is collapsing.
- 36:46
- Unidentified speaker: It is collapsing everywhere, not just in PHP, right?
- 36:49
- Unidentified speaker: In Windows, in Mac OS, in anything.
- 36:53
- Unidentified speaker: Because right now there are people who have like what we call like end factory, you know, they're just waiting for batches from Microsoft, from Apple, from important critical software.
- 37:08
- Unidentified speaker: Like, I'm sure tens of people are watching PHP today.
- 37:12
- Unidentified speaker: After what happened, they're watching PHP, Python, Java, all of all of those important languages.
- 37:19
- Unidentified speaker: And if there is any sign of like, oh, there's a new vulnerability, they're gonna like you know their AI would go pro and then you know, bundling tokens to five, identify that vulnerability, pre-produce it, be an exploit.
- 37:36
- Unidentified speaker: So that changed the um the practice, right?
- 37:40
- Unidentified speaker: Yeah.
- 37:41
- Unidentified speaker: I'm curious what um you thought after the hugging face incident, because after that moment, there were sort of two schools of thought that I thought there was or that I perceived.
- 37:50
- Unidentified speaker: There's one that was like AI creature.
- 37:53
- Unidentified speaker: There was this sort of the, I guess maybe the anthropomorphism Asian view, AI creatures, they want to escape their cages, they're devious, they're tricksters, they want to show, come up with a way to cheat on the right answers, but cheat on the way that the greater, you know, all these stories.
- 38:10
- Unidentified speaker: And then there are other people that were just like, guys, build a stronger sandbox.
- 38:15
- Unidentified speaker: Stop talking about all this stuff, just build a stronger sandbox.
- 38:17
- Unidentified speaker: Jensen Long is kind of in that category.
- 38:20
- Unidentified speaker: I loved his interview with Ezra Klein, where he was basically like, what's your problem?
- 38:24
- Unidentified speaker: Just make models that don't hammock.
- 38:26
- Unidentified speaker: And he's like, just don't ship it, fix your model, don't let them hammock.
- 38:30
- Unidentified speaker: And I love that because it's just like just fix it.
- 38:33
- Unidentified speaker: And then people are like, no, no, no, we can't.
- 38:35
- Unidentified speaker: You don't understand.
- 38:36
- Unidentified speaker: We have a solid alignment.
- 38:37
- Unidentified speaker: It's very tricky.
- 38:38
- Unidentified speaker: And he's like, just don't, just make, just fix the model.
- 38:40
- Unidentified speaker: And I'm sort of curious, as people who have come out of the traditional cyber world, but are now in the AI world, where do you land on this question?
- 38:50
- Unidentified speaker: Can you just like, guys, just fix the sandbox, or is there something deeper here?
- 38:54
- Unidentified speaker: I don't think that is simple to fix the sandbox.
- 38:57
- Unidentified speaker: That's that's the problem.
- 38:58
- Unidentified speaker: People think that, you know, maybe people are not in security.
- 39:01
- Unidentified speaker: They think that there's a simple solution to cybersecurity.
- 39:04
- Unidentified speaker: Just do this thing and you're save.
- 39:06
- Unidentified speaker: And that's how we have so many problems, you know.
- 39:09
- Unidentified speaker: Vendors go out and tell people, just buy this box, this magic box, yeah, and put it in your network and you you'll be safe.
- 39:17
- Unidentified speaker: No, it's not like that.
- 39:18
- Unidentified speaker: Even that box has problems.
- 39:20
- Unidentified speaker: You know, if you look into like there's a famous idea in security, like product security is not the same as security product.
- 39:29
- Unidentified speaker: Like security products may not have product security.
- 39:32
- Unidentified speaker: Even like the the firewalls that you buy, they may have problems.
- 39:36
- Unidentified speaker: Yeah.
- 39:37
- Unidentified speaker: And they do.
- 39:38
- Unidentified speaker: And uh what I mean here is so what we found, like the hacking phase and open AI incident was uh fascinating, right?
- 39:46
- Unidentified speaker: To watch.
- 39:48
- Unidentified speaker: And it's difficult.
- 39:49
- Unidentified speaker: Enterprise security is very difficult.
- 39:51
- Unidentified speaker: Uh anybody saying, like, oh, just buy this thing, just do that thing, you're gonna be safe.
- 39:57
- Unidentified speaker: No, it's just like very complicated.
- 39:59
- Unidentified speaker: And the reason is like um, and then people say, oh, just fix the model.
- 40:04
- Unidentified speaker: And that's a cat come back to the fix the model, just make one that doesn't hack.
- 40:08
- Unidentified speaker: That's that comes back to the incentive again.
- 40:10
- Unidentified speaker: And because if you look at like how AI companies, and Jensen must be careful here, right?
- 40:16
- Unidentified speaker: Because if AI company is not making money, he's not making money as well.
- 40:19
- Unidentified speaker: Yeah, of course.
- 40:20
- Unidentified speaker: And um the AI companies, they're making money by uh selling tokens.
- 40:26
- Unidentified speaker: And most of the tokens are used for coding today, yeah like programming, building things, right?
- 40:31
- Unidentified speaker: And if you build, if you want to be really good at coding, you need uh an agent on AI that understands code, understands technical problems, can debug things, can control computers.
- 40:44
- Unidentified speaker: But that's basically the definition of a hacker.
- 40:47
- Unidentified speaker: So you cannot really escape this, you cannot have a cake and eat it too, right?
- 40:52
- Unidentified speaker: You need to be good at coding, then you have an AI that is good at hacking.
- 40:58
- Unidentified speaker: So unless you don't want to spend more tokens or you don't want to get into the coding business anymore, you focus on like scientific discovery, physical AI, whatever, then maybe you can not, you know, you can train a model that is not good at hacking uh anymore.
- 41:18
- Unidentified speaker: But as long as you want the automate knowledge work, you know, then eventually you must, your AI must be able to use computer and control computer, understand technical problems.
- 41:30
- Unidentified speaker: And that's definitely of someone like Bruce.
- 41:33
- Unidentified speaker: So it's unavoidable.
- 41:36
- Unidentified speaker: So actually, on this note, I'm curious, can you just talk about what it's like being a hacker in the age of AI versus, say, even three or four years ago, right?
- 41:47
- Unidentified speaker: Like it's crazy how quickly stuff has shifted, but how has your day-to-day changed?
- 41:53
- Unidentified speaker: Yeah, well, just to give you some like some background.
- 41:55
- Unidentified speaker: So, like I wrote a textbook with some friends uh on reverse engineering.
- 42:00
- Unidentified speaker: It used to be like, you know, like you try to like reverse engineering is the practice of you know understanding binary like software without any source code or hardware systems.
- 42:09
- Unidentified speaker: That used to be like uh I wrote this textbook, and then I think with the advent of AI, I think what like you know, whatever, nine months ago, as of nine months ago, like that entire book is deprecating.
- 42:21
- Unidentified speaker: It takes like you know, two prompts or one prompt, and then bam, it'll generate the textbook for you, and it's even better, you know, and it took us you know a long time to write.
- 42:29
- Unidentified speaker: That just gives you an idea.
- 42:30
- Unidentified speaker: And then like I talked to friends who've been doing this for decades, right?
- 42:35
- Unidentified speaker: They're like specialists in this area, and they tell me that, oh yeah, man, like I spend, you know, whatever, five dollars on some open source model and it reverse engineered the firmware on you know, some Canon or like you know, HP printer, and it hacks it for you know, whereas before it would take me, you know, like months to figure this some some some stuff out.
- 42:55
- Unidentified speaker: You know, that's just uh just to give you like the like the you know a glimpse into uh uh you know what's happening, like and how how quickly things change.
- 43:04
- Unidentified speaker: Uh front, you know, like this this guy, um uh Matt Swish, he's uh you know, yeah, and he this dude.
- 43:09
- Unidentified speaker: He's a friend of the show.
- 43:11
- Unidentified speaker: A long time friend of the show, multi-time guest.
- 43:14
- Unidentified speaker: Yeah, keep going.
- 43:17
- Unidentified speaker: Oh, okay.
- 43:17
- Unidentified speaker: Yeah, it's you know, like we like I'm known for a long time, and like now he's like he showed us he gave a talk to the team the other day and they showed us rack, you know, like in like in like um in his apartment and is sitting there, you know, like hacking stuff, like Kwami's sitting there talking.
- 43:33
- Unidentified speaker: It's just insane.
- 43:35
- Unidentified speaker: Like something that uh we have people that say, you know, for example, like we've been hacking the iPhone, okay?
- 43:41
- Unidentified speaker: To build a full chain on the iPhone, it's not an easy task.
- 43:45
- Unidentified speaker: It's sometimes it takes several people to do this, right?
- 43:48
- Unidentified speaker: As you um and our team was able to do this within like a few days, you know, as a one person.
- 43:57
- Unidentified speaker: And we have multiple of these chains.
- 43:59
- Unidentified speaker: It really compresses the the time, like the uh the uh the exploitation, uh the time to to build um uh exploits.
- 44:08
- Unidentified speaker: Um, we see customers getting hacked uh by AI, uh, you know, in the financial uh in the like banks basically.
- 44:16
- Unidentified speaker: You know, defenders cannot, you know, type actually ty you want to tell these like these stories.
- 44:21
- Unidentified speaker: You know, it just a little bit happened in the last few weeks, you know.
- 44:24
- Unidentified speaker: Yeah, like how things have changed.
- 44:26
- Unidentified speaker: Yeah, basically we saw like you know, in the uh open AI and hooking face incident, they have one detail that that is very interesting.
- 44:34
- Unidentified speaker: Uh I'm not sure if people notice, but open AI note that like uh the people who handled incident on the OpenAI side, they said that they have handled a lot of incidents, but this is the first time they saw that you know an incident where you have like a hundred points within your networks that were compromised at the same time.
- 44:56
- Unidentified speaker: Yeah.
- 44:57
- Unidentified speaker: Like there were a hundred like hackers within your system.
- 45:01
- Unidentified speaker: Normally you have maybe one or two, right?
- 45:03
- Unidentified speaker: Now you have a hundred.
- 45:05
- Unidentified speaker: And I saw that, I was like, wow, that was pretty interesting, but maybe because OpenAI have lots of tokens, so their agents can just go, you know, hack everything.
- 45:14
- Unidentified speaker: And then just two weeks ago, we have like two clients in the banking industry, and they got hacked and they called us to help them with like the incident response.
- 45:26
- Unidentified speaker: We came in because we do hacking as well.
- 45:28
- Unidentified speaker: We know like usually it took us like maybe a month or something once we come in a network and then took a month for us to spread out.
- 45:36
- Unidentified speaker: For the AI, it took like they came in, they they broke the network, they broke inside the network on Monday, by Tuesday, they're everywhere, everywhere.
- 45:44
- Unidentified speaker: It's like cancer, it spread like at a machine speed.
- 45:47
- Unidentified speaker: And it was like mind-blowing, right?
- 45:49
- Unidentified speaker: Because how could you defend against that?
- 45:51
- Unidentified speaker: Like in in the past, we we were telling our clients, you know, you have to invest in detection.
- 45:58
- Unidentified speaker: Well, what would how can you help yourself?
- 46:00
- Unidentified speaker: If you see it, you can detect it, but there's a hundred of them.
- 46:04
- Unidentified speaker: Like noted, we have a hundred hackers in our system right now.
- 46:07
- Unidentified speaker: It's not really helpful.
- 46:08
- Unidentified speaker: Exactly.
- 46:09
- Unidentified speaker: How could you do it, right?
- 46:10
- Unidentified speaker: And then you have to shut down everything, but shut down is not an option because you are a bank.
- 46:15
- Unidentified speaker: So that's actually requires to sit down and think about how to redesign like defense system, right?
- 46:22
- Unidentified speaker: What we have right now is not working very well.
- 46:25
- Unidentified speaker: But like to come back to your question, like as a hacker, you know, how does it feel?
- 46:30
- Unidentified speaker: I love history, so I always come back to history to say, like, does it did it happen before?
- 46:36
- Unidentified speaker: It actually did happen um, you know, with like Jess, for example.
- 46:40
- Unidentified speaker: You remember like uh in 1997, you know, IBM Deep Blue defeated uh Gary Gasparo.
- 46:47
- Unidentified speaker: Um that was a big moment, right?
- 46:50
- Unidentified speaker: And then 20 years later, Anpha Go defeated like Lisi Do in the game of Go.
- 46:56
- Unidentified speaker: But those two moments are actually quite different.
- 46:59
- Unidentified speaker: Like the first moment, the Deep Loo moment, after Deep Blue, the best chess player wasn't a computer.
- 47:07
- Unidentified speaker: Because Deep Blue was a mainframe, right?
- 47:10
- Unidentified speaker: Not something that you can run on your computer.
- 47:13
- Unidentified speaker: And it took 10 years for someone to go from Deep Blue to develop a computer software that you can run on your computer that befitted everybody.
- 47:23
- Unidentified speaker: Uh but the 10 years, it took 10 years.
- 47:26
- Unidentified speaker: During that 10 years, the best jazz player was a Centaur player.
- 47:29
- Unidentified speaker: This is a term coined by Gary Kasparov.
- 47:32
- Unidentified speaker: It's a combination of computer and human.
- 47:35
- Unidentified speaker: It took 10 years.
- 47:37
- Unidentified speaker: For AnphaGo, it took like three years.
- 47:40
- Unidentified speaker: Like after AmphaGo, then 2019, somebody came up with a computer software that defeated anybody without any human input at all.
- 47:50
- Unidentified speaker: At Caliph, we think like you know, we are in the era of the central hackers.
- 47:55
- Unidentified speaker: Basically, the best hacker right now is a combination of AI and humans.
- 48:01
- Unidentified speaker: And you know, our work and our friends' work as well show that AI is powerful, it can hack anything, uh, but it still requires a human expert to rivet, right?
- 48:13
- Unidentified speaker: And we don't know how long this is gonna last.
- 48:16
- Unidentified speaker: Uh, maybe three months.
- 48:18
- Unidentified speaker: Maybe you know, we we come back here and like by the end of the year, we tell you we retire now.
- 48:25
- Unidentified speaker: There's nothing else for us to do.
- 48:27
- Unidentified speaker: Maybe it's last like three years, and this is actually pretty exciting to find out.
- 48:31
- Unidentified speaker: That's exactly what we want to do.
- 48:33
- Unidentified speaker: And we think like that's like pessimistic view, right?
- 48:39
- Unidentified speaker: And AI is gonna do everything.
- 48:41
- Unidentified speaker: Let's just like retire and hate AI.
- 48:44
- Unidentified speaker: Or you can see AI as a tool.
- 48:47
- Unidentified speaker: It's like like humanity just discovered fire.
- 48:50
- Unidentified speaker: Yeah.
- 48:50
- Unidentified speaker: And without fire, we wouldn't be here today.
- 48:53
- Unidentified speaker: So, of course, we have to think about how to control fire.
- 48:56
- Unidentified speaker: At the same time, a very important question that we need to ask is what should we cook, right?
- 49:03
- Unidentified speaker: That's very important, and we want to find out what we can cook with AI and um how can we do it in a safe way that helps you know everybody?
- 49:12
- Unidentified speaker: Protein that makes our brains get bigger.
- 49:15
- Unidentified speaker: Um all right, well, Ty and Bruce from Caliph, thank you so much for the video.
- 49:20
- Unidentified speaker: That was a lot of fun.
- 49:21
- Unidentified speaker: You're both excellent.
- 49:22
- Unidentified speaker: Thank you both.
- 49:23
- Unidentified speaker: Thanks.
- 49:24
- Unidentified speaker: Thanks, Joe.
- 49:24
- Unidentified speaker: Thanks, thank you for the night.
- 49:41
- Unidentified speaker: Not exactly uplifting.
- 49:43
- Unidentified speaker: I was surprised at Ty's response.
- 49:46
- Unidentified speaker: I thought maybe, maybe people would say we're in this weird, like awkward moment where a lot of like legacy issues are being discovered.
- 49:53
- Unidentified speaker: But once AI actually gets deployed and starts redesigning some of the architecture, it'll be a lot more resilient.
- 50:00
- Unidentified speaker: Apparently, that's not the case.
- 50:02
- Unidentified speaker: That's a little bit worrying.
- 50:03
- Unidentified speaker: I agree.
- 50:04
- Unidentified speaker: There was so much in there that was interesting.
- 50:06
- Unidentified speaker: And like, I could, you know, all the stuff about how yes, you could have provably secure software, but it's small.
- 50:12
- Unidentified speaker: You can't even do it for something that says the browser browser.
- 50:15
- Unidentified speaker: I love that last point about how we're in the centaur age.
- 50:19
- Unidentified speaker: Yeah.
- 50:25
- Unidentified speaker: Three months a year, we have no idea, but the presumption should be that it won't last very long, et cetera.
- 50:31
- Unidentified speaker: Also, the idea of like a computer, an agent that can code is inherently hacker.
- 50:39
- Unidentified speaker: And so you can just tell it to be not to hack it.
- 50:43
- Unidentified speaker: Teach it not to heck.
- 50:44
- Unidentified speaker: What's wrong with you?
- 50:44
- Unidentified speaker: Just teach it not to heck.
- 50:45
- Unidentified speaker: If you if compute, if coding is going to be big business, which it is, you're gonna be building a lot of good hackers.
- 50:50
- Unidentified speaker: Yeah, and so the hackers proliferate.
- 50:53
- Unidentified speaker: On that note, shall we leave it there?
- 50:54
- Unidentified speaker: Let's leave it there.
- 50:55
- Unidentified speaker: This has been another episode of the All Thoughts Podcast.
- 50:57
- Unidentified speaker: I'm Tracy Alloway.
- 50:58
- Unidentified speaker: You can follow me at Tracy Alloway.
- 51:00
- Unidentified speaker: And I'm Joe Weisenthal.
- 51:02
- Unidentified speaker: You can follow me at the Stalwart.
- 51:03
- Unidentified speaker: Follow our producers, Carmen Rodriguez at Carmen Armin.
- 51:06
- Unidentified speaker: Dash O'Bennet at Dashpot, Kale Brooks at Kale Brooks, and Kevin Lozano at Kevin Lloyd Lozano.
- 51:12
- Unidentified speaker: And of course, you can chat about all of these topics 24-7 in our Discord, discord.gg slash hodlots.
- 51:18
- Unidentified speaker: And for more Odlots content, you can check out our daily newsletter, which you can find at Bloomberg.com slash hodlots.
- 51:24
- Unidentified speaker: And if you enjoyed this conversation, if you like it when we talk about hacking, then please leave us a positive review on your favorite podcast platform.
- 51:31
- Unidentified speaker: And if you're watching on video, then please leave a comment or like, or better yet, subscribe.
- 51:36
- Unidentified speaker: And remember, if you're a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free.
- 51:41
- Unidentified speaker: All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there.
- 51:47
- Unidentified speaker: Thanks for joining.
- 52:43
- Unidentified speaker: Subscribe to the Masters in Business podcast on Apple, Spotify, or anywhere you listen.
More from Odd Lots
The Impossible Trilemma That Hangs Over the Nvidia Crisis
Episode discussion: NVIDIA Leadership Change Signals Broader 'Impossible Trinity' of Sovereignty, Spending, and Sound Money
Speakers’ claims: Joe Weisenthal and Tracy Alloway discuss the resignation of NVIDIA CEO Jensen Huang and its market impact, using it as a lens to explore Tian Yang's 'Impossible Trinity' thesis. Yang argues that geopolitical rivalry and domestic social spending force governments to sacrifice sound monetary policy, leading to inflation and financial repression. The hosts analyze how this macro environment affects AI financing, noting that rising rates and inflation create headwinds for the sector's capital-intensive growth. Weisenthal suggests that government backstops will favor top-tier 'Frontier Labs' and domestic suppliers, while physical constraints may eventually force arms control agreements. The episode concludes that aging populations and entitlement obligations further strain resources, making the sustainability of both defense and social spending without compromising currency value increasingly difficult.
How Wall Street Is Bracing For the Post-Jensen Huang Nvidia Era
Episode discussion: Analysts assess NVIDIA's leadership transition risks, China exposure, and competitive landscape post-Jensen Huang
Speakers’ claims: Joe Weisenthal and Tracy Alloway discuss NVIDIA's upcoming earnings and the 'key man risk' surrounding CEO Jensen Huang, citing analyst Stacy Rasgon's downgrade due to internal relationship disclosures. Rasgon details NVIDIA's geographic revenue shifts, noting minimal direct sales to China due to regulatory restrictions, and argues that selling older chips maintains US technological dominance. The hosts analyze talent retention risks as employees face liquidity events and question whether major cloud providers like Google and Anthropic are successfully diversifying away from NVIDIA's hardware and CUDA software ecosystem. Weisenthal presents revenue projections for NVIDIA, Broadcom, and AMD, highlighting the massive scale required to sustain current growth rates amid increasing competition.