How AI is Redrawing the Battle Lines of Cybersecurity

00:00:00: Welcome to Signals, the podcast where we step back from a hype and look at what's shaping technology.

00:00:05: In each episode We talk with industry experts to surface trends question assumptions and unpack why things work.

00:00:11: The way they do today were focused not just on whats new but On the decisions developments that got us here And What To Expect Down The Road.

00:00:27: I'm joined Today by Andrew Rodin An expert in this security.

00:00:32: Andrew thank you for joining me today.

00:00:33: Yeah it is great to be here

00:00:34: Mike.

00:00:35: So tell Me A little bit about your background.

00:00:37: Well, I'm an ex-security engineer at CrowdStrike for quite a while.

00:00:43: We've been taking and intentional career break For four years now.

00:00:49: i've been doing derivatives trading did some serious miles on some long trails But mostly have spent time with my two kids And raising them kind of in transition.

00:00:59: right now might be going back into the industry.

00:01:03: I've just been really privileged and not just with spending time with my kids, but being able to step away from that institutional pressure.

00:01:11: You know you kind of get um...and it's just really provided a lot of clarity following the recent developments in security

00:01:18: right.

00:01:18: so i know that when you were working at cardstrike you're really at the bleeding edge of what's happening in security and i know you well enough to know Staying on top of everything that's happening.

00:01:31: So tell me a little bit about kind of the state and industry And how do you see things shaping up over the next few months?

00:01:40: Oh, man It's an exciting time.

00:01:44: so just like last week in anthropic Did their Claude stuff.

00:01:49: I'm not Just discovery of exploits in Firefox which has all been patched in one four eight but the actual exploit, which is brand new.

00:02:01: Now don't like freak out and write sensational headlines.

00:02:05: they disabled The Sandbox in Firefox to get it to do anything interesting.

00:02:10: And then only one of things I would love To talk about how that exactly worked.

00:02:18: But yeah It's been so cool.

00:02:22: It's an exciting time kind of like back into two thousand and thirteen that Time Of Air right when fuzzer really started impacting the industry And finding those ex-finding those vulnerabilities.

00:02:36: So talk to us for a second.

00:02:37: What is a fuzer?

00:02:39: Oh, yeah Like I love The Fundamentals it's one of my favorite things.

00:02:45: um so A Fuzzer Is more or less just a tool That providing a lot of random noise at software, with the intent to get it to crash.

00:02:57: Because once you understand that your software can crash It's likely there is going be an exploit There.

00:03:06: I think thats good summary.

00:03:07: Yeah That

00:03:08: makes sense.

00:03:09: Its a little smarter than that Like its not just like random noise But it's there, right?

00:03:15: Right.

00:03:16: It's a focused noise on an endpoint and the more that you can focus than The better results are gonna get.

00:03:23: yeah sure

00:03:23: but we're changing right now with large language models.

00:03:26: yes That's kind of.

00:03:30: You can still use that in.

00:03:31: oh man There some interesting things like where I would want to say is You know experience as a curl developer He's recently had to remove the bug bounty on curl.

00:03:46: Um, so I mean they've only given out maybe a hundred thousand dollars.

00:03:50: So it's not like disastrous But Daniels been really complaining about it and It's interesting because you have uh...it was just last year he complimented somebody On using new tools on essay ASTs To dig into vulnerabilities in curl.

00:04:10: um And Well, that was very productive last year and you can read in Joshua washers blog I'm butchering his last name.

00:04:20: But I'm gonna do it for everybody.

00:04:22: so um But that also led to just so much of a flood of AI slopping the bug reports.

00:04:32: It's just overwhelmed them.

00:04:33: And now that's gone.

00:04:35: I think that's a net negative for security industry sure And as we're transitioning with like Anthropic, with their Claude Opus four six which is the only one that's successfully done this at least out of them.

00:04:48: That was just last week.

00:04:51: it was published.

00:04:51: bleeding

00:04:52: edge okay

00:04:52: yeah well I mean you weren't wrong.

00:04:56: i do keep a little bit of an eye on things.

00:04:58: so not for sure incredibly boring but its fascinating to me.

00:05:07: So, do you feel that the loss of some bug bounty programs and some user-based emissions is going to be replaced by AI tools in real time?

00:05:20: Is it just a different experience or truly negative long term.

00:05:27: Right now its a net negative because all over the slot.

00:05:32: I think Anthropoc did really good job When they just published and they interacted with Firefox.

00:05:38: They wrote a beautiful blog up about it.

00:05:42: What ended up happening is?

00:05:45: They had around a hundred reports from their agent.

00:05:49: There's bit more.

00:05:50: things are on a hundred and twelve books And as I was submitting into Mozilla for firefox It just turned out that they were like, okay this is good stuff, send all of it.

00:06:02: Now that resulted in and I like being precise.

00:06:05: so i wrote down... That resulted in twenty two CVEs and then one extra just in one.

00:06:12: four eight zero Wow!

00:06:13: ...that was released now.

00:06:15: And of those uh..of those active CVE's One was able to be exploited by the AI agent.

00:06:21: Okay ..and that puts us at least really dangerous time, right?

00:06:25: Because you have a lot of bug bounty programs and also I've got reports where your being flooded by that AI slop.

00:06:34: And developers the LLMs do great job!

00:06:39: It looks like real report.

00:06:40: it needs somebody to verify...and at same time we can talk about these new tools especially with expert guidance not just now finding a potential vulnerability, but exploiting it with a ton of handheld.

00:07:00: Right?

00:07:00: I mean you're talking.

00:07:02: let's run down the list here.

00:07:05: You had multiple...you add all those tokens to find the vulnerabilities scanning the codebase right?

00:07:11: Your then looking at like four thousand dollars in tokens Just defined some of those exploits, right?

00:07:20: And this is to escape the CN box.

00:07:22: Right and then you got well over two hundred something attempts.

00:07:25: I think it was two hundred fifty-six because everybody loves powers of two!

00:07:30: And that was in WebAssembly.

00:07:33: It's a fascinating thing.

00:07:35: Yeah Well

00:07:37: yeah can...I mean like hey guys Can for sure You could edit me down For

00:07:43: your audience.

00:07:44: I don't mind but i want to talk about.

00:07:47: This is cool.

00:07:48: Do you mind go for it?

00:07:50: Okay, so What they found?

00:07:53: yeah You asked me what a fuzzer was

00:07:55: Yeah.

00:07:55: Yeah,

00:07:56: you know why?

00:07:56: no of course I'm gonna do this different.

00:07:58: I'm going to talk about Super Mario.

00:08:00: okay We'll get there Mario now we're Gonna Get There.

00:08:03: i know where this Is Going through.

00:08:04: i love It.

00:08:05: i Know i like i said i've Been Away from the industry and that Means That you get To do Whatever you want.

00:08:11: yes

00:08:12: So speed Running

00:08:14: yep

00:08:15: And when you really think about it, that is a microcosm of security.

00:08:20: So it's great way to talk what the technique was here and actually used in ways not overwhelming somebody who just wants know whats going on.

00:08:30: so in Super Mario speedrun this back in the nineties.

00:08:36: but you got Yoshi.

00:08:37: And do you remember there's the football player in those games?

00:08:43: Yeah, yeah.

00:08:44: All right I can't remember his name but i'm sure somebody could look it up for me and Yoshi can eat him.

00:08:50: okay yep good But there is the...I think its eleven or twelve star speedrun.

00:08:56: Can't remember Doesn't matter anyway what You can Do as you go In and There'S a way to get The game To have Yoshi's Tongue Eat A coin And when that occurs, you're gonna hand waving here on and with intent.

00:09:17: You know go look it up There's lots of stuff on this.

00:09:21: You'll effectively get the game to think that you ate Mr.. Football player.

00:09:25: Wow Okay now This is great because Effectively that football player by accident in The Code Base Because all sprites In That Game are effectively power-ups or enemies, it's easy to think about this.

00:09:38: You know Mario is a sprite.

00:09:39: you got your Koopas and everything else jump on their head...you also have your mushrooms.

00:09:44: they're power ups.

00:09:46: so what's occurring in the game when you get Yoshi to eat This enemy that you're not supposed be able to eat In memory of them never bothered dealing with.

00:09:57: They coded it as a power up.

00:10:00: Now In the code that handles, is it a power-up?

00:10:05: It was only meant for what?

00:10:06: four or five power ups in the game.

00:10:09: You know we don't need to really talk about the cool feather and the cape... Sorry I'm darking a little but That's so fascinating here.

00:10:19: Is this case of getting The Game To believe you're eating something And A Power Up But its not there.

00:10:25: Now your doing array out of bounds because it's okay, which power up is this?

00:10:32: And its you know one hundred whatever.

00:10:34: I think it actually set me for.

00:10:36: but doesn't matter have a table that only supposed to be four or five long eight if i'm remembering correctly.

00:10:44: and now your overflowing the table and then You go to next thing of calling a function in game If anybody really interested lower level security.

00:10:57: And they're just looking to be like, is this going to be entertaining?

00:11:01: By all means look into video games because it's a great way to kind of test your feet.

00:11:08: you get in assembly and it's the nineteen eighties, nineteen nineties assembly.

00:11:12: It's not the crazy stuff that we see today.

00:11:16: What happens then?

00:11:17: you have your jump.

00:11:20: That brings us back.

00:11:22: So now You're eating a football player.

00:11:26: It's pretending it is power-up and then you've got your pointer

00:11:29: jump.".

00:11:30: That what Claude found, that was what he did in JavaScript.

00:11:37: we can go down that path if they really want.

00:11:40: but trust me I'm not even interested telling about the intricacies of this binding and Firefox optimization on it In WebAssembly context.

00:11:52: But that's the setup you need is effectively by importing modules in a clever way, Claude found.

00:12:02: From first principles from the code base and C++.

00:12:05: not like fuzzing it.

00:12:06: That's the big difference here.

00:12:08: as your looking at source code now Okay but by tricking to say hey we're just binding this And then incorrectly allowing the types to bleed through for the web assembly.

00:12:22: Wow, that's the bug.

00:12:23: now Claude found that It couldn't quite jump itself to having it do the pointer bit.

00:12:31: so that makes a little bit less scary You know but we're you know what?

00:12:34: The future is here.

00:12:35: Just not well distributed right?

00:12:36: yeah Yeah.

00:12:37: So they had to load it with you and train all on previous Firefox CD ease And all this and they prompted the agent with Here is the bug that you successfully found it.

00:12:49: And we're going to start you off on an exploit, We are gonna give you something that crashes the web browser which effectively is You call a function with number four and then because its javascript object pointer That's going be called as a function.

00:13:10: It obviously crashes the browser From there once they disable the Firefox sandbox.

00:13:16: So again, don't freak out.

00:13:18: They were able to get it to successfully read a file by using array bounding where effectively all that was doing is using a pretty and they don't publish this but you're gonna use a pretty standard payload for reading the file off disk in how to do that in a clever way And then... That's It!

00:13:37: You just put it near Array and you got the web browser to call that function?

00:13:42: Then boom!

00:13:45: you know, ace.

00:13:46: You've got arbitrary code execution on there and that's the kind of thing I spent my career looking for is we were looking for evidence when it happens.

00:13:59: so if i'm going to show for crowds right this would be a moment

00:14:03: because

00:14:08: But I think that's going to be an interesting thing is as we're going through this period of finding all these bugs, if you are a security practitioner.

00:14:19: You will need to look for the behavior and how they break your security perimeter.

00:14:29: Anyways...I'd love to go deeper on anything.

00:14:37: There's a lot to unpack there.

00:14:40: So when you are in the space of trying to prevent attacks and breaches, what are some activities that your doing?

00:14:50: To try catch this activity.

00:14:52: Oh gosh!

00:14:53: That is fun one.

00:14:54: I

00:14:54: got

00:14:55: patents on that.

00:14:56: It

00:15:00: was almost like i knew it.

00:15:02: Fair enough.

00:15:07: I think if we're gonna back up it's always best to do so.

00:15:13: One of my favorite vulnerabilities that everything clicked for me was in PHP, okay yeah i'm punching down.

00:15:23: and what happens when an attacker has found a way to exploit things?

00:15:29: And just call out the shell.

00:15:31: you also saw this way.

00:15:37: Yeah, the list goes down.

00:15:39: Long list of there.

00:15:40: what does an attacker do when they think that they've successfully got into a shell and They immediately are going to go?

00:15:49: which user am I?

00:15:50: Which in a UNX world is who am i?

00:15:53: so if you're like looking for The most...I'm not telling You anything about security industry right everybody knows it.

00:16:02: you would look for who am I calls, right?

00:16:05: Especially at a system call level.

00:16:07: That's what most... I was about to use the acronyms-I shouldn't do that.

00:16:14: but advanced persistent threats those types of actors like at CrowdStrike.

00:16:19: You Would Say Fuzzy Bear and stuff like that.

00:16:22: Fancy Bear!

00:16:23: Fancy bear my god!

00:16:26: Fuzzy is different one but fancy is the one that Fancy and Cozy Bear are The Russian Actors that do this kind of stuff, but That's what you're looking for is.

00:16:37: You're looking For those indicators of attack.

00:16:40: Then if you're Looking to dive deeper into This you have your miter Structure Of at What layer of the attack chain?

00:16:49: If you're too close To their force kill chain Are you finding The threat actor in?

00:16:55: and by Looking for Those behaviors you can then Detect something's gone wrong without knowing how it went wrong.

00:17:05: Sure, right?

00:17:06: And yeah It's kind of obvious like if we were living twenty years ago.

00:17:10: still Right when you see Microsoft Outlook calling out to you know shell commands.

00:17:16: Something has gone wrong, right But its more advanced than that.

00:17:20: these days especially around persistence most attacks when you're finding them are suspicious behaviors because most actors get into your computer and onto your network, their persistent goal is to look like they're normal.

00:17:36: And only do unusual

00:17:37: things.".

00:17:40: I don't know that we want to rehash CrowdStrike's early history cause thats kind of politically fraught with the impeachment.

00:17:49: but That's the bread and butter.

00:17:53: One of things that I did, but I was proud with on research at Crosstrike... I think they call it... I wrote down Counter-Adversary Operations as what is called now.

00:18:06: It has a mouthful?

00:18:07: Yeah!

00:18:08: But its exact product if felt can overwash.

00:18:11: And thats'the group i worked in very early on in

00:18:18: two thousand fifteen

00:18:19: and sixteen months And that group is what specialized and advanced persistence rats.

00:18:26: That's the group where you start debating, What do we put onto the sensor?

00:18:31: Again I've not worked there for years now so things have obviously changed.

00:18:37: but...what do you put on a sensor?

00:18:39: or what do we hold back from adversaries?

00:18:43: Right!

00:18:44: If your ask anybody-What are looking forward to an attack?

00:18:48: thats whats you get at One of thins proud of at that time period, they relates to AI.

00:18:54: is the word vectorization problem.

00:18:57: So I was part of that group and we did some of their earlier AI work...I think i was at least first to deploy some AI models into a cloud environment.

00:19:11: In that process, we did pretty novel stuff on word vectorization in a security context as opposed to an LLM context.

00:19:21: And then look at the effectively system calls that we would see an adversary doing and you'd say like oh look your asking what user am I right?

00:19:35: That's suspicious.

00:19:36: let stop it.

00:19:40: What that translated to is we also did that with machine learning and the most of our stuff was on testing.

00:19:49: So unfortunately, that was mostly just catching red teams not too many adversaries because it's kind of hard to get that emergent behavior especially until you have a really strong training set in your sense now.

00:20:06: so You've touched on a number of things, and I kind want you to help me paint the picture.

00:20:12: Of this scope of The security industry.

00:20:16: little bit so.

00:20:17: at the first level your trying To identify that there is an attack or something malicious happening.

00:20:26: Then you're trying to identify who the actor Is.

00:20:28: you try to identify?

00:20:29: The mechanism That they are using.

00:20:31: Your wanting to stop This right as quickly As possible and maybe even eventually lead to future prevention.

00:20:40: There's a lot of different steps in the process, how much are we trying to automate versus manually manage at this stage?

00:20:49: That is hot question isn't it?

00:20:53: Oh boy Mike!

00:20:56: I think that industry is rapidly approaching Each step as fast as possible.

00:21:04: I don't think that you can skip out anything.

00:21:09: when I started to cross strike, right?

00:21:13: This was a little bit more novel than it especially would be today which is ten years later Right.

00:21:20: so at that point catching an actor five minutes like what happened at rack space or So on That was really good especially if you're talking about somebody burning zero days.

00:21:37: So a lot of my experience and my touch points with the security industry happened at that advanced persistent threat level, so it's more difficult for me to answer your question because I'm like well by and large yeah obviously just want to automate it.

00:21:56: You wanna shut down behaviors as we see scrape the dark web and say, oh we're being targeted.

00:22:06: you identify what kind of actions You expect that threat actor might be taking against you And then you would tune your agent to shut That down.

00:22:16: To Be more protective Of that kill chain.

00:22:22: So yeah I mean obviously want to automate it But there's still i think a role for humans There um...and theres always going to be.

00:22:31: in my opinion What we did at CrowdStrike is a lot of the behavioral, well.

00:22:38: The reason why I think it was successful Is that We operated with people first and used That to them Automate their work to automate what they were finding then would look for the emergent behavior out of machine learning.

00:22:58: So it was, how do we take what you've already seen and find new things?

00:23:02: And put in front people to confirm or deny that's?

00:23:06: I mean that is what the Falcon Overwatch products still does because they are not up-and down is to automate as much possible away.

00:23:22: So, you know we would set up stuff for hey... We see your red team noted but it's not-we also know that you're testing it Not because they told us But Because It's Obvious.

00:23:36: And That Happened

00:23:37: A Lot.

00:23:38: That Was a Big Manpower Sync That We Automated Away and I think thats answering Your Question In What Is The Posture Of The Security Industry?

00:23:49: I hope that's answering your question.

00:23:50: Yeah, absolutely

00:23:51: yeah.

00:23:51: That makes sense.

00:23:52: and so when we look at Automation and implementing AI or seeing a on both sides of this right?

00:24:01: We're seeing AI leveraged to prevent identified prevents attacks And that helps us automate and accelerate in lots of great things.

00:24:13: But you're also going to be seeing and the other side.

00:24:16: And so I've got two kind of questions about the risk assessment.

00:24:21: On one hand, what happens if you have this automation that is incorrectly identifying valid activity

00:24:30: as

00:24:31: an attack?

00:24:32: What are risks there and how do we resolve them?

00:24:36: on other side How to identify attacks dynamically changing more advanced AI adversaries.

00:24:48: Oh, I'm gonna go on a little journey as i've done this whole time right?

00:24:54: Um...I think it's actually best to answer with the most recent like huge oh my god The world is going to burn!

00:25:03: Right?!

00:25:04: And unfortunately for me, Veritasium on YouTube did a great video just recently.

00:25:12: But are you familiar with XZ which is a library on compression?

00:25:18: Yeah yeah this is a fascinating story.

00:25:21: so effectively what occurred Is that you have Jin Tan.

00:25:27: It doesn't matter it's made up name right.

00:25:30: That much as obvious and advanced threat actors harassing a poor guy, Colin.

00:25:40: I think he's actually in Germany if i remember correctly.

00:25:44: um He was the maintainer of that library this open source library and what they did.

00:25:52: And my tie into why this story is so dear to me Is I Remember In I was working with trying to integrate XB into a Java thing.

00:26:05: We were just experimenting and i came across the message, this is right at the end of my tenure that CrowdStrike wrote it down as in This Is A Project That Deserves Help Like I'm just summarizing here somebody posted sorry about your mental health issues But important to be aware of your own limits.

00:26:31: Why not pass maintenance of XZ for C so that you can give XC for Java more attention?

00:26:41: And like, That's the message I remember So clearly and it just like tears me up.

00:26:46: then i never did anything.

00:26:48: Like y'know when I went hike back elation trail I spent time with my kids don't regret any of them but still poles at your heart strings, hearing that's what was going on.

00:26:58: And the end results of this to spoil the story is that he's faith... This team successfully got trust from.

00:27:09: The Maintainer wrote an incredibly beautiful sophisticated attack That would have compromised every Linux system On internet.

00:27:20: It almost made it into Red Hat enterprise edition.

00:27:23: It did make it into their testing, I can't remember the name at the moment got me on speaker A and also made in to Debian.

00:27:35: not Debian stable obviously but you know Debian testing so one.

00:27:40: So its scary that attack would have allowed SSHing with a specific private key.

00:27:50: any system of world was legitimately deployed like that.

00:27:56: Wow, and The question I think you're raising about where are we with AI is best answered by Where these weak points?

00:28:05: And i think when talking about the agents Is a big fan of saying future is already here distributed.

00:28:16: You've got team people probably working for A government.

00:28:24: It's reported that it was unlikely to be

00:28:26: China.

00:28:28: Okay,

00:28:29: and That led To them getting caught But it was a team with government funding.

00:28:39: Now think about what they had to do And let's put that in the context of where AI agents are taking us from an adversary point-of-view One.

00:28:49: you have The social engineering.

00:28:53: I imagine that's not too hard to imagine.

00:28:55: training an agent to do locally, you know getting on HUD and FACE.

00:29:00: You can even use compression techniques now.

00:29:03: does it need as much memory?

00:29:04: It is cool stuff!

00:29:08: Then you've got your code bit right so gaining trust in open source contributing to repos stuff like.

00:29:20: I think your agents are getting closer to doing that, especially if you're knowledgeable enough.

00:29:25: So now you've got one person gaining that trust, doing a social engineering and like we just talked about from last week You know seeing these exploits of in the case this compression library.

00:29:40: it was looped into system D pushed into SSH on the conditional login.

00:29:52: We see that now and we know why they were rushed, part of maybe why they are rushed is because it's a performance.

00:29:59: penalties covering their tracks likely had been not so clever might have gotten caught which was crazy to think about.

00:30:08: but thats where ended up an AI agent just at the cusp being able do just at the cusp.

00:30:21: And I think that's what's scary.

00:30:23: is.

00:30:23: you now have this story, right?

00:30:30: That requires a team of people years of effort like three years total minimum for social engineering and building social trust or code execution we don't know necessarily not hijacked.

00:30:50: It might have been a hijacked account, but it's likely that is was not because of the social engineering.

00:30:58: you've got now in position where we're talking about doing with one person.

00:31:01: I think thats what scary and i think thats change were seeing its becoming more accessible.

00:31:07: to do that.

00:31:09: tools on defense will catch up.

00:31:13: you know, there'll be some rough patches.

00:31:16: But there's always been.

00:31:18: I mean... You saw similar things like with the solar wind hacks if you remember that for sure.

00:31:23: um and yeah The world didn't burn right?

00:31:28: i think it's easy to think all of these tools And all this change in upheaval is gonna burn.

00:31:34: but To make a joke Um three d printing It's.

00:31:40: it's a lot like saying AI is going to just replace everything or everybody's gonna vibe code their own app now.

00:31:46: Yeah, there's A LOT like saying yeah nobody can go to Walmart anymore we have three print everything!

00:31:52: It's a really cool thought but it doesn't make sense any more does it?

00:31:55: And

00:31:55: anybody that runs the threedy printer knows how ridiculous That sounds because they're lucky if their printers are all running All The time.

00:32:02: I mean There's always maintenance.

00:32:04: There's Always things you Have To do and bring to the table to just make this stuff happen.

00:32:11: Yeah, it's

00:32:11: no magic

00:32:13: but at one point do you remember that now?

00:32:15: Oh

00:32:15: yeah!

00:32:16: It is easy get wrapped up in it.

00:32:20: You know I'm not as equipped to talk about it there.

00:32:26: the AI researcher oh man god he brilliant um... He was recently moving and leaving where currently working.

00:32:38: What's fascinating is he's basically saying that CNN might the large language models were in.

00:32:46: from a perspective of super like The self-improving models It may be bit about dead end.

00:32:54: So I think there's it's really interesting time to know the fundamentals and That if anybody out, they're as college student I'm sorry Because of like, i think the future is doing the hard bits.

00:33:11: Yeah And i think that's awesome.

00:33:13: So but i'm also sorry

00:33:17: That's fair.

00:33:18: so What would you say?

00:33:22: Were some of the pivotal moments that got us to the current state we're in From the security industry perspective whether like big were they big breaches or the big technological advances?

00:33:34: Like what we're some of them milestones.

00:33:36: They got us here

00:33:39: I'm not sure I'm great at answering this question, I mean think I have too wide of a perspective.

00:33:48: Like, i'm thinking back to you know the two thousands when somebody might hack your server and leave a text file about how to fix it?

00:33:56: You know like- And...I saw that happen!

00:34:00: So It's interesting.

00:34:02: if I'm like thinking about that grand sweeping arc.. I Think its more industrialization Of both The value That can be had And the equalization of those tools and that knowledge.

00:34:24: I'm not sure if this is when it changed, but for me social engineering versus advanced technical attacks Certainly have taken a different direction.

00:34:47: I think we're We're little bit better protected.

00:34:51: it would be hard to say, right?

00:34:53: Then you know the early two thousands.

00:34:55: yeah like.

00:34:56: that's pretty clear on The technical perspective.

00:35:00: But i think of this social side were almost more vulnerable and i don't Know there is all the political polarization.

00:35:08: There's social media And I think that's where, i would say is a pivotal moment as you see the rise of Facebook.

00:35:19: You see the eyes of Twitter and maybe it's downfall.

00:35:26: those are kind of identifying more pivotals in security industry because they identify to me like this risk.

00:35:38: This is how we exploit people and you're still seeing some of that isolation like were just talking about on email lists, I think the switch from it's a purely technical sport to well now its valuable.

00:35:55: Now there are stuff where they can make money.

00:35:58: You saw if i had to pick what was Oh man, you're wracking my brain like.

00:36:05: I think that would be around two thousand fifteen to two thousand sixteen might have been later.

00:36:10: um That you saw the ransomware?

00:36:11: Yeah i Would put.

00:36:13: that is kind of an interesting pivot point because i think it became clear To everybody at that time Is a computer security software security was A problem in every industry and then It Was profitable.

00:36:29: yes Yes I

00:36:30: think if i was gonna pick one that

00:36:32: makes sense because

00:36:33: there

00:36:33: were real dollars changing hands during that time period and you're right every industry.

00:36:41: That use technology with us, you know susceptible to this.

00:36:45: yeah you also got a similar impact with the pig slaughtering kind of scams where your pretending to have much longer relationship for that unfortunate

00:37:00: Yeah, person.

00:37:02: I think those are the kind of social contracts that got

00:37:08: broken

00:37:09: but not broken.

00:37:10: But um That's the wrong word for it.

00:37:13: The invisible hand does invisible hands things Um...but i think thats what..I would put.

00:37:21: Is that monetization?

00:37:24: That realizing people were still the weakest?

00:37:28: There's an ex-KCD, I don't even know a decade ago at least talking about information security.

00:37:35: We can have a team of Reachers searchers and spend a million dollars trying to break into the system where we could buy five dollar wrench and hit that guy until it gives us the credentials like...

00:37:44: Yeah what?

00:37:44: And

00:37:44: its still kind there i feel like What

00:37:47: yeah!

00:37:48: It is getting exciting because were hitting that inflection point Where both that hammer potentially where you can imagine that there's some evidence of Geotown might have been like an actual chance or something, maybe in the XZ.

00:38:08: Oh

00:38:08: no I went all Ontario on New York City hacks but it also... So we're seeing both that hammer as well.

00:38:17: is that incredible amount of spend to accomplish these goals?

00:38:23: so yeah absolutely fascinating.

00:38:28: So if somebody is Oh, yeah.

00:38:34: No I want to talk about one more thing.

00:38:35: Okay so on AI models there's There's an interesting bit here On what did they call it?

00:38:44: My memory is subliminal Learning.

00:38:48: this would be put in okay, so if you're not familiar.

00:38:51: There's knowledge district distillation.

00:38:56: You can imagine ChatGPT, almost every company does this.

00:39:01: But you've got one model and changed your arrangements right?

00:39:06: And you're looking to train another agent Right?

00:39:10: You effectively use Agent A to train Agent

00:39:14: B.

00:39:14: Okay There's an interesting effect that was discovered last year.

00:39:21: It pertains our conversation because it is interesting.

00:39:26: What can occur is let's say you're just sending random numbers.

00:39:32: Okay, you're literally taking your one agent and saying Complete here are three numbers complete the sequence right?

00:39:40: And now sending that in training.

00:39:42: Your new agent on it.

00:39:45: okay what occurs?

00:39:47: but what are the impacts of them?

00:39:50: and What surprising as if take teaching agent tell it, you know.

00:39:57: I think the example might have been an animal and eagle.

00:40:01: Eagles are the best okay?

00:40:04: Eventually your child agent when you're training off of that will start saying eagles or the best.

00:40:11: now You're only training with random numbers right And there's no message they're.

00:40:18: That's what shocking.

00:40:19: It's not that you're sending anything through.

00:40:21: another.

00:40:22: some fascinating math proofs about this is why calculus, remember your series expansions and calculus?

00:40:30: This is why it all matters.

00:40:33: You think back in the day.

00:40:34: I'm never going to use this then sure enough.

00:40:38: but what's fascinating about effect is that its because you're...all of your embedded nets.

00:40:48: they are all linked right so effectively.

00:40:50: even though training a small portion

00:40:54: a

00:40:55: larger output set, you're still getting the behaviors of your training

00:41:03: agent.

00:41:05: What's interesting about this is what happens when an adversary controls Your Training Set?

00:41:13: What does that mean?

00:41:14: When You Can Say To An Agent That Hey I want to behave in This Way Then You Can Produce Training Data.

00:41:26: You know, I mean we might get into touchy ground here about how much the content of the internet is from what?

00:41:31: Right.

00:41:34: But you now are polluting right and because Of The Nature if the two agents If you Know That Original Seed And You've Used The Original Randomization Seeds.

00:41:45: that's a big ask I like for sure but it's an interesting thought.

00:41:51: yeah Is Like How How permissive and how sophisticated are we worried about these attacks?

00:42:02: And what does it occur.

00:42:06: It's kind of a variant, in much more I think realistic variants than the idea.

00:42:11: you know putting a code comment like whatever agent if see this or this function.

00:42:17: You know in UTF hidden characters.

00:42:20: But the idea, I think is there and it's an interesting direction that we're headed in where were not worried about.

00:42:29: Now i'll go ahead and say...I love this concept!

00:42:34: And just really wanted to talk about it but its probably much closer a side channel information link which has been published alot of times.

00:42:46: But I love the idea of it, isn't that fascinating?

00:42:49: Yeah.

00:42:49: That's

00:42:50: wild!

00:42:51: You build an agent you know your adversary is releasing their agent.

00:42:57: so take it and further refine its training to change its behavior then post knowing where they're scraping this stuff purely by accident at random seeds being same.

00:43:11: When the training data picks up your additional malicious content, you may have some influence on The resulting agent.

00:43:21: Wow.

00:43:21: Yeah I just find that Just so fascinating.

00:43:26: And the math of it makes perfect sense as you see It.

00:43:29: yeah right like you can imagine again You've got a very large opposite small bits of random nonsense.

00:43:37: But because of all the connections, like a way off screen right?

00:43:41: Yeah you're going to be able to influence those connections which ultimately influenced the agent.

00:43:49: Mathematically it makes sense from.

00:43:52: I could see that being way outside of the expectations of typical engineer when they are doing their day-to-day life Right!

00:44:00: Oh yeah i don't think so.

00:44:02: I guess this is not practical but It's

00:44:04: not practical now.

00:44:06: Yeah, that's I guess.

00:44:08: That's accurate like solar.

00:44:10: I mean hell it would cross right across.

00:44:12: yeah It didn't seem practical to me at

00:44:16: one point.

00:44:17: So if somebody is in the industry but not really into security at this point What would you see say as something that?

00:44:25: You need to know today To protect yourself and to write better software.

00:44:33: I'm a child of history.

00:44:36: I like pretending, at least to be an old soul.

00:44:40: So i think...i'm gonna misattribute the quote so im not going to attribute it.

00:44:45: but your code is either obviously bug free or has no obvious bugs.

00:44:50: and especially if you're doing something that's central become familiar with AI tools, let it help you.

00:45:01: But if you're not being able to direct it exactly what you want with why you should use this framework or not this framework and your dead in the water?

00:45:10: And then don't be afraid that You have to understand those fundamentals on making sure that code is blatantly obvious.

00:45:19: because I mean like If you are Not In The Security Industry Right!

00:45:24: Your guarding against these bugs still Hell if you're using regular expressions, right?

00:45:32: And I think it's fair.

00:45:34: I think regular expressions are well known but most programming languages they're not safe on a performance bound.

00:45:41: so If your'e careful about back references and stuff like that It can be exploited.

00:45:47: So You have to still have strong understanding.

00:45:53: Don't get too easily subdued into not understanding the code because either you Obviously have no bugs or we have no obvious bugs.

00:46:04: I think what?

00:46:05: We're seeing in the industry is going to prove that To be pretty true.

00:46:10: Because well, we might be able to use a agents to Find stuff to add new features too.

00:46:17: You know vibe of code right and like heck I do it.

00:46:21: It's fun feel like a superpower Right but if you You are so much more effective if you're truly telling it what to do.

00:46:30: And I think that's where, If somebody was like hey What should i take away from this?

00:46:37: It is still matters.

00:46:40: Somebody in all of these models they just numbers Whether want even say we will decimate the software and engineering industry or its already occurring.

00:46:53: Gosh, I hope

00:46:54: not.

00:46:54: That

00:46:58: you know.

00:46:58: there's still somebody reflective in fact like and it is human.

00:47:02: And i think that's why we saw when were talking about bug bounties and the shame they're with AI slot Is even when these models are producing really good-looking things?

00:47:13: Even When You give them a A test harness to say hey is this breaking Right, it's still a lot of work to set that up To have it worked correctly and still needs human verification.

00:47:29: right because not every You know naive string copy is only real problem.

00:47:35: Yeah Stir copy yeah can be used safely but It's really easy to train an agent.

00:47:41: we like hey look there's no end their

00:47:45: Well in.

00:47:45: one of the things you touched on a number of times is kind of the software supply chain.

00:47:50: And one of them, problems that I've seen even before AI supported development is developers especially newer developers coming in and they find something on Stack Overflow or In this case They ask Claude Or some agent to help And it's like, hey there is this library over here you can just implement and its going to do the thing.

00:48:13: You are cool!

00:48:14: It solves my problem.

00:48:15: The problem now I end up with a hundred libraries in my software when ten would have done fine and every single one of those is a potential security risk, right?

00:48:27: Yeah.

00:48:27: It's you're expanding your supply chain in

00:48:29: big ways not to mention license risks

00:48:31: sure yeah we don't talk about that nearly enough but

00:48:34: well I mean it matters right.

00:48:36: um especially if we are practitioners i personally do like...I need money.

00:48:44: so I think yes specially java stripped..i find anything i do on Javascript tends just kind go pew Yeah, and we even saw exploits in that.

00:48:57: Especially fat fingers intentional on the spellings where a library is hijacked most likely includes all of their original source Even updates for scary vulnerabilities right?

00:49:11: Wow That includes some kind of supply chain attack like in JavaScript.

00:49:17: I'm just picking one from my memory Where you would steal environmental variables.

00:49:23: So you can imagine during your deployment phase of the library, grabbing those environmental variables sending them off to a seat.

00:49:31: I'm sorry i almost said c two command and control kind of server for an adversary yes And capturing him in trying to stay quiet about it?

00:49:41: I think that's a big risk and You've got a lot of programming languages where.

00:49:47: how How do you bring in those dependencies?

00:49:52: And you know, having been on both sides of the security team table.

00:49:58: You know as a just I want to get this working.

00:50:01: we have customers who wanna give us money As well as being on the wolf's sled and their buddy It.

00:50:09: it's frustrating right because going through an auditing every library that you're using yeah like Log I'll pick on Java now log for J

00:50:21: Right

00:50:22: where it was a feature that almost nobody used.

00:50:27: Where you would then have the XML put in to get a bypass on reaching out, than getting downloading the Chavico.

00:50:40: Man!

00:50:41: It's so cool talking about this thing now.

00:50:45: I'm sorry but no its real risk.

00:50:49: And how do you audit this?

00:50:51: How do you make sure that your staying up to date, like I know anybody who's working in Java is probably on an outdated JVM still.

00:51:01: It's the scourge of that platform.

00:51:06: so making sure you're staying up-to-date on both your security patches as well and stay up-the-date with technology based.

00:51:14: so it can be risky when talking about just your logging platform being compromised and having me a, you know, critical nine or ten where you're talking about on the next kernel.

00:51:28: You had two cows not that long ago.

00:51:31: okay maybe I was nine years ago but it's-it's a lot.

00:51:36: yeah It really is.

00:51:38: Yeah.

00:51:38: And i think that uh...It's a reason why when I run stuff on the internet..I don't generally do I guess, well how long?

00:51:49: You know fifteen years.

00:51:52: Fifteen-sixteen years ago i ran my own mail server right and it wasn't weird!

00:51:59: I mean you still use slash dot.

00:52:01: yeah so dating myself a lot but that wasn't weirder.

00:52:06: um Yeah...I guess that makes it..that advice on How do you understand this stuff?

00:52:16: It's a lot harder.

00:52:17: Um I mean, oh I'm just thinking about all i know.

00:52:24: but email right and You know how do you actually send email?

00:52:29: How does spam detection work in all that good stuff?

00:52:32: How would I advise somebody to do that?

00:52:35: um, how would I advice somebody on?

00:52:39: For a while cross-track ran dns platform.

00:52:43: some of that means felt me I was inherited.

00:52:48: It's not something i like to talk about but you know how do you get good at running bind servers?

00:52:53: um and yeah oh it's for real,

00:52:58: feels somewhat niche

00:52:59: exactly.

00:53:01: so yeah its a really your asking.

00:53:05: the hard question what would you do for advice or somebody right because he can't just say go

00:53:15: And I feel like that's one of the biggest differences in the software industry today versus when you and i got started, which is we were free to do whatever.

00:53:27: We ran mail servers...I wrote my own database from scratch!

00:53:38: That's not really a thing right now.

00:53:40: It's not that you can't do some of those things, but anything that you do the intersects with the internet is innately risky.

00:53:47: Well it'd be virtual machines have come

00:53:48: along way.

00:53:49: this true

00:53:49: yeah.

00:53:50: so I was like if i'm thinking about it in that weight like II remember back then Like I want to play I'm gonna play devil's advocate and remember back when The idea writing Linux kernel module Was insane?

00:54:06: I mean the first time i installed linux, i don't know about you.

00:54:09: I worried about blowing up my crt.

00:54:10: now it feel really old but The idea is sound.

00:54:16: um You know with that had been a virtual machines?

00:54:18: I've written quite a few just Please for concepts on Linux kernel modules and file system drivers The idea That you could do any of that.

00:54:30: twenty years ago Or maybe I'm gonna say this let me rephrase Obviously people were doing it.

00:54:36: Yeah, but i mean that It was as accessible and safe.

00:54:40: yeah Is is crazy.

00:54:42: so we've definitely gone back on the networking.

00:54:46: That's what you are getting at sure.

00:54:48: But I think We've come forward a little bit On that.

00:54:51: operating should no security?

00:54:54: maybe okay, I am weird like not everybody come.

00:54:58: I'm just thinking hey i can compile GCC and contribute to KDE make it do new effects for my kids.

00:55:06: so they think im cool that kind of behavior is a lot more accessible.

00:55:15: but yeah the idea running a bind server network facing utilities A lot of REST APIs and stuff like that.

00:55:25: And yeah, it's a lot more scary when I've seen some remediations That are just like mind-blowing.

00:55:32: so Yeah, I mean oh man There's like some stuff for like SQL injection in to get so embedded In the platform but you end up putting up antler and then parsing The sql and Then making sure that the where clause is limited by your input only what's allowed by the API documentation because, you know history.

00:55:58: If that's a remediation I mean heck but that's pretty scary isn't it?

00:56:03: when your kind of putting in all together?

00:56:06: and yeah these agents aren't always secure?

00:56:10: So so i think first of all Andrew want to say thank you again for being here.

00:56:15: this is a fascinating conversation And I hope that everyone really takes away from this That there's as much as there's a lot of changes in the industry.

00:56:26: The security industry has been using AI for a long time.

00:56:30: Yeah, we've been trying to yeah

00:56:33: and and the state-of-the-industry is Like with anything both good and bad.

00:56:39: There's good and Bad things about what's going on but it still something that i think all developers should know have some basic awareness of security and their role in it, but the prognosis or industry is still quite optimistic sounds.

00:56:55: Well I would hope...I mean i'm an optimist now..i kind-of view things as not coming.

00:57:02: nothing's going to blow

00:57:03: up

00:57:05: cool.

00:57:06: so any final words?

00:57:09: And Is there a way that people can reach out?

00:57:11: if they want?

00:57:12: keep them in touch with you.

00:57:14: Sure, I mean i probably will have a website rodent.me.

00:57:18: maybe My final word though is A little bit more personal.

00:57:24: if you don't mind of course um If You or anyone?

00:57:26: You know Is really struggling in the united states.

00:57:30: nine eight eight suicide prevention and crisis line And it's not just for people In crisis It's also For those trying to help them.

00:57:39: Um internationally Find To Help Line.

00:57:42: calm Will Help You.

00:57:44: just you're not alone and it's okay.

00:57:48: And your loved.

00:57:50: Thank you very much, Andrew.

00:57:51: appreciate the conversation.

00:57:53: Appreciate the sentiment at the end.

00:57:55: um...and thank you all for joining us for another episode of Signals from DevMio.

00:58:02: do check out DevMIO forward to seeing in the next episode.

00:58:06: so..

00:58:07: Thank You again Mike!

00:58:08: Thanks take care.