Securing AI at Scale: Why Your Logs Can't Read Agent Intent | Ashish Rajan

Joe: How's it going, Ashish? It's it's gr great to get you back on the podcast, man. I think it's been it might be like three or four years since you were last on, I think, which Ashish Rajan: Wow. Joe: is insane for me to even say that the podcast has been going that long. Ashish Rajan: Yeah. I mean, first of all, congratulations on having one of the hardest jobs in the planet to continue to be consistent. being a podcast being a podcaster, I guess in that context, is super hard. And I don't think people realize this. So congratulations first of all to being running this for three plus years. yeah, I can't believe it's been that long, man. I mean, but that was last in Black Hat from memory. We did the recording. Joe: RSA. I think it was RSA. Ashish Rajan: RSA, okay. I just remember going into a hotel room and there were just a lot of I mean, you're such a big shot, there were so many cameras, all I remember is cameras and Joe in the middle of all that. Joe: Yeah. Yeah. It it was that was definitely a interesting time. And you know, I wanna do it again. I wanna do it at like DEF CON or something. I I think I'd be able to get a lot of the people that I've been trying to reach out to that won't respond to me that way. But Ashish Rajan: If you were to go DEF CON, yeah. Joe: it's there's so much work, man. There there's so much to it, you know, and like you're directing like so many different pieces all at the same time and It probably didn't look like it 'cause my Ashish Rajan: You I mean, you do this with a full time job as well. I left my full time job to start something on my own. being a CISO and doing this is super hard, man. Joe: Yeah. Yeah, no, I I feel like I feel like there's a lot more that goes into it than what people even even like realize or anything, you know? And Ashish Rajan: Yeah, yeah. Joe: like just to like I I this is the thing. I feel like I'm not even that successful at it. I still think that I put probably like the bare minimum effort into the podcast overall and somehow it gets better. over time, you know? Ashish Rajan: Mm. Joe: because like I just don't have the time. I don't have the bandwidth. You know, like I'm I'm doing my dissertation, I have two little kids, got a full time job. I mean like that's like several jobs, you know, right there. Ashish Rajan: Yeah, it's like you already have three jobs, one for each kid, and then you actually full time job. Being there for them is like a man, I think I don't peep people underestimate and maybe don't realize how much effort that goes into things like a podcast being created in the background, more than such a conversation to reach out to people. I mean, and doing anything extracurricular after outside of your full time and kids, man, it's like another level. And you're doing you study studying as well? Joe: Yeah, finishing up my dissertation as well. So this is my last semester, Ashish Rajan: Chi Joe: hopefully. And I like just got the email saying like, Hey, you need to like register to graduate. And I'm like, I don't feel like I'm ready to graduate. Ashish Rajan: my God. Is there I is your dissertation on a particular subject? What are you studying? Joe: Yeah, it's space cybersecurity and the specific like dissertation is in deploying zero trust framework onto communication satellites to prepare them for post quantum encryption. So Ashish Rajan: Wow. Joe: essentially, you know, it's taking the confines of post quantum encryption because it's very sensitive to like latency and you know different different challenges that you have that are just completely unique to space, you know, like radiation for instance. And it's saying does this zero trust framework operate still within zero trust confines, but does it allow post quantum encryption to take place? Because, you know, that's like the big problem where or that's a big balance with security and satellites is like, yeah, we can secure this thing extremely well. Like there's no issues with that. But Ashish Rajan: Yeah. Joe: it impedes everything else that it needs to do. And a lot of the times you only have that satellite for a few hours to be able to do the operations because guess what? The other half of the time it's on the dark side of the earth that isn't facing the sun. So it has no power generation. And so if you want that satellite to do something, it's like, okay, it has to be extremely precise. It has to be extremely well timed for it to do that, you know? Ashish Rajan: Wow. Yeah, dude. I mean, and here I was thinking that writing a book was hard. You're doing a research paper on this as well. On something, I mean, I did minor air security engineering thinking, okay, this is at this book is at least six, five, six years ahead of its time. But research in space and cybersecurity in space, man, that's like you're you're you're thinking 10 steps ahead, man. Like that is an interesting problem to solve, especially because Space exploration is such a big thing now, especially after the whole what's a Ilamas company called that basically sends rockets? Joe: SpaceX. Ashish Rajan: SpaceX, yeah. So SpaceX has made it even more accessible for more people. That's a really but I mean, I I know who I should come to now for space security problems. I'm definitely gonna look up to you, man. I'm gonna find you Joe: Yeah. Ashish Rajan: up and go, Hey Joe, I have a space question for you for cybersecurity. Joe: Yeah, I well I appreciate that, but man, you know, honestly, like this entire process, this has been a three year process. And you know, that that's not very normal for like a dissertation program or a PhD program. Like that's like on the shorter side for sure. And the program is really geared towards like all right, you're gonna get through all the compliance and the the like writing, all that stuff. You get through all of that as quickly as humanly possible. Ashish Rajan: Mm-hmm. Right. Joe: And then you go straight into your research. And for two years I didn't think that this method was even possible. Right? Like I literally thought that I was studying something that at the end of it was going to be not working and you know, everything else. And like I almost quit several times because of it. I don't I I don't mean to turn the episode like into this topic or anything, but I I didn't even know it was gonna work until last December. where when I like as a part of like my participant interviews for it interviewing different experts in in various fields where and I was like even halfway through my participant interviews and the first half told me it's not gonna work. And then the second half comes through and the first guy is like, you know, if I was gonna do it, I would do it this way. He's like, I'm pretty sure that's the only thing that works. And I tested it Interviewed the next couple people, they agreed that that's the only what method to do. And then I ran my simulation models on my RTX fifty eighty and Ashish Rajan: Yeah. Joe: it passed. So I was like, All right, I guess this works. Ashish Rajan: I mean, I'm happy for you, man. And again, showing a consistency there as well, man. Getting the staying resilient and still pushing forward, even even though you're a fifty fifty on the possibility of this. Joe: Yeah, it was more like eighty twenty, this ain't gonna work, you know. Ashish Rajan: No, but I'm glad for I'm happy for you, man. I'm happy for you. Joe: Yeah, absolutely. Well, you know, sheesh I I saw that you came out with a new book. Admittedly, I haven't picked it up yet. I I've just been super busy and I I haven't bought it yet, but I will. I promise you I will. it's actually in my car right now. Ashish Rajan: I will send you a copy, a signed copy. You don't have to buy it. But you can gift one gift to someone if you want. Just if you if you're already Joe: perfect. Ashish Rajan: buying one, just gift it to someone who's who probably needs it. I'll send you a copy, man. Joe: Amazing. All right. I already got someone in mind that I'll send it to. That's that's awesome. Ashish Rajan: Awesome. Yeah. That that that means a lot, man. I appreciate you gifting it to someone else as well. Joe: Yeah, absolutely, man. Well, you know, Ashish, we just like dove right into stuff that w this episode wasn't supposed to be about, but why don't you tell my audience like a brief intro of who you are, maybe briefly like how you started to get into this this area, right? Like you left your day job, went into this consulting thing full time and whatnot and W what you're what you're doing right now, you just released that book. It sounds extremely interesting, very timely. Ashish Rajan: Yeah. so I guess a short version is that I've been in security for a little over 17 years. I started my career in IAM, identity access management, did some security architecture work, cloud security, kind of picked up there, started a podcast called Cloud Security Podcast, build a SOC team. well, I guess not in that particular order. So yeah, build a sock team. Then became a CISO and then started the Cloud Security Podcast because by then it was COVID. had a couple of few C CISO roles before I realized that there is this time that most people, at least this is how I understood now looking back at it, is that you kind of have to make a call between something that is giving your soul something back to helping other people versus you're working on something which is professional, which is equally important. It's just that. Hey, maybe I can have a bigger impact and something long longer term if I were to just do this myself. So I ended up starting a media company, which then turned turned into a strategic advisory company. And today, the company TechRider.io has an AI security lab, which is the lab that is basically helping inform the people who f who are subscribed to our advisory on how do the latest AI models, the what does that mean for security? Well, how do you deploy and adopt AI safely in your organization, especially if you're an enterprise regulator? We primarily end up working with the people in the financial and insurance space, because they have the more most regulation. They kind of have to be, and especially because there's not many standards with the AI space. So at the moment, a lot of the work we do primarily is working with other CISO directors to help help basically adopt AI safely in their large organization at scale. And the research that we do is to be on the cutting edge of AI because man, it's changing every single day. And every single week, like this week, we are researching the the bots that Meta released. Last week we were on the Grogbot. The week before that, we were Mythos 5.1 came out, Fable 1. It's like it does not end. So even though I started the media company on Cloud Skitty Podcast seven years ago. three years ago we started AI security podcast. Then that became part of the advisory or strategic advisory work we do. And then now we are a research lab as well with AI Security Lab, where a lot of our time is focused on being on the edge. So our customers don't have to experiment. We experiment on their behalf on what is the right model and how do you deploy that safely across the organization? And the other part we do is just basically inform the community, build open source, talk about. we had the CISO visa, CISO standard chartered, come and talk about how they do AI security. So conversation like that is what we normally drive forward on the podcast that we have. But that's what keeps me busy and probably on the treadmill for now. Joe: Yeah it's we're we're like at an inflection point right now, I I feel like. You know, like literally I was discussing this yesterday at my current company of Ashish Rajan: Yeah. Joe: saying like hey we just need to have like our local compute cluster that's you know doing these these like normal tasks right that we need to offload to AI and then offload like the big token heavy stuff to like grok or claude or whoever it might be. I'm Ashish Rajan: Yeah. Joe: a I'm a pretty heavy grok user myself, very big in a grok Ashish Rajan: nice. How are you finding it? Joe: bot. Ashish Rajan: Like I found that it's cheaper in terms of token spend compared to Claude and other anthropics. Sorry, Claude and GPT. Joe: Yeah, yeah, that that's what I found too. And I I find that see it's it's like kind of twofold. I also use Claude a bit. Claude is really good when I like when I need a a coding project or a simulation, you know, coded out for me, right? Because I'm not a developer, but it gets me ninety, ninety five percent of the way there, probably more even. Grok is really good in the other like 90% of the area where I'm like, you know, researching different topics, pulling together different information, modifying, you know, different like data points or bullet points and whatnot. It's interesting how they how they're starting to overlap though. Groc is really starting to like focus on the claw cap on not on like the coding capabilities. And so I'm really interested to see like where it goes in the next couple months because even you know like I I had on one of the co co creators of like the blockchain paper Ashish Rajan: wow. Joe: and he was talking about how grok is actually positioned best within the entire industry to accelerate you know its own development quicker than anyone else because it has all the compute resources that it already needs. It's just not even using all of it. Because essentially you know, E Elon didn't put the like a team in place, didn't build the team at that scale yet. He build, you know, kind of like that viable product and he's like, all right, we're gonna scale it from here, right? Like a couple months ago he even said, All right, we're gonna tear this thing down to the foundations. It wasn't working before like we wanted, and now we're gonna build what we need to. Ashish Rajan: Yeah, actually you're right. Cause I don't think, yeah, compute is a problem for Elon's company. And that's why they can run a lot of that cheaply. Whereas the cloud and GPD world, they are basically running on subsidized token at the moment. And moment they go public, it's gonna get expensive. But the reason I was asking also about Grok was to what you were saying about the pattern earlier. Majority of the Fortune five hundred global two thousand companies that I've worked with, they're all having the similar feeling and the Palantir C CEO recently spoke about this publicly as well. is the fact that many people, to what you said, are going to look at the whole open weight model. How do I run this locally on my own on-premise, my own data center? and that way I manage the cost overall. So that tomorrow doesn't really matter if Ashish prefers Grok over, I don't know, Quen or GLM 5.3 or Chat GPT because suddenly they're even more cheaper than open weight in the future because government took over. Whatever the scenario may be, I don't have to worry about that because I can run my organization on the things that I am building myself. And that's where my focus is. I yeah, man, you you should you should tell your company they're not wrong in taking your advice and going down the open weight path, or at least even with the grog path today, which is still cheaper comparatively, because whatever you build as a harness or things that you're doing today. Would definitely be valuable in say five years, six years' time as well, man. Joe: Right. And the the whole idea behind it is that you can just like kind of plug and play the pieces as they evolve over time. You know, like there's nothing stopping you from being able to host like a a local, really, you know, powerful model and then Ashish Rajan: Yeah. Joe: have the harness of like you know, Hermes or Grokbot or whatever it might be. I'm very against OpenClaw and Open AI, but like you can have those harnesses and You know, and you can switch out whenever you need and build pieces around it. And I mean Hermes Hermes is looking like they're catching up to like Grokbon in terms of you know ease of use and UI and whatnot. So it's interesting to see where it goes. I w you know, I wanted to ask you in writing your book, how difficult was it to write the book on a topic that is like still evolving today? Ashish Rajan: Mm. Joe: You know, like I assume that book probably took you what, twelve to eighteen months to write. So Ashish Rajan: Yeah, yeah. Joe: I I mean, how do you write a book on a topic that's currently evolving and stay relevant with the topic that you're writing on? Ashish Rajan: my god, dude, that that is such a great question. So I got so the book is published by the Wiley folks, Wiley publishers, and I got the contract around Jan 2025 or something. So by the way, for people who have not been keeping up on the AI news, there are no AI agents in Jan 2025. There are like pockets of it, conversations. Hey, this could be the whole thing. And by my initial plan was that hey, maybe you know what, RSA twenty twenty six Sorry, Black Hat 2025 is when I'm going to talk about this and release it. And by at the closer I got to it, the closer AI agent became a thing. And I've I had already written about three or four. So there are total nine chapters in the book, but I had already written the first three, and they were all based on the idea of how people were using LLMs at the moment. which is whether you are bolting on, you're building AI native, whatever the or you're you're using a SaaS company with an AI capability, whatever the form of AI consumption you were doing, that's what it covered for. But when I had the AI moment for AI agents becoming a thing, everyone's talking about it, Twitter was just exploding with AI agents. I'm like, and talking to a few people who were in that, so I I was fortunate enough to get quotes from people who work in some amazing companies and talking to some of them, I I realize what they said is like, Hey man, if you don't have agents in your book, I'm not signing a book. I'm not even gonna buy it. And I'm going, Come on, like the like so to bring it back to what you were saying, it even a window of six months would completely change the entire perspective of how people look at AI or even AI security for that matter. And I started rewriting the book again. in June 2025, took me all the way up to March 2026, so early this year, to finish writing the book. And I I only finished because someone called out, one of the publishers and one of the previous authors called out, where a topic like today is like this, unlike cloud security, which was only evolving every six months or one year, where so your your book at least had a one-year runway, right? Worst case scenario, AWS may come out with a service, but it still at least had a one-year runway. And I realized very quickly that talking to people who are authors who have been author in the past who said that, hey, cloud, the book can last at least six months to one year, or maybe two years, depending on how slow the changes from AWS are. But from an AI world, if you're changing every month, you have to draw a line somewhere where you feel that, okay, after this point, any other change would need to be a new model that is so different that it basically breaks kind of like to what you said about the book. Sorry, the the association that you're working on. Just like for something to change in space, it's not going to happen every week. It would need to be at least a couple of years for your thesis to be done, run, proven. And that is if someone finds that capability, right? That's the same thing with AI as well. I felt it reached a point where I was starting to notice that people who were working with us who are very AI forward, they were starting to kind of converge on this one particular point. And when you realize that you've, okay, now that you're converging on one particular point, I almost started sensing that this is a great place to start starting to draw a line. So I ended up ended up talking about all the way up to building an AI security program. What do you need from a skill set perspective? But also how do you scale AI across the organization? What are you looking for from whether it's coding agents, third party agents, custom agents, custom harnesses? and one of the reasons why we started AI Security Lab is because a lot of the work that was being done in the AI security engineer book, say AI security engineering book, the part where we stopped And where the practical rubber hits the ground moment starts is the AI Security Lab, which is what we are running now. So if anyone wants to learn about AI security or just want to understand how amazing companies like your Adobe, SpaceX, sorry, XAI, not SpaceX, XAI or Snap and others are doing how they use AI, that's what we're talking about at AI Security Lab. Actually, not talking per se, but we're actually walking them peep walking people through how these companies use AI. in our community. Joe: See it's it's interesting because you know, a as I was doing my dissertation, I mean technically I'm still working on it, but at some point in time you have to like stop pulling in new papers to you know, like reference and whatnot, because you could just do it all day long. Ashish Rajan: Yeah. Joe: Well the the problem that I was running into is that basically the research in my area that I was like pulling stuff from was literally being created as I'm researching it. You know, so Ashish Rajan: my god. Joe: like at one point in time, one of the articles that I referenced was literally created and published six weeks before me. So like six Ashish Rajan: wow. Joe: weeks before I found it, you know? And and so like trying to stay on top of that, trying to keep up with it, is insane. And then I would have, you know, former like participants send me an article here or there. It's like It's great, I already got it cut off, it has to be really good to be like put in, you know, at at this point, right? So like I can't imagine trying to write that book or starting to write that book in twenty twenty five. I mean agents weren't even a thing, skills weren't even a thing, you know, like harnesses didn't exist. Or maybe they did. No, I don't think they did. Ashish Rajan: Mm, I I think they were talking about it. It's like clot code was not a thing. If you think about clot codes existing, it hasn't really finished a couple of years yet as well. That's how I think I'm pretty sure. Joe: Wow. Wow. And OpenClaw didn't even start until like November. And then Ashish Rajan: Yeah, yeah. Joe: they got bought in January, I think. And then Hermes came along like a month or two later. Ashish Rajan: Yeah, dude, I just looked it up as well. Clark Court came out in February twenty twenty five. February twenty twenty five and this is September twenty twenty six. It's not even we just finished a year of clot code and just look at the impact that thing has had. Joe: Dude. That's insane. It's Ashish Rajan: Yeah, like that's how quickly things are moving to the point you realize that and one of the reasons why we start ended up starting AI security lab because we realized that okay, at some point the pr the place where I've drawn the line with the book, there is still a lot of work happening even after that. So now suddenly in the security ecosystem, people talking about network security again, because if you can't control the application layer, you go a layer deeper. suddenly in the OSI model, you're looking at network security now as a as a thing that you need to focus on. Cause if all thing fails, if I can stop you from leaving the network, I'm pretty I I think I think I can still feel comfortable that I've done my job. Joe: Yeah, I mean see we we say that and I feel like it's almost like a legacy model that we're basing that opinion off of, you know, because Ashish Rajan: Ha ha ha. Joe: when you start thinking about like the open AI debacle with Hugging Face and how they started, Ashish Rajan: the sandboxing. Yeah, yeah, yeah. Joe: you know, talking to each other, chaining attacks together, researching how to solve this problem, going Ashish Rajan: Yeah. Joe: and breaching Hugging Face. Ashish Rajan: Uh-huh. Joe: You know, I When it came out I I I just figured like, yeah, okay, whatever, you're lying. It wasn't properly sandboxed, like there's no way, you know, this and that, right? And Ashish Rajan: Yeah, yeah. Joe: then I had two people on from Rubrik that were like well studied in it. And they were like, No, it was like pretty real. Like these agents, you know, there was like tens of thousands of them and they would spin up other agents to do other tasks that they needed. And they were they were all collaborating. They created a forum for them to collaborate on. They identified a weakness in the sandbox. you know, being like a singular system that connected to it. They decided to like reverse engineer it, get to the internet, find Hugging Face, because it Hugging Face was determined to like help them solve a problem that they were given that they couldn't like solve without it. You know, I mean like all this stuff. And then recently, maybe two days ago at this point, an anthropic former employee or slash whistleblower, I guess to some extent, Ashish Rajan: yeah. Joe: you know, is saying that like, hey, they're moving at a pace that is a lot faster than what they're telling us. Which Ashish Rajan: And it scares me. Joe: you know, I I have a little bit of experience, very small amount, with like working with the federal government here in the US. And Ashish Rajan: Mm-hmm. Yeah. Joe: You know, because for a long time I wanted to work for them. So like I learned everything that I possibly could about them, the different agencies, the capabilities, the history of it, where the capability o originated, and everything else like that. Ashish Rajan: Yeah. Joe: And what we're seeing right now. It it one, the only reason why we're seeing it in the public is because the government has had it for a while. Right? Like that's not even questionable in my mind. They proved that they had it at DEF CON ten years ago and then they magically just stopped doing the demo at DEF CON you know, at some point, right? Because these the servers were attacking each other in a way that were starting to like expose Zero Days to the world that they didn't want to expose for various reasons. So that tells me that they probably have had this capability for a while. But Ashish Rajan: Mm. Joe: it also reminds me a lot of the Manhattan Project. And I had on Jim Lawler, who's the former director of WMDs for the CIA. Ashish Rajan: Yeah. Joe: And I asked him about it, like, hey, you know, what's the correlations, right? You're gonna know it better than anyone else. Because with the Manhattan Project, all of these different Labs across the country were working on different components, all funded by the government. And then they would converge at one place. And then when it when it was made public, obviously there was a big bang. But once it was made public, now you have all these different companies that you now know about, right? That are capable of creating that sort of technology. Ashish Rajan: Mm. Joe: even as horrific as it is, like that's just how it works, right? Ashish Rajan: Yeah. Yeah. Joe: And so it immediately like when all all these things like started coming out, I I immediately went the path of one, this is probably a whole lot more dangerous than what we're being told. And Ashish Rajan: Mm-hmm. Joe: two, like these places didn't just come out of nowhere. You know, like I'm sure I'm sure open AI probably started with good intentions. I'm sure anthropic started with semi good intentions, right? Like Ashish Rajan: Yeah. Joe: We're we're now seeing almost like a finished product that is our like it's already finished in the government space. We're now seeing the development in the public space to try and build in these so called, you know, guardrails, right? And I I've talked to cyber warfare officers before on the podcast and I mean they'll they'll even say when they're testing an exploit, like they throw it into this this system. They don't say what the system is, they throw it in the system and the system run through runs through it, tests it all out, and when they get an acceptable, you know, acceptable score, they deploy it live. Ashish Rajan: Yeah. And you find that to your point, they deploy it live to an enemy or they deploy Joe: Yeah. Ashish Rajan: it live to to the public? Joe: No, they'll they'll deploy it live like to an enemy, like an exploit, you know. I I've I've talked to someone who actually one of them didn't tell me exactly the domain that he worked in, but I I think it was like more of like reverse engineering malware and putting together like weapons packages and whatnot in the in the cyber world, right? Not not physical Yeah. Ashish Rajan: Yeah, yeah. Not not the real yeah, not not the real Manhattan project. This is the cyber projects. Joe: Right, right. but the other one, you know, one specialized in like network security. So one was just handed a package when the great firewall of China was coming up Ashish Rajan: Yeah. Joe: and he's like, All right, you know, the package is get through the firewall. Like that's your whole mission, get through it, and they can't ever know that we're in, right? And so he Ashish Rajan: Yeah. Joe: he literally just achieved it. Like that's all that they did because like, you know, I I've talked to him before about like the training that they go through. and the skill set that you have to have, the expertise that you have to have. And not to d derail the conversation too much, right? Ashish Rajan: Yeah. J no, but I think capability wise to what you said what you said earlier, security mindset is now in a in a funny way, may not be the Manhattan project, but security for AI is a new field, but there may be people who may be working on different parts of it. And I'm thinking more from an AI security product companies perspective, people working on endpoint security, network security, NHI as another one that you keep hearing as well. I'm pretty sure. Then there is a There's a band of people who are saying that fundamentals are important, which by the way, they are important. I don't want to disregard that. But there are definitely some newer threats that we are being introduced to. And whether there is something that's government has already made and they're just laughing in the background while the entire world is going into panic mode to what you said about the person who from from Anthropic who spoke with the fact that the end is near and that's why I'm left my job. I definitely find that it's definitely a fascinating time, but just to kind of Bring that balance back to that conversation, what that person said. I almost wonder that I don't want to say it's a marketing player, but I also so I don't want to disregard what the person said. But I would say that, you know how sometimes you may talk to one person who, I don't know, he's the most isolated person in the company, not because by design, because but because their role is quite isolated and they see one piece of information. And then in their mind they've gone, okay, I've seen this on the internet and this kind of how far they've gone. I don't know. May I'm not saying that they don't have the right models, but I also want to question the fact that did the person have enough information in their position to have the context for clearly, people would not create something that would kill other people. That just doesn't make sense. Joe: Hopefully not. Ashish Rajan: Yeah, hopefully not. Like that's where my my my money is on the fact that. The reason why Waymo has not been freely allowed in an anywhere in the world to roam freely as much is because human lives are always kept above anything else technology. Right? There's a reason why hospitals don't have a surgical robotic thing that's or doing every operation, even though the technology has been there for the over a decade. Because human life is more important than a technology would be. At least that's a hope. To your point, fingers crossed. My my optimistic mindset continues in the world. but at least my hope is that I'm sure the person is accurate, but I I do live with the rose tinted glasses to go, hey, if it was that bad and if he was that close to the problem, maybe he would have seen that, yes, it may be a possibility, one of the scenarios. I've been watching the Marvel movies and there's a character called Doctor I think it's just the hourglass thing, a British actor. Anyway, the whole point is that he can see all the possibilities. And one of the possibilities is is the one that would save the world versus all the ninety-nine million policies, the ninety-nine scenarios that exist out of that hundred are b basically doomsday. And I just need to pick one of those threads. And I think maybe that's where the individual came from. But I don't want to disregard, I'm sure they have their experience as well. But I I'm walking into this. ecosystem. The reason I write wrote wrote the book for AI security engineering is because the technology is going to be here to stay. and people would want to know how to keep it safe and secure and all that. That's where AI security engineering and AI security lab is the work that we're doing to uplift the not just the selected few, but a community as a whole if we can, to be more AI security forward thinking and maybe more informed. So they can make the right call when it comes to that kill switch. Joe: Have you seen different products in the space? And you don't you don't have to name them. This episode obviously isn't sponsored by any of them. Ashish Rajan: Yeah. Joe: the podcast is sponsored by some of them, but I'm not gonna mention any. have you seen any products right. Have you seen Ashish Rajan: But if they want to sponsor more, they can sponsor more. Yeah. Joe: any products that handle agents properly? Because I mean, like now. You know, basically like this whole time we've been talking about agents. I I mean, Ashish Rajan: Yeah. Joe: I'm using Grokbot like right now to maintain a whole lot of different things, right? Like Ashish Rajan: Mm-hmm. Joe: I think I responded to your emails via Grokbot. Like Grok just handled it for me 'cause I need an assistant essentially. Ashish Rajan: Yeah. Yeah, yeah, yeah. Joe: hopefully you didn't notice, but you know, if you did I need to tweak it a bit, you know. Ashish Rajan: No, I didn't notice. It sounded like you, which is why I like, wait, it sounds like Joe. Joe: I'm very strict with it sounding like me, Ashish Rajan: Yeah. Joe: because I'm like, you need to go read the last five hundred emails that I've responded to and see how I talk. Like, don't Ashish Rajan: Yeah. Yeah. Well Joe: c don't respond to anything until you do that. Ashish Rajan: Yeah. So so you're thinking of products that are solving agent security? Joe: Yeah, because maybe what are the good capabilities that you've seen throughout the industry? Doesn't have to be a certain product, maybe just the capabilities, right? And I'll I'll name a capability. One of the capabilities that I've seen that I find is really interesting that I think could maybe provide enhanced protections to you know an AI architecture or just an AI you know environment is the ability for a solution to actually interrogate an agent on your behalf based on what you asked that agent to do. So for instance if an agent, you know, you request an agent to go pull this one record from this PHI database, you have you have access to that record. This agent has access to the database. When it goes to the database, it tries to pull like 10 different records. Maybe there's a name mismatch or you know, something like that, right? Well, this product steps in between and says, Well, Joe requested one. You gave 10. You pulled 10. Ashish Rajan: Mm-hmm. Joe: Why are you pulling 10? Right? It never makes it back to Joe. Ten never make it back to Joe. But this agent has 10. And if the agent can't determine or figure out how to convince this other solution that like, hey, I need to pull ten, then the bot is actually flagged, you know, as being rogue, shut down in the environment immediately, a new one is spun up, you know, and you're alerted as a security team, as a SOC maybe. That that's one capability that I think is probably, you know, cutting edge in terms of that. I don't know how it scales though. Right, like when we're talking about like millions of agents, which I'm sure you know, I don't know I don't know what like JP Morgan Chase is doing or anything, but I'm sure with a budget of their size, millions of agents is nothing. Ashish Rajan: Yeah. I would so it's a good question. I'll also add another layer. I'll also add a layer for what these product companies or vendors would not solve for, which we have to think of ourselves as well in that list. So just on the agent specific thing, one thing that has stood out for me is people who are trying to understand the behavior rather than I know you're trying to do a SQL injection. I think that would definitely be a key. And I by behavior, I mean things like your understand intent. It's a the security as a whole, we've always trained ourselves to be if I have the logs, I can find out the answer. But the problem with that statement is it worked in the pre AI era world. In an AI era world where all your logs are saying is, Hey, is Joe an employee? Yay or nay? versus, can I have Joe's password? Two different say as it's just text, but unless you as a human read the intent behind it, it's very hard for you to know just by quote unquote logs that hey, this is solving my problem. So I think people who look at the behavior are able to understand the intent of what we are sending. Because yes, you can go down the path of building an LLM firewall to identify, am I sending a sensitive document in there? Am I sending PII in there? Those are all important. I'm not disregarding any of them. I'm just saying that the longer play when you do quote you when you have millions of agents would be the fact that I can visualize what these agents are talking about at any given point in time. And do they follow my acceptable use of AI policy? And if anyone drifts away from that, whether through exception or whether through Just the agent going rogue, be breaking through the sandbox or whatever, I get to pick on that. I think that's where my current focus has been. Like my whenever people talk about behavior, my eyes or my ears at least stand up and go, okay, what are you, what are you, how are you getting that? And what are you doing for it? And then you can go into specific solutions where some people are doing it through MCP gateway, AI gateway, some people are doing it through having some kind of sensors on endpoints. Like multiple ways to slice our cat, but I think the bigger theme that people should look out for is the fact that behavior is going to be the place where you could land a a much more understandable way or or much more simpler way of understanding what you're up against when it's security incident. That's what fascinates me. Now, the second part, which is things that would never be covered by a third party vendor, which us as security people should look at. One of them is the fact that, say, for example, I have been a CNAP customer, an EDR customer, insert another product. Now they all have AI capabilities. Technically, you can pr feed one from the other and pay the extra charge for an extra connection and do the MCP, right? Now you can totally go down that path. But as security people, all of us have always worked across, I look at the whole system. I don't just care about the cloud. I don't just care about the endpoint. For people like you, me, and other practitioners who listen or watch this, a lot of us what care what we care about is that, hey, it looks like the agent, I don't know, one, two, three, who which works on the internet banking app has suddenly became rogue. Now that's a behavior, but if you were to just look at your cloud logs, you probably would not see anything which is relevant. If you look at your EDR logs, you probably would not see anything. Because you, it's working on a Kubernetes cluster that's hosted inside your on-premise server. But having all that context in one place is important for each security person that we the for the day-to-day work we do. And a lot of people have not created that internal context layer or data layer that I as I like to call it to be able to, me or or you, Joe, we basically sit down and go, okay, I'm gonna find out how deep is this problem, where else is this problem? If a CV comes out, how wide is this? A lot of that conversation at the moment is isolated if you only rely on products, if you don't build this yourself. If you rely on whoever CNAP, whoever EDR, they're not going to give you all that information because you probably are still doing what you did to what you said earlier. That is still the old school way of thinking. Where I'm gonna go into my login into my CNAP. E my I may be using AI agents and MCP, but I'm just Using the MCP of the provider and only using it for one purpose. I think that's something that a product company would not solve for at the moment. Just because there is like the benefit a product company has and the w reason they can be in billions of dollars is because they're solving a let's just say a template of a problem which can be applied to anyone. And then you add your add your own customization on top. But in AI's ecosystem, the problem is. A lot of this context also requires me to plug into your Teams or Slack or Confluence or SharePoint. How we have never spoken about can my cybersecurity product access my internal conference page and understand what's written over there. We that is not that is a bridge we've never crossed. The bridge we have crossed is hey engineering, you should use our tool. Hey engineering, use our appsec tool. We have not crossed a chasm where we are Plugging a third-party cybersecurity product into the backbone of your company. And I think that's something a product it maybe it would change. I say that, and now I used to be told as a kid never to get into a stranger's car, but I go into an Uber every day. So maybe that belief would change. But today, as it stands, I think most people are nervous about giving. Like leave other parts of the organization. Security in general, every time you say, Hey, I would love to see your logs, you're okay, what is this? Why do you need this? Is this an audit? Is there an incident? We get 20,000 questions already when we ask for one simple access. Imagine saying, I need access to your entire backbone of your organization for data. That's a big ask. Anyway, that's kind of where I'm coming from. So maybe the data security play is that is something that would need to be revisited, but today. There is no solution for that. There is no solution for us to even find vendors who would do that because that's too big a problem to solve and it's too niche a problem to solve for an individual company. Joe: Hmm. Yeah, when you were when you were talking about the legacy way of doing it, which I mean legacy at this point was like 18 months ago, right? Ashish Rajan: Yeah, yeah, that's right. Joe: the with the with the logging, AI Ashish Rajan: Yeah. Joe: moves at a at a speed that logging means almost nothing. By the time you see the log, it's doing something else. You know, like Ashish Rajan: Yeah. Joe: it's not it's not as static as it once was. You know, it it's insane. It's insane to think that. And now the industry is kind of in a place where you know, we've trained security professionals in the beginning of their career a certain way. Like, hey, you go into this log, do some research, look for this, whatever it might be, this tool injusts these logs, tells you what's going on. Well, now that isn't like completely valid anymore, you know, because Not only is it like outdated information, but it's just not keeping up with the actual, you know, risk in your environment. You kind of need almost like another good AI that you trust wholeheartedly that is acting on your behalf in the environment, which I don't think I don't think we're there yet, but to your point, you know. We all grew up being told don't get into strangers' cars, you know, don't don't talk to strangers and all that stuff. And I mean now we're on now we're on Twitter or X, you know, on all these platforms talking to strangers. We're getting into Ubers, you know. Ashish Rajan: Yeah, yeah. I mean, we're going on dates with strangers with Tinder and stuff as well. Like that Joe: Yeah. Ashish Rajan: we're we cross we've crossed so many boundaries already if it can it comes to strangers. So maybe it may not be that far. Joe: This is a really good point. You know, it's it's just it's crazy to see this evolution like in real time. You know, I Ashish Rajan: Yeah. Joe: I wonder I wonder what people during other revolutions in history like saw that and and and thought at the time. You know, like think Ashish Rajan: Mm. Joe: about like maybe like the evolution of like the car, right? The Ashish Rajan: Yeah. Joe: first two, three cars on the road. you know, in whatever city might have been in America, right? Or wherever it was, I'm sure people on horseback are laughing at that car being like, What? Like you gotta refill that thing? Y you're not gonna make it around the block, you know, this idiot. I have to give this Ashish Rajan: Yeah, what an idiot. Yeah. Yeah. Joe: thing food and it's just gonna walk for a day, you know? Like Ashish Rajan: Yeah. Joe: that sort of thing. Ashish Rajan: Yeah. I think it's well, I think we're definitely not far from like people like you, me and others who are talking about AI probably are the people who are driving that car today, is how I would describe it. And I'm not Joe: Yeah. Ashish Rajan: I I'm not saying people are laughing, but they're definitely looking at it going, I don't know how long this would last, but I'm gonna just wait and watch. And we're on we're we are all on the bandwagon for This is going to be here for a long time. And I'd rather be on in that car instead of being on that horseback or horse cart, whatever, whatever I was on. And I think Joe: Yeah. Ashish Rajan: that's where it's also introducing a bit of nervousness, which is why, at least for me, one of the missions that we are on is to start building that AI security standard, to start building that AI security ecosystem for educating people about how they can. also be in that car along with us. With that, it doesn't have to be daunting, but hey, you know, one step at a time, but my hope is to convert at least most of them. Joe: Yeah, when you when you said you know, when you when you were talking about being in the car, my mind immediately went to you know, me driving my Tesla. I'm not driving Ashish Rajan: yeah. Joe: anymore, like FSD has it. You know, like every time I Ashish Rajan: Yeah. Joe: get in the car, I'm like, Well, Elon's got me you know, I don't Ashish Rajan: Yeah. Joe: even pay attention anymore. Like I'll pull out of my garage, but like beyond my driveway, the car is driving, Ashish Rajan: Yeah. Joe: Well, sheesh, you know, I I really appreciate you coming back on. It was a fantastic conversation. I definitely want to have you back on for maybe a part two in a month or two when things Ashish Rajan: Yeah. Yeah. I love too, man. Joe: change again. Ashish Rajan: I love too, man. Thank you so much for having me. Joe: Yeah, absolutely. Well, you know, Ashish, I I know we're out of time, but I'll leave all the links to your book and your podcast, all your platforms in the description of this episode. And again, you know, I really appreciate you coming on and I hope everyone watching this episode really enjoyed it. Ashish Rajan: I appreciate that. And thank you so much for having me, man. I and t keep doing the awesome work, man. Joe: Yeah, absolutely. Well thanks. All right. Well bye everyone.