AI amplifies expertise, it cannot replace the craft
John Pirc, cybersecurity product and technology chief at PartnerOne, argues AI amplifies real expertise but cannot replace the craft. He explains why a human must stay in the loop and why AI should collapse the brutal SOC learning curve.
John Pirc is the Chief Product and Technology Officer for PartnerOne's cybersecurity portfolio, a role he stepped into after two and a half years leading product and technology at NetWitness, one of the portfolio's flagship platforms. He describes NetWitness less as a single tool than as a way of pulling every signal into one place. It is a network detection and response platform that also folds in SIEM, SOAR, and EDR, gathering packet-level telemetry so an analyst staring at an alert has the full context to decide whether it is critical or safe to stand down. The goal, in his framing, is to paint a clear picture for the SOC analysts and threat hunters who live inside that noise every day.
Pirc has spent more than twenty-five years in cybersecurity, long enough to have watched the field reinvent itself several times. He studied at the University of Texas at San Antonio, then built a product career across some of the industry's defining names, including product management at HP TippingPoint, network security work at IBM and ISS, and a stint leading security intelligence and product management at HP. Along the way he became a published author on network security and the economics of cybercrime. That path, moving from hands-on network defense into executive product leadership, is why he talks about the work in terms of both the code and the customer, rarely one without the other.
“Just because I have a cordless drill doesn't mean I can build a house with it. These are tools you need to have, but you have to know how to use them.”
He thinks about artificial intelligence the way a tradesman thinks about tools. A screwdriver became a power drill, then a cordless one, and each leap lets a carpenter do twenty times the work, but none of it turns someone into a carpenter who was not one already. For Pirc, the same holds in engineering and security: AI accelerates the people who understand their craft and exposes those who do not. He is adamant that a human stays in the loop, because the inputs and the judgment still have to come from somewhere, and because the models will confidently hand back things that are not true. Knowing where AI begins and where it ends, he argues, is the actual skill.
What animates him now is a problem he thinks the industry keeps ignoring while it fixates on fear. Running a SOC demands near expert knowledge and a brutal learning curve, and he wants AI to collapse that, to greet an analyst at login with a distilled checklist instead of an hour of triage, to answer a configuration question in plain language rather than sending someone digging through admin guides. The adversaries, he notes, are already recompiling their exploits mid attack with stolen API keys, so what worked yesterday may not work today. His answer is not to brace against the change but to keep remaking himself inside it, and to tell anyone who will listen that they should be using these tools too.
The conversation
In this conversation: Josh Rubin (Host, CTO Studio) and John Pirc (Chief Product and Technology Officer, Cybersecurity, PartnerOne).
What do you do, sir?
So I'm the chief product and technology officer for NetWitness. Been in NetWitness probably for about two and a half years. Been in cyber for 25 plus years.
And what is NetWitness?
So NetWitness. We are an NDR company. Set of network detection and response, but we're more of a platform. So with that, there's different technologies that we own. So like SIEM, SOAR, EDR, and then our network platform. All this telemetry that comes into one, and they're able to be in a good picture for people that are using it. SOC analysts, threat hunters, so on and so forth.
What problem is it solving?
It solves a lot of our problems. So we start looking at use cases. You have compliance use cases. You have all the telemetry that we're bringing in. So I talk about NDR, network detection and response. That's all packet data that's coming in. So when you start looking at alerts and trying to triage with alerts that are coming in, having packet data, you just have more context to the security event that might pop up. That could be critical. But after having even more information and context, that can help you either keep it where it's at, deescalated, and so on and so forth.
So has anything in particular changed in cybersecurity in the last, oh, I don't know, year?
Meet this. I mean, among other things. Yeah, a lot of change, I think, with respect to AI and how we approach that problem. I mean, traditionally, when you start looking at the problems that we would have to solve, it took a lot of time. But today, with the AI series that are using AI to move faster from that perspective, means that we have to move faster. So the issues that I see, just bringing up Meet this as an example, is people freak out. All these vulnerabilities that they're purporting about that they found, right? Tens of thousands, as an example, how do you fix that as an under? So we have to use AI as well to triage what's going on from that perspective so we can close that gap. But what's important to note about that is, when you look at vulnerabilities, there's a scoring system. So, CVSS, Common Vulnerability Scoring System, one to 10, 10 being the double. So if anything hits from a seven to a 10, 10, customers are gonna want that fixed, because it's critical. But into the eyes of the people that come up with it, they're absolutely correct that it's bad. But when you look at it from a platform perspective, is that exploitable? Right, so being able to have that information provided to the customer, so on and so forth. But the whole thing with AI has, not only from a threat landscape, has changed things, but it's changed essentially the way we look at solving problems, not only just from a coding perspective, customer perspective, and then how we get features and they're utilizing AI. But it has not everything, it's just a component and tool.
But is this arms race symmetrical? So AI in the front two models can obviously identify vulnerabilities at a far faster pace than we've ever seen before, but can it also plug in? So if somebody's using custom AI, open source tools to launch an attack, are your tools up to keeping up, fighting back? Is this agent versus agent? A lot of this, for people like me, way outside of this realm, just trying to envision what this looks like can be difficult.
Yeah, I mean if you look at it from the speed of exploitation, as you just mentioned, we have to use the same tools and try to respond faster. So when you look at the entire security ecosystem, you have vendors out there that do vulnerability management, some that do code checking, right? So like static code analysis, which is checking for like, did you actually code new bugs or anything that could cause vulnerabilities in it? And then you have dynamic where you can scan, set platform or whatnot to provide vulnerabilities. Those companies are using AI as well. So it's kinda not just like NetWitness is in this boat by themselves, it's the whole industry as a whole. Now for us, from a detection perspective, you can tell things that are manning, that they're going after and things that are basically AI generated. And there's crazy attacks that we've seen that have been AI generated to where an exploit was released. It got to a certain part in the kill chain to where it stopped. It was able to reach back out, recompile itself and finish it. And that's what's using like stolen API keys to like, I think it was Claude or something like that. So the adversaries are getting extremely clever, which means we have to be on top of it. So what we did yesterday may not apply today, right? If that makes any sense, like the tactics, tools and techniques that they're using today are gonna be different tomorrow, right? And certain things that you create today is that really gonna be applicable and be game changing three months from now. It's not, so it's really the way that we have to look at, instead of reacting, respond to what's going on, adopt it, accept it, and there's a whole, it's like a whole new era.
I mean, I assume you're also using your own AI to attack yourself to find vulnerabilities, to get ahead of potential exploits.
Yeah, so when you start looking at the system development lifecycle process, I mean, we've put together, basically how is AI introduced into that whole process? From ideation all the way to when you go GA, when you have releases, but through that, there's all these different checks you have, and specifically when it gets down to QA. So you're doing a lot of your vulnerability, your static code analysis testing, all that's done. So for us using that, that's really good because it gets us ahead of where we're at today, right? And that's the whole thing. You can try to do the most secure code as possible, but somebody's gonna find a vulnerability or gonna find a way to try to exploit that. And then if you just go back and you patch that. And what I mean by that is there's a lot of different packages that we would use in our platform or others that would use that might be open source as an example. SSL, secure socket layer, if you think of HTTPS, sometimes that is totally secure and it's fine for that version. Three months from later, someone finds a vulnerability in it based on that. Then you have to go back, do a patch cycle testing, repackage that up, release it to the customer. So the biggest thing is how can we stay ahead of that even faster? So I mean, we do basically we follow checks probably every day. We run those every day on our own platforms to see kind of where we're at from that perspective.
And how much of this is dependent on people who think differently and are constantly working on this problem, a new AI tooling that's coming up that's constantly working on this problem because they're trying to get at core. Is this a people-solved problem augmented by AI or is this an AI-solved problem eventually that's monitored by people?
I think it's a little bit of both, but I think it's having a clear definition on where AI begins and then AI ends, right? So when you start looking at a lot of the inputs that need to be put in, those have to come in from a human, in my opinion today. And then based on that, there's expected output. So on that output, where's there gonna be AI acceleration and then where's engineering gonna fit into that to make those decisions? Because you always, in my opinion, have to have a human in the loop. And when we start talking about AI as a tool and how people use it, I think we had a fun conversation about this where there's different schools of thought on AI, right? There's those that embrace it and understand it as the tool. There's people in the middle that they really don't know, they don't really care. And then there's ones far over here, the third ones where it's gonna take away the jobs, it's gonna take my job away, et cetera. But that comes back to the point of expertise and understanding what you're doing because we talked a whole thing about tooling to where you have a screwdriver. It's very easy to use, there's a wall socket, you can unscrew it, very easy, right? Most carpenters have a screwdriver. But somebody got clever and they made a power cord, screwdriver, a power drill. From that perspective, instead of doing one, you can do 20 in the time it would take you to do one. That's good, it's efficiency, right? Someone got more clever and made a cordless one. Now you can move around the entire house. But because I have that as JP, I'm not a carpenter or an architect, doesn't mean I can build a house with that. I can help a little bit. But it gets to your point on, as we're doing this evolution, what is in your toolbox from an engineering perspective, product perspective, et cetera. And these are tools that you need to have, but you need to know how to use them, right? And if you look at carpentry, I don't know like a whole lot about it, but I know there's other tools that you can use that you would have to understand how to use those that would help you develop a certain type of trade craft. So those are kind of the things that I see that actually fall into the system development life cycle and really understanding where AI begins and where AI ends.
Well, that's basically, how many is your team right now?
It's big, it's over 100.
Okay, so in that, have you seen the acceleration from these new tools? Because to a certain extent, you've had to because if the AI is coming into attack, you're gonna need to use it to push back. So you must have seen a velocity increase there. One question I ask a lot of people is, you have all the money in the world, and you can hire exactly who you want to hire. Who are you actually hiring for right now? What gaps are you filling? Because often people is like, I would hire, but I don't know what for. If you're running with an org and run 100 people, where are your bottlenecks in the current equation and what are you trying to hire for right now?
I mean, really across the board, I mean, you look at full-stack developers, people with Java experience, C++, back-end experience, UX developers, QA. There's always gonna be a need for that, but in the hiring process, it is looking to see if they're using AI.
But it's also, you need all of that because your customer base is using everything. You also need people that can understand legacy systems. You're not operating in a greenfield environment.
100%, yeah. And you bring up a good point. Our product has been around for 26 years. Could you imagine the amount of lines of code that we have, right? You have to have people who understand that, right? I mean, you're not gonna feed, we're not gonna feed our entire code base up and hey, look, how can we make this more efficient, so on and so forth.
Why?
If it were to get breached, I mean, look what happened with Claude not too long ago. I mean, that's the thing about guardrails. I mean, yeah, you brought up a good point. There is some trust to a level, but there's a level of what we do as a cybersecurity company on throwing that up into the cloud. These are things that we would need to control on-prem in a closed environment, so we'd still use it.
So is there a scenario that you could see where an open source or a model that you could take in-house on-prem, air gapped away from everything else and throw your entire legacy code base at it and say, okay, update this.
Oh, 100%, yeah.
Do you see, is that something you see on the horizon?
Oh, 100%, because you talk about AI. I mean, I remember my last company, we used it. Maybe I started with this maybe four or five years ago, and which only had ChatGPT or Gemini at the time, so you're very limited on the tool sets. But as we've gone through this journey, we've slowly brought in AI, right? So you start off with a pod, how are they using it? Well, giving direction, but how much are they using it? How much is a token burn, et cetera, which can become quite expensive. And then you start ramping up to the rest of your entire team. So when you start having that type of velocity, then you can start seeing more efficiencies, how can we run and move faster? And what's interesting is AI will do a lot of great things for you, right? And it's like, well, hey, it's gonna be a great assistant where you're gonna have to do more. But see, now you're shrinking that window of what you have to do to here. Now you can add more stuff, and you can see you get more velocity on that. So from my perspective is you have to get adoption. Is everybody on board all the time for AI? I gave you those three examples. But from our organization perspective, everybody uses it. They use it every single day to pay upon their job roles and what they need to do.
Are you worried about tech debt anymore?
Yeah, I think so. I mean, I think with anybody who has a product as long as we have, I mean, there's always gonna be some sort of tech debt. When you're rolling out new versions, specifically if you're an on-prem platform, that's different than SaaS. In my opinion, like SaaS has less tech debt. In my opinion, they have it. But I think that window is a lot smaller from that perspective, because they could do updates every two weeks, et cetera. We follow agile, but when you have to do releases every three months, six months, et cetera, it's a bit different. But going back to the thing that you talked about in terms of having everything on-prem, air gapped where we can throw our code in there, we understand what we have from a tech debt perspective and kind of what we need to fix, but I don't think AI is going to really fix that immediately. When you look at using AI as an example as a tool, your startup companies can move a lot faster. They have less tech debt, right? Because they have very small, very lean teams, and they don't have legacy code. So I don't know, that's probably not answering it directly, but I think tech debt will always be there to some degree, do I see in the next couple years, depending on how all this AI is starting to mature, I think that window gap will be shortened significantly.
Yeah, I'm hearing it from, I've talked to Juan Rey over at Cadence out there, building the software that designs the chips that power the AI, that do all these, basically have the opinion on, I don't think tech debt's gonna be a thing for much longer. I think the AI solves it. Legacy systems, but he thinks it's moving so fast that it doesn't matter, it finds a problem, it'll fix it. What you're talking about with the cyber and being able to build tools that can be so incredibly reactive to changes, to a certain extent, kind of means that that feels like where it's going. Because the only way that you can move fast enough is if they're building attack vectors that take advantage of, every time you're patching those holes, you're solving tech debt.
Oh yeah, well it's that too, and then as you plug two holes, two more come open. So I would agree with him, 100%. It's just, if you ask me today, right today, that's my answer. I know that is going to change, and we hope it does, but we know it will.
Well it's funny, it's moving from tech debt being a thing that you look at every few months or every year, okay we're gonna take a week off, and now we're gonna take care of this and clean some of this up to it being, no this is running all the time, because everything's constantly being updated constantly. Which is akin to frankly all of these AI tools which promised us, oh you're gonna get so much time back. Well that was a lie. We're not getting time back, now we just have time to work on more problems that you've identified.
100% yeah, yeah, I totally agree with you on that. I mean, yeah, the whole thing is, yeah, being able to move a lot quicker and a lot faster, which means there's more to do. Yeah.
What are you, someone who's working with cybersecurity, most concerned about that you don't think people are really taking into consideration right now?
Well you know kind of what I see is, we talked about mythos as an example, and that's what everybody's talking about, is like fear, uncertainty, doubt, and I get it. I mean that's the whole premise of why you have security controls, fire alarms, and all the other stuff. You know these things that you need to keep you safe, aware, you know kind of understanding your situation. The things I really don't see a lot of people talking about is using AI from an ease of use perspective. Inside of security as an example, you know doing administration, being a SOC Security Operations Center analyst or a threat hunter requires people who have a very good understanding, almost expert level from that perspective. And there's a massive learning curve. Where I see a lot of people not really talking about us is how you reduce that. And by reducing that, you know from a customer perspective, is time to value, right? How can I get the product up, the platform up, running, in becoming more efficient? And those are things I just don't see a lot of people talking about at least from my industry perspective. And I think that's really important. And the reason why I say that Josh is that I travel the world, we have that conversation. And questions I get all the time is like, can you make this easier? These SOC workflows, can we condense them, right? Not that I'm gonna get rid of a tier one analyst, how can make a hunter's job easier? And you can do that, there's lots of trivial tasks that an individual would have to do. How do we make that faster and quicker as an example to where we can do those workflows for them so when they log in, we can kind of understand what they're doing and what they need to do when they log in. I'll give you an example. So I wake up every morning, what's the first thing I grab? I grab my iPhone, I know it's a floor order, I grab it and I look at it. And I know I shouldn't do that, but there's five things I do. I'll look at my email, personal, corporate, any text messages, look at the weather, and like hey, what happened big today in the news, right? I may not do that in that order, but I'll do those five things. We're creatures that have it. So when you look at analysts, they're the same way. Could you imagine using AI to be able to capture that, understand what they're going to do, and then provide hey, this is what you should do when you come in and it's just a checklist. And then you can move on to your next task and it keeps learning that. Now remember, you take a look at it, so it takes a human review of that. But it just condensed 45 minutes of work or an hour into that. We talked about the complexity of cybersecurity, so doing the configuration and administration. I've seen users where they have to pop out and look at knowledge base articles or look at admin guides. So now your context, which you know, you're wasting time. So what AI provides you from that perspective is being able to pull that in and being able to give you examples how to configure this and how to do this. That's not reinforcement learning, that's like a RAG model. So it's essentially just taking that database of everything that you have, loading it in. They ask a question in natural language, I want to configure a parser, I want to configure X, Y, and Z, and I'll help them do that. So those are the things I really don't see people talking about, they're just talking about the fear, meet those vulnerabilities and the adversaries moving this fast or whatever. I mean, these are all things you should know, otherwise you shouldn't be in this industry, right? You understand that there's a threat and then how do you get ahead of it? But the biggest thing is you have to have people who have an understanding of getting ahead of that threat. And the whole thing is how can you use AI to learn in your platform to help him or her become better at their job instead of hiring, to hire someone like myself or someone who's an expert in industry is expensive, right? You want those people, but the barrier to entry to the market is it's very difficult, especially for some of these kids that are coming out of university, right? But anyway, I hope that answers your question.
I mean, you're talking about the upscaling of expertise via AI platforms. Is college the right model for some of this stuff? How education is gained, how experience is gained through this tool. It's an interesting idea. I'm not totally understanding. Like I get the, you know, you wake up, you know, it makes certain things easier. Like it can prioritize. I don't need to look through all of my emails because it can immediately tell me that these four spam. Don't look at that stuff. It can look at the weather, it can do all of this stuff, prioritize my schedule. What I worry about is I do some of my best thinking when I'm doing some of my least thinking. What I mean by that is, okay, I'm now doing the low power task that doesn't require a bunch of thought. And from that moment in time, I'm suddenly drawing connections that I wouldn't draw otherwise. That's how my ADHD mind works. It needs those moments of distraction, hyper fixation on something that doesn't matter. So it can be free to wander into other places. I worry about the law of unintended consequences when we defer to the AI around certain tasks and certain things. And how do you get people that become the experts, real experts, when the AI is off to do this and that we train them?
That is one of my worries. Yeah, but you bring up a good point on training, right? It's the information that goes in and you absolutely can implicitly trust it 100%, right? That's why I like the whole training of those models, making sure it has all the right information. But that's like, for me, one of the caution to anybody is, you know, look at it at some point, are you not gonna have to do that? Maybe in five, 10 years from now, or it could be soon. I mean, this train is moving like a bullet train, but to your point, you know, absolutely. I mean, those are things I see here. I mean, even if you are a good prompter and put things in, I mean, sometimes you get back stuff that's not, it's not true.
Oh, no, we can lie at scale. I mean, verifiability and learning, teaching people to be discerning, teaching people how to trust and verify, or verify before trusting, frankly, I think is the most important skill. Incredulity is what we need to be optimizing for a lot of these people with these tools. Okay. Let me get to a final question. You know, you spend time everywhere. You spend time in the valley, you spend time outside of the valley. The valley is full speed ahead. It is, I would say, doesn't have time to be anxious about where things are going because they're so just, we gotta keep working, we gotta keep going forward. The anxiety outside of the valley is very, very real. How are you feeling right now?
I mean, I feel good about it. I mean, from my perspective, I mean, living here, I mean, things are moving fast. You have to adapt. You have to be able to recreate yourself from that perspective and embrace it, right? And to your point, yeah, outside the valley, to give an example, when you talk about anxiety, I give roadmap presentations and I'll talk about all the AI capabilities we have, but I've had companies like I want it turned off. I don't trust it. Where's my data going? I mean, very, these questions are good and they're warranted from that perspective. So the anxiety and fear from that perspective, I totally get it. I mean, I feel extremely fortunate to be in the Bay Area and attend these conferences, attend these meetings and stuff on AI and being educated on it and reading about it and formulating my own opinion, right? Because if I don't adapt to it, use it, and say you're just exploited, but I get out of the job. And to your point, I can see, so you mentioned there's three types of people on AI and what their beliefs are. I totally see that, but being in high tech, I mean, it's just something that is every day. Just the other day over dinner, someone was asking, hey, how much do you use AI? And I'm like, every day, you should be using it too.
Be part of the
conversation.
Whether you're a CTO who wants to be featured, a company looking to sponsor, or an engineering leader wanting a seat in the room — there's a place for you here.