Developing Talent in Security Operations

Clarke Rodgers:
When you're looking to hire, either from internal resources or an external hire, what are some of the things you're looking for either the SOC analyst role or other roles within your organization? Is there a particular mindset you're looking for or a particular skillset? What is it?

Tom Avant:
Within our organization, there's 12 different job families. So a lot of people are like, "What? You have an organization with 12 different job families?" We sure do. Everything from PMs, TPMs, support engineers, systems engineers. We have small, tactical dev teams. So, we've got software development engineers, SDMs, security engineers. Look, I'm listing everything off. It goes on and on.

So, across the board, depending on the particular job that you're going in, there's going to be a certain set of BQs, basic qualifications, specific requirements. However, when you talk about working in our organization as a whole, we're looking for high judgment people. We're looking for people who are used to operating in ambiguous situations. We're used to people who have a natural disposition towards being able to do the right thing and... I don’t want to say run into the fire, I don't necessarily like the way that looks, right, but maybe walk into it and assess it before you go in and make sure you enter it the right way.

So, we're looking for those types of people, those people who are already predisposed towards response and want to have that higher calling regardless of what position they're in.

Clarke Rodgers:
And I imagine a certain level of curiosity and “How can I break things so I make sure they don't break again?”

Tom Avant:
Oh, 100%. And also, people who don't like to do the same thing over and over again, because even though that's part of what happens when you run a 24/7 global operation center, you're going to have monotony. I want the people who are tired of it. I want the people who are like, "Man, you've been doing this for six weeks? I don't want to do this anymore." And then they come up with ways to get out of that or more innovative ways to fix it or automate solutions, or, "Hey, maybe we try this process a different way."

Those are the people I'm looking for. The ones that want to question, "Why are we doing this this way?" And if it makes sense, then they're on board. And if it doesn't make sense, then they work with others to find a way not to do it anymore. That's what I'm looking for.

Clarke Rodgers:
So, you mentioned it's a 24 by seven shop. That can be very stressful on a human for quite some time. How do you look at things from a human management perspective to make sure that your folks are not getting burned out, that they do have that ability to rotate through one or more of those 12 job families and have a fulfilling career without just being burned out every day at work?

Tom Avant:
A lot of us in my leadership team have come from different careers, where we had career development as a critical cornerstone to how those careers function. We didn't have that initially when I took over years ago in this space. But it was something that I felt important to incorporate to make sure that people saw a path forward.

Not just level based. It's not about getting to the next level, but skills development, role development, understanding the business. Conversations like that are how we ended up moving from just physical into that fusion space of physical and cyber and logical, right? By saying, “How do we develop these people who are already really performing well? How do we give them greater opportunities? How do we pair with other teams? How can we find more opportunities to help and grow and learn? How do we actually hire and develop the best?” That's the "develop the best" part that needed that focus. So that's really where we dug in.

Tom Avant:
The threat landscape is constantly changing faster than most people out there can even think about. So, if we are only building security experts who are only good in one area, then there's no way that we're going to be able to keep up with the fast changes that are happening in the world.

So, working with my leadership, we started thinking about how can we make what I call a “full spectrum” security analyst. How do we build a person that is well-versed in one discipline and starts to build and develop skills in another discipline? And we start by taking over smaller responsibilities. So we went through some training, we worked with different training programs, sales courses, things of that nature. And then we eventually decided that, okay, we will take on some of the stuff that our other teams are doing because we operate really well at scale.

And the good thing about the SOC is even though we are backed by our people, our first tenant is to always automate for solutions first. Humans are our last resort. And in doing that, we say, "Hey, there are some workloads though that even though with the best intentions of automation, you still need a high judgment individual to be able to take some actions." But some of those, we've got individuals of different skillsets. You don't want to take a super high-skilled comp sci individual and then give them something that doesn't require the full usage of their skill, because that's not a good usage of that resource.

So, if you think about it, it's like resource management. Through having that discussion, we said, "Hey, we can leverage some of those things, building the capacity of our people to be able to operate across the domains." And then at the same time, freeing up some resources for my peers in the other side of the organization so that they could tackle the more ambiguous task.

In terms of the actual day-to-day stuff, we absolutely want to make sure that little simple things that we can do, like have a snack bar available. You'd be surprised that a lot of people who've worked in op centers before, I was like, it really, really is tough when-

Clarke Rodgers:
Pizza goes a long way.

Tom Avant:
Pizza goes a long way. Pizza, a couple of drinks, some chips. People are happy. I'm not saying that to be silly, but I mean really like-

Clarke Rodgers:
No, for sure. It's a real thing.

Tom Avant:
It's a real thing. You're on that ops center and sometimes you're so deep into what's going on, you look up and you've been handling a problem for four hours straight. You didn't even notice that, right? And you need to be able to have some fuels to keep you going.

We also make sure that people have an opportunity, like I said, to rotate roles so they know that, “Hey, here are the requirements to move into a different role.”

We do still have an on-call. On-call again is unavoidable when you're running a global system. But in a lot of places, we've been able to move to where we have that globalized system, where we do a handoff, another team picks up, and then we keep rolling.

Clarke Rodgers:
In preparation for this interview, I looked at your internal team's wiki and you have your tenants published there, and tenants we use throughout AWS to help people have guidance when there isn't something prescriptive down for every particular situation. So, it's how you operate. And you mentioned one earlier, “Scaling through humans is the last resort,” which of course you want to really focus on automation where it makes sense and then the human can make those risk-based decisions. There's a couple more that I'd love for you to speak to because I think they're fantastic.

Tom Avant:
For sure.

Clarke Rodgers:
The first one, "We don't punt, we pass."

Tom Avant:
That's right.

Clarke Rodgers:
Tell me about that.

Tom Avant:
Look, too many times have we seen people get busy, a lot of things are going on, and the first thing that someone says, normally junior in their career, "That's not my job." We don't say that to customers, internal or external. If we are not the ones who own it, then we go and we help you find the right person who owns it, and we give a soft, warm handoff. We don't just say, "Hey, it's not us," and then just punt it away.

And that's internal as well. If you call and you're looking for team A and you call team C, team C doesn't tell you it's team A. Team C walks it over to team A makes it, "Hey, team A, do you have it? You're good?” I've been on the receiving end of it before and after the second or third time that you get transferred, you're just like, “What's up with this organization?”

Clarke Rodgers:
Yeah, “Aren't they the same company? How are they not talking to each other?”

Tom Avant:
How are they not talking to each other? Exactly. So that's exactly why we did that.

Clarke Rodgers:
I love that. And then the other one, which may take a little bit of explaining, "Security is not a secret society." What is that about?

Tom Avant:
I think a lot of people who don't have a cyber background or didn't work in highly technological fields, they think that it's so intimidating. They're like, "Oh my God, I can't learn that," or "I can't code." I'm like, “Shoot, most of the people I know can't code.” I know a lot of people who could code circles around us too.

There's both types, but you need all types in the cybersecurity industry, as we all well know. If you put two military people together, especially if they're the same service, in about five minutes in their conversation, they're speaking a different language because of the acronyms that are flying and the bases that you've never heard of and the different words for TDYs and deployment and this stuff.

So, the same thing happens in the cybersecurity space. So, you have people coming on and they're using all these code names and they're using these acronyms, and if you ask what it stands for, it makes perfect sense. But if you don't know and you're just listening, it just sounds like R2-D2, alphabet soup. And it's so intimidating that people are like, "Oh, I can't do that." And that's actually, nothing could be further from the truth.

So, we seek to demystify, to say, "You know what? Let's actually create an environment where we can learn. Let's create an environment where there's no environment of fear.” Ask the question — I demonstrate this in meetings — if you say an acronym and I've never heard it before, I'm going to ask you. I'm not embarrassed, “What does that mean? What is it?” Now I know and now I can tell someone. You'd be surprised how many times I've asked and a person has said, "Oh, I don't know."

Clarke Rodgers:
Everybody uses it.

Tom Avant:
Everyone gets used to the acronym. So things like that. And then I think the other part of it is, by being able to be more…not just precise with our language, but also understanding of what and how we're using our terms and our words. It creates an environment where more people feel welcome and able to contribute. And the benefit of that — and I really mean this — the benefit of that is there's someone in that room who knows the answer, or there's someone in the room that's thought of something we haven't thought of. And if the only thing holding them back is they're afraid that they might be embarrassed or we might look at them funny because they don't know anything, I want them to speak up. I want to know what that is because that might change the way we do business completely.

Clarke Rodgers:
It would help AWS and help AWS customers and possibly more than that, right?

Tom Avant:
Help the world.

Clarke Rodgers:
Help the world.

Tom Avant:
Absolutely.

Clarke Rodgers:
Well, this has been fantastic, Tom. I really appreciate your time today. Thank you.

Tom Avant:
Thank you so much for having me. I really appreciate it as well.