The Search for Good Security Staff

Skilled security administrators are hard to find

The good news: Administering stand-alone mainframe security is easy. The bad: Finding people skilled at securing a mainframe in a distributed environment is very hard.

Since Sept. 11, the word "security" has taken on an additional, political dimension. It's become a strategic hot-button issue for senior executives taking a long, hard and sometimes seemingly hysterical look at enterprisewide security policies and procedures. Hitherto overlooked and under-funded, security staffing is gaining new respect—and often needs to grow rapidly.

That means the number of open security jobs is on the rise, according to Editorial Director Nick Doty of, a technical recruiting and career Web site in Edina, Minn. "From what we've seen on our site, and from what I've seen from the industry, security is—not surprisingly—very much in demand … Last year, employers' security job listings were rather general. Now companies are requiring applicants to have more specific types of experience and certifications. But it's a new category for many tech recruiters."

With executive-level mandates to staff up on security, companies are finally starting to create full-fledged security departments. These departments can protect the organization by training workers, working with developers, writing proper procedures and securing the entire, distributed network perimeter. But finding personnel who are actually qualified in all these areas can be a real headache for an enterprise IT manager. That's partly, Doty says, "because certifications don't necessarily equal experience, and in some cases job definitions are still being written."

"It's very difficult to find solid security skills," notes SANS (System Administration, Networking and Security) Institute Director of Research Alan Paller. "Overall, the number of really qualified security personnel on the market is relatively small. So organizations often have to choose between paying high rates for experienced personnel, or bringing in neophytes and training them."

A Hiring Manager's Security Resources

  • SC Magazine (, a global security portal.
  • The Information Systems Audit and Control Association & Foundation (ISACA) at
    • Certified Information Systems Auditor (CISA) certification
    • ISACA also has local chapters that members can join
  • Information Systems Security Association (ISSA) at
    • ISSA has local chapters members can join
  • List of security certifications at
  • The SANS (System Administration, Networking and Security) Institute at
    • Global Information Assurance Certification (GIAC) training
    • Intrusion detection training
  • High Tech Crimes Network at
    • Computer investigative forensics training
  • (ISC)2 International Information Systems Security Certifications Consortium Inc. at
  • Includes information on Certified Information Systems Security Professional (CISSP)—requires ongoing training to maintain certification—and Systems Security Certified Practitioner (SSCP) examinations.
  • Also read "Does security certification matter?" an article at


Yet hiring the right workers is crucial, because there's no foolproof software or hardware solution for securing a diverse, heterogeneous enterprise. An effective security program is an effective risk-management program, according to security guru Bruce Schneier. By hiring the right personnel and planning against the most likely attacks, adopting strict best practices guidelines at every level and patching the worst known vulnerabilities, a company minimizes the risk of its servers or network being breached by an attack.

Mainframe Isolationists
Historically, mainframe security people haven't been hard to find, says Paller, because their role has been mostly administrative—preventing the help desk from divulging passwords, dealing with privileges and the physical security of the mainframe. That was relatively easy simply because mainframes historically weren't on any network the outside world could access.

Attach the mainframe to a distributed environment—and mainframes are increasingly a part of such networks—and suddenly those administrators encounter a whole different set of problems requiring operational skills. "What you do to protect yourself against a worm is different than what you do to protect against a help desk person," says Paller.

What's needed to secure current mainframe environments are people who understand how mainframes work in today's distributed environments. Unfortunately, "that skill is in short supply," says Paller, who estimates that there have never been more than a few hundred people in the country who really knew operational mainframe security. "There were very few mainframe programmers who could spell ‘security,' and now there are very few programmers [in general] who can spell security," he says.

Isolating the mainframe in a data center can also isolate its managers. "It's rare to find someone familiar with mainframes who also has more than a passing interest in the Internet, Unix, etc. and vice versa," says Jim Keohane, president of consulting company Multi-Platforms Inc., based in Levittown, N.Y.

While mainframe-security-savvy people aren't needed to fill administrative functions, they're needed to secure any code that developers create that touches the mainframe, as well as to defend the company's overall network—mainframe, PCs, servers—from outside attack. In order to create a unified organizational defense against outside attacks, IT managers need to hire workers who understand the Internet, Unix, Windows 2000 and mainframes.

Find someone who does, however, and you'll generally also find someone with deep experience and a broad set of skills. They don't exactly come cheap. "Many [people with mainframe knowledge] have already branched out and broadened their abilities," moving into higher levels of management or outside consulting, says Rob Clyde, chief technology officer at Symantec Corp. in Cupertino, Calif. "To get someone with that level of experience, realize that it might cost you as much as your director of security," he says.

When planning for security hires, don't forget the day-to-day security work either. Those daily functions will often be handled by an IT or network operations group that runs the machines, installs the patches and staffs the help desk.

But these groups don't work independently, and truly effective organizational security requires that groups that dislike each other—IT, network operations, security, application development programmers—actually work together. With that in mind, Stu Henderson, head of Henderson Group, a mainframe and Internet security and consulting company in Bethesda, Md., suggests that a top candidate is "someone who knows more than one platform, and someone who can work with other departments, across departmental boundaries."

Certifications in Development
Filling that right mix of skills is also difficult because there's currently no "security analyst" degree or standardized test, although some IT associations and educational institutions are developing them. Some security certifications can provide guidelines for employers.

Current certifications come in two flavors: Knowledge-based and tools-based. Among the knowledge-based, CISSP and CISA are to operational security what a CPA is to accounting: The candidate should understand the fundamentals and theory, but may not know much about hands-on tools. A candidate with either certification, however, has earned an understanding of security practices that should speed the security learning curve.

Either type of certification can increase a job candidate's salary by 10 to 25 percent. Note that starting security analysts earn, on average, $54,000, while those with ten or more years' experience might average $82,000, though those figures would substantially increase in urban areas. (See "Average Salary: Security Analyst" for security personnel figures.)

Alternately, tools-based certifications from vendors such as Symantec, Checkpoint, Counterpane and Cisco can demonstrate that the candidate can administer a particular firewall or intrusion detection software system. But tools certification doesn't always ensure that someone knows how to deploy a firewall, say, for maximum safety.

Some certifications, such as Cisco's CCIE (Cisco Certified Internetworking Expert), can include security aspects and require that the student undergo rigorous testing in both the classroom and hands-on lab. SANS Global Information Assurance Certification (GIAC) bridges the tools and knowledge gap by providing training in both. "That's the type of stuff that we teach—not just how to operate a firewall, but [how] to make sure a firewall is blocking all the key attack vectors," Paller says.

Average Salary: Security Analyst

Entry (less than 1 year of experience): $54,090

Junior (1-2 years of experience): $55,958

Emerging (3-5 years of experience): $60,842

Experienced (6-9 years of experience): $75,043

Advanced (10+ years of experience): $82,163


Where to Find Good People
When combing for potential hires, Clyde suggests looking into local chapters of such organizations as the Information Systems Audit and Control Association & Foundation (ISACA, or the Information Systems Security Association (ISSA, Both are professional groups for security personnel. Online job boards such as and are also good sources for candidates, although the hiring manager and recruiter must exercise great care in building correct job descriptions and crafting effective searches. (For a complete list of these resources, see "A Hiring Manager's Security Resources.")

2001's massive tech downsizing may actually be good news for security recruiters. For example, Paul Raines, global head of Information Risk Management at British-based Barclay's Bank, says he has found excellent security personnel who have been let go from consulting companies. Sometimes he also finds recruits straight out of universities and builds them into security workers. "We've gotten a few people directly out of university. We'll typically start them off as interns and if they do well we'll give them a junior-level position," he says.

When he does hire entry-level security personnel, however, he prefers British university graduates. "I think it's a function of the colleges over here having a more robust IT training program than they have in the States. In the States, most people are coming out of computer science backgrounds," he says. "I think a lot of the UK universities are turning out people with a specific focus on security and then placing them into security internships." He's had the best luck with graduates of Royal Holloway and Cambridge University in the United Kingdom, and MIT, Stanford and Carnegie Mellon in the United States.

Even with the downturn in the economy, we're not seeing large numbers of ready professionals that have a great deal of experience in the job," says Clyde. It's still an employee's market for experienced security workers, who often will prefer larger companies or large service providers where the pay and benefits are generally better and where continued training—extremely important in the security field—is readily available. Small- to medium-sized companies may have trouble finding and supporting security staff. In that case, they should seriously consider simply outsourcing their security needs.

New Security Models

The model for an effective security group determines the skills for which companies need to shop, of course. Exactly what those requirements are varies widely between industries and sometimes from company to company. But there are some common

alities. The security organizational model that Paul Raines developed when he was at the Federal Reserve Bank of New York moved with him, with a few refinements, when he became the global head of Information Risk Management at Barclays Capital in London, part of Barclays Bank PLC, a year ago.

Raines divides his security organization into three groups—policies/awareness training, business consulting and compliance—and has developed very different categories of hiring requirements for each group. The policy writing and awareness training group, for example, requires people who "know how to write well," said Raines.

Business consulting, in contrast, requires very technical people who can answer any questions that the business group leaders have about products they're thinking of deploying. They then work with programmers, who might not know how security works in the Java application they're coding. "Incidentally, that's probably the biggest and most overlooked part of security—application security—because there have been so many flaws that come up in applications—from buffer overflow to flaws in cryptography," said Raines.

Finally, at Barclays, the compliance group checks the effectiveness of the other two groups by doing penetration testing—literally trying to hack into their company's servers and databases, just like real hackers or malevolent viruses would. "That again [requires] people with a different skill set. You want people with operational experience—but also wild hair—who know how to overcome obstacles," he says. While Raines' model isn't a fit for every organization, it does at least hint at the range of skills necessary to implement a robust security plan.