Wed 5 Aug 2009
Measuring Security Professionals Performance
Posted by andyitguy under information security
[3] Comments
My Buddy Pete Lindstrom has a question for us. He wonders about our ability to measure security professionals performance. My first thought is “Of course we can”. We just look at what they are tasked with doing, how much is being completed (if it is a “completable” task), what level of risk reduction is being attained due to their tasks, what is the ratio of blocked attacks vs. attempted attacks divided by sucessful attacks, are users better able to do their jobs or are they hindered from doing their job, how many internal systems are compromised, how many are being used to compromise others…….. hmmmmm, maybe this isn’t quiet as easy as it seems.
As difficult as it may appear I do think that performance can be measured. The problem isn’t the ability to measure performance but determing what will be used to measure it. Actually there is a bigger problem; communicating the requirements from Management to the employee. Then there is the whole idea of actually sticking to those factors as you measure performance. What about the out of scope factors? The things that come up day by day and prevent you from completing or working on measurable tasks. These are the things that come into play that can throw a curve ball at you just when you least expect it. It is important that we communicate to management the need to keep these things in mind BEFORE they come up to haunt you.
Management often does not understand the day to day tasks that compromise what employees do. They know what they expect and they measure on that and that alone. It is our job to communicate to them the additional things that we are doing. We must document them and communicate them. We need to ensure that they are part of our performance apprasial. Security is still a young profession and many people who are now leading security teams don’t fully understand what all is involved. Especially as quickly as things are changing. Many now in management are spending more and more time in meetings with the business and less and less time understanding the daily grind of IT and security.
So, Yes Pete, there is a meausre clause but just like Santa Clause it doesn’t just magically appear and leave you a great apprasial. You have to ensure that the list he is working off of is the same one you are working off of.
3 Responses to “ Measuring Security Professionals Performance ”
Trackbacks & Pingbacks:
-
[...] IT Guy… Measuring Security Professionals Performance Andy ponders the thoughts of analyst Pete Lindstrom’s and the ability to measure security [...]


The problem with metrics is that they aren’t very useful for InfoSec. I’m sorry to say.
If you remain 99.99% safe from hackers that sounds good but if that .01% is a hack in which all your customers credit cards goes for a walk and your CEO has to appologise during his year end speech – not good.
You patch 100% of critical servers and 99.9% of non-critical servers, but the server that is not patched shares a database with your finance system and allows someone to break into that database and transfer money out..
Your users safely browse 1.3 billion web pages monthly but only one has a CSRF that transfers money into a hackers account.
What is the better option? I have no idea..
So Allen’s comment is an excellent example of my concern. IMO, it is not good enough to simply say it is too hard, or it doesn’t work. We have to find a way that works. Sometimes this is simply changing .01% to some notion of, say, six sigma, which is much smaller than that.
I am a big believer in measurement, though not to the point of ignoring its weaknesses.
Pete