“The purpose of risk management is to improve the future, not to explain the past. Security metrics are the servants of risk management, and risk management is about making decisions under uncertainty. Therefore, the only security metrics we are interested in are those that support decision making about risk for the purpose of managing that risk. I urge the Congress to put explaining the past, particularly for the purpose of assigning blame, behind itself. Demanding report cards, legislating under the influence of adrenaline, imagining that cybersecurity is an end rather than merely a means — all these and more inevitably prolong a world in which we are procedurally correct but factually stupid. A clearinghouse review of what we know how to measure and how good what we know is at predicting the future would be a good start as we do not even know what it is that we do not know.” — Daniel E. Geer, Jr., Subcommittee on Emerging Threats, Cybersecurity, and Science and Technology
Gunnar Peterson from 1 Raindrop blog posted the above quote. It is an interesting quote and provides me an opportunity to share a few resources.
If you are interested in a book on security metrics, Andrew Jaquith has written, “Security Metrics: Replacing Fear, Uncertainty, and Doubt.” I personally like having a book in hand, but in the electronic world there is always Safari Books Online. For those not familiar with the site, it is a joint venture between O’Reilly Media and the Pearson Technology Group. Security Metrics is available from the Safari site. Prof. Joseph M. Jacobson of the MIT Media Lab on Apr 8, 1988 in the N. Y. Times made the wise observations:
If books had been invented after the computer, they would have been considered a big breakthrough. Books have several hundred simultaneous paper-thin, flexible displays. They boot instantly. They run on very low power at a very low cost.
Of course, one of the nice thing about belonging to Safari Books is that you do not have to lug the books around between home and work.
The author, Andrew Jaquith, is the program manager for Yankee Group’s Enabling Technologies. Security Wire Weekly, had Andrew on the Feb. 14, 2007 podcast. There are a few additional podcasts, for those interested in security metrics. Stephanie Losi & Julia Allen discuss “The ROI of security” on the CERT podcast series. CSOonline talked with George Campbell, retired CSO for Fidelity Investments on “How to Connect With Metrics.” Laura Koetzle, Vice President, Forrester Research, released the podcast, “Why Security Metrics Matter.”
A few additional publications that might be of interest. If you are doing any work for the US government, you should always consult the NIST Special Publication site. They have released the draft, Special Publication 800-80, “Guide for Developing Performance Metrics for Information Security“. Back in 2003, they also published SP 800-55, “Security Metrics Guide for Information Technology Systems.” In the security application world, the Open Web Application Security Project (OWASP) created a category, “OWASP Application Security Metrics Project.” Anyone unfamiliar with OWASP, it is an open-source project dedicated to finding and fighting the causes of insecure software. The Institute for security and Open Methodologies (ISECOM), also provides information on security metrics in their “SECURITY METRICS – RAVs (Risk Assessment Values) section.
Finally, a few websites. Andrew Jaquith runs the site, securitymetrics.org.
To be honest, there are more links and information on that site than I could ever provide. With that in mind, I will finish by pointing to the CSO website. Jeff Jones has begun the “Security by Numbers” blog.
That should be enough to get a person started/swamped with security metrics. While metrics might have a bad reputation, especially to the IT crowd, there is no arguing that they are key in making decisions and controlling risk. IT security professionals need to understand and do their best to provide good and insightful metrics. Otherwise, CIO will force the very kind of metrics that cause us all to question the intelligence of our organization.