CISM: Domain 2 - Module 4


The course is part of this learning path

Metrics, Monitoring, and Reporting - Part One

This is the final module of domain two and looks at metrics, monitoring, and reporting within the sphere of risk management. We look at how to measure risk through key risk indicators (KRIs) and how monitoring of these can help us to avoid risks in the first place.

We also look at the controls that can be put in place to reduce risk overall in our organizations.

Learning Objectives

  • Understanding how to monitor and measure risk
  • Mitigate risk through controls

Intended Audience

This course is intended for anyone preparing for the Certified Information Security Management exam or anyone who is simply interested in improving their knowledge of information security governance.


Before taking this course, we recommend taking the CISM Foundations learning path first.


We continue now with our review of the CISM exam preparation seminar, with Section 41, "Information Risk Management, Metrics, Monitoring, and Reporting."

Risk monitoring is defined as ongoing activities, including control effectiveness assessments and risk assessments, to observe changes in risk over time. Security managers perform risk monitoring to report risk levels to executive management and to identify unexpected changes in risk levels.

Typical activities that contribute to risk monitoring include internal audits, controls, self-assessments, vulnerability assessments, and risk assessments. Because the primary audience of risk monitoring is executive management, the reporting of risk monitoring is often done using dashboards such as you see on this slide, which may be part of a GRC system's risk management function that permits drill down when desired. 

Information security managers should periodically meet with senior managers and executives to give them an update on key changes in the information security program and on changes to key risk indicators. Executives should also be notified of key security related and risk related events, such as security incidents and changes in compliance risk. And as you see on the slide, the red amber-green gauge can quickly convey through a visual the status of the program. These heat charts or stoplights serve as very quick grasps of the current status of the system.

Other sorts of charts can also be used, but it is the visual that communicates virtually instantaneously what the status is. They will obviously need to be regenerated periodically to show changes and this can help senior management readily grasp the status of the program.

We move into the subject of key risk indicators or KRI's. Now these are measures of information risk used to reveal trends related to levels of risk of security incidents in the organization. Key risk indicators are security metrics that are designed to serve as early indicators of rising or falling risk and of the rising or falling probability of various types of security incidents and events.

Now, there is no standard set of KRS that can be used across all enterprises of all types. Rather, these are selected by each organization and they develop their own set of KRS based on specific information requirements from regulators, board members, senior executives and other parties. KRI's are often derived from operational activities throughout the IT and business environment.

On their own, those activities are often meaningless to executives and others, but when properly developed, they can serve as valuable risk signposts that help executives and others better understand information risk in the business over time.

Going further and adding remediation information can transform the metric from a number of vulnerabilities identified, which in its native state is useless to executives, and to an even better metric such as the time to remediate critical vulnerabilities.

When we select KRI's, we need to look at four basic criteria. The KRI should be the indicator of risks with high impact. For two equivalent KRI's, if we have the choice, choosing the easier to measure one so long as we don't lose accuracy in the process, KRI's should reliably predict outcomes and have a high correlation with the risk. And it must accurately model variances in risk with high sensitivity.

Now, as we look at KRI's, we should examine them as a group, and we should look at the different stakeholders in the selection process as well as selecting KRI's that give a balanced approach and appearance of the risk, making sure to have proper relativity and relationships between them. We have to be sure that the KRI's can drill down to the root cause in the event that additional and more detailed information may be wanted by its audience.

Now, part of the normal reporting, monitoring, and metrics gathering processes should be reporting significant changes in risk through an escalation process. Now, this process should trigger a report to senior management in the event of a breach or any other significant change, usually more negative than positive, but definitely anytime there is a significant change. Also, triggering a risk assessment afterwards should be part of this particular process.

Now, security events oftentimes result from a lack or a failure of a control, or the human element involved in any process. As with all of the processes we've discussed, there should be documentation to show how the process works and capture all the various metrics and reporting items.

To document, we start with objectives, what should be achieved so that we can report on any variance from their achievement. It has to be targeted for the specific audience. It should capture the information resources involved, all the assumptions need to be expressed fully and with validation, and how these act upon the decision-making process so that we can show the logical flow.

Now, the following sections for risk management should also be included. The risk register, where all significant risks are tallied. We should have a description of the consequences and the probability of a given type of compromise. There should be the initial risk rating, so that we can show the products as of any change in that risk and relate it back to the events that propagated it. We should have documentation showing any vulnerability to internal or external factors. There should be the inventory of assets. We have to have the risk management plan and the mitigation plan documented as well, and there should be a description of how the monitoring and auditing documentation will be generated, and the processes that do so. There should always be, of course, a version control process to ensure that everyone involved, all audiences, all executives, and all management involved will have the most current version at any one given time. 

Now, as we've discussed in the past in this course, the majority of training and awareness needs to address the various issues and subjects within our own risk environment. We know that the majority of security incidents happen because of human error. Further analysis of these incidents indicates that there are several factors that may contribute to these incidents occurring.

One would be the lack of awareness of the risks associated with general computing and internet use, something we call computing hygiene. Also the lack of training and experience in the configuration and operations of systems and applications. The lack of training and awareness in key business processes and procedures.

A lack of information on workers responsibility for reporting problems and incidents. A security awareness program is essential. On this, there can be a little argument. It is an essential ingredient in every organization because it provides information on safe computing, policy requirements, procedures to ensure that this is properly performed, and any specifics on workers' security related responsibilities that can be reported to all workers through training programs, messaging, and any other means.

Now these of course need to be targeted to the specific audiences. In general though, end-user training should include a description of the importance of following policy, and the expectations of the given policy. How to respond to emergencies, again, with respect to the employee's particular role. The importance of restricting access, privacy and confidentiality requirements, recognizing and reporting security events, and recognizing and dealing with social engineering.

About the Author
Learning Paths

Mr. Leo has been in Information System for 38 years, and an Information Security professional for over 36 years.  He has worked internationally as a Systems Analyst/Engineer, and as a Security and Privacy Consultant.  His past employers include IBM, St. Luke’s Episcopal Hospital, Computer Sciences Corporation, and Rockwell International.  A NASA contractor for 22 years, from 1998 to 2002 he was Director of Security Engineering and Chief Security Architect for Mission Control at the Johnson Space Center.  From 2002 to 2006 Mr. Leo was the Director of Information Systems, and Chief Information Security Officer for the Managed Care Division of the University of Texas Medical Branch in Galveston, Texas.


Upon attaining his CISSP license in 1997, Mr. Leo joined ISC2 (a professional role) as Chairman of the Curriculum Development Committee, and served in this role until 2004.   During this time, he formulated and directed the effort that produced what became and remains the standard curriculum used to train CISSP candidates worldwide.  He has maintained his professional standards as a professional educator and has since trained and certified nearly 8500 CISSP candidates since 1998, and nearly 2500 in HIPAA compliance certification since 2004.  Mr. leo is an ISC2 Certified Instructor.