Part Five: Performance Metrics
Start course

In this course, we take a look at company culture and how to make it focus on security. We look at how to close the gap between current state and desired state and how to build security in your environments. You'll also learn about security in the cloud before moving on to look at the various metrics available for assessing security in your workflows.

If you have any feedback relating to this course, feel free to contact us at

Learning Objectives

  • Learn how to create a security-aware culture in your organization
  • Understand how to close the gap between current state and desired state
  • Learn how to build security into your IT architecture
  • Learn about security in the Cloud in relation to PaaS, SaaS, and IaaS
  • Understand how to use metrics to measure the performance of our infrastructure

Intended Audience

This course is intended for those looking to take the CISM (Certified Information Security Manager) exam or anyone who wants to improve their understanding of information security.


Any experience relating to information security would be advantageous, but not essential. All topics discussed are thoroughly explained and presented in a way allowing the information to be absorbed by everyone, regardless of experience within the security field. 



So now we leave the cloud and move into a more in-depth discussion of the Metrics Development in Section 23. So, as we you discussed earlier, we need to have metrics in order to measure the performance and the lack of performance or outright failures of our environment. In determining that we need metrics, A major decision is, which ones will we need that provide us the actionable intelligence required to properly manage our operation?

So we ask these questions: who needs to know what? What exactly do they need to know? And when do they need to know it? So we have before us, the question of defining who, what, and when. The product, of course, is to be provided is actionable intelligence that will lead to better informed decision-making on a variety of subjects.

So metric comes in different forms at different levels. We have our strategic level metrics, which will provide us insight to guide decisions at a senior management level, and to let us know whether or not the programs that we're operating are headed in the right direction and achieving the goals decided upon.

We have management metrics which are more tactical in nature. These will help determine if a security program is in compliance and aligns with the business goals and it supports their achievement. It also helps us by looking at tactical metrics as part of performance monitoring and measurement. We have a group called operational metrics which are comprised of technical and procedural metrics, showing how well particular tasks or operations are being performed or how far out of alignment they might be.

So, as we look at metrics, generally speaking, let us look at the attributes that will help us decide which ones we need and which ones are distractions. Manageable. Day-to-day should be readily collectible and understandable. Meaningful. Meaning the data should be relevant to the goals we are attempting to monitor or measure. Actionable. It should indicate what actions to take or what actions to stop. Unambiguous. Data be clear and interpreted in a very direct, actionable manner. Reliable. Data must be provide the same information for the same circumstances and be considered trustworthy. Accurate. The data represents the truth of a given situation. Timely. The data is delivered when and as needed. Predictive. Data must give clear indication of where are you're heading. And authentic. That data must not have been manipulated or in any way altered to misrepresent reality.

It should be noted that simply because the data being collected meets all of these criteria does not mean that such data automatically becomes necessary or useful. That determination must be based on the attributes of performance or business operation that drive us to identify the metrics and collect them in the first place.

So in looking at metrics selection, we need to calculate the overall value of a given metric before choosing to use it, and prioritizing metrics once chosen based on how well they meet all of these attributes and provide us the actionable intelligence about our progress or condition.

As I said, metrics are not necessarily useful even if it meets all of the attributes, it's relevance to what we are trying to measure or monitor in relation to the business objective or operational performance we are attempting to measure, will really be the best indicator of that metrics value.

If we take, for example, two metrics that appeared to measure the same thing, then we need to compare them to see if that apparent similarity is really true. In the example that you see on the slide, filling a 20-gallon gas tank and driving it until it is half full, the metrics that may indicate things we need to know might be the number of miles driven down to the half full level, and whether or not the gauge correctly reads out.

If our measurements are in agreement, our overall metrics and measurement of these two will appear to have validated both. When we look at metrics, this sort of corroboration needs to be established so that we can rely on the metrics that we have selected to provide us authentic and accurate information over the conditions being monitored.

Now we move into Section 24, where we will examine the business model for information security as constructed by ISACA. The BMIS defines an approach to security beginning with the business and modeling relationships that the business uses every day, using systems theory.

Another context then, with systems theory, is a complex network of events, relationships, reactions, consequences, technologies, processes, and people that interact in often unseen and unexpected ways. Now we take the system as a complete and functioning unit, more than just a sum of the individual parts that make it up.

This system thinking involved here explains why the whole operates as greater than the sum of its parts. Systems thinking in this regard gives us to think of the system, its components and delve into all of the different interactions that will occur within the aggregated whole.

Looking at this system in this way, it gives us a better understanding of what is actually going on in these interactions within the system taken as this whole construct. Represented that way, it may be easier to relate how security functions within the system and how it meets the various needs that arise that may not otherwise be evident. Here on this slide, we see a graphic depicting the relationship between the BMIS and the COBIT framework.

The first describes the relationships of the components, the system, and the environment in which it operates. And the assurance of the COBIT elements provide the examining of that operation in that context. This interaction will illustrate how to monitor and measure all the relationships within the system and identify changes that occur, and how the indicators reflect that stability will or will not have suffered as a result.

The first element of this system is design and strategy. These must align with your organization, its goals, its strategy, its desired endpoints. The strategy, as we've discussed, is made up of the goals to be reached and the values to be adhered to in the pursuit of these goals.

Once the strategy is constructed, a design will be shaped to show how the organization will actually implement that strategy to achieve the identified goals. This of course will include the various aspects of process, culture, and architecture. The strategy will identify the resources that will be required to accomplish this, and inform the design process that will produce the architecture.

The phrase, people, process, and technology originates from Harold Leavitt's 1964 paper, "Applied Organization Change in Industry". In it, he posits a four-part diamond model for creating change in an organization that involves tasks, structure, technology, and people. These are defined as, structure, how a group of people is organized. Tasks, what people do. People, who they are, and technology, what the people do work with.

Now, people of course are humans, and the security problems they bring with them are part of that. The humans are the ones who say who will implement strategy, require that the security manager worked closely with various departments, and must address these variations through the process of hiring, employment, and termination.

Processes are how to, of have this how to activity, and how it will happen. The basic requirement will be to meet the requirements, stated, adaptable, be well-documented, and undergo periodic review. Technology is the set of tools, applications, and other infrastructure items that facilitate actual work accomplishment.

These are the things that people will use in the processes to do the actual work, but are themselves entered without people to use them. They can be items that are tactical in nature or mechanical in nature, but we should not overlook those aspects of this that are non-technical in nature.

Now, in the setup that we have discussed throughout this module of your preparation seminar, we have examined a number of dynamic and different interactions. We discussed a number of different points that are involved in those dynamic interactions. And we have looked at a number of aspects that can make those interactions more or less secure if not properly adopted.

So let's sum up these discussions in the following way. What we have described is various types of relationships between people, processes, and technologies. We have discussed a number of ways in which these can be employed to accomplish the goals, or if not properly employed, to enable failure or of achievement of these goals.

So we have discussed various types of relationships between these three general components of people, processes, and technology, including governance, which describes the relationship between organization and process. Culture, the relationship between organization and people that populate it. Enablement and support, the relationship between process and technology. Human factors, the relationships between people and technology, and architecture, describing relationships between organization and technology.

From this, we will design an architecture, in which we will put defense in depth as its guiding principle. This defense in depth will be done through multiple layers as exemplified by these five outermost: firewall, watching internet traffic. First inward step, secondary firewall behind the first. Second inward, step encryption of databases. Third inward step, requiring identification and authentication for access to databases, and innermost, passwords being stored in hashed form. And actual defense in depth strategy will of course function in as many layers or as few that the given enterprise may require, that the principle of defense in depth is still followed.

Enterprise defense does not rely on a single layer of protection, but multiple ones, to ensure a spreading of protection, strength, and mutual reinforcement of one layer by others. In describing these dynamic interactions and relationships, we have laid the foundation for a business aligned, operationally relevant, and compliant security structure. And with that, we come to the conclusion of section one, Security Foundations.

This section has been presented as a prelude to prepare you for our in-depth discussion of the CISM four domains. These domains present the concepts, knowledge, experience, and principles that the candidate should have mastery of before embarking on taking the exam.

Please keep track of your questions, as I mentioned earlier. No doubt, you have some, and these, of course, are extremely important that we get them answered. I think you will find that it will be most likely that we will answer these in the upcoming modules. So we're going to pause here a short break and then we're going to begin our in-depth discussion of the CISM and it's four domains.

Thanks for being with us for this first portion of the CISM exam prep seminar, and we look forward to engaging with you again in the next section. So, to be continued.

About the Author
Learning Paths

Mr. Leo has been in Information System for 38 years, and an Information Security professional for over 36 years.  He has worked internationally as a Systems Analyst/Engineer, and as a Security and Privacy Consultant.  His past employers include IBM, St. Luke’s Episcopal Hospital, Computer Sciences Corporation, and Rockwell International.  A NASA contractor for 22 years, from 1998 to 2002 he was Director of Security Engineering and Chief Security Architect for Mission Control at the Johnson Space Center.  From 2002 to 2006 Mr. Leo was the Director of Information Systems, and Chief Information Security Officer for the Managed Care Division of the University of Texas Medical Branch in Galveston, Texas.


Upon attaining his CISSP license in 1997, Mr. Leo joined ISC2 (a professional role) as Chairman of the Curriculum Development Committee, and served in this role until 2004.   During this time, he formulated and directed the effort that produced what became and remains the standard curriculum used to train CISSP candidates worldwide.  He has maintained his professional standards as a professional educator and has since trained and certified nearly 8500 CISSP candidates since 1998, and nearly 2500 in HIPAA compliance certification since 2004.  Mr. leo is an ISC2 Certified Instructor.