Failure Mode Analysis Pt. 2


CSSLP Domain 6:2

The course is part of this learning path

Start course

This course looks at the second section of Domain 6 of the CSSLP certification and covers pre-release activities which include implementing the actual testing process, the actual conductive test, and the variations of the test that will be employed at this stage.

Learning Objectives

  • Understand the pre-release activities to carry out before launching software
  • Understand the pre-release testing process

Intended Audience

This course is intended for anyone looking to develop secure software as well as those studying for the CSSLP certification.


Any experience relating to information security would be advantageous, but not essential. All topics discussed are thoroughly explained and presented in a way allowing the information to be absorbed by everyone, regardless of experience within the security field.


Now, continuing on talking about our test program implementation and the various types of tests we want to use, let's talk about scalability testing. Now this is an augmentation of performance testing. Its main objectives are to identify the loads which can be obtained from load testing and to mitigate any bottlenecks that will hinder the ability of the software to scale, to handle more load or changes in the business processes or the technology.

Performance test baseline results are usually used in testing for the effectiveness of scalability. Degraded performance upon scaling implies the presence of some bottleneck that will need to be addressed either tuned or eliminated or possibly marked as an exception and thus committed to a corrective action plan.

Our next form of testing is usability testing. Now this is a general testing approach that examines user ergonomics or as we say, user friendliness. This tends to be a very subjective test, but can play an important role in terms of gauging user acceptance likelihood or concerns. Now, as we move further through our test program implementation, we come to the alpha test. Now, this is a type of acceptance testing. It is performed to identify all probable issues and bugs before releasing the final product to the end users.

Alpha testing is carried out by the testers who typically are internal employees of the organization. Now the main goal here is to identify the tasks that a typical user might perform and test those. Although it is not as common as its brother method, alpha testing is still an important part of easing the product into the real world of use. This sort of testing is typically done by end users and others but definitely not by the programmers or tactical testers. This testing takes place when a product is nearing delivery.

It is understood that minor design changes will still be made as a result of alpha testing. Thus, the product is not widely circulated during an alpha test. Now, some specifics about alpha testing include that reliability and security testing are not typically part of an in-depth alpha test. Alpha will typically involve both white box and black box techniques. Sometimes it is a typical experience that alpha testing will involve a fairly long execution cycle.

Now as mentioned, critical issues or fixes can be addressed by developers immediately in alpha testing. And it's meant to ensure that the quality of the product is there before it moves on to beta testing. Now, beta testing is by far the most common method of pre-release testing for any new product in the real world. Beta testing takes place when development and testing are essentially completed and final bugs and problems need to be found out before the final release. It is typically done by end users or others, but again, not by the programmers or testers themselves.

To be maximally effective though, there has to be a direct reporting link that hooks the testers to the people who are responsible for the final polishing prior to release. Now, throughout all of this testing, there are of course tools that must be used to do various things in the way of exercising the program, it's code and it's functions.

Now, as mentioned before, it is not important for the CSSLP to have a thorough understanding of how each security tool is used. That is there is no need for them to show that they have expertise in all the tools that are possible. This is the case at least with regard to the examination itself. But they must show that they are familiar with what the tool can be used for and how they can impact the overall state of the software's security.

Now, knowing the specifics of a tool's operation is of course important if the tester is to be successful in performance and attain the results that are accurate and reflective of the target's state of readiness. No tools should therefore be used by a novice without training and the supervision of someone who is experienced in its use. Now here we see a list of the types of tools that are typically used. Some are used for reconnaissance, that is information gathering, and then a fairly typical array, vulnerability scanners, fingerprinting tools, sniffers or protocol analyzers, password crackers, various forms of web security tools, wireless, if that's part of the program, reverse engineering tools, including assemblers, disassemblers, buggers, decompilers and the rest, source code analyzers, vulnerability exploitation tools, such as Metasploit, security-oriented operating systems in the environment and whether there are any vulnerabilities there that interact with the program itself and privacy tools.

About the Author
Learning Paths

Mr. Leo has been in Information System for 38 years, and an Information Security professional for over 36 years.  He has worked internationally as a Systems Analyst/Engineer, and as a Security and Privacy Consultant.  His past employers include IBM, St. Luke’s Episcopal Hospital, Computer Sciences Corporation, and Rockwell International.  A NASA contractor for 22 years, from 1998 to 2002 he was Director of Security Engineering and Chief Security Architect for Mission Control at the Johnson Space Center.  From 2002 to 2006 Mr. Leo was the Director of Information Systems, and Chief Information Security Officer for the Managed Care Division of the University of Texas Medical Branch in Galveston, Texas.


Upon attaining his CISSP license in 1997, Mr. Leo joined ISC2 (a professional role) as Chairman of the Curriculum Development Committee, and served in this role until 2004.   During this time, he formulated and directed the effort that produced what became and remains the standard curriculum used to train CISSP candidates worldwide.  He has maintained his professional standards as a professional educator and has since trained and certified nearly 8500 CISSP candidates since 1998, and nearly 2500 in HIPAA compliance certification since 2004.  Mr. leo is an ISC2 Certified Instructor.

Covered Topics