- [Instructor] Welcome back. In this lecture, we'll review continuous testing and how it should be used to ensure readiness and quality within your DevOps projects. 

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. When building software, it's important to know whether or not the end product as designed works as expected. In past times, this was the role of a software tester, someone who manually tested the product as if they were the end user. This wasn't a problem when there was one release at the end of each month. But nowadays with builds producing releases every hour, manual testing no longer scales. Additionally, it didn't take much for a misstep to occur as the tester followed the testing manual, resulting in a defect being propagated into production. Applications have also become much more distributed with many types of internal components that require other types of testing, not just tests performed from the front end of the application. In order to address these requirements, the concept of continuous testing was proposed as a method of executing automated testing, often and throughout the entire build and deployment process, producing faster feedback that developers could address. Continuous testing builds on test automation, helping businesses mitigate risks associated with high-velocity release schedules. 

Adopting continuous testing requires you to first identify key quality objectives for your software application. Additionally, consider any potential business risks should your application malfunction and what would be required to minimize or indeed eliminate these risks. Also, consider aligning your testing objectives to known business goals. From here, define a test automation flow and embed it within your build and deploy pipeline, making sure that your feedback channels to the development team are in place for every important stage of the pipeline. Culturally speaking, we need to ensure that the testing process is part of everyone's mindset, adopting a shift-lift mindset to testing to test early and fix early. Next, we identify and implement the types of automated tests we want to perform, such as unit tests, performance tests, functional tests, security tests, workflow tests, and/or end-to-end tests to name but a few. We develop these tests and store them alongside our application code base in our version control system. Finally, we take our different types of tests and combine them into an automation test suite or suites and configure them to execute within the desired stages within our centralized build and deployment pipelines. 

The benefits attributed to continuous testing are we're able to deploy new features faster and with less risk. Provides actionable feedback appropriate for each stage of the delivery pipeline. It reduced the time and effort needed to be spent finding and fixing defects. And it establishes a safety net that protects the end user. 

Typical challenges associated with using continuous testing to automate the execution of tests are test dependencies. Modern applications are often highly distributed, and this can create challenges when dependencies exist between components being tested. You have front end cost. Upfront investment is required and returns are not immediately obvious to business stakeholders. Test maintenance. The amount of frequent change in the code base due to new features being built causes increased test maintenance. Execution time. Application complexity can lead to large test suites, which in turn can be time-consuming to run, slowing down feedback to developers. Environment stability. Test environments undergoing constant change cause tests to fail and/or provide false positives and/or timeouts, et cetera. 

The following list of continuous testing tools and/or frameworks can be used to provide solutions to the various types of testing. It is by no means an exhaustive list. Selenium, an opensource browser automation framework and ecosystem. JUnit, a unit-testing framework for the Java programming language. Karma, a test runner for JavaScript that runs on Node.js. PhantomJS, a scripted, headless browser used for automating webpage interaction. Jasmine, an opensource testing framework for JavaScript. Gauntlt, a ruggedization framework that can be used to do security and rugged testing. Serverspec, an infrastructure-testing framework that works over protocols such as SSH and WinRM. JMeter, an Apache project that can be used as a load-testing tool for analyzing and measuring the performance of applications. And Cucumber, used to run automated acceptance tests written in behavior-driven development style. 

In the following continuous testing example, we have a centralized continuous integration server, which automates a number of test suites at various stages of our build and deploy pipeline. In this case, our continuous integration server first executes unit tests authored using JUnit. Next, Cucumber is used to execute integration tests using a set of behavior-driven development stories to confirm the overall functional aspects of the application. PhantomJS is used to execute Selenium tests in a headless browser to confirm overall end-user functionality. Finally, we use an infrastructure-as-code process to provision the infrastructure that the build runs on. And we test is using Serverspec to confirm the end-state and Gauntlt to security test it. 

Taking the previous example and viewing it from the perspective of a sequence diagram, we can see that as each set of automated tests complete, regardless of whether they pass or fail, feedback is always provided back to the developers. This accommodates early detection and resolution. 

For continuous testing to be effective, you'll need to ensure that you have the right types of automated tests included within your build and deployment pipeline and that they execute at the right stages, providing early feedback. The quality and readiness of your application will be entirely dependent on the quality of tests and the coverage the tests provide. Better testing will directly result in a better application and user experience of it. 

Okay, that completes this lecture on continuous testing. Go ahead and close this lecture, and we'll see you shortly in the next one.