International DevOps Certification Academy™
Why Do You Need Test Automation In Your DevOps Organization?
If your software delivery department doesn’t have decent test automation yet, then there must be a clear disconnect between your development and testing efforts. Your developers are just getting late feedback about the quality of their work while they are already busy with other tasks. They usually miss the cause and effect relationship between code they have written (or they haven’t written) and defects. The efforts your developers put for this context switch to turn back to times when they introduced defects clearly reduce productivity. Because the connection between systems and defects are complex and fuzzy after late feedback, getting quality fixes without workaround flavour is neither realistic nor expected.
It is very clear that without test automation, your DevOps organization would need more time and money to deliver quality products and services. This is no longer a scalable business model for any technology organization in 21st century. No matter what business your organization is in.
You Can Automate Your Tests,
But You Can’t Automate Creating Quality
Your DevOps team should automate as many test cases as it is possible. Because you can automate your tests, but you can’t automate the cognitive ability of your DevOps team to create quality, you shouldn’t waste your human capital for any repetitive and repeatable work that software can do for you.
DevOps Principles For Good Test Automation
- Test Automation should give quick and early feedback about your quality of work.
- Tests should generate consistent, deterministic and repeatable results provided same conditions for different test runs.
- Tests shouldn’t generate false positives.
- Small number of automated and reliable test cases are better than high number of non-automated, unreliable tests which can give inconsistent results even though same conditions for different test runs are provided.
- Test Automation should first focus on the validation of key features and benefits your systems provide to their internal and external clients. Even this level of test automation is far more better than no test automation.
- Afterwards, focus on automating as many test cases as it is possible to get the best use out of your human capital in your DevOps teams.
- With your test automation, avoid slow and periodic feedback. What you need is fast feedback whenever you or your developer attempts to check-in code to your trunk.
- If your automated tests are not fast enough, find ways to speed your tests and/or your software. This is not only critical to have a reliable continuous delivery platform and deployment pipeline, but also to have a great quality of service for your clients. To get even faster parallelize your automated tests.
- Consider Test Driven Development. First write your automated tests, then build your software.
Test Automation Is Your Major Enabler For Continuous Delivery and Deployment Pipeline
As we have covered before, DevOps methodology gives your developers the ability to deploy their code to your production systems. And yet observations with early DevOps teams in organizations have proven that: Although developers are quick enough to check-in their code to trunk and close tasks and tickets, they are quite reluctant to push the button to release their code to production systems. They are kind of afraid to break production and get blame for it.
Only after DevOps teams have built comprehensive and reliable test coverage with test automation, developers could overcome this fear.
As an example: In Google, when a code is checked-in to trunk by a developer, after initial checks such as static code analysis, duplication analysis and test coverage analysis, this code is validated with almost a million automated test cases. If all checks and test cases pass, the build is generated from trunk, put it to production and replicated to all other production servers. This is pretty much how 5,000 small and independent Google teams stay productive and make ten thousands of production deployment every day. Without test automation this couldn’t be an option.
Fast and reliable feedback is the only way to have a safe deployment pipeline and to protect its integrity. When deployment pipeline is broken due to failed automated test cases, your DevOps team fixes erroneous codes and configuration before they check-in any other work to your trunk. Furthermore, whenever you find an error in your deployment pipeline that could have been identified beforehand and prevented with an automated test case, you also create and add this test case to your test automation repository.
Types of Tests You Can Automate
- Unit Tests: Validate single functions, classes or modules of your applications.
- Acceptance Tests: Validate features and benefits your applications provide to their clients.
- Integration (Service) Tests: Validate interplay of your applications and their end to end service flows.
- Performance and Stress Test: Validate how your services scale (or don’t scale) with expected and unexpected client loads. This is particularly important for the most frequently used services.
- Non-Functional Tests: To validate security, availability, capacity and scalability.
If you and your team find it difficult to automate unit and acceptance tests, but heavily rely on automation of integration (service) tests, probably you have tightly couple architecture. You should identify ways to decouple elements of your architecture.
CONCLUSION
In this chapter we covered the importance of test automation in DevOps. Test automation is key element to deliver safely and fast.
Regardless your organization adopts DevOps or not, without test automation there is no way to have a world class successful, productive and profitable IT organization which manages to serve well to its internal and external clients.
DEVOPS-CERTIFICATION.ORG
