What is Testing in Production?

According to Compuware, 48% of users are less likely to use an app again if it does not perform well. 88% of Americans form a negative opinion of a brand if it has a poorly performing app ^[1]. Releasing software with bugs has real-world consequences for businesses. From the loss of money and revenue to the company going out of businesses depending on the severity and impact of the issue. The importance of testing and validating software and delivering bug-free code to users cannot be understated. One of the new tools that software companies have in their validation toolbox is testing in production.

Testing in production is a risk management strategy that is gaining traction thanks to changes in software development and the types of risks developers must contend with. In this white paper, CircleCI CTO (Cheif Technology Officer) Rob Zuber shares valuable insights on what risk management looks like in the modern software development landscape and provides a framework that can be used by companies of all sizes to well-informed and logical choices with regards to their testing.

See also:  Planning for Office 365 Gaps  

What is software risk management?

Any time a software's codebase is updated there is a risk that it will start to behave in a way that was not intended. There are also risks during the development process, the risk of development not getting done in time, increase in the cost of production, etc. Software risk can be broadly classified into two categories: internal and external risks. Internal risks are risks that are somewhat in control of the project manager and external risks are out of the manager's control. Risk management is the process of identifying risks and mitigating them. To mitigate risk in software development companies perform testing and validation before they release code into production i.e add it to the environment where users interact with the software and can be impacted by the updated code. Traditionally all testing and validation were done pre-production. However, these days more organizations are recognizing the value of testing in production.

What is testing in production (TIP)?

Instead of testing software in a staged environment, TIP is the practice of testing code with live user traffic. It is part of the continuous deployment model. It is very difficult, if not impossible to find all software bugs in a pre-production environment. This is because, in pre-testing, developers have to simulate the live production environment through the use of mock data and user-flows. Even after a large amount of time spent in testing, it is possible that testers have overlooked a niche use case or flow. Hence, it is no guarantee that investing a large amount of time in testing and validation will result in 100% bug-free software. Modern software and production systems are complex, there is an increased reliance on third-party software and large databases, hence effectively simulating production environments has become very difficult. This has given rise to the practice of testing in production. By using strategies like feature flags, companies can roll out features and updates to a small section of their user base and test for any bugs. It is important to understand that TIP is not a replacement for conventional pre-production testing and validation, it is another stage in that process.

Download this ebook and let CircleCI CTO, Rob Zuber walk you through what a thorough risk assessment looks like in today’s software landscape. Subscribe to Whitepapers.online for more quality resources by industry experts like this.

Source:

1. n.d., N. Roberts, ' How Software Bugs Impact Your Company ', Global App Testing [available online] available from: https://www.globalapptesting.com/blog/how-bugs-impact-your-company-infographic [accessed Mar 2021]