Here's how automated testing maintains a fast and accurate pace for 5G

Over the last few years, I’ve had the chance to discuss the challenges of 5G with some of the world’s leading operators.

A common theme today is developing and deploying new software with the goal to provide new innovative services to the end users. However, despite the market demanding shorter time to market, operators, in reality, face longer deployment cycles due to 5G’s complexity and the varying level of existing automation within delivery.

That’s why DevOps, or its telco customized version in Nokia DelOps (Delivery and Operations), is the norm for reducing the time and effort to create software, and CI/CD (Continuous Integration, Continuous Delivery) is the enabler to deliver the promised benefits. Continuous, automated testing is a crucial component of CI/CD success.

Continuous, automated testing maintains the pace

We all know how important software testing is and have experienced how painful and time-consuming testing execution can be. Conventionally, in the so-called waterfall method, software is tested in a single phase at the end of a lengthy development cycle. With the emergence of CI/CD and its process of continuous, rapid development, integration and delivery, automated testing was casted an essential role.

While testing is performed throughout the entire pipeline, let us take a closer look at acceptance level testing, which is performed towards the end of deployment, but can be used in live networks as well. Our automated acceptance level test cases are structured into four crucial modules.

1. The first module, Lifecycle management validation, is making sure that all the network elements in the system are up and running, well connected and the necessary configuration is applied.
2. The second step, Functional regression testing examines how the software works in the network. Although this is usually done through simulators in the earlier phases of the CI/CD pipeline, Nokia performs Functional regression testing using real devices, making it much more realistic. This testing module ensures that not only does the software work as specified, for example performing a normal call or a conference call, but also that it will not compromise the existing network. Unlike manual methods, automation offers increased speed of testing, allowing us to cover the entire regression test plan.  
3. Once we have made sure, that the system is functionally correct, we enter the Capacity benchmarking stage, ensuring that the system’s capability can handle mass traffic. We use industry standard simulators for the access interface and compare over 27000 KPIs to the previous executions and the dimensioning specifications. During the R&D System verification, we are typically using a 500 kBHCA load to the system, which is already providing a good regression load on system level. Individual products though are typically loaded even more, for example 2million BHCA and above.
4. After we know that the system can handle the defined capacity, we start simulating failure scenarios that operators may encounter in their production environments to ensure the solution’s robustness. In this Robustness testing module, we are provoking the system with application and infrastructure level failures and restarts to verify disaster recovery. We also introduce infrastructure and cloud level maintenance actions to help guarantee service continuity.

The above four modules constitute the entire automated solution validation cycle. Such automated testing is fast, taking hours instead of days and weeks, allowing us to validate that the software is performing safely and reliably. At the end of the test cycle detailed automated verification reports are also generated. In short, we can be sure the update is ready to be deployed. 

Faster, cheaper and better

These automated methods accelerate testing dramatically, significantly reduce the time to validate new software, produce valuable cost savings and improve reliability. Manual testing often leads to faulty software being introduced into the network, potentially damaging the customer experience. Automation cuts human error, giving more consistent results.

Another major benefit is that Nokia provides test automation ‘as a service’, cutting the need for operators to invest in their own testing tools.

Continuous, automated testing is a vital part of the CI/CD pipeline, delivering the agility and pace needed to compete in the 5G world. To learn more, please watch our short video, where we demonstrate how we execute a complete system level validation cycle, which can be used for acceptance testing both in pre-production and production environments.