What is a Soak Test or Endurance Test?
YouTube Video: Soak (Endurance) Test
Soak Test is a type of non-functional test which helps to identify the memory leakage into the software system. Another name of the Soak Test is ‘Endurance Test’. In the soak test, a significant load is applied on the server for an extended period of time. Generally, the test duration of the soak test is in between 8 to 24 hours, but it may vary as per project requirement. A longer period performance test with an average load provides information about the behaviour of Garbage collector and memory management.
How to calculate the Average Load?
The business analyst calculates the average load by referring to historical data. He may consider past days, weeks, months or years statistics to get the average number. He calculates the figure by using some mathematical formulae. To understand the average load calculation refers to the below graph. It is a typical graph; not a practical one:
Above graph shows the historical data of a month. On calculating the average of the given figures a number comes which is 100. Hence 100 is the average user load for the soak test.
In some cases, if data is not available for the calculation then 50%, 60% or 75% of peak load can be considered for the test. But before conducting the test get the confirmation on these volumes from the project team.
For a new application: Since a new application does not have any historical data, so the above-suggested % of peak load can be used for soak testing with proper approval from the project team. The purpose of the soak test does not change for a new or an existing application.
Purpose of Soak Test:
- Verify the sustainability of the application
- Check if there is any spike in the response time
- Identify the memory leakage
- Check the behaviour of Garbage Collector
- Identify if there is any bottleneck
- Check the type of error due to the prolonged duration of the test and get the error percentage
- Check whether the resources (CPU, Memory and Disk) do not breach the defined performance limit
Approach:
Many clients are not aware of the advantages of the Soak test. Hence they skip this test and do not provide any specific requirement for a soak test. Soak Test is one of the important performance tests and helps to detect memory leakage. NFR document must have a separate set of NFRs for the endurance test. These NFRs are related to the average user load count, response time, transactions per second etc. In the absence of an average load, conduct the test at 50%, 60% or 75% of peak load and analyse the behaviour of the application.
A performance tester designs the longer duration workload model using defined or calculated NFRs and executes the test. Ideally, the duration of the endurance test is 8 to 24 hours (excluding ramp-up and ramp-down period). A typical endurance test user graph is:
This endurance test graph has a steady state of 23 hours along with 30 minutes ramp-up and 30 minutes ramp-down period. Therefore the test will run for 24 hours. After completion of the test, a performance tester verifies the result against the defined load test NFRs.
It is recommended to plan at least 2 soak tests in a single performance testing cycle and if both the tests have consistent results then only jump on to next type of performance test.
You may be interested: