Little’s Law in Performance Testing -

Problem Statement

To simulate the real-world scenario in Performance Testing, a performance tester needs to develop a workload model which comprises user load, the request rate sent per hour and the response time of the page. These three metrics are adjusted by Little’s law so that the desired load can be generated on the server. Many times, when a performance tester uses Little’s law (as it states) to build an accurate workload model then he finds some unexpected results and after that, he needs to do manually some reverse or complex calculations to generate the desired load on the server. As a solution to this problem, this article covers:

What mistake did you do while calculating the workload model metrics using Little’s law?
How to correct the mistake?
What is the new (extended) form of Little’s law which will calculate the correct workload without any manual (on-paper) calculation?

Problem Description

Performance testing is used to measure the performance of an application or a system in terms of user load handling capacity, sustainability and responsiveness of the server. In today’s world, every client is keen on the user experience of the application. As per the study if the website page responds in more than 10 seconds then the user either terminates the page or moves out which directly impacts the revenue of the company. To meet user expectations, companies are deeply concentrating on responsiveness along with the user load-bearing capacity of the website.

Before moving ahead let’s understand the terms used in this article:

User’s load: The load applied to the server during testing in terms of virtual users.
Response Time: The time taken by the system to respond to the user request (input). It also includes network latency.
User Journey (or Iteration): The user navigation on the website like Home Page -> Login -> Search Product -> Order Product -> logout
Transaction: Refer to user activity on a page.
Think Time: The time taken by a real user on a webpage before submitting the request.
Pacing: The delay time added by a performance tester between two journeys of users to achieve the defined load.

In 1954, John Little published a paper in which he described the queueing theory by a new law. Later on, it is named Little’s Law. He formulated 3 basic terms: the average number of customers, customer arrival rate and the average time spent by a customer in the system.

Little’s Law Statement: The long-term average number L of customers in a stationary system is equal to the long-term average effective arrival rate λ multiplied by the average time W that a customer spends in the system. Expressed algebraically the law is:

L = λ * W

The same formula was accepted in the performance testing to simulate the real-world load in the testing environment. The above formula in the performance testing term relates 3 basic metrics of performance testing which are the number of users, transaction rate and response time. Hence,

Little’s Law in Performance Testing Term: The number of users (U) active on an application is equal to the rate of transactions (T) sent by the users multiplied by the average response time (R).

So, the formula becomes:

U = T * R

User Load = Transactions per second * Response Time

Extension of Little’s Law for correct workload modelling in Performance Testing

The above-mentioned formula based on Little’s law looks simple so what are the issues faced by a performance tester during test scenario creation and what is the need to make this formula more simplified and accurate?

Let’s consider an example to understand the missing points. A client has given the requirement of 100 users which hits the servers at the rate of 5 transactions per second and the expected response time for each transaction is 2 seconds. Now, put these figures into the above formula. We have 100 as a user load, 5 as transactions per second and 2 seconds as a response time. Hence,

=> 100 = 5 X 2
=> 100 = 10 ………. (1)

Equation (1) does not satisfy the relationship between the metrics. Then what is missing?

To make the scenario realistic, a performance tester adds think time. As described above Think Time is the time taken by a real user to provide the input on a page. It includes the time required by a user to read the content of the page or fill out the form or select a value from the given list etc. Let’s say 5 seconds is the think time that a user takes to read the content of each page and provides input in the form of clicking the link or submitting the details. Since there are 5 pages (Home Page, Login Page, Search Product Page, Product Order Page and Logout Page), the total Think Time will be 20 (= (5-1) X 5). The last page is a logout page which does not require Think Time. Hence 4 (= 5 -1) pages are considered. A point to be noted here is Think Time will also be considered as response time, so as per Little’s law the equation becomes

=> 100 = 5 X (2 + ((5-1) X 5))
=> 100 = 5 X (2 + (4 X 5))
=> 100 = 5 X (2 + 20)
=> 100 = 5 X 22
=> 100 = 110 ………. (2)

Still, something is missing which is restricted to satisfy the equation. Another key value which is called Pacing (delay between two journeys of a user) is used by a performance tester to achieve the desired TPS. Let’s consider 70 seconds as the pacing time which will also be included in the response time, so the equation becomes

=> 100 = 5 X (2 + 20 + 70)
=> 100 = 5 X 92
=> 100 = 460 ………. (3)

Again, LHS is not equal to RHS and to equalize the equation 460 user load will be required which is not as per the client’s requirement. But after considering think time and pacing, the modified formula based on Little’s law becomes

U = T * (R + TT + P)

User Load=Transactions per second * (Response Time + Total Think Time + Pacing)

This is a basic formula which a performance tester used for workload modelling, but when the result comes then they identify that the client’s expectation does not meet or more than expected load is generated on the server due to the wrong calculation. In such cases, they are bound to calculate the figures manually.

The reason behind the wrong calculation

After conducting my research on 50+ samples, I found that in Little’s law, the average arrival rate and the waiting time are related to a person who is waiting in the queue. But this is not the case when performance testing metrics are calculated. When waiting time is referred then this is not the time of a single transaction, but that is the total time of user stayed on the website and complete his end-to-end journey i.e. from login to logout.

Resolution

In the above example, a user navigates 5 pages (or transactions) during his end-to-end journey. As per requirement, 2 seconds is the response time of each page. Hence the total response time throughout the journey is the sum of the response time of all the pages (= 2 + 2 + 2 + 2 + 2) i.e. 10 seconds. Now, the extended formula becomes

User Load = Transactions per second x [Overall Response Time + Total Think Time + Pacing)]

=> 100 = 5 X [(2 + 2 + 2 + 2 + 2) + (4 X 5) + 70]
=> 100 = 5 X [10 + 20 + 70]
=> 100 = 5 X [100]
=> 100 = 500 ………. (4)

Here, another finding is that the given transaction rate belongs to 1 page, but the response time metric belongs to the end-to-end journey (also called as iteration). Hence transaction per second should be divided by the number of pages to convert it into iterations per second. Hence:

=> 100 = (5/5) X [(2 + 2 + 2 + 2 + 2) + (4 X 5) + 70]
=> 100 = 1 X [10 + 20 + 70]
=> 100 = 1 X [100]
=> 100 = 100 ………. (5)

Hence the modified formula for workload modelling in Performance Testing based on Little’s Law is:

User Load = (Transactions per second/Number of pages or requests) * (Overall Response Time + Total Think Time + Pacing)

Where was the problem?

As per Little’s Law, only 3 metrics satisfy the formula of queuing theory, but in performance testing, 6 metrics are used to simulate the real-world situation and satisfy the equation.
Think time and pacing were not considered and hence equation (1) was false.
Total Think time had been added to the response time, but the pacing was missing in equation (2).
In equation (3), the total think time and pacing are added with the response time of a single transaction (not overall response time) which makes equation (3) wrong.
In equation (4), iteration metrics (total think time, pacing and total response time) are multiplied by transaction metric (transaction rate) and hence RHS value is more than the LHS value and is unable to satisfy the equation.
Iteration metrics (total think time, pacing and total response time) are multiplied by another Iteration metric (transaction rate/no. of transactions) and hence equation (5) is true and derived a new formula.

The benefit of Little’s Law Extension

As per Performance Testing standards, the number of expected users, transaction rate and response time is provided by Client (Project) during requirement gathering. These metrics are called as Non-Functional requirements. The number of transactions figure comes up once the script is finalized. The think time is an arbitrary value which depends on the nature of the page. In the practical scenario, 5 seconds think time is considered as a standard value. So, the extended formula is used to calculate the Pacing. Pacing is an important metric which can be adjusted to simulate an accurate workload model. The wrong pacing calculation directly impacts the expected user load and transaction rate. Therefore the true performance of the website cannot be identified. The newly derived formula provides the following benefits over the restriction of Little’s law:

Calculating the correct pacing
Achieving the desired workload
No manual on-paper calculation is required

Conclusion

Little’s law can be used for the queuing system, but in performance testing, other metrics also impact the equation and hence base formula does not calculate the correct figures for the workload model. Many performance testers still use manual calculations on paper because they do not know the mathematical logic behind the law. Also, they are afraid of not meeting the SLA by calculating the figures using the original Little’s Law.

I did my research on this and collected 50+ samples to find out the problem and at last able to identify the above-mentioned loopholes which were the root causes of the miscalculation and came up with a new formula. The extended form of Little’s Law will help all the performance testers to calculate the correct metrics in just a few seconds.