How to make a Performance Test more realistic?

by

Performance Testing of any application or system has two primary objectives:

  1. To find out the capacity of the system at the desired load
  2. To find out the performance bottleneck which restricts the system to achieve the expected load.

On the basis of these two main goals, performance testing tools are designed. One point to be noted here is that all the performance testing tools follow the SEA pattern that means:

  1. Scripting (Design)
  2. Execution (Workload Modelling and Test Execution)
  3. Analysis (Result)

SEA Pattern

The way of implementation of the SEA could be different for the different tools but the fundamental concept will be the same for all. To fulfil the objectives of the performance test, these tools simulate the behaviour of real-world users. The keyword here is a real-world user. The way of scripting and workload modelling plays a vital role to make a real-world performance test scenario. A performance test would be more realistic if it has the below items.

1. Real User Behaviour

In performance testing, the design of the test script reflects the behaviour of real-user. A test script is made up of the steps which a user follows in the real world. A real user cannot be restricted to choose a particular option available on the website. He can click any link, input any character, jump to any tab, take its own time to read the page content, fill in the details etc. To simulate a real user behaviour in performance testing, there are five important aspects which need to keep in the mind while creating a test script: 

  1. Random Navigation: If there are 1000 users accessing a web application, it is not mandatory that all of them using the application for the same purpose. For example, a banking application can be used by the users either to transfer funds, check their balances, add beneficiaries, download statements and many more. Each of these navigations should be recorded as a separate script. One more question may arise here, does a performance tester need to script all the real user flow available on the website? The answer is No; performance tester needs to identify the critical business flow on the basis of expected user load and responsiveness. For example, 80% of users check their bank balance after login irrespective of purpose they are logged-in. Hence it is a critical business transaction because the request of account balance enquiry is hitting more than other requests. Another example of critical business transaction in terms of responsiveness is that an intrabank fund transfer should be completed within 3 seconds.
  2. Random Input: An application can have thousands of visitors or registered customers. Different people log in with different user name and passwords, search for different products, enter different shipping and permanent addresses etc. To simulate this behaviour, parameterization helps a lot in the test script. The parameter file should have a list of users and associated test data during the load test. In an ideal scenario, the number of unique test data should be equal to or more than the expected number of iteration in the test.
  3. Random Selections: All the customers on a website do not have the same navigation flow. Once the users enter into the application, their selections are different (from a dropdown, radio buttons, checkboxes etc). For example, if two users are searching for a flight from Mumbai to New Delhi and a list of flights appears then in the real-world situation they are free to choose any flight from the given list. The selection will be as per their convenience. To simulate this aspect of real user behaviour, correlation is helpful. You can capture all the suitable values via correlation or regex and pass a random value from the captured values. 
  4. Random Think Time: It is a normal behaviour of a real user to stay on a web page for some time to read the content, fill in the information etc. This behaviour leads to a delay between two successive requests in the test script. For example, the real user launches an online retail application (Home Page Transaction) and then he enters the username and password and clicks on login (Login Transaction). So the user has to stay at the homepage until he inserts the credentials and clicks the login button. This staying time denotes User Think Time. A user does the same behaviour on all the pages. The only difference is the type of activity will be different. But, surely he will spend a significant time on each web-page. This think time is not constant for all the users. Because in the real world each user has its own typing or reading speed. A random think time adds a realness to the script. 
  5. Browser Simulation: Performance Testing tools do not have real browser, although they simulate a real browser by adding broswer details in request header, by enabling cache and storing cookies etc. A real web browser has cache feature to reduce the traffic over the Internet. It also reduces the usage of network bandwidth that makes faster communication between clients and servers. Similarly, cookies are text files. The client computer stores cookies with various client’s information like search interest. By default, each request is considered as a new request. Once a cookie is added to the response from the servlet and it is stored in the cache of the browser then it is sent along with the next request. Thus, the server recognizes the user as the old user. These options can be enabled or disabled as per end user behaviour or application configuration to simulate real browser in the script.

2. Real World Load

After the completion of the script by considering the real user behaviour, another important aspect of a realistic performance test is to simulate the real-world load patterns. The real-world scenario preparation comes under workload modelling (or execution) which includes:

  1. Transaction Rate: Identify the user load in peak and average hours and prepare the scenarios which simulate the real-world load. As per requirement load, stress and other performance tests can be conducted.
  2. Geographical Location: Internet has been spread all over the world. It is good practice to identify the geographical location from where high traffic is coming on the website. In case 60% to 80% traffic reaches the application from a particular region then deploy the Load Generators to that locations or use a WAN simulator to simulate the users.
  3. Bandwidth: Though few users are from the same location, there are possibilities that their Bandwidths are different. Keep this point in the mind while generating the user load on to the system. Various performance testing tool has an option to set the required bandwidth during the test execution.
  4. Transaction Type: Each business transaction has its own percentage of accessibility rate (refer to the previous example). Hence identify the load percentage of each business functionality and design the workload accordingly.

Performance Testing is the only testing where the objective of the testing is fulfiled in an artistic way. An art of simulating real-world scenario, an art of simulating real-user behaviour, an art of analysing the results, an art of bottleneck finding and at last, an art of giving the assurance to the client that the application will not fail due to performance issue.

You may be interested:

Leave a Comment