JMeter Script Generation through AI

by

AI is getting better in many areas, so why hasn’t it been used much in performance testing? Creating performance test scripts with AI isn’t very hard. The real challenge is instructing AI, exactly what you need, so it gives you the right results. This is where prompt engineering becomes important.

In this article, we’ll show how a beginner gives instructions to AI and what kind of test scripts the AI creates. We’ll also look at what’s missing in those instructions and how to improve them.

Let’s start, so we have

Website: https://petstore.octoperf.com/actions/Catalog.action

AI Tool: Chat GPT

Scripting for Tool: JMeter 5.6.3

Script Type: Basic (To show how you instruct AI in layman’s terms and what it generates).

Instructions to AI for JMeter Script Generation:

  1. Write a JMeter script.
  2. JMeter Version: 5.6.3
  3. Host URL: https://petstore.octoperf.com/actions/Catalog.action
  4. Navigation Steps:
    • Home Page (https://petstore.octoperf.com/actions/Catalog.action)
    • Enter username ‘testPerf’
    • Enter password ‘testPerf’
    • Click Login
    • Click on Fish (First option in the left menu)
    • Click on ‘FI-SW-01’
    • Click on ‘EST-1’
    • Click on ‘Add to Cart’
    • Click on ‘Proceed to Checkout’
    • Click on ‘Proceed’
    • Click on ‘Confirm’
    • This message should appear ‘Thank you, your order has been submitted.’

AI Response:

Summary of the output

JMeter Script Generation through AI

JMeter Script Content, which you can copy into a file and save the file with the .jmx extension.

JMeter Script Generation through AI

Instructions for use

By following the above instructions, how does the JMeter script look like?

You can download the script from the Google Drive location.

JMeter Script Generation through AI

Do you consider this script perfect for a test run? Your answer will be ‘No’. So, let’s analyse the script:

Test Plan

JMeter Script Generation through AI
  1. AI initially names the ‘Test Plan’ element as ‘Petstore Order Test Plan’. However, the correct name of the application is JPetStore. Therefore, include a prompt that instructs AI to name the Test Plan based on the test case name.
  2. The application URL is stored in a variable named BASE_URL, which includes the https protocol. This causes issues when referencing the variable in the ‘Server Name or IP’ field of the HTTP Sampler. To address this, AI introduces a workaround logic (to be explained later).
  3. The Test Plan currently defines the variables, making them global across all Thread Groups. If multiple Thread Groups target different applications, this can lead to conflicts when the host URL changes. To avoid this, include a prompt directing AI to define all variables within a User Defined Variables element inside each Thread Group.
  4. Additionally, to make the script more flexible across both http and https environments, a separate variable for the protocol should be introduced. Add a prompt for AI to define this variable explicitly.

HTTP Request Defaults

  1. AI attempts to extract only the domain part for the ‘Server Name or IP’ field by splitting the full URL and removing the protocol (http or https).
  2. It does this by using the expression ${__BeanShell(${BASE_URL}.split(“//”)[1].split(“/”)[0])}. However, this is logically incorrect because the BeanShell interpreter treats the unquoted URL as a literal Java expression, leading to a syntax error. The correct syntax should be: ${__BeanShell(“${BASE_URL}”.split(“//”)[1].split(“/”)[0])}.
  3. To avoid such errors and improve reliability, include a prompt explicitly instructing AI not to use URL-splitting logic for extracting the domain. Instead, define the protocol, domain, and path as separate variables from the start.

HTTP Cookie Manager

JMeter Script Generation through AI
  1. If you want to clear cookies on each iteration, include a prompt instructing AI to enable the “Clear cookies each iteration” option.

Thread Group

JMeter Script Generation through AI
  1. The Thread Group has been renamed to ‘Use Thread Group’, but ideally, it should reflect the test case name for clarity and consistency.
  2. The rest will be the default thread group setting unless you explicitly instruct AI.

HTTP Sampler

  1. GET and POST requests are captured correctly.
  2. To include parameters in requests, you must either provide explicit instructions in a prompt or add them manually as needed later.
  3. Prompts are also required to add specific assertions for certain pages, such as what was done for the ‘Confirm Order’ page.
  4. The HTTP Header Manager is currently missing; therefore, a prompt should be added to ensure it is included when necessary.

Response Assertion

  1. AI correctly adds generic response assertions.
  2. For custom or specific assertions, you must provide precise instructions to guide the assertion logic.

Execution

After generating the Row JMeter script through AI, it was observed that no listener was included. Therefore, the first step is to add a listener – preferably the “View Results Tree”- before executing the script. Once added, run the script to observe the outcomes.

Initially, all requests failed due to incorrect logic used for splitting the application URL. This needs to be corrected by updating the expression in the HTTP Request Defaults. So, replace ${__BeanShell(${BASE_URL}.split(“//”)[1].split(“/”)[0])} by ${__BeanShell(“${BASE_URL}”.split(“//”)[1].split(“/”)[0])} in HTTP Request Defaults.

After making this change, rerun the script. The requests will start to pass; however, the login event may still not occur.

To address this, compare the request with the Chrome DevTools output and ensure all required fields are included in the request.

Once these modifications are done, the script is executed successfully.

The updated script is now available at the Google Drive location.

Conclusion

Using AI to generate JMeter scripts is a big step in making performance testing easier and faster. While AI can help create scripts quickly, the output quality depends on how clearly and correctly you give instructions. This is why learning how to write good prompts—called prompt engineering—is so important.

As we’ve seen, even beginners can get useful results from AI tools like ChatGPT. With practice and better prompts, the results can become even more accurate and closer to what you need. AI won’t replace performance testers, but it can be a helpful assistant in the scripting process.

In a future article, we’ll show you how writing accurate and detailed prompts can help generate much better JMeter scripts. This will help you get the most out of AI in your performance testing tasks.

The future of performance testing will likely include more AI support, making it easier for teams to test faster and focus more on analysis and improvements.

You may be interested: