JMeter is an open-source performance testing tool, but in today’s world, it is competing to paid licensed tools with its improved features. One of the main drawbacks in the old version of JMeter was reporting and test result analysis. There were very few listeners available which took time to carry out a proper analysis. After completing the analysis, performance tester had to create a manual test report which was again a time-consuming task.
JMeter 3.0 onwards, you can find a new reporting feature in JMeter called Dashboard Report. This is an HTML report which has a summary dashboard including all the basic graphs. JMeter Dashboard Report solves the reporting problem in JMeter. Now, let’s have a look at how you can generate a correct JMeter HTML Dashboard report?
While creating a Test Script for a Test Scenario in JMeter, the best practice is to keep the group of samplers belong to a particular user action inside a Transaction Controller. By default, when you record a JMeter script there is an option to wrap the related samplers into a Transaction Controller. Once the recording is completed then perform “Apply Naming Policy” action to all the samplers inside each Transaction Controllers. Without proper naming of the samplers, you may face challenges to read JMeter HTML Dashboard Report.
Some Important Points before proceeding:
- It is mandatory as well as recommended to uncheck the below options of all the Transaction Controllers.
- Generate parent sampler
- Include duration of timer and pre-post processors in generated sample
By default they are unchecked.
- It is mandatory as well as recommended to perform “Apply Naming Policy” action for each group of samplers inside the respective Transaction Controllers. This can be done by “right-clicking” on each Transaction Controllers.
- Always, remember to restart JMeter on modifying any of the .properties, .bat or .lib file or adding a new plug-in in JMeter folder.
- Disable the unwanted elements of the script like the debug sampler, listeners etc. before running a test in GUI or Non-GUI mode.
- Use “Backslash” (i.e. “/ “) while giving File path or Directory path. JMeter recognises the forward-slash (i.e. ” \ “) as Escape Character.
1. If you select the option “Generate parent sample” in Transaction Controller and run the test in Non-GUI mode then you will not see the Test Summary. Test will execute, but Test Summary will not generate.
2. Furthermore, the CSV (or JTL) file will be having some missing information on some samplers and some Graphs/Tables will be empty or will not have enough details in the HTML Report, examples:
a. Top Errors
b. Top Errors by Samplers
c. Response code over time
JMeter v5.1.1 onwards, there are no such pre-defined changes that need to be applied or remembered for HTML Report generation. If you have JMeter older than v5.1.1 then follow the pre-requisite mention in this post (Link).
This post shows the generation of the Dashboard Report through JMeter v5.1.1 and JPetStore.jmx. The illustration goes as:
Generation of Dashboard Report in Non-GUI mode
Step 1: Goto to “bin” folder of JMeter and open “user.properties” file using Notepad or Notepad++. Then go to the section “Reporting configuration” in the file. Make the changes as mentioned below:
Default values were:
Modified values are:
Note: Apart from the above-suggested changes all other settings will be the same.
Explanation of the changes:
- The property “jmeter.reportgenerator.overall_granularity” shows the granularity of the graph. This value should never be less than 1 Sec (or 1000 ms). For a short duration test, it is good to keep minimum granularity value i.e. 1000 (denotes 1000 ms or 1 Sec), so that graphs and errors are visible correctly. In case, if you run an endurance test which has longer duration then you may keep it 60000. The default value of this property is 60000 (means 1 minute).
- “jmeter.reportgenerator.apdex_satisfied_threshold” represents the response time NFR which is applicable to all the sampler. The response time above the satisfied threshold value is considered as NFR breached.
- “jmeter.reportgenerator.apdex_tolerated_threshold” represents the bearable maximum response time. Ideally, both satisfied and tolerated threshold values should come from the client or taken as a benchmark value before conducting the test.
Example: A client has provided 3 seconds as response time NFR and 5 seconds as the maximum response time NFR, so 3 seconds becomes satisfied threshold value and 5 seconds become tolerated threshold value.
- On the basis of satisfied and tolerated threshold APDEX (Application Performance Index) is calculated.
- The property “jmeter.reportgenerator.exporter.html.series_filter” represents the name of transactions which you want to show in the Dashboard Report. Here, you have to give the name of all the Transactions Controllers under your Test Plan. The easiest way to get the Transaction Controller names from Test Plan is JMeter Menu -> Tools -> Export transactions for report.
Now, copy the whole lines and replace the value of
- “jmeter.reportgenerator.exporter.html.series_filter” property in the file. In addition, if you do not want to include some of the Transaction Controllers in the report then remove them from the line. The illustration goes as:
In addition, you can modify a few more properties of “user.properties” file under “Reporting configuration” as per your requirement but they are optional:
To show an appropriate name of JMeter HTML Dashboard Report, update the value of below property:
jmeter.reportgenerator.report_title=JPetStore JMeter Dashboard Report
To change the time format of “Start Time” and “End Time” in Report update the value of below property:
jmeter.save.saveservice.timestamp_format= dd-MM-yyyy HH:mm:ss
Step 2: Save the “user.properties” file
Step 3: Set the number of threads, loop count and test duration. Execute the test in Non-GUI mode using below command:
To run the test in Unix/Linux box. Use below command:
./jmeter.sh -n -t "<test JMX file>" -l "<test log file>" -e -o "<Path to output folder>"
Example: ./jmeter.sh -n -t “JPetStore.jmx” -l “JPetStoreLog.csv” -e -o “/home/JPetStoreProject/JPetStoreResult”
To run the test in Windows box. Use below command:
jmeter -n -t <test JMX file> -l <test log file> -e -o <Path to output folder>
Example: jmeter -n -t JPetStore.jmx -l JPetStoreLog.csv -e -o C:\JPetStoreProject\JPetStoreResult
The HTML Dashboard report generates in the provided folder. As per the above example, “JPetStoreResult” folder will have the generated Dashboard Report.
Step 4: Open the “index.html” file inside JPetStoreResult folder using Google Chrome (as Google Chrome supports the best) and navigate through different graphs present on the left side of the Report. You will get all the test stats and graphs in the correct format.
You may be interested: