Analyze Test List Results
The Results tab in Test Studio provides provides an overall view of all test list runs. Along with that you can explore each separate test list result in details, check if there are rerun tests, or elements found by their image as fallback from the set find expression, resolve any failures and many others.
This article will guide you through the possible messages you can see in test list results and how to act upon these.
Open Test List Result
Choose a result to explore in details and double click it to see the Test Results panel on the right side of the calendar.
The Test Results panel shows results for the entire test list, the machine on which it was executed, the overall result of each individual test in the test list.
By selecting a test from the test list, you can see and copy the execution log for the test run (1) or switch to its steps in the Test (2) ribbon.
The Test Results panel allows you to traverse test execution results, drilling down to the individual tests' steps and back up again to the test list level. Double-click a test from the list to see the result of each step in it. The breadcrumb trail at the top of the panel shows the test list followed by the test name. Click the top level in the breadcrumb trail to jump directly back to the test list result.
Failed Test List Result
If one test in the test list fails, the overall status of the test list is reported as Failed. To find out details about what caused the failure, you can drill down to the failing test and step. The faulty step will be marked with red cross sign to indicate that it wasn't successfully executed. Double click the red cross and open the step failure details, which provide useful information for the failure.
Performance Test List Results
When you drill down the results of a Performance Test List execution, there is an additional icon listed in the Test Results panel - this is the Stopwatch.
When you click the Stopwatch icon the step results switch to show the overview details for the performance run - Total Time, Client and Server Time, Size (of transferred data) and Http Errors. These are the details, which can be explored in the Overview view in the Performance tab.
Tip
If you need to review the performance test runs in further depth and compare these with previous runs, ensure that the performance results are stored in the same folder for each run - this is the folder, which you specify when setting up the performance run. Then the performance data will be available in the Perfrmance tab for the respective test in project.
Test as Step Results in a Test List
If a test contains another test executed as step, the column More indicates that there are more details to drill down. Double click the test as step to view the detailed results for this test as step. The breadcrumb trail at the top of the panel now shows the test list followed by the main test name and then the name of the test as step. Clicking on any of the tests in the breadcrumb trail will get you back to the selected level.
Element Not Found Results
One of the common scenarios in a troublesome test run is related to the elements used in the Test Studio tests and that these cannot be located on current window in the application under test. Test Studio provides different options to identify elements during run-time - you can use a find expression, or an image. Whichever is set, the other will be used as a fall back search option for the test run to find the element in the active application. Based on this specific mechanism for Test Studio, there are few different results you can see in regards the element identification apart from a successful test list run.
Successful Test List Run with Warnings
The automatic fall back to image based search of elements will log a warning in the test list result if it was used. The test list result is marked as successful as all elements are located, but if any element failed to be located by its default find expression, it will be listed for your convenience.
Tip
If the project is set to first use images to locate the elements, the warnings will appear, if the image was not found and the element was located using its find expression.
Double click the result to see details for the warning in the test list result.
Resolve the Warning
Test Studio provides direct access through the button in the More column to open the Element Edit Pane and modify the element's find expression.
Failed Test List Run
An element, which failed to be located by its backup search, will fail the test and the overall test list result will be reported as Failed. The particular test's result provides details for the failure in the step failure details, including if the backup search was used.
In Development Tests Results
If the test list setting ExecuteTestsInDevelopment is set to false, the tests flagged 'In Development' will be skipped in the test list execution. Opening the results details will inform you that the test is in development and it wasn't executed.
Automatic Re-run of Failed Tests Results
If the test list setting RerunFailedTests is enabled, the failed tests in a test list run will be automatically executed after the rest of the tests are finished. The results of the rerun test execution will be displayed in the overall list result and placed under the same test's first execution results, but will be marked with an exclamation mark and flagged as 'Rerun'.
Test List Screen Recording Results
If the test list settings to capture the screen during test list execution are enabled and a video is available, a 'Screen Recording Info' icon is displayed in the overall test list results for the tests which triggered video recording.
Click the 'Screen Recording Info' icon for any of the tests which triggered video recording, and choose to play the video directly or copy to clipboard the output folder and navigate to files directly in the File Explorer.
