Once there is a set of test scripts designed and configured to run consistently and smoothly as standalone tests, these can be included in a test list. Test lists can be executed as scheduled jobs, both locally and on remote machines.
Test Studio provides a rich set of test list settings to help for stable execution of the test suites and easier debugging in case of failures. Below are listed some of the most useful settings, which can be applied to enhance the implemented automation process.
Enabling the setting to automatically rerun the failed test will be helpful in the cases of a temporary issue with connectivity or application accessibility. Any failed tests will be executed automatically after the test list execution finishes. Hence, the amount of fake failures will be signifcantly reduced.
The results of the second automatic execution will be listed under the initial failed result of the same test. It will be marked with an exclamation mark.
Note: The tests need to be written as autonomous tests - created in a way to be executed on their own, without the use of previous tests run before them in a list to make them work properly.
Observing the test list execution usually turns out to be helpful to understand what caused the faulure. Of course, there is no way to always keep an eye on the execution. The test list execution recording feature of Test Studio, though, allows you to capture the desktop. You can choose between disabled screeen recording and enabled always or on failure only.
Below are listed the settings related to Screen Recording of test list execution with additional notes related to each one.
- Recording Codec
Supported codecs are:
After installing XVID, "Display encoder status" encoder setting must be disabled. From Windows Start menu open "Encoder Config" ->
Click "Other Options.." -> Uncheck "Display encoder status"
The default selected codec MJPEG can be used without any further adjustments.
In order to use Xvid or X264 codecs, they must be installed on the execution machine (for local and Visual Studio runs on the local machine, and for remote runs, on the machine where test list will be executed)
Recording FPS - Sets captured video frames per second. Lower number produces smaller files.
Recording mode - Sets recording mode. There are three different recording modes:
Off (Default) - In this mode no video recordings are created.
Always - Video recordings for all executed tests are saved.
OnFail - Only video recordings for failed tests are saved.
Recording output location - Sets path to output location for all video files. If no value is set, for local test list runs default location is project results folder. Always set location for remote or Visual Studio runs.
For remote executions best practice is to use a shared folder accessable from all remote execution agents.
Recording scale - Sets downscaling of the recorded video in percents. From 10 to 100
- Example: If set to 100 the original screen resolution is preserved. When the scale is set to 50 % the resolution of video is halved.
Lower number produces smaller files, but decreases video quality.
- Recording size limit - Sets file size limit in megabytes, 0 is unlimited size.
If the limit is reached, video recording will be stopped before the test execution ends.