Docker Container Testing with Test Studio
Test Studio test lists can be executed in Chrome Headless mode within a Docker container. Read below how to prepare the environment and how to start testing in a Docker container.
Environmental Prerequisites
The prerequisites from Test Studio and Docker side are listed below. Make sure that you comply with them, before proceeding with the next steps.
- Docker container can run on a physical or virtual machine with Windows installed.
- Installed Docker on the machine, where tests will be executed.
- Active Test Studio Run-Time license and .msi installer for it. The minimum version of Test Studio is 2021 R3.
Disclaimer
Executing tests in a Docker container has its specifics and limitations. It is better to know about them before you proceed.
- Tests in a Docker container can be executed only in Headless mode, because the container does not have UI. This is the Chrome Headless browser type in Test Studio.
- Element's image find logic is not supported in Docker container test runs.
- Image verification step is not supported in Docker container test runs.
- Text from image verification step (OCR) is not supported in Docker container test runs.
Install Docker
Download Docker from the official website. Then, follow the instructions and install it on the machine that will be used for executing tests.
You can refer to the Docker documentation for commands and questions related with that software.
Setup Docker Container for Testing
Below are listed the steps to setup a Docker container for testing.
1. Switch Docker to use Windows containers, instead of the default Linux containers. Once you switch it successfully, the option will show as "Switch to Linux containers...", meaning that the current state is Windows containers.
2. Pull the official Microsoft Windows Docker image from the Docker Hub. Use the command that is provided in the Docker Hub and make sure to specify the exact build to be pulled.
//For Windows build 1909, use the following command
docker pull mcr.microsoft.com/windows:1909
Note
The build version of the Windows image to install in the Docker container must match the Windows build on the current host machine.
The Microsoft Windows Docker image does not support "latest" tag. For further information check the documentation on the DockerHub's page for this image.
3. Create a container from the official Microsoft Windows image. To do that, go to Images in the Docker desktop application and click on the Run button.
Then, expand the Optional Settings and specify Host Path and Container Path. The Host Path is a folder on the machine, which is available to the Docker container and the Container Path is how you will reference this resources from the Host Path within the container.
4. Copy the .msi installers of Test Studio Run-Time and Chrome Enterprise in the Host Path folder. They become available in the Container Path in the Docker container and are ready to be installed. Open the Command Line Interface (CLI) of the container and navigate to the Container Path folder.
Install both Test Studio Run-Time and Chrome browser enterprise edition in passive mode, because there is no UI in the container and active installation can not complete. You can use the following command, but make sure that each installer is finished without errors before you proceed with the next one.
msiexec.exe /i msiInstallerFileName.msi /passive /le c:\tools\errorLogForThisInstaller.txt
Note
Additional information about the msi installers can be found on the Microsoft documentation.
Execute Tests
Note
Tests and test lists execution in the Docker container is only supported for Chrome Headless mode.
After completing the above steps in this article, you need to copy the project, from which you want to execute tests and test lists, to be available in the Docker container. Place the project folder in the Host Path folder on your local machine and they also show up in the Container Path folder.
Open the container's Command Line Interface (CLI) and use the ArtOfTest.Runner.exe to start the execution. This test runner provide multiple options for execution and output, which are all explained in the linked documentation. The below example command triggers a test list execution.
//navigate to the Test Studio installation bin sub-folder
ArtOfTest.Runner.exe /list="full path to the test list file with extension .aiilist"
You can apply specific settings for Docker container test list execution, like the browser type. Since the test list execution runs only in Headless mode, you can choose different settings file to be used from the ArtOfTest.Runner.exe. Using such file allows you to not change the Test List Settings on your original test list and still execute it in Headless mode in the container.