Test Studio Tests in Continuous Integration Environments

Continuous Integration (CI) and Continuous Delivery/Continuous Deployment (CD) practices are essential in modern software development. The CI/CD pipeline is a crucial part of the DevOps lifecycle nowadays which cannot be complete without testing phase. And this is where you can implement the functional UI tests designed with Test Studio.

This article provides the requirements from Test Studio perspective to setup the execution of functional UI tests as part of an automated pipeline.

• Test Studio Tests in Continuous Integration Environments
  • Execution Machine Requirements
  • Setup Automated Testing Agent Machines
  • Examples of Implementing Test Studio Tests in Different CI Pipelines

Execution Machine Requirements

• Active user session.

  Tip: The only exception is if the executed tests are web test run in headless browser mode.

• CI agent app running in console mode.

  Tip: The only exception is if the executed tests are web test run in headless browser mode.

• Test Studio installation - minimum the Test Studio Run-time Edition.

• Test Studio Execution client running.

• Latest browser(s) or WPF/Desktop application under test installed.

Setup Automated Testing Agent Machines

1. Log onto the build machine (from Test Studio point of view we also call it testing or execution machine) using an account with permissions needed to run the CI build agent.

2. Install the CI build agent from the CI tool in use. Follow the instructions from vendor.

3. Run the CI agent in console mode - use command line or a shortcut link. Follow the instructions from vendor.

   Important!
   Do not run the CI agent as a service unless you intend to execute web tests in headless browser mode.

4. Install Test Studio software on the build machine - the minimum required installation is Test Studio Run-time edition.

5. Leave the machine running and logged into the account.

6. Ensure active desktop session on the build machine. Most UI tests require active desktop session. The only exception is running web tests in headless browser mode.

   Tip
   Use the Test Studio built-in options to control the user session. These options are available in the Test Studio Test Runner application which gets installed with the Run-time Edition.

Examples of Implementing Test Studio Tests in Different CI Pipelines

Find out few examples of setting a pipeline on specific platform and follow the instructions on how to implement Test Studio test runs in it.

• Azure DevOps Classic Pipeline using self-hosted agent
• Jenkins CI
• TeamCity
• Bamboo
• CruiseControl .Net
• Testing in Docker Container - applicable for headless browser testing only.
• Azure DevOps Classic Pipeline using Microsoft-hosted agent - applicable for headless browser testing only.