VS Team Services Test Execution

VS Team Services supports integration with external command line utility where ArtOfTest.Runner.exe could be plugged in.

1.  Create a new plain build definition:

Empty Build Definition

2.  Choose default agent queue on the next step of creation:

Default Agent Queue

3.  Add a build step:

Add a build step

4.  Choose Utility from the catalog on the left and select a Command line step:

Command line build step

5.  Once the step is added you need to close the catalog and provide the ArtOfTest.Runner.exe location and arguments to pass to it. The default location of the tool is C:\Program Files (x86)\Telerik\Test Studio\Bin. Further details about the required arguments could be found here.

Configure CLI step

When you choose to save the step you could rename it with a meaningful to you name.

Save and Rename CLI step

6.  An agent should be configured and mandatory run in interactive mode. Further details how to download, install and configure the agent could be found here.

Once the agent is successfully installed run it in interactive mode. It will be available in the Agents Queues view in the VS Team Services under the Default queue and will be marked green in case it is up and running or red in case the agent could not be detected.

Agents Queue

7.  Back in VS Team Services queue a new build for the newly created CLI step:

Queue a build for the CLI step

8.  The build should now successfully start the ArtOfTest.Runner and execute the tests or test lists from the provided arguments.

Build succeeded