TFS 2017 Builds Test Execution

TFS 2017 supports integration with external command line utility where ArtOfTest.Runner.exe could be plugged in.

1.  Create a new build definition:

Empty Build Definition

2.  Choose Empty process template on the next step of creation:

Empty Process Template

3.  Add a new build task. Choose Utility from the catalog on the left and select a Command line task:

Add a build task

4.  Once the step is added you need to provide the ArtOfTest.Runner.exe location and arguments to pass to the command line. The default location of the tool is C:\Program Files (x86)\Telerik\Test Studio\Bin. Further details about the required arguments could be found here.

Please note that as of version 2017 R3 (v.2017.3.1010) the default installation path for new installation is C:\Program Files (x86)\Progress\Test Studio.

Configure CLI step

The project to execute a test list from should be accessible from the TFS server but is not mandatory to be included under source control on the same server. The current example executes a test list from project stored on a shared location.

5.  A TFS agent should be configured and mandatory run in interactive mode. Further details how to download, install and configure the agent could be found in the Agent Queues section on TFS server.

Agent Queues View

Once the agent is successfully installed run it in interactive mode. It will be available in the Agents Queues view under the Default queue and will be marked green in case it is up and running or red in case the agent could not be detected.

Agents Queue

6.  Back in the Task view for the newly created build definition you could queue a build to run:

Queue a build

8.  The build should now successfully start the ArtOfTest.Runner and execute the tests or test lists from the provided arguments.

Build succeeded