My Project Runs Locally but Does Not Remotely

Apart from the communication issues that might be encountered in a remote setup there are few reasons related to some project specifics why a test list is successfully executed in a local run but fails to be started and compiled in a remote one. To easily determine whether the remote connections are successfully built create a test list with a basic web test preferably in a sample plain project and try to run it remotely. In case this execution is successful you might find a possible solution in the below listed scenarios.

Unstable tests

Such a problem could be caused by an unstable test. If a test is created and ran exception free on one machine it is not a given that the same test will pass on the remote machine. Basically, the reason of an unstable behavior could be a timing problem - the test fails when the action is performed over an element that is still not present on the page. To ensure stability of the test one has to use an "execution delay" or "wait for exist" steps on the element that will be automated in the next step.

An Execution Extension is implemented

To compile such a project remotely the respective dll needs to be available and accessible on the execution machine as well. The extensions library file(s) need to be copied to the installation folder of Test Studio in \Bin\Plugins sub folder. There is a limitation in this scenario to be considered – both the Scheduling and Execution servers need to be configured on the remote machine to be able to use the execution extension during a remote run and therefore such kind of project could be sent to a single remote executor only.

Additional data resources used in the test run

If the project uses additional local data resources they have to be placed at the same location as on the local machine.

The project is in TFS source control and a test list is scheduled with "Get latest version of tests automatically from TFS" enabled

Important note here is that both Scheduling and Execution servers require access to TFS - the Scheduling server loads the recent available test list from TFS to determine which tests to distribute to executors as part of the currently running job and loads these tests. On the other hand the Execution server requires the access to obtain the latest checked in test files since the local project copy may not be the recent one from TFS.

An issue might appear if the required class library files for Test Studio to communicate to TFS do not persist on the Scheduling or Execution machines. They will be automatically installed with the installation of Team Explorer 2013 (free download) or Visual Studio 2013 or earlier.

One of the tests in the test list has very long name

This issue could appear in a local run as well. The name of a test file is interpreted along with its full path location on disk. Windows OS limits this string up to 260 symbols and if it gets longer it cannot be resolved and used. Even if a local execution is possible a remote one could fail due to that string length limitation and the reason behind is that the project files are copied to a temporary folder on the Execution machine. In general this temp folder’s name contains the job ID which is a sequence of letters and numbers which are added to the file’s full path and increase its length significantly. To avoid that scenario keep your local project on a default location on your drive and keep the test names steady and short.

Note: If the issue is at least once encountered there might be corrupted values stored in the MongoDB storage. To get rid of these the DB collections should be maintained accordingly - with the help of Robomongo the corrupted test list could be located and deleted or alternatively the whole test list collection could be dropped.