Jenkins CI

Build and execute Test Studio tests in Jenkins CI by following these steps:

1.  Install the Java Web Archive (.war) for Jenkins CI. Test Studio tests should not be run from a Windows Service account. Because the Windows native package for Jenkins CI runs as a Windows Service, Test Studio requires the Java Web Archive version to run properly.

2.  Run the Jenkins Listener on a custom port. By default, the Jenkins Listener runs on port 8009 -- the same port as the Test Studio Scheduling Service. When installing the .war, use a command line argument to configure a different, non-conflicting port number for the listener (that is, avoid port numbers 8009, 8010, 8011, 8012, and 8013). For example:

java -jar jenkins.war --ajp13Port=8019

As of release R1 2018 (v2018.1.301) you could install and use the Jenkins Test Studio Plugin.

3.  Create a free-style software project. Create a job and select 'Build a free-style software project'.

4.  Add a Windows batch command build step. Under the 'Add build step menu,' select 'Execute Windows batch command'.

5.  Input ArtOfTest.Runner command line command to execute tests. Under 'Execute Windows batch command', input the ArtOfTest.Runner command to execute your test or test list, including full path to ArtOfTest.Runner.exe and appropriate arguments.

Your test execution step is now ready to save and run.

Build a Visual Studio project or a Solution

1. Configure MSBuild Plugin to build coded tests.

   1.1 Under the Jenkins menu, open Manage Jenkins > Manage Plugins.

   

   1.2 Select and install the MSBuild Plugin.

   

   1.3 Configure MSBuild Plugin according to Jenkins CI instructions.

2. Add MSBuild Step. Under Build, open the Add build step and select Build a Visual Studio project or solution using MSBuild.

   

3. Specify a Visual Studio project or solution for your test. Save the build configuration.

   

Attach the results files to the Job

1. Add "out" parameter in the batch command. It should point to the default jenkins workspace (C:\Users[userName].jenkins\jobs\test\workspace\%JOB_NAME%%BUILD_NUMBER%) so the results files are outputted in a folder with an unique name (Job Name + Build Number).

   

2. Add a post-build action of type "Archive the artifacts".

   

3. Type ${JOB_NAME}${BUILD_NUMBER}\** in the "Files to archive" field so all the results files can be taken.

   

4. After the build completion all the results files will be attached to it.

   

5. Click Build Artifacts and download the files in a .zip file (it will keep the folder structure) and open the *.aiiresult* file.

   

Your Jenkins CI build is now ready to run.

Disclaimer:

In case you are running Test Studio tests using Jenkins slaves you may experience the following exception:

The process is not running in 'Interactive Mode'! The 'DialogMonitor' has been disabled for this Browser.

TestExecuteProxy.ExecuteTest() : EXCEPTION! (see below)
Outer Exception Type: System.TimeoutException
Message: Wait for condition has timed out
HRESULT: 0x80131505 (Official ID (if app.) = COR_E_TIMEOUT, Error Bit = FAILED, Facility = FACILITY_URT, Code = 5381)
Call Stack:
at ArtOfTest.Common.WaitSync.CheckResult(WaitSync wait, String extraExceptionInfo, Object target)
at ArtOfTest.Common.WaitSync.For[T](Predicate`1 predicate, T target, Boolean invertCondition, Int32 timeout, WaitResultType errorResultType)
at ArtOfTest.Common.WaitSync.For[T](Predicate`1 predicate, T target, Boolean invertCondition, Int32 timeout)
at ArtOfTest.WebAii.Core.Manager.WaitForBrowserToConnect(Int32 browserIndexToWaitFor)
at ArtOfTest.WebAii.Core.Manager.LaunchNewBrowser(BrowserType browserToLaunch, Boolean waitForBrowserToConnect,
ProcessWindowStyle windowStyle, String arguments)
at ArtOfTest.WebAii.Design.Execution.ExecutionEngine.InitializeWeb(ExecutionEngineCreateParams initParams)
at ArtOfTest.WebAii.Design.Execution.TestExecuteProxy.CreateAndInitializeEngine(Test test, ExecutionEngineCreateParams cp)
at ArtOfTest.WebAii.Design.Execution.TestExecuteProxy.ExecuteTest(ExecuteTestCommand command)

The problem is that Jenkins slaves cannot be ran as service. This will cause the automation to fail with the above exception. In this case you should start Jenkins service headless.

Example:

java -jar slave.jar -jnlpurl http://$HOST:$PORT/computer/$SLAVEMACHINE/slave-agent.jnlp

or

javaws http://$HOST:$PORT/computer/$SLAVEMACHE/slave-agent.jnlp