Execution Extension

Test Studio Dev has enhanced extensibility support for test execution. This model helps integrate Test Studio Dev better into your environment that contains custom results reporting and code defect tracking.

To demonstrate the execution extensibility model, let's build a simple Execution Extension for Test Studio Dev that writes the results of a test list to a text file.

1.  Create a Class Library project in Visual Studio. This example uses C#.

2.  Add references to three DLLs from C:\Program Files (x86)\Progress\Test Studio\Bin\

• ArtOfTest.WebAii.dll

• ArtOfTest.WebAii.Design.dll

• Telerik.TestStudio.Interfaces.dll

Also add the following .NET references:

• System.Runtime.Serialization

• System.Windows.Forms

3.  Add the following using statements to the class file:

using System.IO; 
using System.Data; 
using System.Data.OleDb; 
using System.Windows.Forms; 
using ArtOfTest.WebAii.Design.Execution; 

Imports System.IO 
Imports System.Data 
Imports System.Data.OleDb 
Imports System.Windows.Forms 
Imports ArtOfTest.WebAii.Design.Execution 

4.  The ArtOfTest.WebAii.Design.Execution namespace contains an IExecutionExtension that our class needs to implement:

namespace ClassLibrary1 
{ 
    public class Class1 : IExecutionExtension 
    { 
    } 
} 

Namespace ClassLibrary1 
    Public Class Class1 
        Implements IExecutionExtension 
    End Class 
End Namespace 

5.  Right click on IExecutionExtension and select Implement Interface > Implement Interface. This displays all the methods and notifications exposed by Test Studio. Here are definitions for each IExecutionExtension member:

• The OnInitializeDataSource() function cannot be left empty and should return null if not being used. Test Studio takes into consideration only the first non null data table (if you are using more than one plugin, the first one to return non null data table is used).

• The rest of the functions, which you're not using, should be left empty (remove throw new NotImplementedException).

namespace ClassLibrary1 
{ 
    public class Class1 : IExecutionExtension 
    { 
        #region IExecutionExtension Members 
 
        // After each test is completed, ArtOfTest.Runner calls this method. 
        // <param name="executionContext">The execution context the test is running under.</param> 
        // <param name="result">The actual result of the test.</param> 
        public void OnAfterTestCompleted(ExecutionContext executionContext, TestResult result) 
        { 
            // Your custom implementation here 
        } 
 
        // Not applicable in the context of Test Studio Dev 
        public void OnAfterTestListCompleted(RunResult result) 
        { 
            // Your custom implementation here. 
        } 
 
        // Not applicable in the context of Test Studio Dev 
        public void OnBeforeTestListStarted(TestList list) 
        { 
            // Your custom implementation here 
        } 
 
        // Before a test is about to start, ArtOfTest.Runner calls this method. 
        // <param name="executionContext">The execution context the test is running under.</param> 
        // <param name="test">The test we are about to start running.</param> 
        public void OnBeforeTestStarted(ExecutionContext executionContext, ArtOfTest.WebAii.Design.ProjectModel.Test test) 
        { 
            // Your custom implementation here 
        } 
 
        // Use this to return your own data source. 
        // <param name="executionContext">The execution context.</param> 
        public System.Data.DataTable OnInitializeDataSource(ExecutionContext executionContext) 
        { 
            // Your custom implementation here 
            return null; 
        } 
 
        // Called only on a step failure. 
        // <param name="executionContext">The execution context a test is running under.</param> 
        // <param name="stepResult">The step result that just failed.</param> 
        public void OnStepFailure(ExecutionContext executionContext, ArtOfTest.WebAii.Design.AutomationStepResult stepResult) 
        { 
            // Your custom implementation here 
        } 
 
        #endregion 
    } 
} 

Namespace ClassLibrary1 
    Public Class Class1 
        Implements IExecutionExtension 
        #Region "IExecutionExtension Members" 
 
        ' After each test is completed, ArtOfTest.Runner calls this method. 
        ' <param name="executionContext">The execution context the test is running under.</param> 
        ' <param name="result">The actual result of the test.</param> 
        Public Sub OnAfterTestCompleted(executionContext As ExecutionContext, result As TestResult) 
            ' Your custom implementation here 
        End Sub 
 
        ' Not applicable in the context of Test Studio Dev 
        Public Sub OnAfterTestListCompleted(result As RunResult) 
            ' Your custom implementation here. 
        End Sub 
 
        ' Not applicable in the context of Test Studio Dev 
        Public Sub OnBeforeTestListStarted(list As TestList) 
            ' Your custom implementation here 
        End Sub 
 
        ' Before a test is about to start, ArtOfTest.Runner calls this method. 
        ' <param name="executionContext">The execution context the test is running under.</param> 
        ' <param name="test">The test we are about to start running.</param> 
        Public Sub OnBeforeTestStarted(executionContext As ExecutionContext, test As ArtOfTest.WebAii.Design.ProjectModel.Test) 
            ' Your custom implementation here 
        End Sub 
 
        ' Use this to return your own data source. 
        ' <param name="executionContext">The execution context.</param> 
        Public Function OnInitializeDataSource(executionContext As ExecutionContext) As System.Data.DataTable 
            ' Your custom implementation here 
            Return Nothing 
        End Function 
 
        ' Called only on a step failure. 
        ' <param name="executionContext">The execution context a test is running under.</param> 
        ' <param name="stepResult">The step result that just failed.</param> 
        Public Sub OnStepFailure(executionContext As ExecutionContext, stepResult As ArtOfTest.WebAii.Design.AutomationStepResult) 
            ' Your custom implementation here 
        End Sub 
 
        #End Region 
    End Class 
End Namespace 

A few notes about the code above:

• ExecutionContext - The context is an object that gives you mostly all other objects in context of the current execution. For example, from this object you can access the run-time Manager object, the ActiveBrowser, the Log object, the Find object, etc. These are the run-time objects are used to execute your test. It also exposes the ExecutionContext.ExecutingTestAsStep property, which distinguishes between tests being run normally or as a subtest.

• Result objects - RunResult contains a list of “TestResult” objects for each test that is executed. Each TestResult has an AutomationStepResult list that represents the result for each step that executed. You have access to all the metadata of the execution so you can generate your own reports.

• OnInitializeDataSource - Bind a test to a custom data source by returning a DataTable from this method. Use that DataTable to data drive the test. It will be used regardless if the test is data bound or not.

6.  Once you have included the custom compile the class library.

8.  Deploy the extension by copying the DLL from the %Project Folder%\ClassLibrary1\ClassLibrary1\bin\Debug to the following directory:

• %ProgramFiles%\Progress\Test Studio\Bin\Plugins\

9.  Now when a test is being executed the methods before or after execution will be called and executed.

Example of OnInitializeDataSource Method

Let's see an example using the OnInitializeDataSource method. This assumes that your test is already bound to an Excel file, and each Excel file has matching column names.

1.  Add the following code to that method:

public System.Data.DataTable OnInitializeDataSource(ExecutionContext executionContext) 
{ 
    System.Data.DataTable table = null; 
    var thread = new System.Threading.Thread(obj => 
    { 
        try 
        { 
            System.Windows.Forms.OpenFileDialog ofd = new System.Windows.Forms.OpenFileDialog(); 
            ofd.Title = "Open Excel File"; 
            if (ofd.ShowDialog() == DialogResult.OK) 
            { 
                string excel = ofd.FileName; 
                DataSet foo = ImportExcelXLS(excel, true); 
                table = (foo.Tables[0]); 
            } 
        } 
        catch (Exception ex) 
        { 
            NativeWindow a = new NativeWindow(); 
            a.AssignHandle(ArtOfTest.WebAii.Core.Manager.Current.ActiveBrowser.Window.Handle); 
            MessageBox.Show(a, ex.Message); 
        } 
    }); 
    thread.SetApartmentState(System.Threading.ApartmentState.STA); 
    thread.Start(); 
    thread.Join(); 
    return table; 
} 

Public Function OnInitializeDataSource(executionContext As ExecutionContext) As System.Data.DataTable 
    Dim table As System.Data.DataTable = Nothing 
    Dim thread = New System.Threading.Thread(Function(obj)  
    Try 
        Dim ofd As New System.Windows.Forms.OpenFileDialog() 
        ofd.Title = "Open Excel File" 
        If ofd.ShowDialog() = DialogResult.OK Then 
            Dim excel As String = ofd.FileName 
            Dim foo As DataSet = ImportExcelXLS(excel, True) 
            table = (foo.Tables(0)) 
        End If 
    Catch ex As Exception 
        Dim a As New NativeWindow() 
        a.AssignHandle(ArtOfTest.WebAii.Core.Manager.Current.ActiveBrowser.Window.Handle) 
        MessageBox.Show(a, ex.Message) 
    End Try 
 
End Function) 
    thread.SetApartmentState(System.Threading.ApartmentState.STA) 
    thread.Start() 
    thread.Join() 
    Return table 
End Function 

2.  Now add the following ImportExcelXLS method within the same public class:

private static DataSet ImportExcelXLS(string FileName, bool hasHeaders) 
{ 
    string HDR = hasHeaders ? "Yes" : "No"; 
    string strConn = null; 
    if (FileName.Substring(FileName.LastIndexOf('.')).ToLower() == ".xlsx") 
    { 
        strConn = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + FileName + ";Extended Properties=\"Excel 12.0;HDR=" + HDR + ";IMEX=1\""; 
    } 
    else 
    { 
        strConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + FileName + ";Extended Properties=\"Excel 8.0;HDR=" + HDR + ";IMEX=1\""; 
    } 
 
    DataSet output = new DataSet(); 
 
    using (OleDbConnection conn = new OleDbConnection(strConn)) 
    { 
        conn.Open(); 
 
        DataTable schemaTable = conn.GetOleDbSchemaTable( 
        OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" }); 
 
        foreach (DataRow schemaRow in schemaTable.Rows) 
        { 
            string sheet = schemaRow["TABLE_NAME"].ToString(); 
 
            OleDbCommand cmd = new OleDbCommand("SELECT * FROM [" + sheet + "]", conn); 
            cmd.CommandType = CommandType.Text; 
 
            DataTable outputTable = new DataTable(sheet); 
            output.Tables.Add(outputTable); 
            new OleDbDataAdapter(cmd).Fill(outputTable); 
        } 
    } 
    return output; 
} 

Private Shared Function ImportExcelXLS(FileName As String, hasHeaders As Boolean) As DataSet 
    Dim HDR As String = If(hasHeaders, "Yes", "No") 
    Dim strConn As String = Nothing 
    If FileName.Substring(FileName.LastIndexOf("."C)).ToLower() = ".xlsx" Then 
        strConn = (Convert.ToString((Convert.ToString("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=") & FileName) + ";Extended Properties=""Excel 12.0;HDR=") & HDR) + ";IMEX=1""" 
    Else 
        strConn = (Convert.ToString((Convert.ToString("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=") & FileName) + ";Extended Properties=""Excel 8.0;HDR=") & HDR) + ";IMEX=1""" 
    End If 
 
    Dim output As New DataSet() 
 
    Using conn As New OleDbConnection(strConn) 
        conn.Open() 
 
        Dim schemaTable As DataTable = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, New Object() {Nothing, Nothing, Nothing, "TABLE"}) 
 
        For Each schemaRow As DataRow In schemaTable.Rows 
            Dim sheet As String = schemaRow("TABLE_NAME").ToString() 
 
            Dim cmd As New OleDbCommand((Convert.ToString("SELECT * FROM [") & sheet) + "]", conn) 
            cmd.CommandType = CommandType.Text 
 
            Dim outputTable As New DataTable(sheet) 
            output.Tables.Add(outputTable) 
            New OleDbDataAdapter(cmd).Fill(outputTable) 
        Next 
    End Using 
    Return output 
End Function 

3.  Rebuild the class library, copy the resulting DLL file, and paste it into the Plugins folder (overwriting any other existing class library file).

4.  When you execute a data driven test, you are prompted to select an Excel file (.xls or .xlsx). You have two choices:

• Select a new Excel file and press OK.

• Press Cancel to use the original data source.

CI Setup

In the context of Continuous Integration setup - running a build using the ArtOfTest.Runner.exe on the execution agent machines with test option the methods will be executed.