Skip to main content
Version: Release 22.3

The Continuous Testing plugin allows users to execute Android, iOS and web application test cases that will accelerate your Release Cycles with our SeeTest Continuous Testing Platform. SeeTest can execute hundreds of manual or automated tests in parallel on real iOS & Android devices in the cloud.

Features

  • Execute Espresso test cases by providing .apk files.
  • Execute XCUI test cases by providing .ipa files.
  • Get the run status and results of your Espresso or XCUITest execution.
  • Create test views to house result sets.
  • Get test view results from your Appium test jobs.

Setup the Continuous Testing Server

To setup a server connection, do the following:

  1. In Release, from the navigation pane, go to Configuration > Connections on Continuous Testing instance, click on “+".

  2. Under the New Continuous Testing: instance, fill-in the required fields below:

    1. Enter a title in the Title field. (ex: Continuous Testing USCloud Server)

    2. Enter the URL where the Continuous Testing server is running. This can be a local instance you have configured on premises or one of the DAI Cloud SaaS instances. (ex: https://uscloud.experitest.com)

  3. Under the Authentication section, fill-in the fields below:

    1. Enter the Continuous Testing username in the username field. (example: admin)

    2. Enter the Continuous Testing password in the password field. (example: testing123)

  4. Under the Proxy section, fill-in the appropriate fields. Please be advised that these fields are optional and only required if dictated by your organization.

  5. Click on Test to test the availability of the Continuous Testing Instance.

  6. Click on Save.

Save Connections

Usage

Available Tasks: Trigger Espresso Test Case, Trigger XCUITest case, Get Test Run Status And Result For Unit Tests, Create Test View, Get Test View Results

Trigger Espresso Test Case

Note: This task is used to run test cases for Android applications (.apk)

To create a Trigger Expresso Test Case task, do the following:

  1. Navigate and select the respective Release Template where you want to create the Trigger Expresso Test Case task.
  2. Under the Phase, click on Add task.
  3. Enter the Title in the Task title field.
  4. Select Digital.ai Continuous Testing > Trigger Espresso Test Case using the dropdown.

Select Trigger Espresso test case

  1. Click on Add.

  2. Click on the created task (Trigger Espresso Test Case).

  3. Enter the required properties in the opened Trigger Espresso Test Case task.

  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.

  5. Select the required Running Type.

    Note: Here we define the execution mode. Coverage Running Type is used to run ALL test cases on devices and fastFeedback is used to spread the test cases across devices.

  6. Enter the URL where the APK application can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/filename.apk -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  7. Enter the URL where the APK test cases can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/test-filename.apk -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  8. Enter the appropriate Device Query.

    Note: the details for constructing a device query can be obtained here: https://docs.experitest.com/display/TE/Device+Queries

  9. Close the Trigger Espresso Test Case task.

Close Trigger Espresso test case

Trigger XCUI Test Case

Note: This task is used to run test cases for iOS applications (.ipa)

To create a Trigger XCUI Test case task, do the following:

  1. Navigate and select the respective Release Template where you want to create the Trigger XCUI Test task.

  2. Under the Phase, click on Add task.

  3. Enter the Title in the Task title field.

  4. Select the Digital.ai Continuous Testing > Trigger XCUI Test Case using the dropdown.

  5. Click on Add.

  6. Click on the created task (Trigger XCUI Test Case).

  7. Enter the required properties in the opened Trigger XCUI Test Case task.

  8. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.

  9. Select the required Running Type.

    Note: Here we define the execution mode. Coverage Running Type is used to run ALL test case on devices and fastFeedback is used to spread the test cases across devices.

  10. Enter the URL where the IPA application can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/filename.ipa -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  11. Enter the URL where the IPA test cases can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/test-filename.ipa -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  12. Enter the appropriate Device Query.

    Note: the details for constructing a device query can be obtained here.

  13. Close the Trigger XCUI Test Case task.

Close Trigger XCUI test case

Get Test Run Status and Result for Unit Tests

Note: This task provides the test results for Espresso and XCUI Test tasks.

To create Get Test Run Status and Result for Unit Test task, do the following:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Digital.ai Continuous Testing > Get Test Run Status and Result for Unit Tests using the dropdown.

Select test run status and result for unit test

  1. Click on Add.
  2. Click on the created task (Get Test Run Status and Result for Unit Tests).
  3. Enter the required properties in the opened Get Test Run Status and Result for Unit Tests task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Enter the Test Run ID. This value is an output property from the Trigger Test Case (Espresso / XCUI Test) tasks. Leverage variables to pass this value between the two tasks.
  6. Close the Get Test Run Status and Result for Unit Tests task.

Close the get test run status

The Output properties shown above will be populated after the tests have completed executing in your SeeTest instance.

Create Test View

To create a Create Test View task, do the following steps:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Digital.ai Continuous Testing > Create Test View using the dropdown.
  4. Click on Add.
  5. Click on the created task (Create Test View).
  6. Enter the required properties in the opened Create Test View task.
  7. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  8. Enter a string value in the Test View Name field.
  9. Close the Test View Name task.

Close the Test View Name task

Get Test View Results

To create Get Test View Results task, do the following steps:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Digital.ai Continuous Testing > Get Test View Results using the dropdown.
  4. Click on Add.
  5. Click on the created task (Get Test View Results).
  6. Enter the required properties in the opened Get Test View Results task.
  7. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  8. Enter the Test view ID in the Test View ID field. Test View ID is an output property from the Create Test View task. Use release variables to pass this value between tasks.
  9. Enter the Jenkins Build Number in the Jenkins Build Number field. You can get this value from the Jenkins task. It is an output property of the Jenkins task. Again, use release variables to pass this value between tasks.
  10. Close the Get Test View Results task.

Close the get test view results task

Create Manual Report Template

To create Create Manual Report Template task, do the following steps:

  1. Under the Phase, click Add task.

  2. Type the Title in the Task title field.

  3. Select the Digital.ai Continuous Testing > Create Manual Report Template using the dropdown.

  4. Click Add.

  5. Click the created task (Create Manual Report Template).

  6. Enter values for the required properties in the opened Create Manual Report Template task.

  7. By default, the server field picks the created server. If you have multiple configured Digital.ai Continuous Testing servers, you can pick the oneyou want to use.

  8. Enter the appropriate Device Query.

    Note: For more information about constructing a device query, see https://docs.experitest.com/display/TE/Device+Queries

  9. Enter a Test Name in the Test Name field.

  10. Enter the Manual Test Steps in the Test Steps field as a json list.

Sample Steps:

{
 		       "name": "Test Step1:Launch the application",
 			   "description": "Open the Latest Version available",
 			   "expectedResult": "Application opens and stays visible in the foreground"
 		   },
 		   {
 			   "name": "Test Step2:Open the accounts page",
                "expectedResult": "Accounts page is Visible"
}

Close the get test view results task

  1. Close the Create Manual Report Template task and run the release. When the tasks is running, it displays the link to the created Manual Report with the specified Test Steps.

Click on the link displayed

  1. Click the link, complete the manual testing of all test steps and click Save Report.
  2. Now the task in Release is complete and a link to the completed manual testing report is displayed.

Release is complete and a link to the completed manual testing report is displayed