Manage Test Run with the API

Continuous Testing Cloud exposes Rest APIs to execute Espresso and XCUI based Tests.

This page provides the details of the API.

info

The role of the user performing the Rest Operation is specified by the Authorization header. For more information see How To Execute Rest API.

info

All actions are also available for Project Administrators and users.

Status of the API Run

Gets the status of the Espresso/XCUI test run.

GET /api/v1/test-run/\<id\>/status

id - Test run ID

Response

Response Status: 200 OK
[
    {
  		"status": "SUCCESS",
  		"data": {
    		"Test Run Id": "3520316",
    		"Test Run State": "Finished",
            "Number of tests in file" : "11",
            "Number of requested tests" : "11",
    		"Total number of tests": "11",
    		"Number of passed tests": "3",
    		"Number of failed tests": "8",
   			"Number of skipped tests": "0",
    		"Number of running tests": "0",
    		"Number of queued tests": "0",
			"Number of ignored tests" : "0",
    		"Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520316%22}"
  		},
  	"code": "OK"
	}
]

info

Number of tests in file - the number of tests in the uploaded file
Number of requested tests - the number of tests that were selected for execution after applying includeTests / ignoreTestFile.
"Number of skipped tests" - tests that were not executed because of timeout/cancellation.
"Number of ignored tests" - tests that were not executed because they were ignored by the user (ignoreTestFile parameter)

Running Async Espresso/XCUITest

Enables users to run Espresso or XCUITest asynchronously, the response received is immediate.

POST /api/v1/test-run/execute-test-run-async

Parameters

Name	Type	Mandatory	Description
executionType	String	Yes	Specified if it is Espresso or XCUITest
runningType	String	Yes	coverage / fastFeedback 1. Coverage - all of the tests will be executed on all of the selected devices. 2. Fast feedback - all of the tests will be spread across the selected devices.
app	File	Yes (either app or appUrl or CloudApp)	The application is under test.
appUrl	String	The URL location of the application is under test.
cloudApp	Long	The ID of the application saved in cloud application service
testApp	File	Yes (either testApp or testAppUrl or CloudTestApp)	espresso / XCUI tests(.zip extension only).
testAppUrl	String	The URL Location of espresso / XCUI tests. (.zip extension only)
cloudTestApp	Long	The ID of the test app saved in cloud file repository service
deviceQueries	String	No	For coverage mode: List of strings (XPath queries). The test run should run on devices that match provided queries (device per query). Multiple queries should be passed in the standard way by repeating the parameter multiple times. For Fast Feedback: Only one device query must be provided on the list. The number of devices used is decided by maxDevices and minDevices parameters. If the maxDevices parameter is not provided then the execution will try to reserve the maximum number of devices according to license restriction. It is recommended to use maxDevices parameter always
uuid	String	No	Provision profile UUID to sign the Application and the test application. Relevant to iOS execution-only
runTags	String	No	Map as JSON - wanted tags to upload to the reporter.
overallTimeout	Long	No	The number of milliseconds. Timeout for the overall test run. If not provided - default value (14,400,000) - 4 hours - should be used.
creationTimeout	Long	No	The number of milliseconds. Timeout for test requests in queued status. If not provided - default value (14,400,000) - 4 hours - should be used.
reservationDuration	Long	No	The reservation time of each session in minutes. The default time is 120 minutes * the reservationTime cannot exceed the max reservation time configured for the cloud or project.
includeTests	List of Strings	No	Run only the specified tests. The supported formats: * fullClassName, methodName * fullClassName.methodName * fullClassName * methodName * packageName For cucumber/Cucumberish tags, run only the matching tags. Each tag will be provided separately: * @tag1 * @tag2
ignoreTestFile	File	No	This file specifies the tests to exclude. iOS * xcscheme file Android * the file containing full class and method name, separated with comma, e.g. `<br/><br/>com.example.app.SomeTest, testFoo<br/><br/>com.example.app.AnotherTest, testBar<br/><br/>`
maxDevices	Integer	No	The maximum number of devices to use during the execution. default value - max available devices license. Note: Only use in case of fastFeedback
minDevices	Integer	No	The minimum number of devices to use during the execution. Default value - 1. Note: Only use in case of fastFeedback
retry	Integer	No	An integer from 0 to 5 - Specifies how many times failed test should be retried Default Value - 0 (means no retry)
additionalAppIds	List of Longs	No	List of application ids that should be installed on the device/emulator before starting the Espresso/XCUI test. Application id is returned when uploading application with Rest API or using the Get Applications REST API.
retryDifferentDevice	boolean	No	A boolean value specifies whether or not to run a failed test on different devices according to the "retry" value. Default Value - false (means will not try to run a failed test on different devices). If "maxDevices" is specified, then "retry" might be limited to the min between ("maxDevices" - 1) and "retry".
useUIAutomator	boolean	No	For Android only A boolean value decides whether or not to disable Continuous Testing's UIAutomator. Default Value = false - UIAutomator will not be disabled. True - UiAutomator will be disabled on Test Start and will be enabled after it ends
useTestOrchestrator	boolean	No	For Android only A boolean value decides whether or not to run the test in Test Orchestrator Mode. Default Value = false - test will not run in Test Orchestrator Mode true - test will run in Test Orchestrator Mode
clearPackageData	boolean	No	For Android only If useTestOrchestrator is true, then clearPackageData makes sure to remove all shared states from the device's CPU and memory after each test. Default Value = false - don't clear shared state data true - clear shared state data

Response

{
    "status": "SUCCESS",
    "data": {
        "Test Run Id": "3520546",
        "Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520546%22}"
    },
    "code": "OK"
}

Running Sync Espresso/XCUITest

Enables users to run Espresso or XCUITest synchronously, the response received is after the test run.

POST /api/v1/test-run/execute-test-run

Parameters

Name	Type	Mandatory	Description
executionType	String	Yes	Specified if it is Espresso or XCUITest
runningType	String	Yes	coverage / fastFeedback 1. Coverage - all of the tests will be executed on all of the selected devices. 2. Fast feedback - all of the tests will be spread across the selected devices.
app	File	Yes (either app or appUrl or cloudApp)	Application under test
appUrl	String	The URL location of the application is under test.
cloudApp	Long	The ID of the application saved in cloud application service
testApp	File	Yes (either testApp or testAppUrl or cloudTestApp)	espresso / XCUI tests.
testAppUrl	String	The URL Location of espresso / XCUI tests(.zip extension only).
cloudTestApp	Long	The ID of the test app saved in cloud file repository service
deviceQueries	String	Yes	For coverage mode: List of strings (XPath queries). The test run should run on devices that match provided queries (device per query). Multiple queries should be passed in the standard way by repeating the parameter multiple times. For Fast Feedback: Only one device query must be provided on the list. The number of devices used is decided by maxDevices and minDevices parameters. If the maxDevices parameter is not provided then the execution will try to reserve the maximum number of devices according to license restriction. It is recommended to use maxDevices parameter always
uuid	String	No	Provision profile UUID to sign the Application and the test application.
runTags	String	No	Map as JSON - wanted tags to upload to the reporter.
overallTimeout	Long	No	The number of milliseconds. Timeout for the overall test run. If not provided - default value (14,400,000) - 4 hours - should be used.
creationTimeout	Long	No	The number of milliseconds. Timeout for test requests in queued status. If not provided - default value (14,400,000) - 4 hours - should be used.
reservationDuration	Long	No	The reservation time of each session in minutes. The default time is 120 minutes * the reservationTime cannot exceed the max reservation time configured for the cloud or project.
includeTests	List of Strings	No	Run only the specified tests. If no matching tests were found an error is reported. The supported formats: * fullClassName, methodName * fullClassName.methodName * fullClassName * methodName * packageName For cucumber tags, run only the matching tags. Each tag will be provided separately: * @tag1 * @tag2
ignoreTestFile	File	No	This file specifies the tests to exclude. iOS * xcscheme file Android * the file containing full class and method name, separated with comma, e.g. `<br/><br/>com.example.app.SomeTest, testFoo<br/><br/>com.example.app.AnotherTest, testBar<br/><br/>`
maxDevices	Integer	No	The maximum number of devices to use during the execution. default value - max available devices license. Note: Only use in case of fastFeedback
minDevices	Integer	No	The minimum number of devices to use during the execution. Default value - 1. Note: Only use in case of fastFeedback
retry	Integer	No	An integer from 0 to 5 - Specifies how many times failed test should be retried Default Value - 0 (means no retry)
additionalAppIds	List	No	List of application ids that should be installed on the device/emulator before starting the Espresso/XCUI test. Application id is returned when uploading the application with Rest API.
retryDifferentDevice	boolean	No	A boolean value specifies whether or not to run a failed test on different devices according to the "retry" value. Default Value - false (means will not try to run a failed test on different devices). If "maxDevices" is specified, then "retry" might be limited to the min between ("maxDevices" - 1) and "retry".
useUIAutomator	boolean	No	For Android only A boolean value decides whether or not to disable Continuous Testing's UIAutomator. Default Value = false - UIAutomator will not be disabled. True - UiAutomator will be disabled on Test Start and will be enabled after it ends
useTestOrchestrator	boolean	No	For Android only A boolean value decides whether or not to run the test in Test Orchestrator Mode. Default Value = false - test will not run in Test Orchestrator Mode true - test will run in Test Orchestrator Mode
clearPackageData	boolean	No	For Android only If useTestOrchestrator is true, then clearPackageData makes sure to remove all shared states from the device's CPU and memory after each test. Default Value = false - don't clear shared state data true - clear shared state data

Response

{
  "status": "SUCCESS",
  "data": {
    "Test Run Id": "3520409",
    "Test Run State": "Finished",
    "Number of tests in file" : "11",
    "Number of requested tests" : "11",
    "Total number of tests": "11",
    "Number of passed tests": "3",
    "Number of failed tests": "8",
    "Number of skipped tests": "0",
    "Number of running tests": "0",
    "Number of queued tests": "0",
    "Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520409%22}"
  },
  "code": "OK"
}

Cancel all tests of the executed suite

Cancel all tests of the executed suit.

POST /api/v1/test-run/<id>/cancel

Where id is Test run id.

Java Code Examples

The examples below using the Unirest HTTP library. To compile then use the following dependencies in your Java Project.

Maven dependency:

<dependency>
    <groupId>com.mashape.unirest</groupId>
    <artifactId>unirest-java</artifactId>
    <version>1.4.9</version>
</dependency>

Gradle dependency:

dependencies {
	compile group: 'com.mashape.unirest', name: 'unirest-java', version: '1.4.9'
}

Following are the code examples:

executeTestRunAsync Expand source

import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import java.io.File;
import org.junit.Test;


public class ExecuteTestRunAsync {

    String urlBase = "http://cloudHost:cloudPort"; // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password"; // modify the admin password as applicable your environment
    String url = urlBase + "/api/v1/test-run/execute-test-run-async";
    String pathToTestApp = "pathtoTestApp"; // path to XcuiTests or espresso tests
    String pathToApp = "pathToFIle";

    @Test
    public void executeTestRunAsync() throws UnirestException {
        File app = new File(pathToApp);      
        File testApp =  new File(pathToTestApp);
        HttpResponse<String> response = Unirest.post(url)
            .basicAuth(user, password)
            .queryString("executionType", "espresso")
			.queryString("runningType", "coverage")
			.queryString("deviceQueries", "@os='android'") // can be repeated if multiple queries
            .field("app", app)
            .field("testApp", testApp)
            .asString();
        System.out.println(response.getBody());
    }
}

executeTestRun Expand source

import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import java.io.File;
import org.junit.Test;


public class ExecuteTestRun {

    String urlBase = "http://cloudHost:cloudPort"; // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password"; // modify the admin password as applicable your environment
    String url = urlBase + "/api/v1/test-run/execute-test-run";
    String pathToTestApp = "pathtoTestApp"; // path to XcuiTests or espresso tests
    String pathToApp = "pathToFIle";

    @Test
    public void executeTestRun() throws UnirestException {
        File app = new File(pathToApp);      
        File testApp =  new File(pathToTestApp);
		Unirest.setTimeouts(0, 0); //set infinity timeout for post request
        HttpResponse<String> response = Unirest.post(url)
            .basicAuth(user, password)
            .queryString("executionType", "espresso")
			.queryString("runningType", "coverage")
			.queryString("deviceQueries", "@os='android'") // can be repeated if multiple queries
            .field("app", app)
            .field("testApp", testApp)
            .asString();
        System.out.println(response.getBody());
    }
}

cancelTestRun Expand source

import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import org.junit.Test;

public class TestRun {
    String urlBase = "http://cloudHost:cloudPort";   // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password";  // modify the admin password as applicable your environment
	Long testRunId = testRunId; // need to put here the test run id
	String url = urlBase + "/api/v1/test-run/" + testRunId +"/cancel";



    @Test
    public void cancelTestRun() throws UnirestException {
        
        HttpResponse<String> response = Unirest.get(url)
            .basicAuth(user, password)
            .asString();

        System.out.println(response.getBody());

    }

}

getTestRunStatus Expand source

import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import org.junit.Test;

public class TestRun {
    String urlBase = "http://cloudHost:cloudPort";   // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password";  // modify the admin password as applicable your environment
	Long testRunId = testRunId; // need to put here the test run id
	String url = urlBase + "/api/v1/test-run/" + testRunId +"/status";



    @Test
    public void getTestRunStatus() throws UnirestException {
        
        HttpResponse<String> response = Unirest.get(url)
            .basicAuth(user, password)
            .asString();

        System.out.println(response.getBody());

    }

}

cURL Command Expand source

curl "https://Cloud_URL/api/v1/test-run/execute-test-run-async?deviceQueries=@os='android'" -H "Authorization: Basic YOUR_CREDENTIALS" -F "executionType=espresso" -F "runningType=coverage" -F "app=@APP_PATH" -F "testApp=@TEST_APP_PATH"

# here "deviceQueries" parameter is passed via query string, as it contains "@", which curl treats as a file name indicator.
# multiple device queries can be specified by adding the parameter multiple times: "https://Cloud_URL/api/v1/test-run/execute-test-run-async?deviceQueries=...&deviceQueries=...".
# if device query contains special characters (e.g. spaces) they should be URL encoded.

Status of the API Run​

Running Async Espresso/XCUITest​

Running Sync Espresso/XCUITest​

Cancel all tests of the executed suite​

Java Code Examples​

Status of the API Run

Running Async Espresso/XCUITest

Running Sync Espresso/XCUITest

Cancel all tests of the executed suite

Java Code Examples