Skip to main content

View Test Results in Test Reporter

Overview​

After executing tests with Smart Agent, you can analyze results in the Test Reporter UI. This walkthrough guides you through filtering, analyzing failures, and debugging using real execution data.

Scenario​

You executed a test run with:

Test Execution Results​

Build IDRun IDTest Case IDTest NameStatusRetryLast Error
1234run-001TC-101iOSNativeLoginTestFailed2Element not found
1234run-001TC-102iOSBankPaymentTestPassed0—

1: Open Test Reporter​

  1. In the Digital.ai Testing cloud, navigate to the Reports page.

    This page shows all executions across users and projects.

    Reports

2: Filter by Build ID​

  1. Click the filter option and type Build ID and press enter:
  2. Apply the filter to narrow down results.

Result​

Displays only tests executed as part of this specific run.

Filter by Build ID

3: Review Test Results​

You will now see filtered results:

  • iOSNativeLoginTest (Failed)
  • iOSBankPaymentTest (Passed)

Filtered Results

4: Analyze a Failed Test​

  1. Click on the failed test to open detailed results.

Example Failure

  • Error: Element not found

What this means:
The test could not locate a UI element—likely a locator or UI issue.

5: Check Retry Attempts​

Scroll to view retry attempts for the failed test.

Retry Attempts

You may see:

  • Attempt 1 → Failed
  • Attempt 2 → Failed
  • Attempt 3 → Failed

Insight:
If all retries fail, the issue is likely not flaky and requires fixing.

6: Filter by Test Case ID​

To analyze test history:

  1. Copy the Test Case ID
  2. Apply the filter

Filter by Test Case ID

What you learn:

  • Past executions
  • Stability trends
  • Flaky behavior

7: Filter by Run ID (Advanced)​

If multiple agents are used:

  1. Locate Run ID filter
  2. Select a Run ID

Filter by Run ID

Use case:
Isolate results from a specific Smart Agent execution.

8: Debug Using Logs​

  1. Open logs for the failed test.
  2. Review execution flow, retry attempts, and failure reasons.

Goal: Identify the root cause quickly.

End-to-End Flow​

  1. Execute: Trigger tests using the Smart Agent.
  2. Trace: Use the build.id to isolate the specific execution in the Cloud Reporter.
  3. Analyze: Identify failures and investigate logs or video recordings for root cause analysis.
  4. Validate: Review automated retries to distinguish between environmental flakiness and genuine code defects.
  5. Optimize: Debug the script or environment, apply the fix, and re-run the targeted tests to confirm resolution.
Last updated on