Skip to main content

Testing and Quality Reports

Testing and quality reports provide visibility into defect patterns, test progress, and overall product quality. Use these reports to track defect trends across different dimensions, monitor test execution, identify quality issues early, and make data-driven release decisions.

Report Categories​

Defect Trend Reports​

Track defects over time using various breakdown dimensions to identify patterns and assess quality trajectory.

Defect Trend Reports: Comprehensive guide covering three defect trend dimensions:

  • Defect Priority Trend Report: Tracks defects by priority level (Critical, High, Medium, Low)
  • Defect Resolution Trend Report: Tracks defects by resolution type (Fixed, Won't Fix, Cannot Reproduce, etc.)
  • Defect Status Trend Report: Tracks defects by workflow status (Ready for Dev, Developing, Testing, etc.)

Best for: Release planning, quality monitoring, bottleneck identification, and resolution effectiveness tracking.

Issue Trend Reports​

Track issues (different from defects) over time to monitor non-defect work items affecting quality or delivery.

Issue Trend Report: Displays total issues within a project over time, breaking them down by status. Issues represent impediments, risks, or other tracked items distinct from defects.

Best for: Risk tracking, impediment management, and monitoring non-defect quality concerns.

Test Reports​

Monitor test execution, coverage, and results from both internal and external testing systems.

Test Trend Report: Shows how test statuses change over time within Digital.ai Agility. Accessible from the Testboard for sprint-specific test tracking.

Best for: Sprint-level test monitoring, internal test status tracking, and test completion trends.

Test Runs Report: Displays aggregated test results from external testing systems, showing pass/fail/not executed counts and elapsed time. Supports integration with third-party testing tools.

Best for: External test system integration, comprehensive test suite tracking, and cross-system test reporting.

Choosing the Right Report​

By Use Case​

GoalReport to UseWhy
Assess release readinessDefect Priority TrendShows critical/high-priority defect counts and trends
Identify workflow bottlenecksDefect Status TrendReveals where defects accumulate in your process
Evaluate defect resolution qualityDefect Resolution TrendShows how defects are resolved (fixed vs deferred vs duplicates)
Track sprint test progressTest Trend ReportMonitors test status changes within the sprint
Monitor impediments and risksIssue Trend ReportTracks issues affecting delivery
View external test resultsTest Runs ReportAggregates pass/fail data from testing tools
Make go/no-go decisionsDefect Priority + Test TrendCombines defect severity with test coverage
Improve QA processDefect Resolution + Status TrendsIdentifies resolution patterns and bottlenecks

By User Role​

Product Owners & Release Managers:

  • Defect Priority Trend Report: Monitor critical defect counts before releases
  • Test Runs Report: Verify external test pass rates
  • Defect Resolution Trend Report: Ensure defects are being fixed, not just deferred

QA Leads & Test Managers:

  • Test Trend Report: Track test execution progress within sprints
  • Test Runs Report: Monitor external testing tool results
  • Defect Status Trend Report: Identify bottlenecks in defect resolution workflow

Scrum Masters & Team Leads:

  • Defect Status Trend Report: Identify workflow issues and capacity constraints
  • Issue Trend Report: Monitor impediments affecting the team
  • Test Trend Report: Track sprint test completion

Development Teams:

  • Defect Priority Trend Report: Understand defect severity distribution
  • Defect Status Trend Report: See where defects are in the workflow
  • Test Trend Report: View sprint test status

Process Improvement:

  • Defect Resolution Trend Report: Analyze resolution effectiveness
  • Defect Status Trend Report: Identify systemic workflow problems
  • All reports combined: Comprehensive quality process assessment

By Review Frequency​

Daily:

  • Defect Priority Trend: Check for critical defect increases during active release periods
  • Test Trend Report: Monitor daily sprint test progress

Weekly:

  • All Defect Trend Reports: Review quality patterns and identify early issues
  • Issue Trend Report: Track impediment resolution progress
  • Test Trend Report: Assess weekly test completion rates

Sprint Review:

  • Defect Status Trend: Present defect workflow health to stakeholders
  • Test Trend Report: Show sprint test coverage and results
  • Defect Resolution Trend: Demonstrate resolution quality

Monthly / Release Planning:

  • Defect Priority Trend: Assess long-term quality trajectory
  • All Defect Trends Combined: Comprehensive quality assessment for release decisions
  • Test Runs Report: Review external test suite pass rates over time

Common Use Cases​

Release Planning​

Assess Release Readiness:

  1. Review Defect Priority Trend for critical and high-priority defect counts
  2. Check Defect Status Trend to see how close defects are to resolution
  3. Examine Test Runs Report for external test pass rates
  4. Review Defect Resolution Trend to ensure defects are being fixed, not deferred
  5. Combine data to make informed go/no-go decisions

Establish Quality Gates:

  • Maximum acceptable critical defects (e.g., 0 critical, < 5 high-priority)
  • Minimum test pass rate (e.g., 95% pass rate on Test Runs Report)
  • Required trends (e.g., decreasing critical defects over last 2 weeks)
  • Workflow criteria (e.g., minimal defects in "Ready for Dev" status)

Sprint Execution​

Daily Standups:

  • Review Test Trend Report for today's test status
  • Check Defect Priority Trend for new critical defects
  • Monitor Issue Trend Report for impediments

Sprint Planning:

  • Use Defect Status Trend to understand defect backlog
  • Plan defect resolution capacity based on Defect Priority Trend
  • Allocate testing capacity based on Test Trend Report history

Sprint Review:

  • Present quality metrics using defect and test trends
  • Demonstrate resolution effectiveness with Defect Resolution Trend
  • Discuss impediments tracked via Issue Trend Report

Process Improvement​

Identify Workflow Bottlenecks:

  1. Use Defect Status Trend to find where defects accumulate
  2. Analyze root causes (capacity, dependencies, process issues)
  3. Implement improvements
  4. Monitor Defect Status Trend to validate changes

Improve Resolution Quality:

  1. Review Defect Resolution Trend for patterns (high "Cannot Reproduce," many "Duplicates")
  2. Address root causes (better defect reporting, improved triage)
  3. Track improvements in resolution distribution over time

Enhance Testing Process:

  1. Compare Test Trend Report with Test Runs Report to understand internal vs external testing
  2. Identify test coverage gaps
  3. Improve testing practices
  4. Monitor trends to validate improvements

Cross-Report Analysis​

Comprehensive Quality Assessment:

  • Defect Priority + Status Trends: Do high-priority defects move faster through workflow?
  • Defect Resolution + Status Trends: Where do bottlenecks affect resolution quality?
  • Test Trend + Defect Priority: Test coverage vs defect severity correlation
  • All Trends Combined: Complete quality picture for release decisions

Quality-Speed Balance:

  • Compare Defect Priority Trend with velocity reports
  • Assess whether increased velocity correlates with decreased quality
  • Find sustainable balance between speed and quality

Best Practices​

Establish Review Cadence:

  • Set regular intervals for reviewing quality reports (daily, weekly, sprint, release)
  • Include quality metrics in sprint reviews and retrospectives
  • Make quality trends visible on team dashboards
  • Use trends to drive continuous improvement discussions

Maintain Data Quality:

  • Update defect status regularly (at least daily)
  • Assign defect priority consistently using clear criteria
  • Use resolution values accurately
  • Update test status promptly
  • Close defects and tests when complete

Combine Multiple Perspectives:

  • Never rely on a single report for quality assessment
  • Use defect trends, test reports, and issue tracking together
  • Correlate quality data with delivery metrics (velocity, cycle time)
  • Consider context (releases, team changes, new features increasing defects)

Act on Insights:

  • Don't just review trends—take action on patterns
  • Address critical defect growth immediately
  • Investigate and resolve workflow bottlenecks
  • Improve processes based on resolution and test data
  • Celebrate quality improvements

Set Clear Thresholds:

  • Define acceptable defect counts by priority
  • Establish minimum test pass rates
  • Set quality gates for releases
  • Document criteria for go/no-go decisions
  • Review and adjust thresholds based on experience

Prerequisites​

For Accurate Trend Reports:

  • Minimum 2 data points (trend requires comparison over time)
  • Consistent data entry (regular status updates, accurate priorities)
  • Proper configuration of work item types (defects, issues, tests)
  • Correct sprint/iteration assignments
  • Team assignments for filtered views

For Meaningful Analysis:

  • Clear understanding of your organization's defect workflow
  • Documented priority and resolution criteria
  • Team awareness of reporting and data quality importance
  • Regular cadence of report review
  • Action plans to address identified issues

Weekend Settings: Your "Show Weekends" setting affects data point calculation. If weekends are excluded and your selected duration includes weekend days, ensure you have sufficient weekdays for data points.

Technical Considerations​

Report Performance:

  • Trend reports over long periods take longer to generate
  • Use filters to focus on relevant timeframes and teams
  • Export large reports for offline analysis
  • Consider shorter intervals for faster rendering

Custom Workflows:

  • Custom team processes affect Defect Status Trend Report (may show duplicate statuses when viewing "All Teams")
  • Resolution and priority values can be customized by System Administrators
  • Test statuses may vary by team configuration
  • Filter by individual team for clearest status visualization

Data Aggregation:

  • Reports show real-time or near-real-time data
  • Historical trends require time for data aggregation
  • Changes to work items update trend calculations
  • Deleted work items may affect historical trends