Delivery Patterns Tutorial

Delivery Patterns in Release allow you to create pipelines where each team can work at its own timelines and tools, and converge at synchronized points.

With delivery patterns you can perform the following:

• Merge all delivery items at the same time to production, with all items being synchronized in individual phases
• Manage approvals between pipelines
• Sync releases together before moving to another stage of development
• Define dependencies
• Scale up your deployments

Designing the components

This section will walk through the process of creating a basic delivery pattern and release templates to create a set of different microservices from Jira tickets with their own releases, run builds and tests, and deploy them synchronously with Deploy. This will help you to conceptually understand how to use delivery patterns to scale up your releases.

Create a delivery pattern

You can model this by defining your initial pattern. See Manage delivery patterns for more information on this process.

In this case, the delivery flow uses three stages and two transitions, with automated completion and schedule conditions on the transitions.

For more information on how to add a Stage Owner, see Stage Owner.

Note If you want, you can also predefine the set of tracked items you want to use in the delivery. Select the pattern and click Tracked items in the left navigation bar, and then click Add tracked item to create the list of items.

Note that, the options in the Show drop-down list is moved to the left navigation bar when you select a Pattern or a Delivery.

Delivery View

Pattern View

Create release templates

This scenario will use two main templates. In a real production scenario, all teams would use their own templates to work at their own pace, and sync up in the delivery stages.

Create microservices template

We can create one 'master' delivery release template to create the release items, start the delivery, and generate tracked items.

In the example here, the release will:

• Use a Jira query to fetch a set of tickets, and use a list variable to create a set of tracked items from the list. Each microservice component will use a separate release to fetch its own set of tickets.
• Create a delivery using the pattern above, using the tracked items you created
• Wait until the delivery moves to end-to-end testing. This will allow each release containing the tracked items to complete its necessary steps and move to testing and then pushing to production.
• Notify stakeholders and update ServiceNow change request tickets as the release progresses.

In the first phase of this release, we have added:

• A Jira query ticket to fetch fetch a set of tickets
• A script task to parse the tickets into a list variable
• A Create delivery task with the ID of the delivery pattern
• A Register tracked items task that uses the list of Jira tickets as variables for creating tracked items

The next stages of the release start with a Wait for stage task, to allow the delivery approval milestones to happen before triggering the release actions.

Create microservice release process template

A second release template can be used to release the tracked items picked up by the first template as a set of component releases.

This template does the following things:

• Allows you to define the specific microservice for each release
• Obtains the list of Jira tickets and parses them for the microservice, then uses them to create the set of tracked items for this release. This will register the release in the delivery and force the release to wait when it reaches the testing stage.
• Run sanity checks and build the services, then update the tracked items when the build is successfully completed.
• When the end-to-end testing phase of delivery starts, run testing tasks and update the tracked items when testing has been successful, then wait until the delivery moves to pushing to production
• Deploy the microservices via Deploy, run a smoke test, then mark the tracked items as complete

Run the releases

Run the create microservices release

1. Create a new release from the create microservices template. In the Name of the new delivery field, enter the name of the delivery you want to create.
2. Start the release. Assuming that everything has been correctly configured, the first phase of the release will complete, and you will have started a new delivery with a set of tracked items you picked up from the Jira tasks, and the release will pause at the beginning of the next phase.

Open the new delivery

1. Click the delivery link in the Delivery: Wait for Stage task to open the new delivery and view the tracked items you created.

Run the release to deploy the microservices

1. Create a set of releases from the Microservice release process template, using one component tag for each release:
2. When the release runs, if every step was successful it will pause at the Test: Wait to start end-to-end testing phase.
3. Run all the releases until they get to this stage.

Approve the build and test stage and move the delivery to testing

1. Open the delivery when all the releases are waiting. You will see that the tracked items are all marked as complete, and the Wait and Sync approval phase is ready to transition the items.

   Note Any items that were not completed can be skipped or manually completed by expanding the stage, clicking the menu to the right, and selecting the option. Skipped items will not be part of any further stages.

   

   This view will also show you which releases each tracked item is linked to, and you can click the links to navigate to them.

   

2. Click Transition and complete the previous stage. The delivery will move to end-to-end testing, and will trigger the releases to begin their test phases.

Open the releases

1. In the releases, you can now see that if everything was successful, it will have moved to the stage of waiting for the Push to Production stage of the delivery.

2. Repeat the steps above, approving the delivery stage and moving the delivery to its final "Push to Prod" stage.

3. The releases will start the final stage of deploying to Deploy.

   Note If there are any issues with the tracked items during the release, they will be marked as failing in the delivery as well.

4. You can also view the activity logs for the delivery by selecting Activity logs in the left navigation bar, if you want to track the changes or troubleshoot a delivery.

5. When the stages have all completed, the delivery will also complete itself.

Apply the tutorial templates in yaml

You can create the patterns above by applying the following YAML sample. For more information, see Using DevOps as Code with Software Delivery.

Note: to run this sample, you will need to set up your own services such as:

• Jira
• Jenkins
• ServiceNow
• BlackDuck
• Deploy

Refer to the documentation to set up these integrations in Release. However, it is unlikely that you would want to run this entire exercise, and it should be viewed more as a blueprint for how you might want to set up Software Delivery.

---
apiVersion: xl-release/v1
kind: Templates
spec:
- directory: Microservices Delivery
  children:
  - pattern: ECommerce Microservices Delivery
    stages:
    - stage: Build and Test
      transition:
        name: CAB Approval
        type: delivery.Transition
        conditions:
        - type: delivery.ConditionGroup
          operator: OR
          conditions:
          - type: delivery.ConditionGroup
            operator: AND
            conditions:
            - type: delivery.ItemsCompletionCondition
    - stage: end-to-end testing
      transition:
        name: Wait and Sync
        type: delivery.Transition
        conditions:
        - type: delivery.ConditionGroup
          operator: OR
          conditions:
          - type: delivery.ConditionGroup
            operator: AND
            conditions:
            - type: delivery.TimeCondition
    - stage: Push to Prod
  - template: Create Microservices Delivery
    scheduledStartDate: 2019-10-22T09:00:00+02:00
    dueDate: 2020-03-09T12:17:19.602+01:00
    phases:
    - phase: Create delivery
      tasks:
      - name: Create weekly delivery
        type: delivery.CreateDelivery
        ^title: ${DeliveryName}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.deliveryId: ${DeliveryID}
      - name: Get Jira tickets for all microservices in the delivery
        type: jira.Query
        query: project = SAN AND issuetype in (Bug, Story) AND creator in (admin)
          AND labels = SoftwareDeliveryExample
        variableMapping:
          pythonScript.issues: ${JiraTickets}
      - name: Parse Jira tickets
        type: xlrelease.ScriptTask
        script: |-
          TrackedItems = []

          for k, v in releaseVariables['JiraTickets'].items():
              TrackedItems.append('{} {}'.format(k, v))

          releaseVariables['TrackedItems'] = TrackedItems
      - name: Register Jira tickets as tracked items in delivery
        type: delivery.RegisterTrackedItems
        delivery: ${DeliveryID}
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      color: '#00875A'
    - phase: Wait for CI cycles
      tasks:
      - name: Wait for the "End-to-End Testing" stage to finish
        type: delivery.WaitForStage
        stage: end-to-end testing
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
      - name: Send update to Product Management
        type: xlrelease.NotificationTask
        addresses:
        - admin@company.com
       subject: Delivery ${DeliveryName} has reached E2E Testing
        body: Delivery ${DeliveryName} has reached E2E Testing
      color: '#CC4A3C'
    - phase: Approval
      tasks:
      - name: Create change request for production deployment
        type: servicenow.CreateChangeRequest
        servicenowServer: ServiceNow
        shortDescription: Change for ${DeliveryName}
        variableMapping:
          pythonScript.Ticket: ${ServiceNowNumber}
          pythonScript.sysId: ${ServiceNowSysId}
          pythonScript.data: ${ServiceNowData}
      - name: Wait for the "Push to Prod" stage to finish
        type: delivery.WaitForStage
        stage: Push to Prod
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
      - name: Update change request
        type: servicenow.UpdateChangeRequest
        servicenowServer: ServiceNow
        shortDescription: ${DeliveryName} is complete
        sysId: ${ServiceNowSysId}
        variableMapping:
          pythonScript.Ticket: ${ServiceNowNumber}
          pythonScript.data: ${ServiceNowData}
      color: '#FFAB00'
    realFlagStatus: ATTENTION_NEEDED
  tags:
    - ecommerce
    - delivery
    variables:
    - type: xlrelease.StringVariable
      key: DeliveryName
      label: Name of the new delivery
    - type: xlrelease.StringVariable
      key: DeliveryID
      requiresValue: false
      showOnReleaseStart: false
    - type: xlrelease.MapStringStringVariable
      key: JiraTickets
      requiresValue: false
      showOnReleaseStart: false
    - type: xlrelease.ListStringVariable
      key: TrackedItems
      requiresValue: false
      showOnReleaseStart: false
      label: Jira tickets translated to tracked items for the delivery
    - type: xlrelease.MapStringStringVariable
      key: ServiceNowData
      requiresValue: false
      showOnReleaseStart: false
    - type: xlrelease.StringVariable
      key: ServiceNowSysId
      requiresValue: false
      showOnReleaseStart: false
    - type: xlrelease.StringVariable
      key: ServiceNowNumber
      requiresValue: false
      showOnReleaseStart: false
    - type: xlrelease.StringVariable
      key: AccountServices
      requiresValue: false
      showOnReleaseStart: false
    scriptUsername: robot
    riskProfile: Default risk profile
  - template: Microservice Release Process
    scheduledStartDate: 2019-10-22T09:00:00+02:00
    dueDate: 2019-10-22T10:00:00+02:00
    plannedDuration: 3600
    phases:
    - phase: Register with delivery
      tasks:
      - name: Get Jira tickets for microservice
        type: jira.Query
        query: project = SAN AND issuetype in (Bug, Story) AND creator in (admin)
          AND component = ${microservice} AND labels = SoftwareDeliveryExample
        variableMapping:
          pythonScript.issues: ${JiraTickets}
      - name: Parse Jira tickets into tracked items
        type: xlrelease.ScriptTask
        script: |-
          TrackedItems = []

          for k, v in releaseVariables['JiraTickets'].items():
              TrackedItems.append('{} {}'.format(k, v))

          releaseVariables['TrackedItems'] = TrackedItems
      - name: Find this week's delivery
        type: delivery.FindOrCreateDelivery
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.deliveryId: ${DeliveryID}
      - name: Register tracked items in the delivery
        type: delivery.RegisterTrackedItems
        delivery: ${DeliveryID}
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      - name: Notify the Product Owner that the microservice is registered with the
          delivery
        type: xlrelease.NotificationTask
        addresses:
        - admin@company.com
       subject: ${microservice} is registered with ${DeliveryID}
        body: |-
          ${microservice} is registered with ${DeliveryID}

          Tracked items are:

          ${TrackedItems}
      color: '#65747C'
    - phase: Build
      tasks:
      - name: Scan open source libraries
        type: blackduck.checkCompliance
        projectName: ${global.blackduck}
        versionName: master
        securityRiskHighThreshold: 1000
        securityRiskMediumThreshold: 700
        securityRiskLowThreshold: 500
        licenseRiskHighThreshold: 1000
        licenseRiskMediumThreshold: 700
        licenseRiskLowThreshold: 500
        operationalRiskHighThreshold: 1000
        operationalRiskMediumThreshold: 700
        operationalRiskLowThreshold: 500
      - name: Build base artifact
        type: jenkins.Build
        jobName: One-minute
      - name: Build plugins
        type: jenkins.Build
        jobName: One-minute
      - name: Package base artifact with plugins
        type: jenkins.Build
        jobName: One-minute
      - name: Mark tracked items from this release as complete in the "Build and Test"
          stage
        type: delivery.MarkTrackedItems
        stage: Build and Test
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      color: '#0079BC'
    - phase: Test
      tasks:
      - name: Wait for all tracked items (across all releases) to be available in
          the "End-to-End Test" stage
        type: delivery.WaitForTrackedItems
        stage: end-to-end testing
        ^status: NOT_READY
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      - name: Trigger Selenium tests
        type: jenkins.Build
        jobName: One-minute
      - name: Mark tracked items from this release as complete in the "End-to-End
          Test" stage
        type: delivery.MarkTrackedItems
        stage: end-to-end testing
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      color: '#CC4A3C'
    - phase: Deploy
      tasks:
      - name: Wait for all tracked items (across all releases) to be available in
          the "Push to Prod" stage
        type: delivery.WaitForTrackedItems
        stage: Push to Prod
        ^status: NOT_READY
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      - name: Deploy ${microservice} to production
        type: xldeploy.Deploy
        deploymentPackage: ${package}
        deploymentEnvironment: ${environment}
      - name: Smoke test the deployment
        type: webhook.JsonWebhook
        URL: ${global.webhook}
      - name: Mark tracked items from this release as complete in the "Push to Prod"
          stage
        type: delivery.MarkTrackedItems
        stage: Push to Prod
        delivery: ${DeliveryID}
        pattern: Microservices Delivery/ECommerce Microservices Delivery
        variableMapping:
          pythonScript.trackedItems: ${TrackedItems}
      - name: Update Jira tickets
        type: jira.UpdateIssues
        comment: 'Sample comment from release: ${release.title}'
        updateSummaries: false
        variableMapping:
          pythonScript.issues: ${JiraTickets}
      color: '#00875A'
  tags:
    - ecommerce
    - microservice
    variables:
    - type: xlrelease.StringVariable
      key: DeliveryID
      requiresValue: false
      showOnReleaseStart: false
      label: Delivery ID
    - type: xlrelease.MapStringStringVariable
      key: JiraTickets
      requiresValue: false
      showOnReleaseStart: false
      label: Jira Tickets
    - type: xlrelease.ListStringVariable
      key: TrackedItems
      requiresValue: false
      showOnReleaseStart: false
      label: Jira tickets translated to tracked items for the delivery
    - type: xlrelease.StringVariable
      key: microservice
      label: Microservice
      description: Microservice to be released (component name in Jira)
    - type: xlrelease.StringVariable
      key: package
      label: Deployment package
      value: Applications/Samples/ECommerce/
    - type: xlrelease.StringVariable
      key: environment
      label: Environment
      value: Environments/Samples/PROD
    scriptUsername: robot
    riskProfile: Default risk profile
- name: ServiceNow
  type: servicenow.Server
  url: https://servicenow.com
  username: admin
  password: !value "servicenow_Server_ServiceNow_password"