Skip to main content
Version: Release 24.3

Store Reports and Logs on S3—Kubernetes

This topic describes how to configure Digital.ai Release on Kubernetes to store reports and task logs on Amazon S3. This setup eliminates the need for persistent volumes, making storage management easier for Kubernetes-based release instances.

Prerequisites​

  • An Amazon account with permissions to create and manage S3 buckets.
  • AWS credentials via Default credential provider chain.
  • AWS CLI for moving existing files to S3.

If you are using Kubernetes, you will need to update the Release Operator configuration file to enable S3.

Step 1—Update the Operator CR yaml File​

Open the Release Operator Custom Resource (CR) file and add the following configuration under the reportingEngine and jobLog sections:

spec:
  storage:
    reportingEngine:
      s3:
        enabled: true
        bucket: "xlr-saas-test-01"
        baseKey: "xlr-on/reports"
        region: "us-east-1"
    jobLog:
      s3:
        enabled: true
        bucket: "xlr-saas-test-01"
        baseKey: "xlr-on/logs"
        region: "us-east-1"

Where:

  • s3.bucket: The name of your S3 bucket (e.g., xlr-saas-test-01).
  • s3.baseKey: The path in the bucket where logs and reports will be stored (e.g., xlr-on/logs and xlr-on/reports).
  • s3.region: The AWS region where your bucket is located (e.g., us-east-1).

Note: When configuring the s3.region setting, ensure that the chosen AWS region is geographically close to your XL Release nodes. This minimizes network latency, which can significantly impact the performance of storing and retrieving logs and reports from S3. Choosing a region near your deployment reduces delays and improves overall efficiency.

Step 2—(Optional) Update Configuration to Disable Persistent Storage​

If you want to stop using persistent storage for reports, add this configuration:

spec:
  persistence:
    enabled: false
    paths: []
    emptyDirPaths:
      - /opt/xebialabs/xl-release-server/cache
      - /opt/xebialabs/xl-release-server/conf
      - /opt/xebialabs/xl-release-server/driver/jdbc
      - /opt/xebialabs/xl-release-server/driver/mq
      - /opt/xebialabs/xl-release-server/node-conf
      - /opt/xebialabs/xl-release-server/plugins
      - /opt/xebialabs/xl-release-server/work
      - /tmp
      - /opt/xebialabs/xl-release-server/reports

This removes the need for persistent volume (PV) by storing files in local storage (emptyDir), and S3 will be used for reports.

Step 3—Migrate Existing Logs and Reports​

For migrating existing data, follow these steps:

  1. Stop the Release server.

  2. Use the AWS CLI to transfer existing data to S3.

    aws s3 cp --recursive /path/to/reports/ s3://xlr-saas-test-01/xlr-on/reports/
    aws s3 cp --recursive /path/to/job_logs/ s3://xlr-saas-test-01/xlr-on/logs/

Step 4—Update Configuration to Switch to S3​

After migrating the data, ensure that your Operator CR yaml file is configured to switch from local storage to S3. Refer to Step 1 for configuration details on how to set this up.

Step 5—Restart Release Instances​

After updating the configuration and completing the migration, restart Release to ensure the new settings take effect.