Skip to main content
Version: Release 24.3

Store Reports and Logs on S3—JVM Cluster

You can configure Digital.ai Release to save reports and logs on Amazon S3 instead of the local file system. Here's how you can configure S3 storage and move your existing data if necessary.

Prerequisites​

  • An Amazon account with permissions to create and manage S3 buckets.
  • AWS credentials via the Default credential provider chain.
  • AWS CLI for moving existing files to S3.

Step 1—Update the xl-release.conf File​

Update the xl-release.conf file to configure S3 for storing logs and reports. Here's an example configuration:

xl {
  job-log {
    s3 {
      base-key="your-base-key/logs"
      bucket=your-bucket-name
      region=your-region
    }
    type=xlr-s3
  }
  reporting {
    engine {
      s3 {
        base-key="your-base-key/reports"
        bucket=your-bucket-name
        region=your-region
      }
      type=xlr-s3
    }
  }
}

Where:

  • type: Specifies the storage type. Use xlr-s3 to enable S3.
  • s3.bucket: The name of your S3 bucket (e.g., xlr-saas-test-01).
  • s3.baseKey: The path in the bucket where logs and reports will be stored (e.g., xlr-on/logs and xlr-on/reports).
  • s3.region: The AWS region where your bucket is located (e.g., us-east-1).

Note: When configuring the s3.region setting, ensure that the chosen AWS region is geographically close to your XL Release nodes. This minimizes network latency, which can significantly impact the performance of storing and retrieving logs and reports from S3. Choosing a region near your deployment reduces delays and improves overall efficiency.

AWS will automatically handle credentials using the default credential provider chain.

Step 2—Migrate Existing Logs and Reports:​

If you are migrating existing data, follow these steps:

  1. Stop the Release server.

  2. Use the AWS CLI to copy your existing data to S3.

    aws s3 cp --recursive /path/to/reports/ s3://xlr-saas-test-01/xlr-on/reports/
    aws s3 cp --recursive /path/to/job_logs/ s3://xlr-saas-test-01/xlr-on/logs/

Step 3—Update Configuration to Switch to S3​

After migrating the data, ensure that your xl-release.conf file is updated to switch from local storage to S3. Refer to Step 1 for configuration details on how to set this up.

Step 4—Restart Release​

After updating the configuration and migrating data, restart Release to apply the changes.