Build Custom Container Plugin using Python SDK

Digital.ai Release provides a modern SDK experience for building container-based task plugins using popular programming languages and frameworks.

This quickstart guide shows you how to create your first custom container plugin using the Python SDK.

note

This guide uses Docker Desktop with Kubernetes for local development. For production deployments, you'll need to set up Digital.ai Release Runner in a proper Kubernetes cluster as described in Run Release Digital.ai Release Runner on Kubernetes cluster.

Prerequisites

Before starting the development, ensure you have the following components ready:

System Requirements

• Operating system: Windows, macOS, or Linux
  • For Apple M1/M2: macOS Ventura with Rosetta enabled in Docker Desktop
• Memory: Minimum 8GB RAM, recommended 16GB RAM
• Administrator privileges (for editing system files)

Required Software

• Docker Desktop with Kubernetes enabled
• Digital.ai Release server instance
• Python 3.8 with pip
• Git

Development Tools

IDE (recommended options):

• PyCharm Community/Professional - Recommended
• Visual Studio Code with Python extension
• Any Python-capable IDE

Source Code

• Digital.ai Release Integration Python SDK template

Step 1 — Set up GitHub Project for Building Container Plugins

This setup process involves three main steps to get your development environment ready:

Step 1.1 — Create a New Repository

Create a duplicate of the Digital.ai Release Integration Python SDK template project to start developing your own container-based integration plugins.

1. In GitHub, go to the Digital.ai Release Integration Python SDK template.
2. Click Use this template > Create a new repository to create a new repository with the contents of this sample template.
3. Provide a repository name, based on the following naming convention.

The best practice is to use the following naming convention to develop container-based integration plugins:

[publisher]-release-[target]-integration

Sample:

acme-release-workshop-integration

For more details, see how to Create a Git repository.

Step 1.2 — Checkout the New Repository

Open the terminal and navigate to a directory where you want to store your project, and run the following command:

git clone https://github.com/digital-ai/acme-release-workshop-integration.git

The repository is cloned in your local system.

Step 1.3 — Configure the project.properties File

Navigate to the project.properties file in the repository, and update as follows:

PLUGIN=acme-release-workshop-integration

note

This is the name of the repository you created in Step 1.1.

Step 2 — Run a Development Instance of Digital.ai Release

In this step, you will learn how to set up a development environment with Release running on Docker using the Template repository.

1. Navigate to the dev-environment folder in the cloned repository, and start the Release and Runner environments by running the following command:

   docker compose up -d --build
2. Once the docker command is executed, the containers will start. Go to Docker Desktop and check the logs of the dev-environment-digitalai-release-1 container for it to display the following lines:

   2023-04-26 11:48:21 2023-04-26 09:48:21.377 [main] {activemq.broker=embedded-broker} INFO  c.x.x.s.jetty.JettyServerListener - Digital.ai Release has started.
   2023-04-26 11:48:21 2023-04-26 09:48:21.378 [main] {activemq.broker=embedded-broker} INFO  c.x.x.s.jetty.JettyServerListener - You can now point your browser to http://host.docker.internal:5516/
3. Open a browser, and go to http://localhost:5516. The Digital.ai Release login screen opens.
4. Log in with username=admin and password=admin.

At the end of this step, you would have started Release. Additionally, a container registry is created, which will be used in the next steps.

Step 3 — Configure your hosts File

note

This set up will vary for a production environment based on your container registry.

For all the containers to find each other, the easiest way is to use the hosts file.

note

You need sudo or Admin privileges to perform this step based on the OS.

To allow the Release server to locate the container images of the integration being developed, a registry is set up within Docker for the development environment. You must include the address of this registry in your local machine's host's file.

For Unix / macOS

Add the following lines in the hosts file in the following location: /etc/hosts

127.0.0.1 digitalai.release.local
127.0.0.1 container-registry
127.0.0.1 host.docker.internal

For Windows

Add the following lines in the hosts file in the following location: C:\Windows\System32\drivers\etc\hosts

127.0.0.1 digitalai.release.local
127.0.0.1 container-registry
127.0.0.1 host.docker.internal

Step 4 — Set up your Development Environment

Complete the following configuration steps to set up your Python development environment:

To keep all the dependencies of Python isolated and prevent interference with other projects on your computer, you can create a virtual environment in Python.

1. Open your terminal.
2. Navigate to the project directory where you want to create the virtual environment.
3. Create a new Python virtual environment in the project directory by running the following command:

python -m venv venv

This will create a new directory called venv in your project directory, which will contain all the Python dependencies. 4. Activate the virtual environment by running the following command: For Linux / macOS

source venv/bin/activate

For Windows

venv\Scripts\activate

Step 4.1 — Set up requirements.txt file to Manage Dependencies

1. Install the required libraries by running the following command:

   pip install -r requirements.txt 

This command will install the required Python packages listed in the requirements.txt file within the virtual environment. 2. You now have a virtual environment with all the necessary dependencies installed for your project.

Step 4.2 — Set up IDE for Coding

1. Open PyCharm.
2. Click Configure Python interpreter link > Python 3.11. The Add Python Interpreter dialog opens.
3. Click Existing radio button.
4. Click OK.

Step 5 — Develop a Custom Container Plugin

Follow these steps to define and implement your custom plugin:

Define and code a new task, based on the Hello example task.

Step 5.1 — Define a Type in type-definitions.yaml

Firstly, you will define the name and properties of the task, so that Digital.ai Release can handle it.

note

The type-definitions.yaml file is newly introduced to replace synthetic.xml for creating plugins. This quick-start is explained using type-definitions.yaml.

1. Navigate to the type-definitions.yaml file.
2. Rename the following:

• containerExamples.BaseTask to workshop.BaseTask.
• containerExamples.Hello to workshop.Greet.

3. Remove the following:

• containerExamples.SetSystemMessage
• containerExamples.ServerQuery
• containerExamples.Server

types:
  workshop.BaseTask:
    extends: xlrelease.ContainerTask
    virtual: true

    hidden-properties:
      image:
        default: "@registry.url@/@registry.org@/@project.name@:@project.version@"
        transient: true
      iconLocation: test.png
      taskColor: "#667385"


  workshop.Greet:
    extends: workshop.BaseTask  # Don't forget to extend the new base task
    description: "Simple greeter task"

    input-properties:
      yourName:
        description: The name to greet
        kind: string
        default: World

    output-properties:
      greeting:
        kind: string

With this metadata and other artifacts like icons, Release will be able to display the task in the UI and execute it. However, when building, it will be put into a zip file and will be uploaded to the Release server.

Step 5.2 — Refine Python Code to Create Custom Plugin

Based on type-definitions.yaml file, the Python SDK will scan the src directory for Python classes with the same name as the type definition.

To develop a custom container plugin:

1. In the hello.py file, rename the Hello class to Greet class.
2. Rename hello.py file to greet.py file.
3. Delete the following unused files:

• sample_release_api_task.py
• sample_server_task.py

You can now build the plugin.

Step 6 — Build and Run Container-based Plugin

A Container-based integration plugin consists of two parts:

• The metadata in the plugin zip file. You will install this file into Digital.ai Release.
• The container image containing the code that will be executed when a task runs.

Learn how to package a plugin and publish the image.

1. Configure the plugin and registry details in the project.properties file.
2. Open the terminal and navigate to the root directory of project, and run the required command:

For Unix/mac OS run the following commands:

• Builds the zip, image and pushes the image to the configured registry

  sh build.sh
• Builds the zip

  sh build.sh --zip
• Builds the image and pushes the image to the configured registry

  sh build.sh --image

For Windows run the following commands:

• Builds the zip, image and pushes the image to the configured registry

  build.bat
• Builds the zip

  build.bat --zip
• Builds the image and pushes the image to the configured registry

  build.bat --image

Step 7 — Install Plugin zip into Release

You can install the plugin using either the command line interface or the Release UI:

Install Plugin via CLI

Update the Release server details in .xebialabs/config.yaml

Run the command for Unix / macOS:

sh build.sh --upload 

Run the command for Windows:

build.bat --upload 

The above command builds the zip and image and uploads the zip to the release server.

Install Plugin via UI

In the Release UI, use the Plugin Manager interface to upload the zip from build. The zip takes the name of the project, for example release-integration-template-python-0.0.1.zip.

1. On the top-right corner, click Settings > Manage plugins.
2. In the left-navigation pane, click Installed plugins.
3. Click Upload to upload the plugin file from your local machine.
   note

   Browse the build directory of the acme-release-workshop-integration project in your local system. The plugin zip takes the name of the project specified in project.properties file. For example, acme-release-workshop-integration-0.0.1.zip.

4. Select the required file and click Upload. The plugin is now available in your Release instance.

Step 8 — Run your First Custom Container Plugin

1. Create a template in Release.

2. Select the newly built Greet container task and add to the template.

3. Start a Release to run the task.

   note

   • If you change the task logic in Python files, just run the build script with --image flag to update the plugin image.
   • If you change type-definitions.yaml, run the plugin script with --upload flag to update plugin types on Release. No restart is required.