Customize Your Site— Deploy Custom Configuration

Overview

In an operator-based environment, Deploy uses a central configuration server to manage settings for both master and worker nodes. This server generates configuration files from templates located in the central-conf directory. During startup, these templates are processed to create configuration files in the central configuration directory, but only if the corresponding file does not already exist. Starting with version 24.1, the centralConfiguration and conf directories are no longer mounted with Persistent Volumes (PVs), preventing manual storage or modification of files.

Prerequisites

• A Digital.ai Deploy installation on a Kubernetes cluster
• kubectl connected to the cluster
• Deploy operator version 24.1 or later

Configure Central Configuration Pod

Generic update of template-based files:

It is possible to update other text files provided in the application directory. For example, to update the central-conf files:

• deploy-caches.yaml.template
• deploy-cluster.yaml.template
• deploy-metrics.yaml.template
• deploy-oidc.yaml.template
• deploy-plugins.yaml.template
• deploy-reporting.yaml.template
• deploy-repository.yaml.template
• deploy-satellite.yaml.template
• deploy-secret-complexity.yaml.template
• deploy-server.yaml.template
• deploy-task.yaml.template
• logback.xml

or default-conf files:

• boot.conf.cloud.template
• boot.conf.template
• deployit.conf.template
• jmx-exporter.yaml
• logback-access.xml
• logback.xml
• logging.properties
• wrapper-daemon.vm
• xlc-wrapper.conf.common
• xlc-wrapper.conf.posix
• xlc-wrapper.conf.win

Any missing file in the central-conf template directory is created with the default values.

Change configuration file on the central configuration server

Following are the steps to update the configuration file on the central configuration server. The default template file is located in the central-conf directory and is processed during the startup of the central configuration server. To update the configuration file, you must download the template file, modify it, and then create a patch file to update the configuration on the central configuration server.

1. Download the template file by running the following command:

   kubectl cp -c deploy \
     digitalai/dai-xld-digitalai-deploy-cc-server-0:central-conf/deploy-task.yaml.template \
     deploy-task.yaml.template

2. Customize the file deploy-task.yaml.template as needed by running the following command:

   vi deploy-task.yaml.template

3. Create a patch file to update the configuration on the central configuration server using the following:

   cat << EOF >> deploy-task.yaml.template.patch.yaml
   spec:
     centralConfiguration:
     extraConfiguration: 
       "central-conf_deploy-task_yaml_template":
         path: "central-conf/deploy-task.yaml.template"
         mode: 0660
         content: |
           deploy.task:
             in-process-worker: ${XLD_IN_PROCESS}
             queue:
               name: ${XLD_TASK_QUEUE_NAME}
               archive-queue-name: "xld-archive-queue"
               in-process:
                 maxDiskUsage: ${XLD_TASK_QUEUE_IN_PROCESS_MAX_DISK_USAGE}
                 shutdownTimeout: ${XLD_TASK_QUEUE_IN_PROCESS_SHUTDOWN_TIMEOUT}
               external:
                 jms-driver-classname: "${XLD_TASK_QUEUE_DRIVER_CLASS_NAME}"
                 jms-url: "${XLD_TASK_QUEUE_URL}"
                 jms-username: "${XLD_TASK_QUEUE_USERNAME}"
                 jms-password: |-
                   ${XLD_TASK_QUEUE_PASSWORD}
                 rabbitmq:
                   quorum-queue: ${XLD_TASK_QUEUE_IS_QUORUM}
   EOF

4. Patch the configuration by changing the CR for Deploy central configuration pods using the following command:

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file deploy-task.yaml.template.patch.yaml

   Applying the patch command will restart the central configuration server, updating the configuration file. The Deploy master and worker nodes will receive the updated configuration during their next startup.

5. Validate the changes. The CR yaml will have a new section under extraConfiguration key.

   kubectl get Digitalaideploys dai-xld -o yaml -n digitalai

   After restarting, the central configuration pod container uses the new template file. Verify the Custom Resource (CR) changes. It may take up to one minute for the central configuration pods to automatically restart and replace the file with the new template. The duration depends on reconciliation period which is set to one minute by default:

   kubectl exec -it sts/dai-xld-digitalai-deploy-cc-server -n digitalai \
     -- cat /opt/xebialabs/central-configuration-server/central-conf/deploy-task.yaml.template

6. The master and worker pods do not automatically restart when there are changes to the CR. Therefore, a manual restart is required. The recommended approach is to delete the STS and allow the operator controller to recreate the new STS.

   kubectl delete sts dai-xld-digitalai-deploy-master -n digitalai
   kubectl delete sts dai-xld-digitalai-deploy-worker -n digitalai

After restarting master and worker pods, you can validate the configuration file on the central configuration server.

Exception for changing templates for central-conf/deploy-server.yaml.template and central-conf/deploy-oidc.yaml.template

The templates:

• central-conf/deploy-server.yaml.template
• central-conf/deploy-oidc.yaml.template

already exist in the default CR configuration under keys respectively:

• central-conf_deploy-server-yaml-template
• central-conf_deploy-oidc-yaml-template

So for the files use following patch files:

• for central-conf/deploy-server.yaml.template

cat << EOF >> deploy-task.yaml.template.patch.yaml
spec:
  centralConfiguration:
   configuration: 
    "central-conf_deploy-server-yaml-template":
      path: "central-conf/deploy-server.yaml.template"
      mode: 0660
      content: |
        deploy.server:
          bind-hostname: 0.0.0.0
          bind-port: {{ .Values.master.containerPorts.deployPekko }}
          license:
            daysBeforeWarning: 10
          {{- if .Values.oidc.enabled }}    
          security:
            auth:
              provider: "oidc"
          {{- end }}
        pekko:
          io:
            dns:
              resolver: "${DNS_RESOLVER}"
          # loggers:
          #  - "org.apache.pekko.event.slf4j.Slf4jLogger"
          loglevel: "INFO"
EOF

• for central-conf/deploy-oidc.yaml.template

cat << EOF >> deploy-oidc.yaml.template.patch.yaml
spec:
  centralConfiguration:
   configuration: 
    "central-conf_deploy-oidc-yaml-template":
      path: "central-conf/deploy-oidc.yaml.template"
      mode: 0660
      content: |
        deploy.security:
          {{- if .Values.oidc.enabled }}
          auth:
            providers:
              oidc:
                clientId: "${OIDC_CLIENT_ID}"
                clientSecret: "${OIDC_CLIENT_SECRET}"
                {{- if .Values.oidc.clientAuthMethod }}
                clientAuthMethod: {{ .Values.oidc.clientAuthMethod | quote }}
                {{- end }}
                {{- if .Values.oidc.clientAuthJwt.enable }}
                clientAuthJwt:
                  jwsAlg: {{ .Values.oidc.clientAuthJwt.jwsAlg | quote }}
                  tokenKeyId: {{ .Values.oidc.clientAuthJwt.tokenKeyId | quote }}
                  {{- if .Values.oidc.clientAuthJwt.keyStore.enable }}
                  keyStore:
                    path: {{ .Values.oidc.clientAuthJwt.keyStore.path | quote }}
                    password: "${OIDC_CLIENT_AUTH_JWT_KEYSTORE_PASSWORD}"
                    type: {{ .Values.oidc.clientAuthJwt.keyStore.type | quote }}
                  {{- end }}
                  {{- if .Values.oidc.clientAuthJwt.key.enable }}
                  key:
                    alias: {{ .Values.oidc.clientAuthJwt.key.alias | quote }}
                    password: "${OIDC_CLIENT_AUTH_JWT_KEY_PASSWORD}"
                  {{- end }}
                {{- end }}
                ...
EOF

Example for changing central-conf files using variables

Here is an example of how to update the central-conf file using variables to avoid copy-paste mistakes.

1. Define the file to be updated:

   export FILE_TO_UPDATE=deploy-task.yaml.template
   export TEMPLATE_DIR=central-conf
   export TARGET_DIR=centralConfiguration
   export TARGET_FILE=deploy-task.yaml

2. Download the latest template configuration file by running the following command:

   kubectl cp -c deploy \
     digitalai/dai-xld-digitalai-deploy-cc-server-0:$TEMPLATE_DIR/$FILE_TO_UPDATE \
     $FILE_TO_UPDATE

3. Update the downloaded template file by running the following command:

   vi $FILE_TO_UPDATE

4. Create a kubectl patch file by running the following command::

   c=$(cat $FILE_TO_UPDATE) \
     contentPath=".spec.centralConfiguration.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.content" \
     f="${TEMPLATE_DIR}/$FILE_TO_UPDATE" \
     filePath=".spec.centralConfiguration.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.path" \
     yq -n 'eval(strenv(contentPath)) = strenv(c) | eval(strenv(filePath)) = strenv(f)' \
     > "$FILE_TO_UPDATE.patch.yaml"

5. Backup the existing file from the pod from the $TARGET_DIR directory by running the following command:

   kubectl cp -c deploy \
     digitalai/dai-xld-digitalai-deploy-cc-server-0:$TARGET_DIR/$FILE_TO_UPDATE \
     backup/$FILE_TO_UPDATE

6. Delete the configuration file from the podto ensure it is recreated by running the following command:

   note

   This step is only necessary if you have the target folder mounted as a persistent volume. Starting from version 24.1 and later, this is no longer the default option.

   kubectl exec -c deploy -it sts/dai-xld-digitalai-deploy-cc-server -n digitalai \
     -- rm $TARGET_DIR/$FILE_TO_UPDATE

7. Update the configuration on the Deploy central configuration pods by running the following command:

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file $FILE_TO_UPDATE.patch.yaml

8. Restart Deploy pods by running the following commands:

   kubectl delete sts dai-xld-digitalai-deploy-master -n digitalai
   kubectl delete sts dai-xld-digitalai-deploy-worker -n digitalai

Example for updating the type-defaults.properties

To update the type-defaults.properties file, modify the central configuration server's CR configuration. This file is then distributed to masters and workers, similar to other template-based configuration files in the central-conf directory.

Since the type-defaults.properties file doesn't exist in the central-conf folder by default, there's no need to download it. Instead, add the necessary changes directly to the CR. Simply add the changes in the CR, focusing on the specific modifications rather including complete file with all comments, as it is too big.

For example,

cat << EOF >> type-defaults.properties.patch.yaml
spec:
  centralConfiguration:
    extraConfiguration:
      central-conf_type-defaults:
        path: central-conf/type-defaults.properties
        mode: 432
        content: |
            aws.Cloud.allowConcurrentDeployments=true
            aws.api.Method.integrationType=AWS
EOF

You must add configuration properties that are different from the default value. The size limit of the custom file is around 1MB.

After applying patch file to cluster and the automatic restart of the central configuration pod, you must manually restart the master and worker pods.

Configure Master Pod

note

In an operator-managed environment, the central-conf folder inside the master containers is not used for configuration. Instead, all configurations are managed by the central configuration server files.

In Deploy master server, local configurations are stored in the conf directory within the Deploy master folder. During startup, the system processes templates from the default-conf directory to generate configuration files in the conf folder. These templates are applied only if the corresponding file does not already exist in the conf directory, ensuring that existing configurations remain unchanged.

Change configuration file on the master Deploy containers

Following are the steps to update the configuration file on the master Deploy containers. The default template file is located in the central-conf directory and is processed during the master server startup. To update the configuration file, you must download the template file, modify it, and then create a patch file to update the configuration on the master.

1. Download the template by running the following command:

   kubectl cp -c deploy-master \
     digitalai/dai-xld-digitalai-deploy-master-0:default-conf/xld-wrapper.conf.common \
     xld-wrapper.conf.common

2. Customize the file xld-wrapper.conf.common as needed by running the following command:

   vi xld-wrapper.conf.common

3. Create a patch file to update the configuration on the Deploy master pods:

   cat << EOF >> xld-wrapper.conf.common.patch.yaml
   spec:
     master:
       extraConfiguration:
         default-conf_xld-wrapper-conf-common:
           path: "default-conf/xld-wrapper.conf.common"
           mode: 0660
           content: |
               # YAJSW configuration properties.
               # Documentation: http://yajsw.sourceforge.net/YAJSW%20Configuration%20Parameters.html
               
               # WARNING - Do not modify any of the properties when an application
               #  using this configuration file has been installed as a service or daemon.
               #  Please uninstall the service before modifying this file.  The
               #  service must then be reinstalled.
               
               # This is a common configuration file used to run YAJSW on both Linux and Windows
               wrapper.working.dir=${xldeploy_home}
               wrapper.tmp.path = ${jna_tmpdir}
               wrapper.startup.timeout=60
               
               wrapper.console.loglevel=FATAL
               ...
   EOF

4. Patch the configuration by changing the CR for the Deploy master pods by running the following command:

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file xld-wrapper.conf.common.patch.yaml

5. Applying the patch command will not automatically restart the Deploy master pods. Run the following command to restart it manually:

   kubectl delete sts dai-xld-digitalai-deploy-master -n digitalai

After restarting the master pods, validate the configuration file on the master Deploy containers. It should be customized according to the changes made in the CR.

Example for the default-conf files using variables

1. Define the file to update by running the following command:

   export FILE_TO_UPDATE=xld-wrapper.conf.common
   export TEMPLATE_DIR=default-conf
   export TARGET_DIR=conf
   export TARGET_FILE=xld-wrapper.conf.common

2. Download the latest template configuration file by running the following command:

   kubectl cp -c deploy-master \
     digitalai/dai-xld-digitalai-deploy-master-0:$TEMPLATE_DIR/$FILE_TO_UPDATE \
     $FILE_TO_UPDATE

3. Update the downloaded template file by running the following command:

   vi $FILE_TO_UPDATE

4. Create a kubectl patch file by running the following command:

   c=$(cat $FILE_TO_UPDATE) \
     contentPath=".spec.master.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.content" \
     f="${TEMPLATE_DIR}/$FILE_TO_UPDATE" \
     filePath=".spec.master.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.path" \
     yq -n 'eval(strenv(contentPath)) = strenv(c) | eval(strenv(filePath)) = strenv(f)' \
     > "$FILE_TO_UPDATE.patch.yaml"

5. Backup the existing file from the pod from the $TARGET_DIR directory by running the following command:

   kubectl cp -c deploy-master \
     digitalai/dai-xld-digitalai-deploy-master-0:$TARGET_DIR/$FILE_TO_UPDATE \
     backup/$FILE_TO_UPDATE

6. Delete the configuration file from the pod to ensure a configuration is recreated:

   note

   This step is only necessary if you have the target folder mounted as a persistent volume. Starting from version 24.1 and later, this is no longer the default option.

   kubectl exec -c deploy-master -it sts/dai-xld-digitalai-deploy-master -n digitalai \
     -- rm $TARGET_DIR/$FILE_TO_UPDATE

7. Update the configuration on the Deploy central configuration pods by running the following command:

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file $FILE_TO_UPDATE.patch.yaml

8. Restart Deploy pods by running the following commands:

   kubectl delete sts dai-xld-digitalai-deploy-master -n digitalai
   kubectl delete sts dai-xld-digitalai-deploy-worker -n digitalai

Configure Worker Pod

In the Deploy worker server, local configurations are stored in the conf directory within the Deploy worker folder, just like in the master server.

Change configuration file on the worker Deploy containers

Following are the steps to update the configuration file on the Deploy worker containers. The default template file is located in the central-conf directory and is processed during the worker server startup. To update the configuration file, you must download the template file, modify it, and then create a patch file to update the configuration on the master.

1. Download the template by running the following command:

   kubectl cp -c deploy-worker \
     digitalai/dai-xld-digitalai-deploy-worker-0:default-conf/xld-wrapper.conf.common \
     xld-wrapper.conf.common

2. Customize the file xld-wrapper.conf.common as needed by running the following command:

   vi xld-wrapper.conf.common

3. Create a patch file to update the configuration on the Deploy worker pods:

   cat << EOF >> xld-wrapper.conf.common.patch.yaml
   spec:
     worker:
       extraConfiguration:
         default-conf_xld-wrapper-conf-common:
           path: "default-conf/xld-wrapper.conf.common"
           mode: 0660
           content: |
               # YAJSW configuration properties.
               # Documentation: http://yajsw.sourceforge.net/YAJSW%20Configuration%20Parameters.html
               
               # WARNING - Do not modify any of the properties when an application
               #  using this configuration file has been installed as a service or daemon.
               #  Please uninstall the service before modifying this file.  The
               #  service must then be reinstalled.
               
               # This is a common configuration file used to run YAJSW on both Linux and Windows
               wrapper.working.dir=${xldeploy_home}
               wrapper.tmp.path = ${jna_tmpdir}
               wrapper.startup.timeout=60
               
               wrapper.console.loglevel=FATAL
               ...
   EOF

4. Patch the configuration by changing the CR for the Deploy worker pods by running the following command.

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file xld-wrapper.conf.common.patch.yaml

5. TThe patch command will not trigger restart of the Deploy master pods. Run the following command to restart it manually:

   kubectl delete sts dai-xld-digitalai-deploy-worker -n digitalai

After restarting the worker pods, validate the configuration file in the worker Deploy containers. It should be customized according to the changes made.

Extra example for the default-conf files by using variables

1. Define the file to update by running the following command:

   export FILE_TO_UPDATE=xld-wrapper.conf.common
   export TEMPLATE_DIR=default-conf
   export TARGET_DIR=conf
   export TARGET_FILE=xld-wrapper.conf.common

2. Download the latest template configuration file by running the following command:

   kubectl cp -c deploy-worker \
     digitalai/dai-xld-digitalai-deploy-worker-0:$TEMPLATE_DIR/$FILE_TO_UPDATE \
     $FILE_TO_UPDATE

3. Update the downloaded template file by running the following command:

   vi $FILE_TO_UPDATE

4. Create a kubectl patch file by running the following command:

   c=$(cat $FILE_TO_UPDATE) \
     contentPath=".spec.worker.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.content" \
     f="${TEMPLATE_DIR}/$FILE_TO_UPDATE" \
     filePath=".spec.worker.extraConfiguration.${TEMPLATE_DIR}_${FILE_TO_UPDATE//./-}.path" \
     yq -n 'eval(strenv(contentPath)) = strenv(c) | eval(strenv(filePath)) = strenv(f)' \
     > "$FILE_TO_UPDATE.patch.yaml"

5. Backup the existing file from the pod from the $TARGET_DIR directory by running the following command:

   kubectl cp -c deploy-worker \
     digitalai/dai-xld-digitalai-deploy-worker-0:$TARGET_DIR/$FILE_TO_UPDATE \
     backup/$FILE_TO_UPDATE

6. Delete the configuration file from the pod to ensure it is recreated:

   note

   This step is only necessary if you have the target folder mounted as a persistent volume. Starting from version 24.1 and later, this is no longer the default option.

   kubectl exec -c deploy-worker -it sts/dai-xld-digitalai-deploy-worker -n digitalai \
     -- rm $TARGET_DIR/$FILE_TO_UPDATE

7. Update the configuration on the Deploy Central Configuration pods by running the following command:

   kubectl patch -n digitalai digitalaideploys.xld.digital.ai dai-xld \
     --type=merge --patch-file $FILE_TO_UPDATE.patch.yaml

8. Restart Deploy pods by running the following command:

   kubectl delete sts dai-xld-digitalai-deploy-worker -n digitalai