Skip to main content

Bring Your Own LLM

Several Digital.ai products include features that call an AI model to do their work. For example, Digital.ai Arxan Security's Quick Protect Agent analyzes your applications and applies protection to them automatically. By default, every one of those features routes its AI requests through the built-in Digital.ai provider (AWS Bedrock), so there's nothing to set up if you don't want to.

The built-in provider meets the needs of many organizations. However, yours may have requirements it can't satisfy. To solve this, Digital.ai offers a Bring Your Own Large Language Model (BYO-LLM) capability that lets you configure the AI provider used by Digital.ai product features, so AI requests originating from your software flow through your own provider account.

You may want to use BYO-LLM to:

  • Keep AI traffic inside a vendor relationship you already govern
  • Meet regulatory, compliance, and data residency requirements
  • Draw down committed spend with an AI vendor
  • Route requests through a central AI gateway
  • Use models that the built-in provider doesn't offer

The supported vendors are AWS Bedrock, OpenAI, and Anthropic.

note

AI provider settings are available only to account administrators.

Product Features That Use BYO-LLM​

The AI providers you configure here serve the AI features in these Digital.ai products:

note

Some of these features, such as Agility Sage and Release AI Assistant, can also be self-hosted. The BYO-LLM settings described here apply only to SaaS deployments.

Key Concepts​

The Bring Your Own LLM functionality is built around the following important concepts:

  • AI provider: A connection to a third-party AI vendor set up with your own account credentials.
  • Digital.ai provider: The built-in provider your account uses by default. You can't edit or delete it.
  • Default provider: The one provider used for all AI operations. Only one provider can be the default at a time. Until you set another one, the Digital.ai provider is the default.
  • Provider configuration: The set of fields, specific to each provider type, that hold the credentials and settings for a provider connection. For the full field list per type, see AI Provider Configuration Reference.
  • Model alias: A stable, friendly model name that Digital.ai applications use in their requests. The alias stays the same even when the real models behind it change. Digital.ai defines these aliases; as an administrator you map them to real models in your provider.
  • Model mapping: The link between an alias and the real provider model IDs it resolves to. You can list several model IDs in priority order, so if the first one is unavailable, the next one is tried.

BYO-LLM Workflow​

A complete setup runs in this order:

  1. Gather the credentials your vendor requires. See Prerequisites.
  2. Add your provider. See Add a Provider.
  3. Test the connection. See Test a Provider.
  4. Make it the default so AI operations use it. See Set the Default Provider.
  5. Map the Digital.ai model aliases your applications use to models in your provider. See Map Model Aliases to Your Models.
tip

Test a provider before you make it the default. That way you confirm the credentials work while AI operations still run through your previous provider.

Last updated on