Deploy

The DynamoAI platform installs as a set of Helm charts. dynamoai-base carries the shared platform — API, UI, identity, and the datastore wiring — and is always installed first. The product charts install on top of it:

Chart	Contains	Installed
dynamoai-base	API, UI, Keycloak, core services, datastore configuration	Always, first
dynamoai-dynamoguard	Guardrail inference services, fine-tuning, model serving	If your licence covers DynamoGuard
dynamoai-dynamoeval	Evaluation workers and job orchestration	If your licence covers DynamoEval

Which product charts you install depends on what your organization licensed. Some environments run DynamoGuard only, some run DynamoEval only, and some run both. Neither product chart depends on the other, so they can be installed in either order, or one now and the other later. Your DynamoAI Solutions Architect confirms which apply to your environment.

Chart versions are independent of the application version they carry. All three charts currently ship application version 3.26.5.

Before installing, complete the Environment Dependencies checks for your cloud.

Prerequisites

1. Obtain Helm charts from Dynamo AI

   • The Dynamo AI Helm charts are not publicly hosted. You must obtain them directly from the Dynamo AI team. Once you have the chart package (e.g., .tgz file), you can either:
     • Host it in an internal/private Helm repository accessible by your cluster, or
     • Reference the chart locally (e.g., helm install ./dynamoai-base-<VERSION>.tgz).

2. Container images in an accessible registry

   • Dynamo AI container images must be accessible by your Kubernetes cluster. Ensure you have pushed/pulled the correct images to your container registry before deployment. The Dynamo team will share scripts that help you mirror our images in your repositories.
   • Verify your cluster nodes have permission to access this registry.

3. Model weights in object storage

   • Dynamo AI relies on certain model artifacts. You can store these model weights in your existing object storage solution. These will be shared with you at deployment time.
   • Ensure you configure the correct credentials and endpoints in your values.yaml so the application can retrieve the models during runtime.

Dependencies

Install these only where your cluster does not already provide them. Each is labelled Required, Conditional, or Optional; the Deployment Runbook explains the labels.

Dependency	Status
NVIDIA device plugin	Conditional: only with DynamoGuard, or any other GPU workload
Ingress controller	Conditional: not needed on OpenShift, which uses Routes
KEDA	Conditional: needed for event-driven scaling of evaluation and fine-tuning
Prometheus	Optional: without it there is no bundled metrics and alerting
OpenTelemetry	Optional: without it there is no bundled tracing pipeline

NVIDIA Device Plugin (For GPU Clusters)

Conditional. Required only if you intend to run GPU workloads, which in practice means DynamoGuard. Skip it entirely on a DynamoEval-only deployment:

# Example using the NVIDIA Helm chart (standard K8s):
helm repo add nvdp https://nvidia.github.io/k8s-device-plugin
helm repo update

helm upgrade --install nvidia-device-plugin nvdp/nvidia-device-plugin \
  --version <CHART_VERSION> \
  --namespace kube-system

For RedHat OpenShift, the recommended approach is installing the NVIDIA Device Plugin Operator from OperatorHub.

Ingress Controller or OpenShift Routes

Conditional. Dynamo AI services need external access, but the mechanism differs by platform:

• NGINX Ingress (Generic K8s)

  helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
  helm repo update
  
  helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
    --namespace ingress-nginx --create-namespace

• OpenShift – Use built-in Routes; no separate NGINX required.

KEDA

Conditional. Dynamo AI integrates with KEDA to autoscale workloads based on custom metrics and events. Required for event-driven scaling of evaluation and fine-tuning workloads; without it those workloads cannot scale from events:

helm repo add kedacore https://kedacore.github.io/charts
helm repo update

helm upgrade --install keda kedacore/keda \
  --namespace keda \
  --create-namespace

Prometheus (Metrics)

Optional. Prometheus is used for metrics-based monitoring. The platform runs without it; you lose bundled metrics and alerting:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm upgrade --install prometheus prometheus-community/prometheus \
  --namespace monitoring --create-namespace

OpenTelemetry

Optional. If you want distributed tracing:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

helm upgrade --install opentelemetry-operator open-telemetry/opentelemetry-operator \
  --namespace opentelemetry --create-namespace

---

Configuration Variables

Configuration splits in two. Helm values describe the shape of the deployment: names, domains, which components are enabled. Kubernetes Secrets carry every credential — database passwords, the licence, API keys. The charts read credentials only from Secrets, so no password belongs in a values file.

Shared global Values

These are set the same way in every chart you install. A mismatch between charts is the most common cause of services that deploy cleanly but cannot reach each other.

Variable	Description	Example
global.nameOverride	Base name for generated resources.	dynamoai
global.fullNameOverride	Full name prefix used when naming per-application resources.	dynamoai
global.fullnameOverride	Full release name. Note the lowercase n: this is a separate value from fullNameOverride above, read by a different helper. Set both, to the same value.	dynamoai
global.apiDomain	Fully qualified URL for the API endpoint.	https://api.dynamoai.example.com
global.uiDomain	Domain for the UI endpoint.	dynamoai.example.com
global.keycloakDomain	Domain for the Dynamo AI Keycloak IdP, if using the bundled Keycloak.	sso.dynamoai.example.com
global.dynamoguardnamespace	Namespace DynamoGuard references point at. Note the all-lowercase spelling.	dynamoai
global.awsRegion	Region used for AWS SDK calls, including Bedrock. Set it even on Azure and GCP if any AWS-backed service is in use.	us-east-1
global.imagePullSecrets	List of image pull secrets for your mirrored registry.	[ecr-regcred]
global.serviceAccount.name	Service account the platform workloads run as.	dynamoai-service-account
global.metricsServer.address	In-cluster address of the metrics server.	dynamoai-metrics-server.dynamoai.svc.cluster.local:9090
global.config.common	Name of the shared ConfigMap.	dynamoai-common-config
global.config.proxy	Name of the proxy ConfigMap. Leave empty if you do not run an egress proxy.	dynamoai-proxy-config
global.config.api	Name of the API ConfigMap.	dynamoai-api-config
global.secrets.common	Name of the Secret holding the licence and API keys.	dynamoai-common-secret
global.secrets.postgres	Name of the Secret holding PostgreSQL connection details.	dynamoai-postgres-secret
global.secrets.mongodb	Name of the Secret holding MongoDB connection details.	dynamoai-mongodb-secret
global.secrets.redis	Name of the Secret holding Redis connection details.	dynamoai-redis-secret
global.secrets.apikeys	Name of the Secret mounted as a volume of service API keys.	dynamoai-apikeys-secret

Secret names must match across charts

global.secrets.* and global.config.* name existing objects; the charts do not create them. If dynamoai-base writes to dynamoai-common-secret and dynamoai-dynamoguard reads dynamoai-core-common-secret, DynamoGuard starts with no licence and no API keys, and the failure surfaces at request time rather than at install time.

Kubernetes Secrets

Create these before installing. Each Secret must use exactly these key names.

Secret (global.secrets.*)	Key	Holds
postgres	host	PostgreSQL hostname
	port	PostgreSQL port
	name	Application database name
	keycloakDbName	Keycloak database name (a second database on the same instance)
	username	PostgreSQL username
	password	PostgreSQL password
mongodb	host	MongoDB hostname
	port	MongoDB port
	name	MongoDB database name
	username	MongoDB username
	password	MongoDB password
redis	host	Redis hostname
	port	Redis port
	password	Redis password
common	license	Your Dynamo AI licence string, provided by your Solutions Architect
	hfToken	HuggingFace token, if pulling private models
	openai_api_key	OpenAI API key, if used
	mistral_api_key	Mistral API key, if used
	dataGenerationApiKey	API key for the synthetic data generation and judge LLM
	azureApimSubscriptionKey	Azure API Management subscription key, if the utility LLM is fronted by APIM
	defaultUserEmail	Email of the initial platform user
	defaultUserPassword	Password of the initial platform user
	platformAdminPassword	Platform administrator password
	defaultUserApiKey	API key issued to the initial platform user
	pentestServiceUserApiKey	Service API key used by DynamoEval red-teaming. Required when DynamoGuard is installed
	serviceUserApiKeyMap	Map of service-to-API-key assignments
	AWS_ACCESS_KEY_ID	AWS access key, where IRSA or workload identity is not in use
	AWS_SECRET_ACCESS_KEY	AWS secret key, where IRSA or workload identity is not in use
apikeys	—	Mounted as a volume rather than read key by key. Its contents are supplied by Dynamo AI

Create secret values without a trailing newline

Use echo -n or printf when creating Secret values by hand. A trailing newline on the license key crash-loops the API with an error that does not mention the newline.

dynamoai-base Values

Variable	Description	Default
dynamoai-api.enabled	Install the API service.	No default; set explicitly
dynamoai-ui.enabled	Install the UI service.	No default; set explicitly
dynamoai-core.enabled	Install the core services.	No default; set explicitly
dynamoai-redis.enabled	Install the Dynamo AI Redis wrapper.	No default; set explicitly
keycloak.enabled	Deploy the bundled Keycloak. Disable if you front the platform with your own OIDC provider.	true
mongodb.enabled	Deploy the bundled in-cluster MongoDB. Disable when using a managed endpoint such as Atlas; see External MongoDB.	true
postgresql.enabled	Deploy a bundled in-cluster PostgreSQL. Production environments use a managed instance and leave this off.	false
minio.enabled	Deploy bundled MinIO for object storage. Leave off where native object storage is available.	false
kubernetes-event-exporter.enabled	Export Kubernetes events for observability.	true
dbMaintenance.enabled	Run scheduled database maintenance jobs.	false
global.storageClass	Storage class for the bundled datastores' persistent volumes. Consumed by the bundled MongoDB, PostgreSQL, MinIO, and Keycloak subcharts; it has no effect if all of them are disabled.	Cluster default

The four enabled flags without a default are gated by Helm conditions that resolve to nothing unless you set them. Set each one explicitly in your values file rather than relying on chart behaviour.

API environment variables. The API reads these from dynamoai-api.dynamoai-common.deployment.env, not from a global key:

Variable	Description	Default
ENABLE_ENTRA_WORKLOAD_IDENTITY_AUTH	Allow AI systems to authenticate with Microsoft Entra workload identity instead of a stored credential. See Microsoft Entra Workload Identity.	false
ENTRA_ALLOWED_ENDPOINT_HOSTS	Comma-separated hostnames an Entra-authenticated AI system may point at. Required when ENABLE_ENTRA_WORKLOAD_IDENTITY_AUTH is true; the API refuses to start without it. See Integrate AI Systems.	Empty
AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_FEDERATED_TOKEN_FILE	Workload identity the API requests tokens as, to test Entra AI systems when they are saved. Required when ENABLE_ENTRA_WORKLOAD_IDENTITY_AUTH is true. On AKS the workload identity webhook injects them; do not set them by hand.	Injected
AZURE_AUTHORITY_HOST	Microsoft Entra authority. Override only for sovereign clouds.	https://login.microsoftonline.com

Setting an API environment variable

dynamoai-api:
  dynamoai-common:
    deployment:
      env:
        ENABLE_ENTRA_WORKLOAD_IDENTITY_AUTH:
          value: "true"
        ENTRA_ALLOWED_ENDPOINT_HOSTS:
          value: "gateway.internal.example.com"

dynamoai-dynamoguard Values

Set the shared global block exactly as in dynamoai-base, then configure the models and the guardrail services.

Models. Each entry names a model and the path it is fetched from.

Variable	Description	Example
global.models.dataGeneration.name	Model used for synthetic data generation and judging. Needs a provider prefix.	bedrock/llama/us.meta.llama3-1-70b-instruct-v1:0
global.models.finetuning.name / .path	Base model for guardrail fine-tuning.	meta-llama/LlamaGuard-7b
global.models.piiRedaction.name / .path	PII redaction model.	dynamoai-ml/pii_piiredaction_based_version_8
global.models.promptInjection.name / .path	Prompt injection detection model.	Provided by Dynamo AI
global.models.toxicity.name / .path	Toxicity detection model.	Provided by Dynamo AI
global.models.safety.name / .path	Safety model.	Provided by Dynamo AI
global.models.legalAdvice.name / .path	Legal advice guardrail model.	Provided by Dynamo AI
global.models.financialAdvice.name / .path	Financial advice guardrail model.	Provided by Dynamo AI
global.models.compensation.name / .path	Compensation guardrail model.	Provided by Dynamo AI

global.models.dataGeneration.name must carry the LiteLLM provider prefix. mistral/mistral-medium-latest resolves; mistral-medium-latest does not.

Guardrail services. Each guardrail is a top-level key with its own enabled flag, image, and resource requests. Every GPU-bound guardrail requests one whole GPU, so the number of enabled GPU guardrails sets your GPU node count.

Service	Purpose	Default
data-processing	Request preprocessing pipeline	true
moderation	Content moderation	true
pii-redaction	PII detection and redaction	true
prompt-injection	Prompt injection detection	true
toxicity	Toxicity detection	true
fin-advice	Financial advice detection	true
legal-advice	Legal advice detection	true
compensation	Compensation guidance detection	true
finetuning, e5_finetune, mini_finetune	Custom policy fine-tuning	true
safety	Safety classification	false
offTopic	Off-topic detection	false
hallucinationFaithfulness, hallucinationResponseRelevance, hallucinationRetrievalRelevance	Hallucination guardrails	false
og_training	Training service	false
dynamoserve.enabled	DynamoServe model serving	false
minio.enabled	Bundled MinIO for DynamoGuard artifacts	false

The chart also carries customer-specific and region-specific guardrails, all disabled by default. Enable only the guardrails your policies use; each enabled GPU guardrail is a GPU node you pay for.

dynamoai-dynamoeval Values

DynamoEval is a thin chart. It inherits almost everything from the shared global block and adds:

Variable	Description	Default
dynamoai-core.enabled	Install the core services alongside the evaluation workers.	No default; set explicitly
dynamoai-customrag.enabled	Install the custom RAG adapter, for evaluating a RAG pipeline you provide.	false
metricsServer.address	Address of the metrics server the evaluation workers report to.	needed — must be set

DynamoEval runs on CPU nodes and calls an external LLM for data generation and judging. Its throughput is bounded by that endpoint's rate limit, not by cluster capacity: a judge quota below roughly 250k tokens per minute produces 429 responses part-way through an evaluation run.

---

Installing Dynamo AI Charts

1. Prepare the Helm Charts

Since the Dynamo AI Helm charts are not hosted online:

1. Obtain the chart .tgz files (e.g., dynamoai-base-<VERSION>.tgz, dynamoai-dynamoguard-<VERSION>.tgz, dynamoai-dynamoeval-<VERSION>.tgz) from your Dynamo AI Solutions Architect.
2. Store them in a local directory or a private Helm repository accessible to your cluster.

2. Deploy the dynamoai-base Chart

Install this first and wait for it to come up healthy; the product charts assume its services and Secrets exist.

1. Create a dynamoai-base-values.yaml (or equivalent) with the shared global values and the base chart values configured.

2. Install locally, for example:

   helm upgrade --install dynamoai-base /path/to/dynamoai-base-<VERSION>.tgz \
     --values ./dynamoai-base-values.yaml \
     --namespace dynamoai \
     --create-namespace

   If you hosted your chart in a private repo, use:

   helm upgrade --install dynamoai-base my-private-repo/dynamoai-base \
     --version <DAI_VERSION> \
     --values ./dynamoai-base-values.yaml \
     --namespace dynamoai \
     --create-namespace

Confirm the API, UI, and Keycloak pods are Running and Ready before continuing.

3. Deploy the Product Charts

Install the charts your licence covers. Both read the same global block, so copy it from your base values file rather than retyping it.

DynamoGuard:

helm upgrade --install dynamoai-dynamoguard /path/to/dynamoai-dynamoguard-<VERSION>.tgz \
  --values ./dynamoai-dynamoguard-values.yaml \
  --namespace dynamoai

DynamoEval:

helm upgrade --install dynamoai-dynamoeval /path/to/dynamoai-dynamoeval-<VERSION>.tgz \
  --values ./dynamoai-dynamoeval-values.yaml \
  --namespace dynamoai

If you install both, order does not matter. If you install one now and add the other later, the second install needs no change to the first.

---

Post-Installation

Stages 1 to 3 confirm the deployment is up. Stages 4 onward connect it to your organization's identity and mail infrastructure, and are where most of the remaining work sits — plan for them as part of the deployment rather than afterwards.

1. Validate Pods

Required.

kubectl get pods -n dynamoai

Confirm all pods are in a Running/Ready state.

2. DNS & Ingress Configuration

Required.

• If using an NGINX Ingress on EKS, AKS, or GKE, ensure your domains (global.apiDomain, global.uiDomain, global.keycloakDomain) have DNS A/CNAME records pointing to the ingress load balancer.
• On OpenShift, confirm that the Routes are created and externally resolvable. In some cases, the domain will be auto-generated unless you configure custom hostnames.

3. First Sign-In

Required.

Open the UI domain you specified (e.g., https://dynamoai.example.com) and sign in with the credentials from the defaultUserEmail and defaultUserPassword keys of your common Secret.

4. Connect Your Identity Provider

Optional, strongly recommended. The platform works on the bundled Keycloak with local accounts. No production deployment should stop there.

The bundled Keycloak is an identity broker, not the intended end state for a production deployment. Federate it to your corporate IdP so that access follows your existing joiner–mover–leaver process.

1. Keycloak SSO — how the platform uses Keycloak, and what federation changes.
2. Entra ID as an IdP — the common path: register the application in Entra ID and wire it as an identity provider. For a non-Microsoft IdP, Keycloak as an IdP covers the generic OIDC case.
3. Adding Mappers — map IdP claims onto platform roles, so group membership in your directory grants access here.
4. Default IdP Redirect — send users straight to your IdP instead of showing the Keycloak login form.
5. Keycloak Web Console via Entra ID — bring administrative access to Keycloak itself under the same IdP, so the local admin account stops being a shared credential. The generic IdP version covers other providers.

Two failure modes are worth reading before you start rather than after. Entra ID Groups Claim Overage explains why users in many Entra groups silently lose their group claim, and Troubleshooting IdP Logins covers the rest.

5. Configure Access

Required.

With federation working, set up who can do what:

• Understanding Access and Roles — the access model.
• Scopes In Detail — fine-grained permissions.
• User Management — day-to-day administration.

Then activate DynamoGuard for each user who needs it. This step is easy to miss, because it is the only part of onboarding that no role or group mapper can perform, and a user who lacks it signs in successfully and finds no Policies tab:

• Activate Each IdP User on the Platform is the procedure and applies to every identity provider, not only Entra ID.

Confirm before continuing that a user who is not an administrator can sign in, reach the Policies tab, and create a policy. Roles that look correct in Keycloak are not evidence of either.

6. Operational Integrations

Conditional. SMTP is required only if you invite users by email; Vault storage of the OIDC client secret is optional.

• SMTP Integration — required for invitations, password resets, and verification mail. Without it, users you invite receive nothing.
• Storing the OIDC Client Secret via Vault — move the IdP client secret out of Helm values into your secret manager.

7. Integrate AI Systems

Required to realize value. A validated platform guards nothing until an AI system is connected to it.

• For real-time guardrails, configure an external or local AI system in the UI or via CRDs.
• Provide your LLM API keys (OpenAI, Mistral, Databricks, etc.) or local GPU resources if running in-cluster.

Conditional, for Microsoft Entra workload identity. An AI system may authenticate with a token minted from the platform's own managed identity, or from a customer-owned managed identity named on the AI system, rather than a stored credential. Setup is covered in Microsoft Entra Workload Identity. Because the platform sends the token to whichever endpoint the AI system names, the hosts it may reach are restricted by an allowlist.

Set ENTRA_ALLOWED_ENDPOINT_HOSTS to the gateway hostnames your AI systems use. Entries are exact, lowercase hostnames, comma-separated, with no scheme, port or path. A host that is absent is rejected at validation time with Endpoint host "<host>" is not permitted for Entra workload identity authentication on this deployment.

A value with no hostname disables the restriction

The API refuses to start when ENABLE_ENTRA_WORKLOAD_IDENTITY_AUTH is true and ENTRA_ALLOWED_ENDPOINT_HOSTS is empty. A value that contains no hostname, such as a lone comma, passes that check, permits every host, and only logs a warning. Anyone able to create an AI system can then direct a managed-identity token to an endpoint of their choosing. List the hosts you need explicitly.

8. Validate the Deployment

Required. This is the handover gate.

Run the Deployment Validation checks and record sign-off before handing the environment over for product testing. For availability and failover expectations, see Reliability.

---

Cleanup

Remove Dynamo AI charts:

helm uninstall dynamoai-dynamoguard -n dynamoai
helm uninstall dynamoai-dynamoeval -n dynamoai
helm uninstall dynamoai-base -n dynamoai

(Optional) Remove dependencies if they were installed specifically for Dynamo AI:

helm uninstall nvidia-device-plugin -n kube-system   # GPU device plugin
helm uninstall ingress-nginx -n ingress-nginx        # Ingress controller
helm uninstall keda -n keda
helm uninstall prometheus -n monitoring
helm uninstall opentelemetry-operator -n observability