Requirements

Command-line Tools

To provision and manage a Dynamo AI cluster, you will need:

1. Terraform – (If using Terraform for cluster or infrastructure creation.)
2. Helm – For installing and managing Dynamo AI in Kubernetes.
3. Docker – For container image management.
4. Kubectl – For interacting with the Kubernetes cluster.
5. Cloud-specific CLI (optional, but typically needed if provisioning or configuring infrastructure directly):
   • AWS CLI
   • Azure CLI
   • gcloud CLI
   • oc CLI for OpenShift (optional).

Dependent Services

1. Container Registry: A container registry is required to store Dynamo AI container images in the deployment. Dynamo AI images will be copied to the container registry for the cluster to retrieve:

   • AWS ECR
   • Azure Container Registry
   • Google Artifact Registry

2. Model Storage: A storage service is required for storing Dynamo AI models. There are three options to achieve this:

   • Option 1: Granting your cluster access to HuggingFace to download the images directly from the Dynamo AI repo.
   • Option 2: Using in-cluster MinIO to store the images. For this option, PersistenVolume must be configured in the Kubernetes cluster.
   • Option 3: Copy models to an external service the cluster can access, such as AWS S3.

3. Utility LLM: A Utility LLM is required for advanced Dynamo AI features, such as synthetic data generation in the custom policy creation workflow. There are two options for supporting this:

   • Option 1: Leverage an existing LLM service. Dynamo AI can leverage the following services as utility LLM:
     • OpenAI (GPT-3.5, GPT-4, GPT-4o)
     • Mistral
     • Databricks
     • Sufficiently performant LLM service (minimum Llama 8B) compliant with API schemas above.
   • Option 2: Deploy the Utility LLM inside the same cluster. This requires provisioning additional GPU resources.

4. PostgreSQL Database (Optional): Dynamo AI offers an in-cluster PostgreSQL database. Alternatively, integrate with an external managed PostgreSQL database for enhanced data management.

5. Target AI System (Post deployment): Integrate your AI system to evaluate or run guardrail against. This integration typically occurs after cluster deployment. If using DynamoEval or DynamoGuard in Managed Inference Mode, the cluster needs to call your target AI system through API.

   • External AI Systems: If your AI system is OpenAI, Azure OpenAI, Databricks, mistral.ai, AWS Bedrock, or its API is compliant with the OpenAI schema, Dynamo AI supports out-of-box integration in the UI via the "Connect AI System: and "Remote Model" options.
   • Local AI Systems: If the AI system is within the same cluster, configure it in the UI via the "Connect AI System" and "Local model".