Deployment Validation

Run these checks after deployment to validate that platform features are operational in the target environment, before handing the deployment over for product testing. DynamoAI provides executable validation scripts for these checks; the checklists below track sign-off. Checklist state is saved in your browser and can be downloaded as Markdown for the handover record.

Before starting, review the runtime and test-time dependencies; most validation failures trace back to that table (target endpoint guardrails, provider health, rate limits), not to the platform.

The validation script uses a model from the GPT-4 family (4, 4o, 4.1, or 4.1-mini) as the AI system under test.

Identity and Access

Run this first. Every scenario below assumes a user who can sign in and reach the relevant product, and an identity failure presents as a product failure if this is not checked separately.

Perform these as a non-administrator account that arrived through your identity provider. An administrator account, or a local account, passes these checks while federated users remain blocked.

Check	Target Result
Federated sign-in completes	The user reaches the application without a Review Profile prompt
Org-level roles resolve	GET /v1/user/all?includeOrgLevelRoles=true shows the expected roles for that user, and the array is not empty
Platform configuration loads	The interface reports no Unable to fetch platform config or request-id error
DynamoGuard access is active	The Policies tab is visible to the user
The user can create	The user creates a policy and saves it without a permission error
A second user is unaffected	A user outside the mapped directory groups signs in and is correctly limited

Confirm resolved roles for every user

curl -s -H "Authorization: Bearer $ADMIN_TOKEN" \
  "https://api.<domain>/v1/user/all?includeOrgLevelRoles=true" \
  | jq '.users[] | {email, orgLevelRoles}'

Identity and Access0/6

• Federated sign-in completes for a non-administrator
• Org-level roles resolve and are not emptyAn empty array produces a server error rather than a restricted interface.
• Platform configuration loads with no request-id error
• Policies tab visibleConditional: DynamoGuard deployments only. Requires per-user activation.
• Non-administrator creates and saves a policy
• An unmapped user is correctly limited

Do not validate identity as an administrator

An administrator holds every scope and is usually a local account, so it passes each check above while federated users are blocked. This is the single most common reason a deployment is signed off and then reported as broken on the first working day.

DynamoEval: System Policy Compliance

Prerequisites: the model is registered in DynamoEval, user access management is set up, and an input content policy exists.

Check	Target Result
Create a System Policy Compliance test	Test record created with a valid test ID; status reaches PREPARING then RUNNING
Test execution completes	Status transitions to COMPLETED without ERROR
Retrieve deep-dive prompt-response results	Deep-dive API returns HTTP 200; guardrail_evaluation_prompt_responses is non-empty; each record contains prompt and response fields
Retrieve report PDF data	Report data API returns HTTP 200 with a non-null body
Verify model metrics	Model metrics API returns HTTP 200 with non-null data (ASR, success count, attack type per test)

System Policy Compliance0/5

• Test created (valid ID, PREPARING then RUNNING)
• Test reached COMPLETED without ERROR
• Deep-dive results retrieved (non-empty prompt-responses)
• Report PDF data retrieved
• Model metrics present and retrievable

DynamoEval: Policy Jailbreak

Prerequisites: same as System Policy Compliance, plus a healthy AI provider (check the provider status page before the run). Policy Jailbreak tests additionally require DynamoGuard to be deployed; the test exercises guardrail endpoints.

Check	Target Result
Create a Policy Jailbreak test	Test record created with a valid test ID; status QUEUED or RUNNING
Test execution completes	Status transitions to COMPLETED without ERROR; progress stages validate during execution
Retrieve deep-dive prompt-response results	HTTP 200; policy_jailbreak_prompt_responses non-empty; attackTypes and total count present
Retrieve report PDF data	HTTP 200 with a non-null body
Verify model metrics	HTTP 200 with non-null data

Policy Jailbreak0/5

• Test created (valid ID, QUEUED or RUNNING)
• Test reached COMPLETED without ERROR
• Deep-dive results retrieved (attackTypes present)
• Report PDF data retrieved
• Model metrics present and retrievable

DynamoGuard: Safety Policy

Prerequisites: the safety model is deployed and the user has access to the DynamoGuard policy page.

Check	Target Result
Create a safety policy	POST /moderation/policy returns HTTP 201 with a policy ID
Create an AI system	AI system model created in the /ml-model response for the authenticated test user
Apply the safety policy to the AI system	appliedStatus = APPLIED; model ID present in the policy's models array
Test the built-in chatbot with the guardrail applied	Chatbot returns HTTP 201; guardrail evaluates inline in real time (requires the third-party LLM powering the chatbot to be connected)

Safety Policy0/4

• Safety policy created
• AI system created
• Policy applied (appliedStatus = APPLIED)
• Built-in chatbot guardrailed in real time

DynamoGuard: Custom Content Policy

Prerequisites: access to the DynamoGuard policy page. Each check depends on the previous one passing.

Check	Target Result
Create a use-case context	POST /moderation/policy/domain/usecase returns HTTP 201 with a valid usecase ID; title and description match the payload
Create a custom content policy linked to the use case	HTTP 201; policy linked to the usecase ID; action and applicableTo match (INPUT or OUTPUT)
Auto-generate allowed and prohibited behaviors	HTTP 204; policy expanded view returns exactly 5 allowed and 5 disallowed behaviors
Create a synthetic dataset from policy behaviors	HTTP 201; stage = seed; policyId matches; dataset visible via GET
Seed examples ready and dataset review submitted	Datapoints present; review accepted (HTTP 200/201); stage transitions to augment or reviewReady
Trigger guardrail model training	HTTP 200/201 on startTraining
Training completes	trainingStatus = TRAINING_COMPLETE with no TRAINING_FAILED encountered
Policy deploys	appliedStatus = READY_TO_APPLY; resourceDeploymentStatus = DEPLOYED
Scale down the deployed policy	isEnabled=false accepted; resourceDeploymentStatus transitions to SCALING_DOWN or NOT_DEPLOYED

Custom Content Policy0/8

• Use-case context created
• Custom content policy created and linked
• Behaviors auto-generated (5 allowed, 5 disallowed)
• Synthetic dataset created (stage = seed)
• Dataset review submitted
• Training triggered and completed (TRAINING_COMPLETE)
• Policy deployed (DEPLOYED)
• Policy scaled down cleanly

---

After Validation

Validation is the handover gate, not the finish line. A validated platform guards and evaluates nothing until a real workload is connected to it, and that work usually belongs to a different team than the one that ran the deployment. Plan it as phase 8 of the runbook rather than as an afterthought.

If DynamoGuard is licensed, connect the first application: create an AI system, start from the foundation guardrails and add a custom content policy where your requirements are organization-specific, then integrate DynamoGuard into the request path and tune thresholds against real traffic before enforcing.

If DynamoEval is licensed, run the first real test: the jailbreak walkthrough and the system policy compliance walkthrough, then connect your own targets through custom systems integration.

Before handover, agree the operating baseline with whoever will run the platform: logging, metrics and dashboards, health monitoring, an upgrade cadence from Upgrades, and, if you carry an availability commitment, the Reliability requirements.