Deployment Validation

Run these checks after deployment to validate that platform features are operational in the target environment, before handing the deployment over for product testing. DynamoAI provides executable validation scripts for these checks; the checklists below track sign-off. Checklist state is saved in your browser and can be downloaded as Markdown for the handover record.

Before starting, review the runtime and test-time dependencies; most validation failures trace back to that table (target endpoint guardrails, provider health, rate limits), not to the platform.

The validation script uses a model from the GPT-4 family (4, 4o, 4.1, or 4.1-mini) as the AI system under test.

DynamoEval: System Policy Compliance

Prerequisites: the model is registered in DynamoEval, user access management is set up, and an input content policy exists.

Check	Target Result
Create a System Policy Compliance test	Test record created with a valid test ID; status reaches PREPARING then RUNNING
Test execution completes	Status transitions to COMPLETED without ERROR
Retrieve deep-dive prompt-response results	Deep-dive API returns HTTP 200; guardrail_evaluation_prompt_responses is non-empty; each record contains prompt and response fields
Retrieve report PDF data	Report data API returns HTTP 200 with a non-null body
Verify model metrics	Model metrics API returns HTTP 200 with non-null data (ASR, success count, attack type per test)

System Policy Compliance0/5

• Test created (valid ID, PREPARING then RUNNING)
• Test reached COMPLETED without ERROR
• Deep-dive results retrieved (non-empty prompt-responses)
• Report PDF data retrieved
• Model metrics present and retrievable

DynamoEval: Policy Jailbreak

Prerequisites: same as System Policy Compliance, plus a healthy AI provider (check the provider status page before the run). Policy Jailbreak tests additionally require DynamoGuard to be deployed; the test exercises guardrail endpoints.

Check	Target Result
Create a Policy Jailbreak test	Test record created with a valid test ID; status QUEUED or RUNNING
Test execution completes	Status transitions to COMPLETED without ERROR; progress stages validate during execution
Retrieve deep-dive prompt-response results	HTTP 200; policy_jailbreak_prompt_responses non-empty; attackTypes and total count present
Retrieve report PDF data	HTTP 200 with a non-null body
Verify model metrics	HTTP 200 with non-null data

Policy Jailbreak0/5

• Test created (valid ID, QUEUED or RUNNING)
• Test reached COMPLETED without ERROR
• Deep-dive results retrieved (attackTypes present)
• Report PDF data retrieved
• Model metrics present and retrievable

DynamoGuard: Safety Policy

Prerequisites: the safety model is deployed and the user has access to the DynamoGuard policy page.

Check	Target Result
Create a safety policy	POST /moderation/policy returns HTTP 201 with a policy ID
Create an AI system	AI system model created in the /ml-model response for the authenticated test user
Apply the safety policy to the AI system	appliedStatus = APPLIED; model ID present in the policy's models array
Test the built-in chatbot with the guardrail applied	Chatbot returns HTTP 201; guardrail evaluates inline in real time (requires the third-party LLM powering the chatbot to be connected)

Safety Policy0/4

• Safety policy created
• AI system created
• Policy applied (appliedStatus = APPLIED)
• Built-in chatbot guardrailed in real time

DynamoGuard: Custom Content Policy

Prerequisites: access to the DynamoGuard policy page. Each check depends on the previous one passing.

Check	Target Result
Create a use-case context	POST /moderation/policy/domain/usecase returns HTTP 201 with a valid usecase ID; title and description match the payload
Create a custom content policy linked to the use case	HTTP 201; policy linked to the usecase ID; action and applicableTo match (INPUT or OUTPUT)
Auto-generate allowed and prohibited behaviors	HTTP 204; policy expanded view returns exactly 5 allowed and 5 disallowed behaviors
Create a synthetic dataset from policy behaviors	HTTP 201; stage = seed; policyId matches; dataset visible via GET
Seed examples ready and dataset review submitted	Datapoints present; review accepted (HTTP 200/201); stage transitions to augment or reviewReady
Trigger guardrail model training	HTTP 200/201 on startTraining
Training completes	trainingStatus = TRAINING_COMPLETE with no TRAINING_FAILED encountered
Policy deploys	appliedStatus = READY_TO_APPLY; resourceDeploymentStatus = DEPLOYED
Scale down the deployed policy	isEnabled=false accepted; resourceDeploymentStatus transitions to SCALING_DOWN or NOT_DEPLOYED

Custom Content Policy0/8

• Use-case context created
• Custom content policy created and linked
• Behaviors auto-generated (5 allowed, 5 disallowed)
• Synthetic dataset created (stage = seed)
• Dataset review submitted
• Training triggered and completed (TRAINING_COMPLETE)
• Policy deployed (DEPLOYED)
• Policy scaled down cleanly