Guardrail Serving
Applies to DynamoGuard, across a region pair. For in-region zone behaviour see Guardrail Serving Across Zones.
Guardrail serving determines how long a regional failover takes. It also has the least redundancy of any component in the current release.
Redundancy
| Property | Current release |
|---|---|
| Replicas, built-in models | One by default, configurable |
| Replicas, custom policy models | One. The platform sets the count itself |
| Disruption budget | Created automatically at two or more replicas |
| Loss of the only pod | The policy is unavailable until a replacement loads its model |
| Standby GPU capacity | Provisioned during recovery |
| Model weights in the standby | Loaded after the pods start |
Whether a policy runs on CPU or GPU is set by its model class. A request naming an unavailable policy fails as a whole, including the other policies in the same request. The API returns an error rather than a verdict, and the calling application decides what happens to the content: see Failure Behavior.
Built-in guardrail models accept a replica count, and a second replica reaches a second node. Reaching a second zone additionally requires accelerator nodes in more than one zone, which is a property of the cluster rather than of the platform. See Guardrail Serving Across Availability Zones.
Policy States That Serve Nothing
A policy can be absent from serving for reasons other than a failed pod, and they are not equivalent for recovery.
| State | Cause | What a request naming the policy does |
|---|---|---|
| Disabled | An operator disabled it. The platform scales its model to zero | Fails with a 400 naming the policy |
| Scaled to zero | The consequence of disabling. No pod exists | As above |
| Deployment failed | The model never became ready, commonly a missing artifact in the standby | Fails the whole request |
| Not yet loaded | The pod is running and the weights are still loading | Fails the whole request until ready |
| Not resolvable | The policy id does not resolve in this region's policy store | Returns success with that policy silently omitted |
The last row is the one to design around in a paired region. A standby whose policy set is
behind the primary answers requests normally while enforcing less, because a policy that cannot
be resolved is dropped rather than raising an error. Compare the policy list between regions
before recording the pair as ready, and compare appliedPolicies in the response against the
identifiers sent.
Where Recovery Time Goes
| Phase | Typical duration |
|---|---|
| Detect and declare | Minutes |
| Routing withdrawal | Probe interval multiplied by samples, plus DNS propagation |
| Datastore promotion | Minutes |
| Platform pods ready | Minutes |
| Container image pull | Several minutes |
| Model weights loaded to GPU | 5 to 15 minutes for the large generative guards; well under a minute for the small classifiers |
| Readiness delay before the pod is used | Whatever readinessProbe.initialDelaySeconds is set to |
Measure these two separately. A pod that has loaded its model is still withheld from traffic
until its readiness delay expires, so a deployment tuned for the slowest model makes every
policy recover at that pace. A startupProbe removes the trade-off: slow models keep a long
budget, fast ones return as soon as they answer.
The last two phases account for most of the total. A standby holding GPU capacity with weights already resident removes both. A standby built during the incident pays them in full.
These durations are a budget composed from configured readiness ceilings and observed image and weight load times. Measure them with a drill before quoting them.
Custom Policies In The Standby
The Helm release recreates the out-of-the-box models in the standby. Every custom policy has to be redeployed from the UI or API in the standby before it is enforced, and the current release includes no automatic reconciler. Confirm each policy reaches a ready state before directing traffic.
This step belongs in the runbook alongside the parity check described in Traffic Routing.
GPU Availability In The Standby Region
The standby needs the GPU family its guardrail models were sized for. Where the standby region offers a different family, size the fleet against what that region actually provides and record the resulting card count, since throughput per card varies between families. See Region Requirements.