Skip to main content

Managing Policies

This section outlines how to manage, apply, edit, and retrain custom content policies.

Policy Page​

Navigate to the Policies tab to access the policy registry - a list of all policies you have created for guardrailing. Through this page, you can view policy information and apply them to models.

Policy Statuses​

Each policy has three status fields:

  1. Training: reflects the current progress of a custom content policy as it moves through the training pipeline:
    • In Creation: The policy is in the definition step and is being authored
    • In Review: The policy is going through example review
    • Data Generation Queued: Data generation has been queued
    • Generating Data: Synthetic data is being generated for training
    • Data Generation Failed: The data generation job has failed - if you encounter this error, try resuming the job or contact an admin if the error persists
    • Ready to Train: The data is ready and the policy can be queued for training
    • Training Queued: The policy training is in the job queue
    • Training: Policy training is in progress
    • Training Complete: The policy training is complete and the policy is ready to apply
    • Training Failed: Policy training has failed - if you encounter this error, try restarting the job or contact an admin if the error persists
    • New Changes: The policy was updated after training and should be reviewed and re-trained.
  2. Stage: whether the policy is applied to an AI system
    • In Development: The policy is still in the creation process and cannot be applied. (Only for custom content policies)
    • Ready to Apply: The policy is ready to be applied to an AI System
    • Applied: The policy has actively been applied to 1+ AI Systems
  3. Deployment: whether the policy is deployed and ready for use
    • Not Deployed: the policy is not deployed to a compute instance and cannot be used
    • Deployed: the policy is deployed to a compute instance and is ready to use

Policy Instance​

Click on the Manage Policy button to view the policy instance page for a specific policy. This page provides comprehensive informaiton about the policy definition, training data, and incoming feedback. This page can be used to edit and manage individual policies.

Policy Registry

Training Content Policies​

Before applying a content policy, the policy must first be fine-tuned on the synthetically generated training dataset. To train a policy, click the train policy button within the policy management page. The policy will then move to a state of 'Training Queued' and then 'Training Policy'.

Train Policy

Policy Versioning​

When training your policy, you will be asked to create a new policy version. Policy Versioning allows you to track changes made to policies and revert to previous versions if necessary.

Version Policy

Version Policy

Editing Policies​

You can edit a policy’s definition, training data, and feedback at any time. Here’s how to manage those edits:

Editing Policy Definitions​

Policy definitions can be edited from the Policy Management page. Here, edits can be made to the title, description, or associated behaviors. Allowed and disallowed behaviors can also be added and removed. After editing the policy, regenerate the training data to ensure that it aligns with the updated definition.

Policy Editing

Policy Editing

Adding Training Data​

Training data can also directly be added to a policy by clicking the 'Add Data' button in the Training Data tab. Here, data can either be provided one at a time by providing the example and compliance status or by uploading a file containing a larger set of datapoints. Adding training data will also prompt you to regenerate the rest of your training data so that it's better aligned to the uploaded data.

Adding Data

Editing Datapoints​

Existing training data can directly be edited by clicking into the particular datapoint. Here, the datapoint text or compliance status can be edited.

Adding Data

Adding Feedback Data​

After deploying a policy, you can collect feedback on how it performs in the live environment:

  • Navigate to the Monitoring Log to view feedback on datapoints
  • This feedback can be applied to the policy for continuous improvement

Leaving Feedback

Applying Feedback

Retraining Policies​

Whenever you make edits to a policy (such as modifying its definition or adding new training data), you must retrain the policy to ensure that the changes are incorporated. To retrain, navigate to the training tab and click train policy.