Skip to main content

Projects

Introduction​

Projects in Dynamo AI serve as a foundational organizational framework for managing all aspects of your AI applications. For the initial release, this feature enables you to centrally manage and monitor your policies and AI systems. In future versions, projects will enable you to create dedicated workspaces to evaluate (DynamoEval), protect (DynamoGuard), and monitor (more to come) your AI applications.

What are Projects?​

A Project is a container that brings together all the components needed to protect and monitor an AI application:

  • AI Systems: Models that you've created or connected
  • Policies: Content, PII, Hallucination, and other guardrails
  • Monitoring Logs: Records of AI application activities across all AI Systems
  • Access Controls: Permission settings for team members

Key Benefits of Using Projects​

1. Centralized Organization​

Projects provide a structured way to organize all resources related to a specific AI initiative. Rather than managing AI systems, policies, and monitoring separately, you can group them logically within a Project, making it easier to track and manage your AI assets. In the future, Projects will continue to gain functionality towards central management and coordination.

2. Streamlined Governance​

By consolidating AI systems and policies within Projects, you can apply consistent governance controls across related AI systems serving the same application. This ensures that all AI systems are governed corresponding to the risks they present.

3. Enhanced Collaboration​

Projects enable team members to collaborate effectively on AI applications. With cascading role-based access controls, you can easy share all project assets with a new team member with one click.

4. Efficient Monitoring & Review​

Project monitoring logs aggregate logs that once had to be reviewed in multiple tabs or pages, allowing legal or other review team members to work in centralized locations.

Getting Started with Projects​

Creating a New Project​

  1. Navigate to the Projects tab in the Dynamo AI dashboard
  2. Click "Create New Project"
  3. Enter a name for your Project
  4. Click "Create"
  5. You should see your new project at the top of the list. Click “Monitor” (leads to monitoring logs) or “Manage” (leads to Policies) to enter the project.

Project Actions​

Project Actions Menu

Project Actions Dropdown

Project actions can be taken via the “more options” menu in the projects table or via various entry points within the Project. In this initial version, you can:

  1. Edit the Project Name
  2. Assign / Remove AI Systems
  3. Reassign an AI System — Since AI Systems can only belong to at most one project at a time, we made it easy to reassign AI Systems between projects.
  4. Apply / Remove Policies — Currently, we only support Policies that have at least one trained version
  5. Share Project — By default, sharing a project with a team member will automatically cascade view permissions to all assets belonging to a project. This ensures the team member will have the minimum require permissions to work on the project.
  6. Delete Project

Project Structure​

Each Project includes several key sections:

AI Systems Tab

AI Systems Tab

Project AI Systems

The AI Systems tab shows all AI models associated with the Project. Here you can:

  • View all AI systems in the Project
  • Assign / Remove AI systems
  • Access AI system specific pages

Policies Tab

Policies Tab

Project Policies

The Policies tab displays all policies applied within the Project. Here you can:

  • View all policies in the Project
  • Apply existing policies to AI systems — Policies do not have to be applied to all AI Systems within a project and can be selectively applied
  • Navigate to Policy pages
  • Share Policies

Monitoring Tab

Policies Tab

Project Monitoring Logs

The Monitoring tab provides insights into AI system activity within the Project. Here you can:

  • View conversation logs across AI systems in one place
  • Track policy violations & moderation actions
  • Provide feedback to retrain policies on false classifications

Project Advanced Concepts​

Using Policies with Projects​

  • Establish use-case / application-level policies: Define policies that address risks derived from the use case or application the project represents. These will serve as the default policies that apply to all AI Systems.
    • This might be policies such as Prohibit Financial Advice or Prohibit Social Scoring that any AI System in an application could engage in.
  • Establish AI System-level policies: Then, define policies that address risks derived from the nature of specific AI Systems. These can be used to provide targeted controls on unique risks.
    • This might be policies such as Faithfulness that only make sense on RAG-enabled AI systems or Japanese guardrails that only apply to AI systems tailored for a different language.

Log Aggregation Logic​

  • AI System assignment: When an AI System is assigned to a project, all of its logs are inherited by the project and can be reviewed to provide feedback.
  • AI System reassignment: When an AI System is reassigned to another project, the current project retains logs up until the time of reassignment. The project receiving the reassigned system inherits all associated logs.

Providing Feedback & Guardrail Retraining​

Similar to AI System monitoring logs, the Project monitoring logs similarly allows you to review a log item in detail and provide feedback to individual guardrail judgments. The most common use case is correcting a guardrail judgment and having the guardrail retrain on that corrected data. This can be done by:

  • Click on any log item in the monitoring logs
  • Select a content policy to provide feedback to / view definition
  • Choose your judgment of the datapoint and provide any textual feedback
  • Click send feedback

Policies Tab

Project Monitoring Logs - Feedback

The policy receiving feedback will have data available in the “Incoming Feedback” tab that can then be reviewed in detail and incorporated into the training set.

Best Practices for Using Projects​

Project Organization​

  • Create purposeful Projects: Organize Projects around specific use cases or applications. This allows you to take full advantage of permission cascading and log aggregation.
  • Use descriptive naming: Give Projects clear, descriptive names that indicate their purpose
  • Maintain Project hygiene: Regularly review and archive inactive Projects

Project Governance​

  • Implement appropriate access controls: Limit Project access to those who need it
  • Conduct regular reviews: Periodically review Project permissions and policies
  • Document key decisions: Maintain records of important decisions related to the Project
  • Monitoring Log Access Control: The monitoring logs for any project are only accessible to DynamoGuard Admins, so organization roles should be configured properly.

Project Collaboration​

  • Define clear roles: Establish who is responsible for each aspect of the Project
  • Establish communication channels: Set up appropriate channels for Project-related discussions
  • Schedule regular reviews: Conduct periodic reviews of Project progress and performance
  • Share knowledge: Document lessons learned and best practices within the Project