Skip to main content

Hallucination Policies

Overview​

Hallucination policies enable the detection of hallucinated model responses for RAG systems.

  • Input Relevance: Evaluate and detect if the user-provided prompt inputs are aligned with the retrieved context for RAG systems. Inputs that are not related to the context will be categorized as “off topic”.
  • Response Relevance: Evaluate and detect if the model's responses are aligned with user-provided prompt for RAG and non-RAG systems.
  • Response Faithfulness: Evaluate and detect if the model response is faithful and adheres to the provided context for RAG systems.

Hallucination Policy Actions​

You can manage what happens to outputs violating the hallucination policies with the actions below:

  • Flag: flag content for moderator review
  • Block: block user inputs or model outputs containing hallucinated content