Test Suites

What Is a Test Suite?

A test suite is a named group of tests that belong to a single prompt. Each suite has a name, an optional description, and one or more tests with assertions.

Creating a Suite

Open the Evals tab on any prompt and click New Suite. Give it a name that describes what it tests (e.g., “Tone Checks”, “Safety Guardrails”).

POST /api/prompts/{promptId}/eval/suites

{
  "name": "Tone Checks",
  "description": "Verify the prompt maintains a professional tone"
}

Managing Suites

Each prompt can have multiple suites. Use suites to separate concerns:

• Smoke Tests - Quick sanity checks that run fast
• Regression Suite - Comprehensive checks against your baseline
• Safety Suite - Assertions that check for harmful or off-topic output
• Performance Suite - Latency, token count, and cost thresholds

Suite API