EchoStash

Deploy & Eval

PLP v1.1.0 adds deployment and evaluation endpoints. These are optional capabilities — check the server's discovery response to see if supported.

Deployment

Deploy a prompt version to a named environment (e.g., staging, production):

http

POST /prompts/{prompt_id}/deploy

{
  "versionNo": 3,
  "environment": "production"
}

SDK Usage

const result = await client.deploy("welcome-email", 3, "production")
// { promptId: "welcome-email", versionNo: 3, environment: "production", deployedAt: "..." }

Evaluation

Run eval suites against prompts via the PLP API. Send eval content as YAML:

http

POST /prompts/{prompt_id}/eval
Content-Type: text/yaml

suite: "Quality Check"
config:
  target: welcome-email
tests:
  - name: "Has greeting"
    given:
      name: "Alice"
    expect_render:
      - contains: "Alice"
  - name: "Professional tone"
    given:
      name: "Bob"
    expect_llm:
      - llm_judge: "Is the response professional?"
      - sentiment: positive

Eval Response

json

{
  "suiteName": "Quality Check",
  "status": "pass",
  "summary": {
    "total": 2,
    "passed": 2,
    "failed": 0,
    "durationMs": 1862
  }
}

SDK Usage

const evalYaml = fs.readFileSync("eval/tests/smoke.eval", "utf-8")
const result = await client.runEval("welcome-email", evalYaml, { versionNo: 3 })
console.log(result.summary) // { total: 2, passed: 2, failed: 0 }

Dataset Management

// Save a dataset
await client.saveDataset("welcome-email", "customer-data", {
  name: "Customer Feedback",
  parameters: [{ name: "case_1", customer_msg: "Help me!" }]
})

// Retrieve a dataset
const dataset = await client.getDataset("welcome-email", "customer-data")

Capability Detection

const discovery = await client.getDiscovery()

if (discovery.capabilities.deploy) {
  await client.deploy("my-prompt", 3, "production")
}
if (discovery.capabilities.evaluation) {
  await client.runEval("my-prompt", evalContent)
}