Playground

The playground is where prompt iteration meets governance: every invocation is recorded as evidence, and dataset replays run through the same durable eval pipeline as any other run.

Invoke

Run a registry version (pick prompt + version; variables are auto-suggested from the template) or a draft template you edit inline. The result panel shows the completion, token counts, latency, the serving endpoint, and manifest cost. Missing variables render as empty strings and are flagged — the invocation proceeds so you can see exactly what the model received.

Every invocation is recorded: a playground.invoked audit event carries the prompt identity (registry id@version or draft content hash), endpoint, tokens, latency, and cost — and when tracing is enabled for your organization, the invocation also appears as an LLM span in the trace views. Playground traffic is never invisible.

Replay against a dataset

Replay renders a prompt version over every example of a frozen dataset version, completes each through the model, and judges the outputs with the evaluators you select — asynchronously. Submitting a replay returns a pending eval run immediately and takes you to its live run page; no model calls happen inside the request.

Two properties worth knowing:

• One budget. Generation tokens count against the run's maxJudgeTokens alongside judging, under the same hard reservation accounting — a replay cannot silently outspend its budget, and cost_usd covers generation plus judging.
• Same evidence. A replay run writes Quality Center evidence exactly like any other eval run. Replaying a candidate prompt against your regression dataset produces a gate-consultable verdict, which makes "new prompt version + replay + compare runs" a reviewable, governed release process for prompts.

Cancellation works between phases: cancel during generation and no judging starts; partial scores are kept either way.