Concepts
Guardrails
SRM, metric decline, and error-rate checks protect experiments while they run.
Guardrails evaluate whether an experiment should continue running.
The M5.1 SRM watcher uses shared guardrail evaluation code and can invoke a guardrail_breach action when the split looks suspicious.
Common guardrails:
- SRM — sample ratio mismatch; detects assignment or delivery imbalance.
- Metric decline — detects harmful movement in a key metric.
- Error rate — detects variant execution or client-side failure spikes.
The watcher runs as a Fabric Harness/Temporal workflow in staging and production.