Harness agents and Fiber

Fabric Harness is the execution substrate for operational experiment jobs. Fabric Experiments supplies the domain rules, organization boundary, audit log, and Quality history; Harness supplies the finite-job and durable-agent runtime; Fiber supplies the interactive terminal experience.

You do not need a separate Harness server to use the hosted operator. The Experiments API exposes a Harness-compatible, organization-scoped endpoint and fx connects Fiber to it securely.

Start here

Requirements:

• Node.js 22 or newer;
• an organization in Fabric Experiments;
• an Experiments API key with write scope; and
• Databricks CLI authentication when Studio is hosted by Databricks Apps.

Install and connect:

npm install --global @fabricorg/experiments

fx login --studio-url https://<your-studio-host> --api-key fx_key_...
fx doctor
fx harness doctor

fx login discovers the API and organization, verifies the key and role, and stores a private local profile. For a Databricks App it also uses your Databricks CLI login to cross the Apps OAuth gateway. The Fabric API key and Databricks bearer serve different purposes; both checks remain active.

Use Fiber

fx harness fiber

Fiber lists the jobs available to your organization and lets you submit JSON inputs interactively. Use the line-oriented UI when a full terminal renderer is not suitable:

fx harness fiber --plain

fx starts a random-port server bound only to 127.0.0.1. Fiber receives an ephemeral bridge token and the localhost URL. The bridge injects the Fabric API key and Databricks bearer upstream; neither hosted credential appears in Fiber's command arguments. The bridge accepts only operator discovery and finite-job routes, caps request bodies, and closes when Fiber exits.

Run one job from automation

Fiber is optional. CI and scripts can invoke the same typed operator contract:

fx harness list --json
fx harness run quality-status
fx harness run srm-watcher --input '{"apply":false,"metric":"signup"}'
fx harness run analyst --input '{"experimentId":"<id>","deliver":false}'

Job inputs and safety

SRM mutation and analyst delivery are false unless explicitly requested. Every job validates its JSON input, derives a stable action key for governed actions, and writes a tenant-scoped completion event to the audit log. Audit summaries do not contain API keys, gateway tokens, analyst prose, or prompts.

See it in Studio and Quality Center

Open Harness agents in Studio to see the live operator catalog, safety semantics, terminal commands, latest certification result, certified background-agent registrations, and recent durable runs. An admin or owner can install, enable, or disable a certified agent there. Open Quality Center for retained evaluation details.

The certified catalog currently includes srm-watcher, analyst, and eval-runner. Each organization registration pins an immutable implementation digest, Fabric Harness runtime version, model policy, and capability grant. Before every Temporal activity, the worker asks the Experiments control plane to admit that exact executable. A disabled registration or mismatched digest never enters the agent body. Completion stores a worker-build attestation and SHA-256 input/output hashes, not the underlying prompt or result payload. New official worker releases advance a registration to a new immutable built-in version without re-enabling a registration an administrator disabled.

The scheduled certification exercises:

1. operator discovery;
2. a tenant-scoped Quality read;
3. an SRM scan that must remain dry-run and pause zero experiments; and
4. an analyst preview that must not deliver.

It emits standard fabric-cli evidence and publishes it under the exact suite name Fabric Harness operator. The same check runs through the Cloudflare staging API and through the Databricks Apps OAuth gateway. A failing safety assertion produces failed Quality evidence and fails the scheduled run.

You can reproduce the check with your own target:

FX_HARNESS_API_URL=https://<host>/api \
FX_HARNESS_API_KEY=fx_key_... \
FX_HARNESS_ORG_ID=<organization-id> \
node scripts/harness-operator-certification.mjs

fx test publish reports/harness-operator/evidence.json \
  --suite "Fabric Harness operator"

For a Databricks App, use fx login/the stored profile or provide workspace OAuth credentials using the standard CLI environment variables. Never commit keys or gateway tokens.

Durable background execution

The repository's worker supports two execution modes:

• inline runs the SRM watcher and analyst loop in a local development process; and
• temporal runs durable workflows and activities using the task queues in infra/harness/task-queues.yaml.

Audit forwarding dispatches terminal experiment events to the analyst workflow. The consumer starts one workflow per event and advances its cursor only after successful dispatch. The hosted finite-job API and background worker reuse the same operator implementation, so dry-run rules and readout behavior do not drift between Fiber, Studio-triggered operations, and Temporal.

The production worker can run as its own Databricks App in a customer workspace. It uses the App service principal for the Databricks gateway and resources, a Fabric API key for organization-scoped public API calls, and a worker token for internal dispatch/admission routes. See Databricks Harness worker.

Customer-authored defineAgent() executables remain customer-operated. The official worker deliberately does not download arbitrary agent modules into its process. An admin pins a tenant Ed25519 public key and enrolls an immutable signed capability manifest with fx harness register-key and fx harness enroll. If the manifest requests Unity Catalog Agent Services registration, the customer worker reconciles the external service and EXECUTE grants using the published Harness Databricks integration. See Databricks Harness worker. After rotating a release key, revoke the retired key with fx harness revoke-key <key-id> so it cannot authorize a later version.

Developers building those agents should begin with Fabric Harness's Your First Agent guide and Agent anatomy. The Fabric Harness documentation also covers runtime modes, policies, deployment targets, and Databricks integrations.

Roles and tenancy

• A viewer can discover operator definitions through the read-only endpoint.
• An experimenter, admin, or owner can invoke jobs.
• An API key must include write scope for invocation.
• The organization ID is part of the server route and is verified against the resolved session or API-key principal.
• The operator creates its API client with that resolved organization; a job input cannot select a different tenant.

For production automation, create a dedicated expiring API key in Studio Settings rather than using a human session.

Troubleshooting

fx harness doctor reports authentication failure

Run fx doctor. Confirm the profile has the intended organization and that the API key has not expired or been revoked. For Databricks Apps, run databricks auth profiles and refresh the selected login.

Fiber says a job is forbidden

Discovery is readable by viewers, but invocation requires the experimenter role and API-key write scope. Create the Quality publisher or automation key preset in Studio Settings.

Analyst preview works but delivery fails

Preview does not require a delivery integration. Delivery requires FX_SLACK_WEBHOOK_URL on the API/worker runtime. A missing webhook fails the explicit delivery request instead of pretending it was sent.

If an experiment or warehouse read has a transient connection failure, preview returns a clearly labelled degraded readout without delivering it. A missing experiment still returns a not-found error; only connection and timeout errors use the degraded path.

CI has no TTY

Use fx harness run <job> for one-shot automation. Fiber's --plain option is for interactive line-oriented terminals, not unattended jobs.