Tracing

Fabric Experiments ingests OpenInference/OTLP traces, so any instrumentation library that emits OpenInference semantic conventions (OpenAI, LangGraph, LlamaIndex, Vercel AI SDK, and the wider ecosystem) works without modification — point its OTLP exporter at your ingest endpoint. Spans tagged with an experiment and variant power per-variant runtime analytics and online evals.

Ingest endpoint and authentication

Traces are ingested at the edge:

POST https://otlp.experiments.fabric.pro/<org>/v1/traces
Authorization: Bearer fxit_...
Content-Type: application/json | application/x-protobuf

• OTLP/HTTP with JSON or protobuf payloads is supported. gRPC (:4317) is not — configure exporters with OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf (or http/json).
• Ingest tokens are org-scoped, write-only, and rotatable. Mint them in Settings → Ingest tokens; the secret is shown once. Tokens are bound to your organization — a token cannot write into another org's path — and revocation takes effect at the edge immediately.
• Browser (cross-origin) ingestion requires explicitly configured allowed origins; server-side ingestion is unaffected by CORS.

First-party SDK tracing

The Node and Web SDKs include a tracer that emits OpenInference spans and stamps the active experiment assignment automatically:

import { createTracer } from '@fabricorg/experiments-sdk-node';

const tracer = createTracer({
  ingestUrl: 'https://otlp.experiments.fabric.pro',
  tenantId: 'my-org',
  ingestToken: process.env.FX_INGEST_TOKEN,
});
tracer.setAssignmentFromTreatment(treatment); // fx.experiment_id / fx.variant_key

const answer = await tracer.trace(
  { name: 'ChatCompletion', kind: 'LLM', model: 'gpt-4o', input: question },
  async (span) => {
    const out = await llm.complete(question);
    span.setOutput(out.text);
    span.setTokens(out.usage.prompt, out.usage.completion);
    return out.text;
  },
);

In the browser, import from @fabricorg/experiments-sdk-web/tracing — a separate entry point, so the experimentation tag's size budget is unaffected.

Where spans go

Ingested spans land in two places:

1. Hot store (Postgres) — powers the interactive Studio views below.
2. Lakehouse (R2 → Auto Loader → Delta) — the system of record for analytics; join spans against exposures and conversions in your Databricks workspace.

Hot-store data is pruned on a rolling window (default 14 days, legal holds honored); the lakehouse copy is retained per your warehouse policy.

Studio views

• Traces — filterable list (kind, errors-only, experiment, session) with duration, token, and cost columns.
• Waterfall — per-trace span tree with proportional timing bars, input/output values, attributes, and per-span dollar cost.
• Sessions — multi-trace conversation view in chronological order.
• Experiment detail → Traces — recent traces attributed to each variant of a running experiment.
• Annotations — attach human labels (score 0–1 plus explanation) to any span; online eval scores from sampled live traffic appear alongside.

Privacy controls

Per-organization redaction policies apply at the ingestion edge, before storage: drop input.value/output.value entirely, or strip all non-essential attributes to an allowlist. Structural fields (span kind, token counts, model name, session and experiment ids) always survive so analytics keep working on redacted data.