FabricFabric
Testing on Databricks

Publish results to Quality Center

Send BDD, Databricks live-suite, or JUnit evidence from any CI system to Studio and investigate failures with Databricks deep links.

Quality Center is Studio's durable test-results surface. It is separate from A/B analytics:

SurfaceQuestionInputs
ExperimentsDid a treatment change a product or business metric?Assignments, exposures, conversions, covariates, dimensions
QualityDid a workload, table, pipeline, notebook, job, contract, or CLI preflight behave correctly?Fabric BDD, Databricks live-suite JSON, JUnit XML, Fabric CLI check JSON

fx validate, fx plan, fx apply, and fx report continue to update or read the Experiments and Audit surfaces. They are not silently copied into Quality. Publish their test/preflight outcomes explicitly using the generic CLI format below.

Live verification — July 16, 2026: the production Quality list and detail pages were opened with a real tenant session. They showed a generic CLI release verification (3/3 passed), the latest Fabric product BDD run (10/10 passed), and Databricks live suite (20 passed, one skipped), including revision metadata, individual cases, durations, and CI links. The generic run was published through the production API with a temporary Quality publisher key, then the key was revoked.

Hosted entry points

The branded Studio URL is served directly by Cloudflare. fx login discovers the canonical API, and CLI publication uses the organization-scoped Fabric API key.

Target-pack runs retain targetPackId, name, version, maturity, capabilities, certification scopes, cloud, region, runtime and compute in their environment metadata. Quality Center can filter history by pack ID and displays the pack version in the run table and detail header.

fx targets run sql-delta --live --evidence reports/sql-delta.json
fx targets publish reports/sql-delta.json

One-time setup

  1. Sign in to Studio and select the target organization.
  2. Open Settings → API keys and create a key with the Quality publisher preset. This gives the key the experimenter role and read + write scopes without API-key or organization administration. Set an expiry and copy the key immediately; it is stored hashed and displayed only once.
  3. Configure and verify the CLI:
fx login --api-key fx_key_...
fx doctor

The profile stores the discovered API, immutable organization ID, role, and API key with private file permissions. It never prints the secret. The Quality page's Publish another test run panel shows the same safe workflow.

Publish supported evidence

Fabric BDD

The BDD formatter writes reports/evidence.json:

npx @fabricorg/experiments@latest test publish reports/evidence.json \
  --suite "Checkout data contracts"

Studio records scenarios, pass/fail/skip counts, durations, tags, revision/CI metadata, and captured failure evidence. A Databricks statement ID becomes a direct SQL query-history link.

Databricks live suite

@fabricorg/databricks-testkit writes reports/databricks-live/evidence.json by default:

fx test publish reports/databricks-live/evidence.json \
  --suite "Production workspace certification"

The run includes configured, passing, failing, and skipped checks plus cloud, workspace, warehouse, runtime, and compute metadata. Fabric's scheduled live workflow publishes both live-suite and product BDD evidence automatically.

JUnit from any runner

JUnit XML is the compatibility boundary for dbt, Jest/Vitest, pytest, Java, and other enterprise test tools:

fx test publish reports/junit.xml --suite "dbt contracts"

Nested suites, class names, durations, failures, errors, and skips are normalized into the same Quality run model.

Generic CLI or preflight checks

Use this format for checks around fx plan, manifests, deployment policy, data contracts, or another CLI workflow:

{
  "kind": "fabric-cli",
  "suite": "Experiment production preflight",
  "success": true,
  "environment": { "target": "production" },
  "cases": [
    { "name": "manifest validates", "status": "passed", "durationMs": 42 },
    { "name": "plan has no blocked changes", "status": "passed", "durationMs": 18 }
  ]
}
fx test publish reports/experiment-preflight.json

Supported case statuses are passed, failed, skipped, pending, undefined, and unknown. Use --suite to override the file's suite name, --run-url for an artifact/CI link, and --external-id for idempotency outside GitHub Actions.

Production API authentication

For unattended CI, use a scoped Fabric API key against the canonical API:

export FX_API_URL=https://api.experiments.fabric.pro
export FX_API_KEY=<quality-publisher-api-key>
export FX_ORG_ID=<organization-id>

fx test publish reports/evidence.json

Databricks service credentials used to execute or inspect workloads remain separate from the Quality publisher key and are not sent to the API gateway.

Authorization and tenancy

Credential or roleCapability
Fabric API key, experimenter + writePublishes or updates Quality runs for its own organization
Fabric API key, read onlyCan call read endpoints but cannot publish
Studio viewer sessionReads Quality list and detail pages for organizations where the user is a member
Studio admin or ownerCreates, expires, and revokes organization API keys

Every list, detail, insert, and idempotent update includes the authenticated organization ID in its database predicate. A key from organization A cannot publish to or read organization B. An admin also cannot create a key with a role higher than their own.

CI example

- name: Run Databricks BDD
  run: pnpm test:bdd

- name: Publish Quality evidence
  if: always()
  env:
    FX_API_URL: ${{ secrets.FX_API_URL }}
    FX_API_KEY: ${{ secrets.FX_API_KEY }}
    FX_ORG_ID: ${{ secrets.FX_ORG_ID }}
  run: npx @fabricorg/experiments@latest test publish reports/evidence.json

On GitHub Actions, the CLI derives an external ID from repository, run ID, attempt, and evidence kind. Re-running the same attempt updates the existing tenant-scoped run instead of duplicating it. Other CI systems should set --external-id explicitly.

What is stored and displayed

  • Run source, suite, status, duration, summary, environment, revision, and artifact links.
  • Up to 2,000 cases per run with status, duration, tags, bounded errors, statement IDs, resource links, and structured details.
  • Paginated Studio history, 50 runs per page, with tenant-scoped detail routes.
  • Durable evidence rows. The UI intentionally has no casual delete action; organizational evidence should be governed through retention policy rather than removed by experimenters.

Evidence is redacted again at ingest: token-, secret-, password-, authorization-, cookie-, and API-key-shaped fields are replaced, embedded bearers/Fabric keys are removed, and oversized strings are truncated. CI artifact storage remains the source for full HTML, screenshots, and large logs.

Troubleshooting

SymptomMeaning and fix
Fabric 401 UNAUTHENTICATEDThe API key is invalid, expired, revoked, from another deployment, or does not belong to FX_ORG_ID.
Fabric 403 FORBIDDENThe key is read-only or below experimenter; create the Quality publisher preset.
404 on a run detailThe run ID does not exist in the authenticated organization; cross-tenant IDs intentionally appear missing.
Duplicate-looking runsSupply a stable --external-id; GitHub Actions derives one automatically per run attempt and evidence kind.
Evidence format rejectedPublish Fabric BDD/live JSON, JUnit XML, or JSON with "kind": "fabric-cli".

Continue with Advanced Databricks BDD for restricted identities, scratch environments, and certification policy.

On this page