Behavior-driven testing

Status: Available. Engine: cucumber-js with the Databricks step library from @fabricorg/databricks-bdd; running examples live in packages/databricks-bdd/features/ and product scenarios in apps/api/features/ (both run in CI on the local profile). Steps marked Live steps are gated by tags and workspace configuration. Feature files are kept engine-neutral, but the shipped runner and step implementation are entirely TypeScript. Fabric does not require or currently ship a Python Behave/pytest-bdd adapter.

New to the framework? Complete the BDD quickstart before using this page as a feature reference. Moving from Python Behave? Use the migration guide and exact compatibility matrix.

BDD is the natural register for Databricks workloads: data producers, analysts, and platform engineers can all read — and agree on — a Gherkin scenario, and the same scenario is enforced against a local engine on every PR and against a real workspace nightly.

A complete example

Feature: Experiment aggregation attributes conversions correctly

  Background:
    Given catalog "fx_test" and schema "scenarios"
    And a table "exposures" with:
      | experiment_id | subject_id | variant_key | at                   |
      | checkout-cta  | u1         | control     | 2026-07-01T10:00:00Z |
      | checkout-cta  | u1         | treatment   | 2026-07-01T11:00:00Z |
      | checkout-cta  | u2         | treatment   | 2026-07-01T10:05:00Z |
    And a table "conversions" with:
      | subject_id | event_name | value | at                   | tenant_id |
      | u1         | purchase   | 49.0  | 2026-07-01T12:00:00Z | default   |

  Scenario: Conversions attribute to the first exposure
    When I run the aggregate for experiment "checkout-cta" with metric "purchase"
    Then variant "control" has 1 conversions
    And variant "treatment" has 0 conversions
    And the SRM guardrail does not fire

This feature shape runs in Fabric's product CI. A smaller cloneable example is available in the public published-package consumer. Run your suite locally (DuckDB, sub-second) or against a workspace:

DBX_TEST_PROFILE=local pnpm run test:features
DBX_TEST_PROFILE=live DBX_TEST_LIVE=1 pnpm run test:features

Long-running artifacts

Live-only scenarios cover Jobs, serverless notebooks, serverless dbt Workspace projects, cloned Lakebase branches, Volumes, and Lakeflow pipelines with poll-to-terminal steps. Full refresh is permitted only for DBX_TEST_AUTOLOADER_PIPELINE_ID; the runner refuses to full-refresh the production pipeline.

@live @dlt
Scenario: Auto Loader lands exposure files with no schema drift
  Given fixture files uploaded to volume "$DBX_TEST_VOLUME":
    | path                                           | prefix    |
    | features/fixtures/autoloader-exposures.ndjson | exposures |
  When I start a full refresh of pipeline "$DBX_TEST_AUTOLOADER_PIPELINE_ID"
  Then the pipeline update completes within 15 minutes
  And table "exposures" contains rows matching:
    | experiment_id | subject_id |
    | checkout-cta  | u1         |
  And no rows were rescued in table "exposures"

@live @jobs
Scenario: Nightly readout job succeeds
  When I run job "fx-nightly-readout"
  Then the job run reaches SUCCESS

Step library reference

The generated step catalog is the authoritative wording list. The grouped summary below helps you discover the right family.

Given — environment and data setup:

• a Databricks workspace — validates auth + warehouse eagerly on the live profile; a no-op locally.
• catalog {string} and schema {string} — scopes the scenario's execution context (must precede the first warehouse-touching step).
• a table {string} with: — data table; column types inferred from cells (BIGINT/DOUBLE/TIMESTAMP/BOOLEAN/STRING, empty cells → NULL).
• a table {string} with schema {string} and rows: — explicit DDL.
• an empty table {string} with schema {string}.
• a table {string} with schema {string} from fixture {string} — NDJSON file.
• a Lakebase database — production provider, OAuth exchange, and fx,public search path.
• fixture files uploaded to volume {string}: — scenario-prefixed Files API upload with automatic teardown.
• service principal {string} with grants: — creates a secondary OAuth execution context; credentials come from DBX_TEST_PRINCIPAL_<NAME>_*.

When — actions:

• I run the SQL: (docstring; failures are captured for the the statement fails … assertions) · I query table {string}
• I run job {string} (@jobs) · I submit notebook {string}
• I submit the job orchestration: (JSON docstring; waits for terminal state)
• I start the job orchestration: · I cancel the current job run · I wait for the current job run
• I repair failed tasks in the current job run
• I query Lakebase: (docstring)
• I start a full refresh of pipeline {string} (@dlt)
• I start a refresh of pipeline {string}
• I run dbt build for {string}

Then — assertions:

• table {string} has {int} rows
• table {string} contains exactly: / contains rows matching:
• table {string} matches golden {string} ordered by {string}
• the result has {int} rows / the result contains rows matching: / the result matches golden {string}
• the statement fails with {string} / the statement fails with permission denied
• the job run reaches {runState} · the pipeline update completes within {duration}
• task {string} reaches {runState} · the job run has {int} tasks
• no rows were rescued in table {string}
• row count of {string} is within {int}% of {int}
• table {string} matches schema contract {string}

Product-domain steps (aggregates, variant {string} has {int} conversions, the SRM guardrail fires / does not fire) are an extension example living in apps/api/features/support/product-steps.ts — layer your own domain steps on DatabricksWorld the same way.

Tags and profiles

Reports

Runs emit a schema-versioned, secret-redacted evidence JSON via the bundled formatter (@fabricorg/databricks-bdd/evidence-formatter, same spirit as the live check runner's output) so nightly BDD runs double as audit artifacts. JUnit XML, HTML, and rerun manifests ship in both package and product configurations. Failed steps attach truncated, secret-redacted SQL, the Databricks statement ID, and captured process output. The same structured failure record is included in evidence JSON schema v2 so nightly failures are self-diagnosing without opening the HTML report.

BeforeStep captures up to 32 KiB of ordered stdout and stderr per step. That includes console.* and Pino's standard process-stream destination. Passing-step output is discarded; failure output is redacted before it is attached or persisted. Set DBX_TEST_CAPTURE_OUTPUT=passthrough to display and capture output, or DBX_TEST_CAPTURE_OUTPUT=off to disable capture. Child processes that inherit OS file descriptors directly and custom logger transports do not pass through this in-process boundary.

The World provides a reverse-order scenario cleanup stack, typed worldParameters userdata (with DBX_TEST_USERDATA_* fallback), run-level live preflight, AfterAll teardown, run/feature shared fixtures, hierarchical run → feature → scenario state, typed action composition, active capability tags, cardinality fields, and custom {runState}, {duration}, and validated {table} parameters. Local DuckDB scenarios run in parallel; live scenarios remain serial because they share governed scratch resources. The generated steps catalog includes unused definitions and freezes the portable wording contract.

Behave-style shared fixtures

Use addCleanup() for scenario resources. For expensive resources that Behave would create in before_all or before_feature, use the typed fixture helpers:

Given('a seeded reference model', async function (this: DatabricksWorld) {
  const model = await this.featureFixture('reference-model', async () => {
    const created = await deployReferenceModel()
    return { value: created, cleanup: () => deleteReferenceModel(created.id) }
  })
  this.referenceModel = model
})

BeforeAll(async () => {
  await sharedFixtures.runFixture('shared-warehouse', async () => ({
    value: await startWarehouse(),
    cleanup: stopWarehouse,
  }))
})

Concurrent calls share the same setup promise, setup failures can be retried, and cleanup runs once in reverse registration order. featureFixture() is isolated by feature URI. Because cucumber-js has no reliable after_feature event—especially with filters, retries, and parallel workers—feature resources are released by Fabric's AfterAll hook, not immediately after the last scenario in that file. This preserves once-per-feature setup; explicit layered user state is provided separately through the World APIs below.

TypeScript equivalents for Behave context and composition

Fabric exposes the outcomes of Behave's layered context, execute_steps(), active tags, and cfparse cardinalities without invoking Python or reparsing Gherkin at runtime:

import { Given, When } from '@cucumber/cucumber'
import {
  defineAction,
  parseCardinalityField,
  type DatabricksWorld,
} from '@fabricorg/databricks-bdd'

const rememberColumns = defineAction<[string], string[]>(async (world, value) => {
  const columns = parseCardinalityField(value, '+', (item) => item, {
    fieldName: 'columns',
  })
  world.setState('scenario', 'columns', columns)
  return columns
})

Given('catalog policy {string}', function (this: DatabricksWorld, policy: string) {
  // Scenario values override feature values, which override run values.
  this.setState('feature', 'catalogPolicy', policy)
})

When('I select columns {string}', async function (this: DatabricksWorld, value: string) {
  await this.runAction(rememberColumns, value)
  const policy = this.requireState<string>('catalogPolicy')
  // Use policy and the typed scenario action result here.
})

parseCardinalityField() accepts Behave-compatible ?, *, and + cardinalities. Typed actions are ordinary reusable functions, so they retain compiler checking, direct unit tests, useful stack traces, and IDE navigation. They replace the outcome of context.execute_steps() without hidden textual step recursion.

Capability predicates work on inherited Feature, Rule, Scenario Outline, and Scenario tags:

@requires.cloud=azure @requires.compute=serverless
Scenario: serverless SQL contract
  Given a Databricks workspace

@use.with_cloud=azure
Scenario: Behave active-tag spelling also works
  Given a Databricks workspace

Built-in keys are profile, cloud, compute, and stage. Configure custom keys with DBX_TEST_CAPABILITY_<KEY>, for example DBX_TEST_CAPABILITY_UNITY_CATALOG=true. Use @excludes.<key>=<value> or the Behave-compatible @not.with_<key>=<value> form for negative predicates.

Developer workflows

The default profile emits durable CI reports. Two focused TypeScript workflows make local development closer to Behave's ergonomics:

# Colored, source-annotated Gherkin output using Cucumber 13's built-in formatter
pnpm bdd:pretty

# Only @wip scenarios, stop after the first failure, concise summary
pnpm bdd:wip

Cucumber also supplies --dry-run, --fail-fast, snippet generation, language selection, sharding, retries, and tag expressions directly. Fabric adds the Databricks execution contexts, workload steps, cleanup, evidence, and profile policy around that runner.

TypeScript strategy and compatibility boundaries

All shipped framework code stays in TypeScript: hooks, World state, parameter types, workload clients, live checks, formatters, and domain step extensions. Analysts can author readable .feature files without writing TypeScript; a platform team only writes TypeScript when adding new reusable step behavior.

Python is therefore an optional ecosystem bridge, not a feature dependency. The generated steps catalog keeps the Gherkin contract portable if a customer later chooses to implement a Behave or pytest-bdd adapter, but no Python adapter ships today and Fabric's production runner does not invoke Python step definitions.

Python files may still be the workload under test—for example a Databricks notebook or Lakeflow pipeline—but orchestration, assertions, Gherkin steps, evidence, lifecycle management, and CI policy remain TypeScript.

Published-package consumer gate

The public fabric-experiments-consumer repository is intentionally outside this monorepo. Its scheduled workflow installs exact public npm versions, imports the published step and evidence formatter subpaths, runs a DuckDB-backed feature, and uploads JUnit, HTML, and evidence JSON. This catches workspace:* leakage, missing package exports, peer-dependency mistakes, and tarballs that differ from the source build.

To run it yourself:

git clone https://github.com/Fabric-Pro/fabric-experiments-consumer.git
cd fabric-experiments-consumer
corepack enable
pnpm install --frozen-lockfile
pnpm test

TypeScript parity boundaries:

• Fabric provides typed actions rather than interpreting nested Gherkin text. The reusable behavior is equivalent; Python's context.execute_steps() API and its runtime parser are not copied.
• setState() / getState() / requireState() provide run, feature, and scenario precedence. State and fixture cleanup are explicit and typed rather than Python attribute mutation.
• parseCardinalityField() covers ?, *, and +; data tables remain the preferred representation for larger structured lists.
• Feature fixtures are process-local and keyed by feature URI. Under local worker parallelism they are once per feature per worker; live execution is serial, so an expensive Databricks artifact is created once per feature run.

For production extension patterns, required-check composition, identity separation, and CI policy, continue with Advanced Databricks BDD.

Governance scenarios

Least privilege is an executable spec. The grants step applies grants with the primary CI identity, runs subsequent SQL with a separately configured service principal, and revokes grants during scenario teardown:

@live
Scenario: CI principal cannot read outside its schema
  Given service principal "fx-ci" with grants:
    | securable          | privilege |
    | fx_test.scenarios  | USE, SELECT |
  When I run the SQL:
    """
    SELECT * FROM fx_prod.raw.exposures LIMIT 1
    """
  Then the statement fails with permission denied