Databricks target packs

Target packs bundle the code and evidence needed to validate a Databricks workload consistently. Each pack provides typed TypeScript interfaces, offline tests, live checks, configuration diagnostics, documentation, and Quality Center metadata.

Use a target pack when you want a supported starting point for a workload instead of assembling REST calls, polling, cleanup, reports, and evidence publication yourself.

Available packs

live-gated means Fabric has run the pack's required check against its named Azure Databricks certification environment. It does not imply certification for every cloud, runtime, network topology, or optional check.

The lakeflow-jobs adapter and checks are published and a basic Azure serverless two-task dependency run has passed. Its broader condition, loop, retry, cancellation, and repair matrix is not yet certified, so the pack is deliberately labeled shipped rather than live-gated.

List and diagnose packs

Install the CLI or run it directly:

npm install -g @fabricorg/experiments
fx targets list
fx targets list --json

doctor checks configuration without creating or changing a Databricks resource:

fx targets doctor sql-delta
fx targets doctor lakeflow-jobs --json

The output names missing environment variables and checks. Secret values are never printed.

Run a pack

Live execution is fail-closed. You must pass --live or set DBX_TEST_LIVE=1:

fx targets run sql-delta --live \
  --required sql,delta-contract \
  --evidence reports/sql-delta.json \
  --junit reports/sql-delta.xml

Authentication is resolved in this order:

1. DATABRICKS_BEARER
2. DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET
3. Databricks OIDC environment or token file
4. DATABRICKS_TOKEN

Use a dedicated workspace identity and disposable catalog/schema/resources. Required grants depend on the selected pack; doctor reports the resource configuration it can validate before execution.

Lakeflow Jobs orchestration

The Jobs adapter accepts a typed JobsSubmitRequest and validates task keys, dependencies, unknown references, self-dependencies, and cycles before submission.

import {
  DatabricksJobsAdapter,
  type JobsSubmitRequest,
} from '@fabricorg/databricks-testkit'

const request: JobsSubmitRequest = {
  run_name: 'orders-contract',
  tasks: [
    {
      task_key: 'seed',
      notebook_task: { notebook_path: '/Workspace/Shared/orders-seed' },
      max_retries: 2,
      retry_on_timeout: true,
    },
    {
      task_key: 'validate',
      depends_on: [{ task_key: 'seed' }],
      notebook_task: { notebook_path: '/Workspace/Shared/orders-validate' },
    },
  ],
}

const adapter = new DatabricksJobsAdapter(client)
const runId = await adapter.submit(request)
const result = await adapter.wait(runId)

Supported typed task forms include notebooks, Python wheels, JARs, Spark Python, Spark submit, pipelines, SQL, dbt, nested Jobs, conditions, and for_each tasks. The adapter also exposes get, cancel, and repair.

For a live pack run, provide the scenario as JSON:

export DBX_TEST_JOBS_ORCHESTRATION_JSON='{
  "request": {
    "run_name": "orders-contract",
    "tasks": [
      {
        "task_key": "seed",
        "notebook_task": { "notebook_path": "/Workspace/Shared/orders-seed" }
      },
      {
        "task_key": "validate",
        "depends_on": [{ "task_key": "seed" }],
        "notebook_task": { "notebook_path": "/Workspace/Shared/orders-validate" }
      }
    ]
  },
  "expectedResult": "SUCCESS",
  "expectedTasks": { "seed": "SUCCESS", "validate": "SUCCESS" }
}'

fx targets run lakeflow-jobs --live

Use target packs in TypeScript

import {
  builtinTargetPacks,
  createTargetPackRegistry,
  doctorTargetPack,
  runTargetPack,
} from '@fabricorg/databricks-testkit'

const registry = createTargetPackRegistry(builtinTargetPacks)
const pack = registry.get('sql-delta')
if (!pack) throw new Error('sql-delta pack is unavailable')

const diagnosis = doctorTargetPack(pack, process.env)
if (!diagnosis.ready) {
  throw new Error([...diagnosis.missingRequirements, ...diagnosis.missingRequiredChecks].join(', '))
}

const evidence = await runTargetPack(pack, {
  env: process.env,
  evidencePath: 'reports/sql-delta.json',
  junitPath: 'reports/sql-delta.xml',
})

The API is additive to the lower-level live checks. Existing check exports remain available when you need a custom suite.

Publish evidence to Quality Center

Target-pack evidence contains the pack ID, version, maturity, capabilities, certification scope, cloud, region, workspace, runtime, and compute metadata. Credentials and secret values are excluded or redacted.

fx login --api-key fx_key_...
fx targets publish reports/sql-delta.json

Quality Center stores the run inside the authenticated organization. Open Quality in Studio to filter history by target-pack ID and inspect cases, durations, environment metadata, and failure evidence.

Current boundaries

• Azure Databricks is the certified cloud target. AWS and Google Cloud are not currently certified targets.
• A pack's certification is limited to the compute, runtime, identity, and topology named in its evidence.
• Live packs can create, execute, repair, or delete disposable Databricks resources. Use an isolated test environment and least-privilege identity.
• Fabric stores governed summaries and attachments. Specialized tools remain authoritative for detailed Playwright traces, load-test samples, or SARIF.
• Python, R, SQL, and Scala workloads can be tested, but Fabric's extension and step-definition API is TypeScript.

See Certification evidence for verified target boundaries and Quality Center for publication, authentication, tenancy, and production-gate behavior.