Databricks workload support matrix
Exact Fabric Experiments certification coverage, release gates, preview boundaries, and remaining live evidence.
“Supports Databricks” is too broad to be a useful release claim. Fabric
Experiments publishes an exact capability matrix instead. A workload is
release-ready only when its package contract, offline tests, fx targets
diagnostics, required live check, and retained Quality evidence are all green
for the named environment.
Workload matrix
| Databricks workload | Target pack | Package status | Azure live evidence | Release meaning |
|---|---|---|---|---|
| SQL warehouses and Delta Lake | sql-delta | Live-gated | Green | Statement execution, contracts, recovery, performance, backup, and rollback |
| Lakeflow pipelines, Auto Loader, and Volumes | lakeflow-ingestion | Live-gated | Green | Refresh and governed ingestion behavior |
| Jobs, notebooks, dbt, Python/JAR/Spark/SQL tasks | jobs-code, lakeflow-jobs | Live-gated | Green | Existing and disposable multi-task orchestration |
| Streaming tables, CDC, and Lakeflow Connect | streaming-cdc-connect | Live-gated | Green | Freshness, CDC correctness, and managed replication |
| Unity Catalog, secrets, and storage | unity-storage | Live-gated | Green | Positive and negative access controls plus Volume/secret behavior |
| Databricks Apps and Lakebase | apps-operational | Live-gated | Green | App health/resources and operational Postgres behavior |
| AI/BI dashboards and Genie | aibi-genie | Live-gated | Green | Published dashboard and authenticated conversation |
| MLflow experiments and UC model lifecycle | mlflow-lifecycle | Live-gated | Green | Experiment and exact registered model version |
| MLflow 3 tracing, agent evaluation, scorers, monitoring | mlflow3-agent-quality | Shipped preview | Workspace-gated | Two-task native MLflow Job plus four durable SQL evidence assertions; built-in Safety monitor feedback must be observed |
| Feature Engineering and online serving | feature-engineering-serving | Live-gated | Green | Offline table, materialization, and freshness |
| Model Serving and AI Gateway | model-serving-gateway | Live-gated | Green | Readiness, inference, and gateway configuration |
| AI Search, Vector Search, RAG, and served agents | vector-rag-agents | Live-gated | Green | Index readiness, retrieval, and agent inference |
| Managed MCP servers and UC MCP Services | managed-mcp-agents | Live-gated | Green | Streamable HTTP list/call plus an explicit denied call |
| Customer Harness agent enrollment | agent-services-enrollment | Shipped preview | Workspace-gated | Signed control-plane enrollment and UC Agent Services ACLs |
| Agent evaluation delivery | agent-evaluation-pipeline | Live-gated | Green | API/outbox/Temporal/judge/SQL/Quality path |
| Data Quality Monitoring and App OTel | data-quality-observability | Shipped preview | Baseline-gated | DQM freshness/completeness plus UC logs, spans, and metrics |
| Federation, Delta Sharing, and Clean Rooms | federation-sharing-cleanrooms | Live-gated | Green | Governed data access and collaboration metadata |
| Network, cost, policy, and regional recovery | security-cost-dr | Live-gated | Green | Least privilege, cost guardrail, and secondary workspace workload |
| Metric views, materialization, SQL alerts, and performance | semantic-analytics | Shipped | Awaiting live evidence | Native Job plus four durable semantic and operational assertions |
| AI Functions and batch/streaming inference | ai-functions-batch-inference | Shipped | Awaiting live evidence | Output, quality, and lineage assertions |
| Knowledge Assistant, multi-agent, and Responses API | agent-orchestration-responses | Shipped preview | Workspace-gated | Grounded native orchestration plus durable evidence |
| AI Runtime training and fine-tuning | ai-runtime-training | Shipped preview | Workspace-gated | Training, evaluation, and exact UC model assertions |
| Lakebase Autoscaling Online Feature Store | lakebase-autoscaling-features | Shipped preview | Workspace-gated | Online publication, Feature Serving, and freshness |
| Declarative Automation Bundles | declarative-bundles | Shipped | Awaiting live evidence | Candidate provenance, execution, and rollback |
| Predictive Optimization, liquid clustering, and Iceberg | lakehouse-optimization | Shipped | Awaiting live evidence | Three fail-closed table assertions |
| Marketplace and OpenSharing | marketplace-open-sharing | Shipped | Account-gated | Private provider/recipient and governed asset evidence |
“Shipped” means the typed pack, fail-closed doctor/run behavior, mocks, tests,
and documentation exist. It is intentionally not the same as “live-gated.”
Fabric Experiments must not describe a shipped workload as live-gated until
its required live checks produce retained evidence on the exact candidate.
Preview packs may ship with explicit workspace-gated status; they are not part
of the GA live-gated claim. Every advanced pack also requires the
restricted-principal check;
evidence produced with a developer token cannot satisfy the OAuth M2M scope.
Languages and custom workloads
The framework can launch and inspect Python, R, Scala, SQL, notebook, wheel, JAR, Spark-submit, dbt, and pipeline workloads through Databricks Jobs. The test author owns the workload-specific assertion; the target pack owns submission, terminal-state handling, evidence, and failure semantics. Language support does not mean Fabric reimplements each language runtime.
Agent responsibility boundary
Fabric Harness owns agent definitions, policies, tools, durable execution, and Databricks deployment integrations. Fabric Experiments owns enrollment, version admission, evaluation thresholds, certification, and Quality evidence.
Customer agents use defineAgent() and run in a customer-owned worker. The
official Experiments worker never downloads arbitrary customer modules. An
admin first pins an Ed25519 public key, then submits an immutable signed
manifest. The customer worker reconciles the requested external Agent Service
and grants through @fabric-harness/databricks; the Experiments API does not
contain a general Databricks runtime SDK.
See the Fabric Harness Databricks guide and Your First Agent for agent authoring.
Preview and cloud boundaries
- Managed MCP, Agent Services, Data Quality Monitoring, and Apps telemetry are Databricks preview or Beta surfaces. They require explicit configuration and fail closed when absent; Fabric does not silently downgrade them.
- Data Quality Monitoring enrollment is not a health pass. A release requires
a recent latest result with both freshness and completeness
Healthy; a newly enabled table inTrainingremains blocked until Databricks builds its baseline. - Agent Services currently certifies registration, discovery, metadata, and permissions. Databricks does not expose Agent Services runtime invocation in the Beta, so no runtime-invocation claim is made.
- Azure Databricks is the current named live environment. AWS and Google Cloud package compatibility is not a live certification claim until the same required matrix runs there.
- Private Link, customer-managed keys, unusual network egress, regional SKUs, and customer data/model quality remain customer-topology certifications.
Release checklist
Before publishing a candidate:
- run repository, boundary, type, lint, build, docs, OpenAPI, and package verification gates;
- run every required target-pack check against the named Azure workspace;
- run the API-to-outbox-to-Temporal-to-Model-Serving-to-SQL evaluation path;
- verify
fh,fx, and Fiber against candidate artifacts rather than global workstation installs; - publish evidence to Quality Center and retain the CI artifact; and
- publish only after every required row above is green.
See Databricks target packs for configuration, the Databricks enablement checklist for account and fixture prerequisites, and Certification evidence for the latest observed results.