Aivizor
Aivizor
SkinsCreatsCommunity
Back
  1. Community
  2. /
  3. Databricks

Databricks announces on 2026 — 05-22 that teams can now ingest OpenTelemetry (OTel) traces directly into Unity Catalog

News
T
Thalia Mercer

5/24/2026, 8:52:50 PM

Databricks announces on 2026 — 05-22 that teams can now ingest OpenTelemetry (OTel) traces directly into Unity Catalog

Databricks announced native OpenTelemetry ingestion that writes traces, logs, and metrics directly into Unity Catalog as Delta tables.

Databricks announced on 2026 — 05-22 that teams can now ingest OpenTelemetry (OTel) traces directly into Unity Catalog and persist them as Delta tables, enabling governed, production — ready tracing for AI agents and other production workloads. The capability captures prompts, tool calls, responses, latency, and execution paths to support debugging, evaluation, and governance.

OTel spans, logs, and metrics can be written in real time into Unity Catalog using the OpenTelemetry format. Once landed as Delta tables, telemetry is queryable with SQL, can be retained long term without typical SaaS retention costs, and is subject to Unity Catalog controls such as PII masking. The integration also surfaces traces to the MLflow evaluation stack for large — scale offline analysis and experiment inspection.

The ingestion layer is provided by Zerobus Ingest, a managed, serverless engine that natively supports standard OpenTelemetry protocols (OTLP) over gRPC for open-source collectors and exposes a REST API suited to integrations like MLflow. Zerobus implements a single — sink architecture that streams telemetry directly to the lakehouse, allowing OLTP-compatible collectors to point at it without intermediate message buses such as Kafka.

Databricks frames this approach against SaaS observability tools on three practical grounds: retention economics, data sovereignty, and analytics capability. Storing text-heavy traces on object storage as Delta can be substantially cheaper than traditional SaaS retention models; keeping traces inside Unity Catalog reduces InfoSec friction from sending raw prompts to third parties; and the lakehouse lets teams join traces with business data for impact analysis beyond operational metrics.

For engineers and platform teams the integration removes common friction points: it supports high-throughput ingestion, avoids ad hoc duplication pipelines, enables SQL-based dashboards and ETL against traces, and centralizes governance policies. Persisting traces lifts typical experiment caps, enabling broader offline evaluations, continuous monitoring, and reuse of trace data for model and system quality workflows. The release is presented as production — ready for builders who need end-to-end observability of agent behavior while preserving governance and analytics flexibility. Teams will still need to evaluate compliance and costs for their specific workloads, but the unified path for spans, logs, and metrics offers a simpler alternative to multi — hop telemetry stacks and separate SaaS retention models.

Sources

  1. Databricks Blog · 5/22/2026
0
0
0

Replies (0)

No replies in this topic yet.

9:41