Datadog’s new AI Impact ties AI coding tool telemetry to DORA metrics so engineering teams can measure whether AI assistance changes how fast they ship and how often changes fail, enabling comparisons across tools and models for data‑driven decisions.
Datadog has launched Datadog AI Impact, a feature that links telemetry from AI coding tools directly to established software delivery metrics so teams can measure whether AI assistance actually changes delivery outcomes. By associating each commit with the specific tool and model that contributed to it and carrying those attributes through CI/CD, the product lets engineering leaders move beyond usage statistics and judge tools by their effect on shipping speed and stability. This matters because teams can compare tools and models using the same DORA-based measures they already trust and make more informed rollout or rollback decisions.
The new offering addresses a common gap: organizations have rapidly adopted AI coding assistants, but most monitoring surfaces adoption and activity — daily active users, acceptance rates, or lines of generated code-rather than whether those tools improve delivery. Those usage metrics show whether developers use a feature, not whether it speeds up lead time or reduces incidents. Datadog AI Impact preserves concrete delivery measures, making it possible to connect AI assistance to the outcomes engineering teams care about.
On the technical side, Datadog aggregates AI-tool telemetry with standard engineering telemetry so each commit retains metadata identifying the tool and model that influenced it. Those AI attributes persist alongside commits as they travel through pull requests and into deployments, removing the need for manual tagging or custom instrumentation. Because the linkage is maintained through CI/CD, teams can segment and compare delivery performance for AI-assisted versus unassisted changes at scale.
The product surfaces standard DORA metrics such as lead time and change failure rate while also exposing PR-level signals like pull request cycle time and review time. That combination lets teams examine both throughput and trade — offs: for example, whether faster development correlates with higher throughput (measured as pull requests deployed per developer per day), or whether AI-assisted work produces larger changes and longer reviews that could affect stability. Teams can use the same metrics to compare different tools or models and to detect whether a model change alters delivery behavior.
The Datadog blog post outlining AI Impact was authored by Eric Metaj (Product Marketing Manager) and Teddy Gesbert (Product Manager). It presents concrete steps teams can follow to measure shipping speed, evaluate the production stability of AI-assisted code, compare coding tools by delivery outcomes, and run controlled tests of new models before broader rollouts.
Sources
Replies (0)
No replies in this topic yet.
