New Datadog Integration! What it Means for Kubernetes:

What the new StormForge / Datadog integration means for Kubernetes resource optimization, application performance and cost-efficiency

A few common and reasonable questions from the users of observability and monitoring solutions are:

What do I do with all of the observability data that I’m now collecting?
How do I make sense of this information?
What is an actionable step I can take?

Well, these questions haven’t always had crisp answers. After all, collecting data is only the first part of the challenge. Being able to analyze it and make meaningful changes to deployments is what can actually have an impact. The collaboration and integration announced this week between StormForge and Datadog is a game-changer because Datadog users can now easily derive intelligent insights from their data and apply them to cloud-native applications to achieve better performance and resource utilization. It’s the ability to turn observability into actionability with one integrated groundbreaking solution that can automatically optimize complex Kubernetes workloads.

Datadog is a leading observability platform and monitoring service for cloud-native applications. When combined with StormForge’s ML-powered performance and resource optimization solution, development and business teams are able to not only find out where problems reside or may potentially pop up, but now they’ll know what specific actions to take based on that observability data – or choose to have those actions occur automatically. The result is that teams running Kubernetes applications can optimize cloud resources, and achieve cost savings and application performance at any scale.

What’s being announced?

There are two new StormForge tiles within the Datadog platform:

A Datadog Integration tile. This is a set of dashboards that show insight into optimization opportunities for cloud-native applications, driven by Datadog data and StormForge Optimize Live analysis of observation-based data.
A Datadog Marketplace tile. This is a method of purchasing StormForge Optimize Live through Datadog’s Marketplace

What’s the gist of the integration?

Datadog customers can deploy the StormForge platform in under one hour and begin receiving recommendations for optimization within a day. The StormForge machine learning engine queries and then analyzes observability data and recommends resource settings (CPU, memory, replicas) for improved efficiency, scale and application performance. Customers can use a single Datadog dashboard to view actual resource usage and identify the application resource parameters that should be recalibrated to address BOTH over-provisioning and under-provisioning – and thereby avoid both wasted cloud resources and performance risk, respectively. Specifically by addressing over-provisioning, which is the bigger issue for most Kubernetes environments, customers can reduce their cluster size, and reclaim and reallocate resources for other applications or projects.

What’s the benefit for Datadog customers?

Datadog users can view the StormForge insights from within the Datadog interface to better understand and get more value from the observability data they’re collecting. They can also apply StormForge recommendations to improve performance and resource utilization. By applying the recommended optimized configurations for Kubernetes applications, Datadog customers can proactively – and automatically – ensure operational efficiency. They can make the best use of application resources and improve performance at scale without time-consuming, ineffective trial-and-error – and without requiring Kubernetes optimization expertise.

What’s the benefit for StormForge customers?

With the Datadog integration, StormForge users can get out-of-the-box dashboards that offer better insight into how their applications are performing and specific areas of opportunity for optimization. Dashboards will show them the most CPU and memory intensive deployments, usage per deployment over time, and most important, the recommended requests and limits that Optimize Live has generated using ML-based analysis. Customers can slice and dice the data as needed to make smarter decisions about their applications – right from the Datadog platform.

The following examples illustrate the value of the integration in a couple of scenarios. We can see how StormForge insights and recommendations, when applied to Datadog data, can help address potential performance risks associated with under-provisioning resources as well as inefficiencies due to over-provisioning resources.

The example in Figure 1 shows that CPU resources are currently being under-provisioned, so apps need more CPU than what is currently being allocated. We know this because the actual CPU usage is above the Current Requests line, which can put the deployment at risk of performance issues due to resource contention. StormForge Optimize Live is recommending that the requested CPU be increased, as shown by the yellow Optimized Requests line. Similarly, Optimize Live is also recommending that the CPU limit be increased, as shown by the purple Optimized Limits line. These optimized recommendations will ensure that the application(s) get the CPU resources they need to meet performance requirements, and at the same time, not waste CPU resources by over-provisioning them.

Figure 2 reflects the opposite scenario. The Y-axis scale is a bit different, but the chart shows similar values for both Current CPU Requests and Current CPU Limits as the chart in Figure 1. Notice, however, that there’s no CPU activity between those two lines. In fact you can see that our Usage line is way down at the bottom of the chart, below the Current Request line. In this example, our requested CPU amount is much more than what we need, so we are over-provisioning CPU resources – which represents significant waste. Based on analysis of actual usage, StormForge Optimize Live is recommending much lower Requests and Limits, again represented by the yellow and purple lines, respectively. By following Optimize Live recommendations, we can deploy CPU resources that are in line with our actual usage and optimized for our needs. And this frees up these otherwise wasted resources for other applications.