OpenTelemetry is essential to manage modern applications…but it’s not magical for IT teams to manage hybrid environments

OpenTelemetry is an open source, vendor-agnostic and portable standard for metrics, and traces with the plans to support logs in the future. It provides technologists with visibility into modern application stacks, something that their traditional monitoring tools are simply unable to deliver.

This is because monitoring performance is far more challenging in a software-defined, cloud environment, where everything is constantly changing in real-time. Traditional approaches to monitoring are based on physical infrastructure – IT departments operate a fixed number of servers and network wires – they are dealing with constants. This then provides fixed dashboards for each layer of the IT stack. But with cloud computing, organizations are continually scaling up and down their use of IT, based on real-time business needs.

Indeed, the transition to modern applications brings with it huge complexity and massive volumes of data. If you look at it in the most simple way, if you have a single app that is running on premise, most likely a Java app, and you want to bring that app over into a cloud native construct, you need to break the app up into several separate entities. That app will then run as a cloud native, microservice-driven application. And so therefore, you’ve suddenly gone from instrumenting one application to potentially  10, 12 or 20 or more different microservices.

Most traditional, agent-based monitoring solutions simply aren’t able to handle the sprawling, dynamic and highly volatile nature of cloud native environments. And this is where OpenTelemetry comes in.

Using OpenTelemetry, with just a few lines of code during the development stage, businesses can ensure they’re able to get real-time data on application performance once in production. OpenTelemetry offers open standard tools for auto instrumentation of applications built in microservices and Kubernetes environments and easy ways to plug into the CI/CD pipeline.

While adoption of OpenTelemetry remains relatively low, certainly at an enterprise level, from recent conversations I’ve had with IT leaders, most organizations will be deploying it within their IT department within the next couple of years.

OpenTelemetry must be part of a wider observability strategy

For all the attention and focus that OpenTelemetry is getting however, technologists need to recognize its limitations. On its own, OpenTelemetry doesn’t provide IT teams with the insights they need.

OpenTelemetry can collect individual, isolated pieces of data and stitch them together, but it doesn’t deliver holistic insight into performance. Put simply, it’s only concerned with generating data; it doesn’t make sense of the vast volumes of data it creates or visualize the full application picture.

So technologists need a way to consume, process and correlate this mass of telemetry data. They need additional tools to enhance the data they get from OpenTelemetry and combine it with other data points that they derive through agents and data coming in from public cloud environments, such as Kubernetes.

Indeed, it’s also worth recognizing that OpenTelemetry doesn’t provide the same level of visibility as a proprietary agent within on premise environments. It currently only provides traces and metrics so IT teams are still faced with significant visibility gaps.

Critically, OpenTelemetry does not understand business transactions so IT teams are unable to identify and prioritize those availability and performance issues that could do the most damage to end user experience and business outcomes.

Most organizations are still deploying separate tools to monitor cloud native and legacy applications, and this means they have no clear line of sight of the entire application path, where application components are running across on-premise and cloud native environments. This makes it challenging for IT teams to troubleshoot issues quickly so MTTR and MTTX inevitably increase.

This is why it’s essential for IT teams to be able to ingest OpenTelemetry directly into their observability platform, so that they get a unified view on applications running across multi-cloud and hybrid environments. Observability presents the only way for technologists to cut through data noise, make informed decisions and prioritize actions based on potential severity.

IT teams need to adopt a hybrid strategy where OpenTelemetry can be ingested into their observability platform along with the overarching mix of already instrumented applications through traditional agent-based monitoring systems.

Not only that, but organizations need to ensure they have a platform which enables IT teams to correlate IT data with real-time business metrics across both cloud native and on premise environments. Having business context throughout the overarching application flow allows technologists to focus on the things that really matter, and to measure and report on the impact that they are delivering to the business.

As organizations look to accelerate their migration to cloud native technologies, OpenTelemetry will become the foundation for IT teams to manage and optimize application availability and performance. But as a standalone tool, OpenTelemetry isn’t enough. It must be part of a wider, unified observability strategy.