What’s New in OpenTelemetry? | Logz.io

OpenTelemetry (OTEL) is an observability platform designed to generate and collect telemetry data across various observability pillars, and its popularity has grown as organizations look to take advantage of it.

It’s the most active Cloud Native Computing Foundation project after Kubernetes, and it’s progressing at an immense pace on many fronts. The core project is expanding beyond the “three pillars” into new signals, such as continuous profiling. It’s moving past backend into client-side telemetry and real user monitoring, as well as work on eBPF more. 

Keeping track of all these updates can be challenging.

In our recent webinar “What’s New in OpenTelemetry?” I gave attendees a comprehensive overview of OpenTelemetry, its significance, and recent developments. 

Here are the key takeaways from the webinar and why OTEL is becoming increasingly essential for organizations.

Observability and the Vision of OpenTelemetry

First, let’s emphasize the importance of observability, which entails understanding a system’s state based on telemetry data it emits. The ultimate vision of OTEL is to provide unified observability across diverse signal types, including logs, metrics, and traces, from various sources such as front-end and back-end code, open-source tools, and cloud services. 

Achieving this unified observability is crucial for detecting and diagnosing issues effectively.

However, in reality, observability is often fragmented due to the multitude of tools and vendors used in the field. Each tool and vendor typically comes with its proprietary APIs, SDKs, agents, collectors, and protocols, leading to operational complexity and vendor lock-in. 

This fragmentation impedes the correlation of data across silos, which is a fundamental goal of observability.

OpenTelemetry’s Role in Solving Observability Challenges

OpenTelemetry steps in as a solution to these challenges. It’s an observability platform that unifies telemetry data collection across logs, metrics, and traces. By offering a single framework and standardized specifications, OTEL provides a holistic approach to observability. 

Notably, it emerged from the merger of OpenTracing and OpenCensus and is an incubating project under the CNCF.

All major cloud providers and monitoring tools have embraced OpenTelemetry, contributing to its growth and widespread use. Analyst firms such as Gartner also recognize OTEL’s potential, placing it on their hype cycle for emerging technologies and predicting a relatively short path to the plateau of productivity.

Understanding OpenTelemetry Components

Let’s delve into the core components of OTEL:

• OpenTelemetry Specification: This specification governs all OpenTelemetry components, providing a unified data model and semantic conventions for traces, metrics, and logs.

• Client Libraries: OpenTelemetry offers client libraries for various programming languages, each based on the unified specification. These libraries support both manual and automatic instrumentation, making it versatile for different use cases.

• OpenTelemetry Collector: The Collector serves as a crucial bridge between your applications and backend analytics tools. It can collect data from OpenTelemetry SDKs, other sources like databases, cloud services, and export the telemetry data to multiple backends

• OpenTelemetry Protocol (OTLP): OTLP is a general-purpose telemetry data delivery protocol used for communication between SDKs, Collectors, and backend analytics tools. It supports multiple transport methods and encoding formats, making it versatile for various environments.

Status of OpenTelemetry Components and New Developments

Here’s an update on the status of OpenTelemetry components, categorized by telemetry type: 

Traces are the most mature signal, with stable APIs, SDKs, and protocol specifications. Metrics are in the release candidate stage, while logging is undergoing integration with existing logging systems and the development of a new logging format.

Additional exciting developments within OpenTelemetry include:

• Elastic Common Schema Integration: Elastic Common Schema (ECS) is merging into OpenTelemetry’s semantic conventions for logging. This integration brings valuable knowledge from the Elastic community to enhance OpenTelemetry’s logging capabilities.

• Support for New Telemetry Signals: OpenTelemetry is expanding its support to include continuous profiling and real user monitoring (client-side telemetry). These efforts aim to provide a standardized approach to capturing and analyzing these telemetry signals.

• eBPF for Auto-Instrumentation: OpenTelemetry is exploring the use of eBPF (Extended Berkeley Packet Filter) for auto-instrumentation. The Go programming language already has an eBPF auto-instrumentation agent, with plans to support other languages.

• Remote Agent Management: The project is focusing on improving the operational aspects of OpenTelemetry, particularly in remote agent management. This includes using the Open Agent Management Protocol (OpAMP) to configure and update remote collectors and agents.

• Sunset of OpenCensus: OpenTelemetry has achieved feature parity with OpenCensus, signaling the official transition path for users of OpenCensus or OpenTracing.

• OpenTelemetry Demo in GA: The OpenTelemetry community has launched the OpenTelemetry demo in general availability. This demo enables users to experiment with OpenTelemetry easily by deploying a fully instrumented microservices application.

You can read more on these updates in my OTEL roadmap post and KubeCon ‘23 updates post. 

Getting Started with OpenTelemetry

To help organizations navigate the complexity of OTEL’s components, there is a practical guide to getting started. Organizations should start by understanding their stack, considering which programming languages, frameworks, signal types, and analytics tools they use. 

Mapping these to OTEL’s components will help organizations determine the relevant libraries, receivers, and exporters to use. I also recommended exploring the OTEL website, official documentation, and GitHub repository for detailed information. 

Additionally, I encourage you to check out our comprehensive guide on OTEL available on the Logz.io website, providing step-by-step instructions for various programming languages and resources to help users get started.

Discover more about how Logz.io can help with your OTEL environment by signing up for a free trial of the Logz.io Open 360™ platform today.