OpenTelemetry Framework

An OpenTelemetry Framework is an open-source observability framework (provides a unified set of APIs, libraries, agents, and instrumentation to enable the collection and transmission of telemetry data from various applications and services).

• Context:
  • It can (typically) include Components, such as:
    • OpenTelemetry Collector, which is used to receive, process, and export telemetry data [1].
    • OpenTelemetry APIs and SDKs, which provide the necessary tools to instrument applications in different programming languages.
    • OpenTelemetry Protocol (OTLP), which ensures efficient and vendor-neutral data transmission between telemetry sources and backends.
    • OpenTelemetry Semantic Conventions, which standardize the naming and structure of telemetry data for consistency and interoperability.
  • It can (often) be integrated with various backends like Prometheus, Jaeger, and other observability platforms.
  • ...
• Example(s):
  • OpenTelemetry Collector vv0.105.0 (~2024-07-14) [2]
  • ...
• Counter-Example(s):
  • New Relic, which is a commercial observability solution with its proprietary agents and collectors.
  • Dynatrace, another commercial solution providing end-to-end observability but not based on the open-source model of OpenTelemetry.
• See: PostHog, Prometheus, Jaeger

---

---

References

• https://opentelemetry.io/

2024

• https://github.com/open-telemetry/opentelemetry-collector
  • The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process and export telemetry data. In addition, it removes the need to run, operate and maintain multiple agents/collectors in order to support open-source telemetry data formats (e.g. Jaeger, Prometheus, etc.) to multiple open-source or commercial back-ends.
  • Objectives:
    • Usable: Reasonable default configuration, supports popular protocols, runs and collects out of the box.
    • Performant: Highly stable and performant under varying loads and configurations.
    • Observable: An exemplar of an observable service.
    • Extensible: Customizable without touching the core code.
    • Unified: Single codebase, deployable as an agent or collector with support for traces, metrics and logs.

2022

• (Majors et al., 2022) ⇒ C. Majors, L. Fong-Jones, and G. Miranda. (2022). “Observability Engineering.” O'Reilly Media. ISBN:9781492076414
  • QUOTE:

   A Brief Introduction to Instrumentation
   Open Instrumentation Standards
   Instrumentation Using Code-Based Examples
       Start with Automatic Instrumentation
       Add Custom Instrumentation
       Send Instrumentation Data to a Backend System

2022

• https://www.containiq.com/post/auto-instrumentation-and-opentelemetry

2021

• https://www.splunk.com/en_us/data-insider/what-is-opentelemetry.html
  • QUOTE: ... OpenTelemetry is an open source observability framework. It offers vendor-agnostic or vendor-neutral APIs, software development kits (SDKs) and other tools for collecting telemetry data from cloud-native applications and their supporting infrastructure to understand their performance and health.

    Managing performance in today’s complex, distributed environment is extremely difficult. Telemetry data is critical for helping DevOps and IT groups understand these systems’ behavior and performance. To gain a complete picture of their services’ and applications’ behavior, they need to instrument all their frameworks and libraries across programming languages.

    However, no commercial vendor has a single instrument or tool to collect data from all of an organization’s applications. This lack results in data silos and other ambiguities that make troubleshooting and performance-issue resolution more difficult.

     OpenTelemetry is important because it standardizes the way telemetry data is collected and transmitted to backend platforms. It bridges visibility gaps by providing a common format of instrumentation across all services. Engineers don’t have to re-instrument code or install different proprietary agents every time a backend platform is changed. OpenTelemetry will continue to work, too, as new technologies emerge, unlike commercial solutions, which will require vendors to build new integrations to make their products interoperable.

    In the sections that follow, we’ll take a closer look at OpenTelemetry, how it works and how it helps organizations achieve the observability in their distributed systems needed to meet their business goals.

---