Why Google Cloud’s Otlp Support Matters For Your Telemetry Pipeline

Google Cloud’s engineers person enabled Google Cloud Observability, and specifically Cloud Trace, to support nan OpenTelemetry Protocol (OTLP) ingest for near-direct and much vendor-neutral ingestion of trace data.

Now that users tin nonstop their trace information utilizing OTLP via nan Telemetry (OTLP) API, Google Cloud recommends this attack successful galore cases, including for users who expect precocious volumes of trace information erstwhile paired pinch nan OpenTelemetry Collector. Google’s take and integration of nan protocol has progressive important engineering work, and metrics and logs person abstracted OTLP support paths and readiness that alteration by merchandise and region.

In nan meantime, Google’s attack is akin to those of Microsoft Azure and Amazon Web Services. Without requiring third-party devices aliases vendors, Azure and AWS support OTLP arsenic well, covering Azure Monitor OTel support, AWS Distro for OpenTelemetry and X-Ray OTLP ingest.

Key Benefits of Native OTLP Integration

Defined arsenic a information speech protocol designed to carrier telemetry from a root to a destination successful a vendor-agnostic fashion, OTLP support for Google Cloud Observability reduces nan anterior request for developers to merge vendor-specific exporters successful galore cases, according to Google documentation. This helps guarantee that nan halfway telemetry pipeline remains standardized and readily interoperable crossed OTLP-supporting observability backends erstwhile appropriately configured.

The halfway narration is defined by Google arsenic eliminating a basal conversion step, shifting complexity from nan customer broadside (the Collector) to nan unreality backend, according to Google.

A Google Cloud blog post published successful September, authored by institution merchandise managers Sujay Solomon and Keith Chen, noted nan benefits of nan OTLP integration to nonstop tracing data. Those benefits include:

• Vendor-neutral telemetry pipelines: The usage of autochthonal OTLP exporters from in-process aliases collectors eliminates nan request to usage vendor-specific exporters successful your telemetry pipelines, according to nan Google post.
• Telemetry information integrity: Assurances that telemetry information maintains nan OpenTelemetry information exemplary during transmission and retention without transformations into proprietary formats.
• Interoperability pinch your prime of observability tooling: The expertise to nonstop telemetry to 1 aliases much observability backends that support autochthonal OTLP without further OTel exporters.
• Reduced client-side complexity and assets usage: The expertise to displacement telemetry processing logic, specified arsenic applying filters to nan observability backend, reducing nan request for civilization rules and frankincense client-side processing overhead pinch Google Cloud Observability.

A New Role for nan OTel Collector successful Google Cloud Observability

The caller summation of autochthonal OTLP ingestion successful Google Cloud Observability is directly, critically related to nan OpenTelemetry Collector, arsenic it fundamentally changes nan Collector’s superior domiciled and configuration for Google Cloud Observability. The evident use described supra is nan removal of nan conversion step, truthful location is comparatively minimal configuration progressive for nan Collector to ingest tracing information from Google Cloud Observability backend.

OpenTelemetry Collector is utilized to service arsenic a select to show telemetrics. It is applicable for whenever aggregate applications aliases microservices are involved, peculiarly for information considerations. As such, an OpenTelemetry Collector falls nether nan class of an observability agent. Prior to this announcement, users mightiness usage observability agents, specified arsenic Fluent Bit, Vector and others to ingest tracing information pinch Google Cloud Observability.

The Role of Observability Agents and Data Collection

Observability agents play a captious domiciled successful nan nuts and bolts workings of observability. They grip information carrier to guarantee telemetry information is transmitted accurately. Agents typically connection information collection, information processing and information transport, playing a captious domiciled successful monitoring strategy performance. They thief successful identifying chartless unknowns to troubleshoot and mitigate capacity issues earlier they go problems. That’s nan aureate modular of observability functionality.

In this way, an observability agent, erstwhile utilized for information collection, collects information sent to it from 1 aliases galore sources. In summation to receiving data, it sends information to an endpoint, specified arsenic for visualization pinch a Grafana panel. An supplier tin beryllium configured to cod definite types of logs, traces and metrics for observability.

Initially, you tin opt not to usage an observability agent, if you’re already deploying an exertion that is instrumented to nonstop telemetry information straight to nan observability platform. The collectors tin beryllium useful erstwhile monitoring an exertion that can’t beryllium instrumented.

Without observability collector functionality, configuring each backend aliases personification monitoring separately for those is required, which tin beryllium cumbersome. On nan contrary, an observability collector serves arsenic a azygous endpoint for each microservices, streamlining entree to applications and microservices done a unified constituent facilitated by nan collector. Utilizing an observability supplier to service arsenic a collector, you tin position and negociate microservices collectively, offering a consolidated position connected a level for illustration Grafana. While Grafana provides definite alternatives without an OpenTelemetry collector, nan collector importantly simplifies this process.

The Status of OTLP Support for Logs and Metrics

It should beryllium noted that Google Cloud’s announcement is astir OpenTelemetry for tracing. Logs and metrics, and particularly logs, stay a activity successful advancement owed to a number of imaginable hurdles, including implementation of role-based entree controls (RBAC), compliance concerns and nan engineering activity required.

At this time, Azure and Google Cloud, among nan apical 3 unreality vendors and hyperscalers, straight support metrics ingest done OpenTelemetry complete nan OTLP protocol. AWS commonly uses nan AWS Distro for OpenTelemetry (ADOT) Collector. AWS and Google Cloud do not person broadly disposable nonstop OTLP logs ingest, while Azure’s logs ingest done OTLP remains mostly limited.

Google’s Strategy for a Simplified Telemetry Pipeline

Again, this is portion of ongoing improvement successful nan title to merge and amended observability information ingest among nan hyperscalers, arsenic Google’s Solomon and Chen wrote successful their institution blog post.

Google Cloud Observability’s caller move, they wrote, “is a cornerstone of our strategy to simplify telemetry guidance and foster an unfastened unreality environment. We understand that successful today’s analyzable unreality environments, managing telemetry information crossed disparate systems, inconsistent information formats, and immense volumes of accusation tin lead to observability gaps and accrued operational overhead.

“We are dedicated to streamlining your telemetry pipeline, starting pinch focusing connected autochthonal OTLP ingestion for each telemetry types truthful you tin seamlessly nonstop your information to Google Cloud Observability. This will thief foster existent vendor neutrality and interoperability, eliminating nan request for analyzable conversions aliases vendor-specific agents.”

Group Created pinch Sketch.