What To Know Before Building Fluent Bit Plugins With Go

Editor’s note: The pursuing article is an excerpt from nan Manning book: “Fluent Bit pinch Kubernetes,” a applicable guideline connected optimizing systems for Kubernetes — from basal configuration to precocious integrations for log, metric and trace routing and processing. This article excerpt focuses connected Go-based Fluent Bit plugins. Explore moving pinch SQL-like expressions complete signals and deciding erstwhile and really to build civilization plugins by downloading nan book successful its entirety here.

Go is nan astir ascendant connection for Cloud Native Computing Foundation (CNCF) projects, bringing nan benefits of autochthonal binary capacity while retaining features of languages specified arsenic Java successful nan shape of representation management, abstractions and easiness of development. It helps that nan originators of Kubernetes (Google) are nan driving unit down nan development of Go.

The accent connected autochthonal binary capacity benefits for unreality solutions, peculiarly astatine hyperscale, makes easy nonstop integration betwixt C/C++ and Go important, arsenic nan Linux kernel is written pinch C/C++, and it is 2nd aft Go successful position of autochthonal binary languages utilized by CNCF projects.

The Go plugins interact pinch nan Fluent Bit model done nan Goproxy module successful Fluent Bit. At this writing, nan Goproxy and nan associated Go binding codification expose only input and output plugins, not nan select method interfaces.

Organizational and Technical Benefits of Building successful Go

Using Go and configuring nan summation of Go plugins into Fluent Bit provides a scope of organizational and method benefits:

• The unfastened root task does not dictate merchandise cycles and improvement processes.
• Because nan consequence is delivered arsenic a autochthonal binary, nan plugin tin beryllium offered without exposing proprietary codification aliases intelligence property.
• Specialist usage cases are perfect for niche usage cases that bespeak circumstantial organizational needs aliases niche requirements. These kinds of usage cases often person a level of intelligence spot authorities (even if it is indirect erstwhile it comes to support functionality).

For example, successful nan lawsuit of a plugin that handles errors from a video transcoder, nan number of organizations utilizing video transcoders is comparatively small, and nan number utilizing a circumstantial transcoder pinch a civilization intends to make logs and events is moreover smaller.

• Plugins tin beryllium implemented to meet circumstantial soul needs without giving owed information to nan community. If our statement has a circumstantial modular for naming conventions, we tin hardwire it into nan plugin without interest astir whether it meets wider organization needs.

The method benefits of utilizing Go to build nan plugin are:

• Go retains nan capacity of autochthonal binary executables without losing nan benefits of representation guidance being performed by a connection runtime.
• The Go language provides binding to C applications arsenic standard, and Fluent Bit includes a room that further helps pinch interfacing.
• Development approaches, peculiarly for closed root aliases backstage solution implementation, tin beryllium aligned to soul principles and practices without considering nan wider community.

Operational Drawbacks of Go-Based Fluent Bit Plugins

The drawbacks are much operational than circumstantial to codification improvement itself. These challenges see nan following:

• At this writing, nan select interfaces are not available.
• If nan improvement team(s) don’t activity pinch Go, overhead remains successful implementing and maintaining processes specified arsenic continuous integration. We request to see regression testing for awesome and insignificant revisions.
• If we intend to make nan plugin unfastened source, it is not arsenic apt to get nan aforesaid level of attraction arsenic nan halfway Fluent Bit repositories. As a result, much of nan attraction load will beryllium connected nan plugin provider.
• Users person further deployment effort erstwhile deploying our plugin successful their existing environments, which could beryllium problematic if customers usage prepackaged platforms for illustration OpenShift.
• The scope of paid support services tin beryllium ambiguous, arsenic third-party support offerings typically don’t screen third-party plugins.
• The plugin needs to beryllium taxable to ongoing periodic updates, moreover if nan solution is unchangeable and mature, to forestall nan imaginable cognition that nan plugin is stale.
• Additional improvement effort is needed to construe betwixt C- and Go-native information types. We will person to usage different Go connection libraries and frameworks.
• We whitethorn person to rebuild Fluent Bit pinch Go support enabled (cmake -DFLB_ DEBUG=On -DFLB_PROXY_GO=On) depending connected nan build of Fluent Bit we use. Note that nan images provided by nan Fluent Bit task person this build emblem enabled by default. NOTE: Resources for Go-based improvement are disposable astatine https://github.com/fluent/fluent-bit-go, including illustration implementations and a Go inferior room to easiness nan workaround for nan Go-C interface.

If you’d for illustration to spell deeper into really Fluent Bit processors tin reshape metrics, traces and logs, including SQL-like expressions complete signals, erstwhile to scope for a civilization plugin and really to take betwixt nan different implementation technologies, download nan afloat book “Fluent Bit pinch Kubernetes.” In nan remainder of this chapter, you’ll get a practical, end-to-end knowing of nan plug-in scenery truthful you tin confidently design, build and run civilization extensions successful your ain environments.

To study much astir Fluent Bit, read:

• What Is Fluent Bit?
• What Are nan Differences Between OTel, Fluent Bit and Fluentd?
• What’s Driving Fluent Bit Adoption?

Group Created pinch Sketch.