What is Fluentd?

Fluentd is an open source data collector for building the unified logging layer. Once installed on a server, it runs in the background to collect, parse, transform, analyze and store various types of data.

What are the differences between td-agent and Fluentd?

Here's the differences. In one word, td-agent is a stable distribution package of Fluentd.

Fluentd td-agent
QA/Support Community-driven QAed by Treasure Data, Inc maintainers
Support available for Treasure Data customers
Installation Ruby gems or source code RPM / DEB / DMG / MSI packages
Configuration Self service Preconfigured with a couple of recommended settings, and sending data to Treasure Data by default (can be modified)
Adding 3rd party plugins $ fluent-gem install fluent-plugin-xx $ /usr/sbin/td-agent-gem install fluent-plugin-xx
/etc/init.d/ scripts No (the user needs to write shell script to set it up) Yes (shipped with .deb and .rpm)
Chef recipe No Yes
Memory allocator OS default Optimized (jemalloc)

Should I use td-agent or the Fluentd gem?

td-agent prioritizes stability over new features. If you wish to control Fluentd features and updates on your own, using the Fluentd gem is recommended. If you are using Fluentd for the first time or are using it in a large scale environment, using td-agent is recommended. A new version of td-agent is released every 2 or 3 months.

Fluentd is written in Ruby. How's the performance?

The most performance sensitive parts of Fluentd are written in C. The Ruby code acts as a wrapper that provides flexibility to the overall solution. In particular, the networking layer and object serialization layer are written in C (See cool.io and MessagePack, both are maintained by Fluentd maintainers too).

Since Fluentd is not written entirely in C or C++, it may be slow in its Ruby parts. But by giving up a little bit of speed, we have gained many plugins from the Ruby community :). Fluentd’s performance has been put to the test at many large services; in fact, a regular PC box can handle 18,000 messages/second with a single process.

If this number is insufficient for your application, please consider other solutions which are written entirely in C-family or Java. However, please note that you will lose some flexibility as a tradeoff.

What is Fluent Bit?

Fluent Bit is a Lightweight Data Forwarder for Fluentd. Fluent Bit is specifically designed for forwarding the data from the edge to Fluentd aggregators.

What are the differences between Fluentd and Fluent Bit?

Here's the differences. In one word, Fluent Bit is designed for small footprint while Fluentd is designed for flexibilities.

Fluentd Fluent Bit
Scope Servers Containers / Servers / Embedded Systems
Language C & Ruby C
Memory ~40MB ~450KB
Dependencies Built as a Ruby Gem, it requires a certain number of gems and Ruby interpreter Zero dependencies
Plugins More than 650 plugins available Around 30 plugins available
License Apache License v2.0 Apache License v2.0
For more details, please visit Fluent Bit documentation page.

Does Fluentd run on Windows?

Yes, fluentd supports Windows since v0.14. Check download page.

I wrote a new plugin. How to add this plugin to plugin page?

Our script updates a plugin page periodically and this script collects the information of fluent-plugin-xxx gems. If you want to add your gem on plugin page, release it as fluent-plugin-xxx, not fluentd-plugin-xxx, fluent-xxx-plugin and etc.

What does `Certified` mean?

The plugins marked as `Certified` are developed by either Fluentd core committers or companies who made the commercial commitment to Fluentd project.

Since we made it really easy to develop plugins, we have 500+ more plugins and started hearing from people saying it's hard to figure out which plugins are ready to use in production. That's why we introduced this system.

Of course a lot of `Not Certified` plugins are also used by wide veriety of people, but this mark will help you at least navigate through core plugins.


Want to learn the basics of Fluentd? Check out these pages.

Ask the Community

Couldn't find enough information? Let's ask the community!

Ask the Experts

You need commercial-grade support from Fluentd committers and experts?