Here's the differences. In one word, td-agent is a stable distribution package of Fluentd.
|QA/Support||Community-driven||QAed by Treasure Data, Inc maintainers
Support available for Treasure Data customers
|Installation||Ruby gems||rpm / deb / dmg packages.|
|Configuration||Self service||Preconfigured with a couple of recommended settings, and sending data to Treasure Data by default (can be modified)|
|Adding 3rd party plugins||$ fluent-gem install fluent-plugin-xx||$ /usr/sbin/td-agent-gem install fluent-plugin-xx|
|/etc/init.d/ scripts||No (the user needs to write shell script to set it up)||Yes (shipped with .deb and .rpm)|
|Memory allocator||OS default||Optimized (jemalloc)|
The most performance sensitive parts of Fluentd are written in C. The Ruby code acts as a wrapper that provides flexibility to the overall solution. In particular, the networking layer and object serialization layer are written in C (See cool.io and MessagePack, both are maintained by Fluentd maintainers too).
Since Fluentd is not written entirely in C or C++, it may be slow in its Ruby parts. But by giving up a little bit of speed, we have gained many plugins from the Ruby community :). Fluentd’s performance has been put to the test at many large services; in fact, a regular PC box can handle 18,000 messages/second with a single process.
If this number is insufficient for your application, please consider other solutions which are written entirely in C-family or Java. However, please note that you will lose some flexibility as a tradeoff.
Here's the differences. In one word, Fluent Bit is designed for small footprint while Fluentd is designed for flexibilities.
|Scope||Servers||Containers / Servers / Embedded Systems|
|Language||C & Ruby||C|
|Dependencies||Built as a Ruby Gem, it requires a certain number of gems and Ruby interpreter||Zero dependencies|
|Plugins||More than 650 plugins available||Around 30 plugins available|
|License||Apache License v2.0||Apache License v2.0|
The plugins marked as `Certified` are developed by either Fluentd core committers or companies who made the commercial commitment to Fluentd project.
Since we made it really easy to develop plugins, we have 500+ more plugins and started hearing from people saying it's hard to figure out which plugins are ready to use in production. That's why we introduced this system.
Of course a lot of `Not Certified` plugins are also used by wide veriety of people, but this mark will help you at least navigate through core plugins.