Fluentd Blog

td-agent v4.5.0 has been released

2023-05-08

Hi users!</p>

We have released td-agent v4.5.0</a>. td-agent is a stable distribution package of Fluentd.</p>

Changes from td-agent v4.4.2</h3>

Update bundled Ruby to 2.7.8 which is the last version of Ruby 2.7 series.

Only for RHEL 9 & Ubuntu 22.04 jammy, we updat Ruby to 3.1.4 exceptionally.</li> </ul></li>
Update fluentd to 1.16.1</a></li>
Update plugins

fluent-plugin-kafka 0.19.0</li>
fluent-plugin-elasticsearch 5.3.0</li>
fluent-plugin-opensearch 1.1.0</li> </ul></li>
Drop Debian buster support</li> </ul>
About next major version</h3>
We are preparing next major version of td-agent which bundles Ruby 3.2. It's still work in progerss, we'll announce it as soon as we are ready for it.</p>
Please note that the package name will be changed to `fluent-package</code>. Please see the following discussion for more detail:https://github.com/fluent/fluentd/discussions/3860</a></p>`
`Download</h3> Please see the download page</a>.</p>`



Fluentd v1.16.1 has been released
2023-04-20
Hi users!</p>

We have released v1.16.1 on 2023-04-17. ChangeLog is here</a>.</p>

In the previous version v1.16.0, there was a bug that caused Fluentd to fail to start with certain secondary configurations.
In this release, we fixed that problem and added a new feature of in_tcp</code>.
And other some improvements!</p>

Enhancement</h2>


in_tcp</code>: Add message_length_limit</code> to drop large incoming data. #4137</a></li>
</ul>

Bug Fixes</h2>

out_secondary_file</code>: Fix NameError of SecondaryFileOutput</code> when setting other secondaries</h3>

In the previous version v1.16.0, there was a bug that if you set another secondary without ever setting out_secondary_file</code>, Fluentd fails to start with the following error.</p>

/path/to/fluentd/lib/fluent/plugin/output.rb:429:in `configure': uninitialized constant Fluent::Plugin::Output::SecondaryFileOutput (NameError)

          if (@secondary.class != SecondaryFileOutput) &&
                                  ^^^^^^^^^^^^^^^^^^^
</code></pre>

In this release, we fixed this problem.</p>

See #4124</a> for more information.</p>

Buffer</code>: Fix that compress</code> setting causes unexpected error when receiving already compressed MessagePack</h3>

With this release, it is now possible that Fluentd forwards already compressed MessagePack data to another Fluentd as is (without decompressing).</p>

A possible use case is two-stage forwarding.</p>

Forwarder1(out_forward</code>) -> Forwarder2(in_forward</code>, out_forward</code>) -> Aggregator(in_forward</code>)</p>

In this case, Forwarder2 should process the data of CompressedMessagePackEventStream</code> as is (without decompressing) and re-forward the data to Aggregator.</p>

Until this version, a bug prevented us from setting Forwarder2's compress gzip</code> of out_forward</a> or buffer</a>.
It resulted in the following error.</p>

[error]: #0 unexpected error on reading data host="..." port=... error_class=ArgumentError error="unknown keyword: :packer"
</code></pre>

So we had no choice but to decompress and forward it from Forwarder2 to Aggregator.</p>

With this release, we can now set compress gzip</code> for Forwarder2!</p>

See #4146</a> and #4147</a> for more information.</p>

Others</h3>


Server helper: Suppress error of UDPServer</code> over max_bytes</code> on Windows. #4131</a></li>
</ul>

Enjoy logging!</p>


Fluentd v1.16.0 has been released
2023-03-29
Hi users!</p>

We have released v1.16.0 on 2023-03-29. ChangeLog is here</a>.</p>

This release is a new release of v1.16 series.
In this release, we enhanced the feature for chunk file corruption and fixed some bugs, mainly about logging and race condition errors.</p>

Enhancement</h2>

Backup corrupted chunk files at resuming</h3>

Some chunk files in buffer_file</a> or buffer_file_single</a> are possibly corrupted after Fluentd stopped abnormally, such as due to a power failure.</p>

Fluentd has a feature to backup unrecoverable chunks as before.
However, when Fluentd resumes, remaining corrupted chunk files were just deleted.</p>

Since this release, Fluentd also backups those files at resuming.
In addition, Fluentd outputs some useful logs about possibly corrupted chunk files due to an abnormal termination.</p>

Please see the document</a> and the following PRs for more information.</p>


buffer: backup broken file chunk</a></li>
Add warning messages for restoring buffer with flush_at_shutdown true</code></a></li>
Add logs for time period of restored buffer possibly broken</a></li>
</ul>

in_tcp</code>: Add send_keepalive_packet</code> option</h3>

Since this release, you can use send_keepalive_packet option</a> for in_tcp</code>, as well as in_forward</a> or in_syslog</a>.</p>

Please see the document</a> and #3961</a> for more information.</p>

Bug Fixes</h2>


out_secondary_file</code>: Fix race condition. #4081</a></li>
out_secondary_file</code>: Suppress warning using different secondary for outsecondary</em>file. #4087</a></li>
out_forward</code>: Fix error of ack handling conflict with require_ack_response</code>. #4030</a></li>
Windows: Fix bug that Fluentd sometimes tries to use an unavailable port and fails to start. #4092</a></li>
Windows: Fix bug that flushing or graceful reloading cause reopening wrong log file. #4054</a></li>
Fix not to log some initial messages without format</code> setting applied. #4091</a></li>
Fix bug that some system</code> configs are not reflected. #4064</a>, #4065</a>


suppress_repeated_stacktrace</a></li>
ignore_same_log_interval</a> (It was not reflected only to the supervisor process)</li>
</ul></li>
Non-Windows: Fix not to generate sigdump file after receiving SIGTERM</code>. #4043</a></li>
Fix bug that system_config.workers</code> value can be wrongly 1</code> at config check at startup. #4066</a></li>
Fix bug that a variable whose type is not Fluent::Config::Element</code> can be passed to Fluent::Plugin::Base::configure()</code>. #4066</a></li>
</ul>

Enjoy logging!</p>


Fluentd v1.15.3 has been released
2022-11-11
Hi users!</p>

We have released v1.15.3 on 2022-11-02. ChangeLog is here</a>.</p>

In this release, we fixed several bugs.</p>

Bug Fixes</h2>

out_file</code>: Fix the multi-worker check with <worker N-M></code> directive</h3>

In the v1.15.1, we added a new feature to prevent write conflicts between multiple workers.</p>


#3808</a></li>
</ul>

However, this fix didn't work correctly with <worker N-M></code> directive.</p>


<worker N-M></code> directive</a></li>
</ul>

With this setting, the configuration check before starting the supervisor mistakenly determines
that the setting of out_file</code> is invalid and causes Fluentd to terminate.</p>

In this release, we fixed this problem.</p>

See #3942</a> for more information.</p>

Fix log rotation on Windows</h3>

We can configure log rotation setting as follows.</p>


Log Rotation Setting</a></li>
</ul>

On Windows, the log files must be separated by each process.</p>


NOTE: When --log-rotate-age is specified on Windows, log files are separated into
log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation.
Windows does not permit delete and rename files simultaneously owned by another process.</p>
</blockquote>

However, that log rotation setting in the system-directive didn't satisfy this and
Fluentd outputted the logs of the supervisor process into the log file of the worker-0 process.</p>

This could cause errors during log rotation.</p>

In this release, we fixed this problem.</p>

See #3939</a> for more information.</p>

Support glob for !include</code> directive in YAML config format</h3>

In the v1.15.0, we supported YAML config format.</p>


Config File Syntax (YAML)</a></li>
#3712</a></li>
</ul>

However, handling glob of !include</code> directive was leaked.</p>

In this release, we supported it.</p>

See #3917</a> for more information.</p>

Remove meaningless and unsecure oj</code> options</h3>

In this release, we removed the following oj</code> options.</p>


FLUENT_OJ_OPTION_MODE=object</code></li>
FLUENT_OJ_OPTION_MAX_NESTING</code></li>
</ul>

FLUENT_OJ_OPTION_MODE=object</code> was unsecure, so we removed it.</p>


GHSA-fppq-mj76-fpj2</a></li>
</ul>

This option is not used unless a user explicitly sets the environmental variable,
so almost all users do not need to care about this vulnerability.</p>

FLUENT_OJ_OPTION_MAX_NESTING</code> was not supported, so we removed it.</p>


https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031</a></li>
</ul>

See #3929</a> for more information.</p>

Enjoy logging!</p>


td-agent v4.4.2 has been released
2022-11-08
Hi users!</p>

We have released td-agent v4.4.2</a>. td-agent is a stable distribution package of Fluentd.</p>

Changes from td-agent v4.4.1</h3>


Update fluentd to 1.15.3</a>


Fix log initializer to correctly create per-process files on Windows</li>
out_file: Fix the multi-worker check with <worker 0-N></code> directive</li>
Remove meaningless oj options</li>
Support glob for !include</code> directive in YAML config format</li>
</ul></li>
macOS: Support Apple Silicon</li>
Update plugins


fluent-plugin-elasticsearch 5.2.4</li>
fluent-plugin-opensearch 1.0.8</li>
fluent-plugin-s3 1.7.2</li>
fluent-plugin-windows-eventlog 0.8.2</li>
</ul></li>
</ul>

Download</h3>

Please see the download page</a>.</p>

Fluentd Blog

td-agent v4.5.0 has been released

Changes from td-agent v4.4.2</h3> Update bundled Ruby to 2.7.8 which is the last version of Ruby 2.7 series. Only for RHEL 9 & Ubuntu 22.04 jammy, we updat Ruby to 3.1.4 exceptionally.</li> </ul></li>

About next major version</h3> We are preparing next major version of td-agent which bundles Ruby 3.2. It's still work in progerss, we'll announce it as soon as we are ready for it.</p>

Fluentd v1.16.1 has been released

Bug Fixes</h2>

Fluentd v1.16.0 has been released

Fluentd v1.15.3 has been released

td-agent v4.4.2 has been released