Fluentd Blog https://www.fluentd.org/blog Get the Latest Updates on Fluentd and More Fluentd v1.15.3 has been released https://www.fluentd.org/blog/fluentd-v1.15.3-has-been-released <p>Hi users!</p> <p>We have released v1.15.3 on 2022-11-02. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#release-v1153---20221102">here</a>.</p> <p>In this release, we fixed several bugs.</p> <h2>Bug Fixes</h2> <h3><code>out_file</code>: Fix the multi-worker check with <code>&lt;worker N-M&gt;</code> directive</h3> <p>In the v1.15.1, we added a new feature to prevent write conflicts between multiple workers.</p> <ul> <li><a href="https://github.com/fluent/fluentd/pull/3808">#3808</a></li> </ul> <p>However, this fix didn&#39;t work correctly with <code>&lt;worker N-M&gt;</code> directive.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/multi-process-workers#less-than-worker-n-m-greater-than-directive"><code>&lt;worker N-M&gt;</code> directive</a></li> </ul> <p>With this setting, the configuration check before starting the supervisor mistakenly determines that the setting of <code>out_file</code> is invalid and causes Fluentd to terminate.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3942">#3942</a> for more information.</p> <h3>Fix log rotation on Windows</h3> <p>We can configure log rotation setting as follows.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/logging#log-rotation-setting">Log Rotation Setting</a></li> </ul> <p>On Windows, the log files must be separated by each process.</p> <blockquote> <p>NOTE: When --log-rotate-age is specified on Windows, log files are separated into log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation. Windows does not permit delete and rename files simultaneously owned by another process.</p> </blockquote> <p>However, that log rotation setting in the system-directive didn&#39;t satisfy this and Fluentd outputed the logs of the supervisor process into the log file of the worker-0 process.</p> <p>This could cause errors during log rotation.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3939">#3939</a> for more information.</p> <h3>Support glob for <code>!include</code> directive in YAML config format</h3> <p>In the v1.15.0, we supported YAML config format.</p> <ul> <li><a href="https://docs.fluentd.org/configuration/config-file-yaml">Config File Syntax (YAML)</a></li> <li><a href="https://github.com/fluent/fluentd/pull/3712">#3712</a></li> </ul> <p>However, handling glob of <code>!include</code> directive was leaked.</p> <p>In this release, we supported it.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3917">#3917</a> for more information.</p> <h3>Remove meaningless and unsecure <code>oj</code> options</h3> <p>In this release, we removed the following <code>oj</code> options.</p> <ul> <li><code>FLUENT_OJ_OPTION_MODE=object</code></li> <li><code>FLUENT_OJ_OPTION_MAX_NESTING</code></li> </ul> <p><code>FLUENT_OJ_OPTION_MODE=object</code> was unsecure, so we removed it.</p> <ul> <li><a href="https://github.com/fluent/fluentd/security/advisories/GHSA-fppq-mj76-fpj2">GHSA-fppq-mj76-fpj2</a></li> </ul> <p>This option is not used unless a user explicitly sets the environmental variable, so almost all users do not need to care about this vulnerability.</p> <p><code>FLUENT_OJ_OPTION_MAX_NESTING</code> was not supported, so we removed it.</p> <ul> <li><a href="https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031">https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031</a></li> </ul> <p>See <a href="https://github.com/fluent/fluentd/pull/3929">#3929</a> for more information.</p> <p>Enjoy logging!</p> 2022-11-11 <p>Hi users!</p> <p>We have released v1.15.3 on 2022-11-02. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#release-v1153---20221102">here</a>.</p> <p>In this release, we fixed several bugs.</p> <h2>Bug Fixes</h2> <h3><code>out_file</code>: Fix the multi-worker check with <code>&lt;worker N-M&gt;</code> directive</h3> <p>In the v1.15.1, we added a new feature to prevent write conflicts between multiple workers.</p> <ul> <li><a href="https://github.com/fluent/fluentd/pull/3808">#3808</a></li> </ul> <p>However, this fix didn&#39;t work correctly with <code>&lt;worker N-M&gt;</code> directive.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/multi-process-workers#less-than-worker-n-m-greater-than-directive"><code>&lt;worker N-M&gt;</code> directive</a></li> </ul> <p>With this setting, the configuration check before starting the supervisor mistakenly determines that the setting of <code>out_file</code> is invalid and causes Fluentd to terminate.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3942">#3942</a> for more information.</p> <h3>Fix log rotation on Windows</h3> <p>We can configure log rotation setting as follows.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/logging#log-rotation-setting">Log Rotation Setting</a></li> </ul> <p>On Windows, the log files must be separated by each process.</p> <blockquote> <p>NOTE: When --log-rotate-age is specified on Windows, log files are separated into log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation. Windows does not permit delete and rename files simultaneously owned by another process.</p> </blockquote> <p>However, that log rotation setting in the system-directive didn&#39;t satisfy this and Fluentd outputed the logs of the supervisor process into the log file of the worker-0 process.</p> <p>This could cause errors during log rotation.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3939">#3939</a> for more information.</p> <h3>Support glob for <code>!include</code> directive in YAML config format</h3> <p>In the v1.15.0, we supported YAML config format.</p> <ul> <li><a href="https://docs.fluentd.org/configuration/config-file-yaml">Config File Syntax (YAML)</a></li> <li><a href="https://github.com/fluent/fluentd/pull/3712">#3712</a></li> </ul> <p>However, handling glob of <code>!include</code> directive was leaked.</p> <p>In this release, we supported it.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3917">#3917</a> for more information.</p> <h3>Remove meaningless and unsecure <code>oj</code> options</h3> <p>In this release, we removed the following <code>oj</code> options.</p> <ul> <li><code>FLUENT_OJ_OPTION_MODE=object</code></li> <li><code>FLUENT_OJ_OPTION_MAX_NESTING</code></li> </ul> <p><code>FLUENT_OJ_OPTION_MODE=object</code> was unsecure, so we removed it.</p> <ul> <li><a href="https://github.com/fluent/fluentd/security/advisories/GHSA-fppq-mj76-fpj2">GHSA-fppq-mj76-fpj2</a></li> </ul> <p>This option is not used unless a user explicitly sets the environmental variable, so almost all users do not need to care about this vulnerability.</p> <p><code>FLUENT_OJ_OPTION_MAX_NESTING</code> was not supported, so we removed it.</p> <ul> <li><a href="https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031">https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031</a></li> </ul> <p>See <a href="https://github.com/fluent/fluentd/pull/3929">#3929</a> for more information.</p> <p>Enjoy logging!</p> td-agent v4.4.2 has been released https://www.fluentd.org/blog/td-agent-v4.4.2-has-been-released <p>Hi users!</p> <p>We have released td-agent <a href="https://github.com/fluent/fluent-package-builder/releases/tag/v4.4.2">v4.4.2</a>. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.4.1</h3> <ul> <li>Update fluentd to <a href="https://github.com/fluent/fluentd/releases/tag/v1.15.3">1.15.3</a> <ul> <li>Fix log initializer to correctly create per-process files on Windows</li> <li>out_file: Fix the multi-worker check with <code>&lt;worker 0-N&gt;</code> directive</li> <li>Remove meaningless oj options</li> <li>Support glob for <code>!include</code> directive in YAML config format</li> </ul></li> <li>macOS: Support Apple Silicon</li> <li>Update plugins <ul> <li>fluent-plugin-elasticsearch 5.2.4</li> <li>fluent-plugin-opensearch 1.0.8</li> <li>fluent-plugin-s3 1.7.2</li> <li>fluent-plugin-windows-eventlog 0.8.2</li> </ul></li> </ul> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p> 2022-11-08 <p>Hi users!</p> <p>We have released td-agent <a href="https://github.com/fluent/fluent-package-builder/releases/tag/v4.4.2">v4.4.2</a>. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.4.1</h3> <ul> <li>Update fluentd to <a href="https://github.com/fluent/fluentd/releases/tag/v1.15.3">1.15.3</a> <ul> <li>Fix log initializer to correctly create per-process files on Windows</li> <li>out_file: Fix the multi-worker check with <code>&lt;worker 0-N&gt;</code> directive</li> <li>Remove meaningless oj options</li> <li>Support glob for <code>!include</code> directive in YAML config format</li> </ul></li> <li>macOS: Support Apple Silicon</li> <li>Update plugins <ul> <li>fluent-plugin-elasticsearch 5.2.4</li> <li>fluent-plugin-opensearch 1.0.8</li> <li>fluent-plugin-s3 1.7.2</li> <li>fluent-plugin-windows-eventlog 0.8.2</li> </ul></li> </ul> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p> td-agent v4.4.1 has been released https://www.fluentd.org/blog/td-agent-v4.4.1-has-been-released <p>Hi users!</p> <p>We have released td-agent <a href="https://github.com/fluent/fluent-package-builder/releases/tag/v4.4.1">v4.4.1</a>. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.4.0</h3> <ul> <li>Update fluentd to <a href="https://github.com/fluent/fluentd/releases/tag/v1.15.2">1.15.2</a> <ul> <li>out_file: Fix append mode with <code>--daemon</code> flag</li> <li>Drop win32-api gem dependency</li> <li>Please see also <a href="fluentd-v1.15.2-has-been-released">the release announcement of v1.15.2</a></li> </ul></li> <li>Update fluent-plugin-kafka to <a href="https://github.com/fluent/fluent-plugin-kafka/releases/tag/v0.18.1">0.18.1</a> <ul> <li>Fix out<em>kafka2 doesn&#39;t respect `chunk</em>limit<em>records<code>and</code>chunk</em>limit_size`</li> </ul></li> </ul> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p> 2022-08-23 <p>Hi users!</p> <p>We have released td-agent <a href="https://github.com/fluent/fluent-package-builder/releases/tag/v4.4.1">v4.4.1</a>. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.4.0</h3> <ul> <li>Update fluentd to <a href="https://github.com/fluent/fluentd/releases/tag/v1.15.2">1.15.2</a> <ul> <li>out_file: Fix append mode with <code>--daemon</code> flag</li> <li>Drop win32-api gem dependency</li> <li>Please see also <a href="fluentd-v1.15.2-has-been-released">the release announcement of v1.15.2</a></li> </ul></li> <li>Update fluent-plugin-kafka to <a href="https://github.com/fluent/fluent-plugin-kafka/releases/tag/v0.18.1">0.18.1</a> <ul> <li>Fix out<em>kafka2 doesn&#39;t respect `chunk</em>limit<em>records<code>and</code>chunk</em>limit_size`</li> </ul></li> </ul> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p> Fluentd v1.15.2 has been released https://www.fluentd.org/blog/fluentd-v1.15.2-has-been-released <p>Hi users!</p> <p>We have released v1.15.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#release-v1152---20220822">here</a>.</p> <p>In this release, we fixed the problem that <code>out_file</code>&#39;s append mode with <code>--daemon</code> flag failed to flush the buffer.</p> <h2>Bug Fixes</h2> <h3><code>out_file</code>: Fix append mode with <code>--daemon</code> flag</h3> <p>In the previous version v1.15.1, we added a new feature to prevent write conflicts between multiple workers.</p> <ul> <li><a href="https://github.com/fluent/fluentd/pull/3808">#3808</a></li> </ul> <p>However, this fix didn&#39;t work correctly with <code>--daemon</code> flag of Fluentd command.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/command-line-option">Command Line Option</a></li> </ul> <p>With that flag, <code>out_file</code> in the append mode fails to flush the buffer. This is a critical bug for td-agent, because td-agent uses <code>--daemon</code> flag by default.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3864">#3864</a> for more information.</p> <h2>Miscellaneous fixes</h2> <ul> <li>Add a new system configuration <code>enable_jit</code>. <a href="https://github.com/fluent/fluentd/pull/3857">#3857</a></li> <li>child_process: Plug file descriptor leak. <a href="https://github.com/fluent/fluentd/pull/3844">#3844</a></li> <li>Drop win32-api gem to support Ruby 3.2. <a href="https://github.com/fluent/fluentd/pull/3849">#3849</a>, <a href="https://github.com/fluent/fluentd/pull/3866">#3866</a></li> </ul> <p>Enjoy logging!</p> 2022-08-22 <p>Hi users!</p> <p>We have released v1.15.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#release-v1152---20220822">here</a>.</p> <p>In this release, we fixed the problem that <code>out_file</code>&#39;s append mode with <code>--daemon</code> flag failed to flush the buffer.</p> <h2>Bug Fixes</h2> <h3><code>out_file</code>: Fix append mode with <code>--daemon</code> flag</h3> <p>In the previous version v1.15.1, we added a new feature to prevent write conflicts between multiple workers.</p> <ul> <li><a href="https://github.com/fluent/fluentd/pull/3808">#3808</a></li> </ul> <p>However, this fix didn&#39;t work correctly with <code>--daemon</code> flag of Fluentd command.</p> <ul> <li><a href="https://docs.fluentd.org/deployment/command-line-option">Command Line Option</a></li> </ul> <p>With that flag, <code>out_file</code> in the append mode fails to flush the buffer. This is a critical bug for td-agent, because td-agent uses <code>--daemon</code> flag by default.</p> <p>In this release, we fixed this problem.</p> <p>See <a href="https://github.com/fluent/fluentd/pull/3864">#3864</a> for more information.</p> <h2>Miscellaneous fixes</h2> <ul> <li>Add a new system configuration <code>enable_jit</code>. <a href="https://github.com/fluent/fluentd/pull/3857">#3857</a></li> <li>child_process: Plug file descriptor leak. <a href="https://github.com/fluent/fluentd/pull/3844">#3844</a></li> <li>Drop win32-api gem to support Ruby 3.2. <a href="https://github.com/fluent/fluentd/pull/3849">#3849</a>, <a href="https://github.com/fluent/fluentd/pull/3866">#3866</a></li> </ul> <p>Enjoy logging!</p> td-agent v4.4.0 has been released https://www.fluentd.org/blog/td-agent-v4.4.0-has-been-released <p>Hi users!</p> <p>We have released td-agent v4.4.0. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.3.2</h3> <ul> <li>Update bundled Fluentd from v1.14.6 to v1.15.1. Please see each release announcement of Fluentd for more detail: <ul> <li><a href="fluentd-v1.15.0-has-been-released">Fluentd v1.15.0 has been released</a></li> <li><a href="fluentd-v1.15.1-has-been-released">Fluentd v1.15.1 has been released</a></li> </ul></li> <li>Update OpenSSL to 1.1.1q (for macOS and Windows)</li> <li>Remove <code>TD_AGENT_OPTIONS=</code> from td-agent.service. Instead it&#39;s loaded from /etc/sysconfig/td-agent (rpm) or /etc/default/td-agent (deb).</li> <li>Drop Ubuntu 16.04 &quot;Xenial Xerus&quot; support <a href="td-agent-v4.3.1-has-been-released">as we announced before</a>.</li> <li>Update plugins and gems</li> </ul> <p>Please see <a href="https://github.com/fluent/fluent-package-builder/blob/master/CHANGELOG.md#release-v440---20220729">ChangeLog</a> for more detail.</p> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p> 2022-07-29 <p>Hi users!</p> <p>We have released td-agent v4.4.0. td-agent is a stable distribution package of Fluentd.</p> <h3>Changes from td-agent v4.3.2</h3> <ul> <li>Update bundled Fluentd from v1.14.6 to v1.15.1. Please see each release announcement of Fluentd for more detail: <ul> <li><a href="fluentd-v1.15.0-has-been-released">Fluentd v1.15.0 has been released</a></li> <li><a href="fluentd-v1.15.1-has-been-released">Fluentd v1.15.1 has been released</a></li> </ul></li> <li>Update OpenSSL to 1.1.1q (for macOS and Windows)</li> <li>Remove <code>TD_AGENT_OPTIONS=</code> from td-agent.service. Instead it&#39;s loaded from /etc/sysconfig/td-agent (rpm) or /etc/default/td-agent (deb).</li> <li>Drop Ubuntu 16.04 &quot;Xenial Xerus&quot; support <a href="td-agent-v4.3.1-has-been-released">as we announced before</a>.</li> <li>Update plugins and gems</li> </ul> <p>Please see <a href="https://github.com/fluent/fluent-package-builder/blob/master/CHANGELOG.md#release-v440---20220729">ChangeLog</a> for more detail.</p> <h3>Download</h3> <p>Please see <a href="/download">the download page</a>.</p>