Fluentd v1.16.8 has been released

Hi users!

We have released v1.16.8 on 2025-05-01. ChangeLog is here.

This release is a maintenance release of v1.16 series.

This release will be bundled for upcoming fluent-package LTS version v5.0.7!

Bug Fixes

Windows: Stop the service correctly when the supervisor is dead

In this release, we have fixed a bug that Windows service process will continue running even if the supervisor process is dead due to config error or etc.

With fluentd#4942 fixed, it becomes easier to notice config errors and other issues.

formatter_csv: Fixed memory leak

The caching mechanism introduced for performance optimization was caching threads inappropriately in formatter_csv. The in_exec plugin creates a thread each time to emit an event. Then all threads were cached in formatter_csv, causing a memory leak.

With fluentd#4920 fixed, it becomes stable.

Misc news

Windows: Add fiddle as dependency gem for Ruby 3.5

For comming Ruby 3.5, we have added a fiddle gem as dependency.

CI stabilization

We are enforcing stabilizing CI of Fluentd. It has nothing to do with Fluentd's functionality directly, but it is very important to provide stable version of Fluentd continuously.

Enjoy logging!

Read More

Drop Alpine and switch default to Debian for Fluentd docker image

Hi users!

We have planned to change what docker image provides.

Currently, we ship Fluentd docker image for Alpine and Debian (armhf, arm64, amd64). But, Alpine image in Fluentd was already marked as deprecated, so we will make things forward.

In the near future, we will drop Alpine image. This decision affects existing Alpine image users.

Notice about existing Alpine image users

  • fluent/fluentd:edge will be changed from Alpine to Debian image
    • This is a notable incompatible change!
  • v1.19 (not released yet)
    • No more releases for Alpine images
    • Then short tag also changed to Debian images (e.g. v1.19.0-1.0, v1.19-1 should be Debian instead of Alpine)
  • v1.18 Alpine image will not be supported after v1.19 is released (T.B.D.), so recommend to migrate it!
  • v1.16 Alpine image will be supported until Dec 2025, so there is room to migrate gradually.

If you still want to use Alpine, keep v1.16 series or v1.18 series. (not recommended though)

Why not provide Alpine anymore?

It was well known that Alpine has possibility of incompatibility and performance issues in contrast to Debian images. It was a historical reason to provide Alpine images - "In the previous versions, we provided them", so there is no positive reason to continue it.

Provide shorter tags to follow internal version bump

Not only just dropping Alpine image, we will improve tagging rules:

  • Add version-specific tag which will not be affected by internal version bump

In previous versions, there was no version-specific tag which would follow internal version bump. For example, if you want to stick to Fluentd v1.18.0, you can select v1.18.0-1.0, but if internal version was bumped to v1.18.0-1.1 or something, need to update it. There is edge-debian tag, but it does not stick to a specific Fluentd version because it will automatically update when the next major/minor version is released.

For such a purpose, shorter v1.19 or v1.19.x tag will be available in the future release.

Happy logging!

Read More

fluent-package v5.0.6 has been released

Hi users!

We have released fluent-package v5.0.6. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. Bundled Fluentd was updated to 1.16.7.

We recommend upgrading to fluent-package v5.0.6!

Changes from fluent-package v5.0.5

In this release, known Windows issues were fixed.

  • Update bundled Fluentd to v1.16.7
  • Update bundled gems overall
  • msi: Fixed to keep some registry values with update

msi: Fixed to keep some registry values with update

In the previous versions, there had been a known issue that fluent-package could not keep some registry values when upgrading.

Thus, if you modify Fluentd service (fluentdwinsvc) related registry, you need to restore your configuration after upgrading.

Since v5.0.6, fluent-package was fixed to keep some registry values during upgrading package.

Here is the target registry which will be kept during upgrading fluent-package:

  • Start
  • DelayedAutostart
  • fluentdopt (logging path might be modified by users)

In most cases, fluent-package user might want to keep fluentdopt because it stores additional command line option parameters.

In this release, bundled Fluentd v1.16.7 also contains the fixes for Windows.

  • Windows: Fix NoMethodError of --daemon option
  • Windows: Fixed the issues which are related to start/stop Fluentd service

See Fluentd v1.16.7 has been released blog article in details.

Download

Please see the download page.

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.7 on June, 2025. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 reached EOL in Dec, 2023.

After reached EOL, td-agent v4.5.3 on Windows was released because there was a crash bug during startup on Windows. It was backported fix from fluent-package v5 as it is critical in some case. Even though this was a exceptional maintenance release, but there is no change to the fact that we already stopped maintaining td-agent v4.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Read More

Fluentd v1.16.7 has been released

Hi users!

We have released v1.16.7 on 2025-01-29. ChangeLog is here.

This release is a maintenance release of v1.16 series. In this release, some bugs which are related on Windows were fixed with backporting from v1.18.x.

This release will be bundled for upcoming fluent-package LTS version v5.0.6!

Bug Fixes

Windows: Fix NoMethodError of --daemon option

In this release, we have fixed a bug that it will fail when --daemon option is used on Windows.

It was a regression bug since v1.16.0 on Windows, but usually running as a service is recommended way on Windows, so most of Fluentd users were not affected.

Windows: Fixed the issues which are related to start/stop Fluentd service

In this release, we have fixed the following bugs which are related to start/stop the Fluentd service.

  • Fixed an issue when stopping the service immediately after startup could leave the processes
  • Fixed an issue where stopping service sometimes can not be completed forever

These bug were caused because stop event was not handled correctly in the previous releases.

Typically, this bug will happen if you stop the Fluentd service immediately after startup. Then it could leave the processes or stopping service sometimes can not be completed forever.

Misc news

We also enforcing stabilizing CI of Fluentd. It has nothing to do with Fluentd's functionality directly, but it is very important to provide stable version of Fluentd continuously.

Not only Windows: Add workaround for unexpected exception #4747, many CI improvements was merged into master, then backported to v1.16 branch, too.

Enjoy logging!

Read More

fluent-package v5.2.0 has been released

Hi users!

We have released fluent-package v5.2.0. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This release is a new release of v5.2 series.

Changes from fluent-package v5.1.0

  • Support upgrade fluentd service with zero-downtime
  • Update ruby to 3.2.6
  • Update bundled Fluentd to v1.18.0
  • Update bundled gems overall
  • Solved possibility of DoS through "NoMemoryError"
  • msi: set GEM_HOME/GEM_PATH in fluentd.bat
  • deb: suppress service restart by needrestart
  • Fixed SIGABORT error with fluent-plugin-systemd

Support upgrade fluentd service with zero-downtime

Previously, when upgrading the fluent-package, you had to stop the fluentd service, install the new version of the fluent-package, then install any necessary plugins, and finally restart the fluentd service.

Starting from fluent-package v5.2.0, we have introduced the zero-downtime restart feature that automatically installs the required plugins during the upgrade process and allows you to upgrade Fluentd without interrupting the logging service.

Please refer to "Advisory for upgrading with zero-downtime".

Solved possibility of DoS through "NoMemoryError"

In the previous versions of msgpack gem, there was a possibility that cause DoS (Denial Of Service) when crafted message was sent to Fluentd.

When that message was sent, it might cause "NoMemoryError" on Fluentd because there is a case that msgpack try to pre-allocate huge amount of memories.

Above behavior was fixed not to pre-allocate over 32k entries (limit pre-allocated amount of memories) in msgpack 1.7.3 and bundled.

msi: set GEM_HOME/GEM_PATH in fluentd.bat

In the previous versions, fluentd.bat doesn't set GEMHOME/GEMPATH explicitly.

If users set custom GEM_HOME / GEM_PATH environment variables, the batch file will not work as expected. Therefore, this patch will set the appropriate values in the batch file.

deb: suppress service restart by needrestart

We have changed so that Fluentd does not restart when the needrestart package is installed. fluent-package v5.2.0 places /etc/needrestart/conf.d/50-fluent-package.conf.

Fixed SIGABORT error with fluent-plugin-systemd

In this release, the bundled fluent-plugin-systemd was updated to 1.1.0. That release fixed SIGABORT error with inconsistency of memory allocator handling.

Advisory for upgrading with zero-downtime

The zero-downtime restart feature can be configured by FLUENT_PACKAGE_SERVICE_RESTART environment variable. Please refer to following section for more details.

  • Add FLUENT_PACKAGE_SERVICE_RESTART environmental variable
    • This section explains FLUENT_PACKAGE_SERVICE_RESTART environment variable.
  • Automate Plugin Install for Update on Demand
    • This section explains the mechanism that installs the plugins automatically during the upgrade process.

Add FLUENT_PACKAGE_SERVICE_RESTART environmental variable

We have introduced FLUENT_PACKAGE_SERVICE_RESTART environment variable to configure the zero-downtime restart feature. The variable exists in the following file:

  • RPM: /etc/sysconfig/fluentd
  • DEB: /etc/default/fluentd

Example:

FLUENT_PACKAGE_OPTIONS=""
# Control method to upgrade service (auto/manual) restart
FLUENT_PACKAGE_SERVICE_RESTART=auto

auto (default)

The service automatically restarts with zero-downtime restart feature when all of the following conditions are met:

  1. The service was active before updating.
  2. The installed and upgrading versions are required 5.2.0 or higher (both sides must support this feature).

The plugins will be automatically reinstalled if needed if your environment has an active online connection. For more details, please refer to "Automate Plugin Install for Update on Demand".

manual

You can use this approach if you prefer to manage plugins manually or if your environment does not have offline connection. If the service was active before the update, it will not restart automatically with this configuration. You need to restart the service manually.

  • The zero-downtime restart:
    • Send a SIGUSR2 signal to the supervisor process, such as with kill -USR2 <PID>
  • Normal restart:
    • Restart the service normally, such as with systemctl restart fluentd.

Automate Plugin Install for Update on Demand

Automate Plugin Management

If you prefer to manage plugins automatically, please set FLUENT_PACKAGE_SERVICE_RESTART to auto (default).

When the service restarts automatically using the zero-downtime restart feature, any missing plugins are automatically detected and reinstalled before the restart.

Previously, automatic restarts after updates were not recommended due to potential issues. If you manually install plugins, you need to reinstall them before restarting when the embedded Ruby version is updated.

To address this, this feature enables automatic plugin installation during the restart process. The steps are as follows:

  1. Collect the current list of plugins before the update.
    • Recognize gems as plugins that has fluent-plugin- prefix.
  2. After installing the updated package, detect missing plugins.
    • Compare the collected plugin list with the default plugins to identify any missing ones.
  3. Install the missing plugins.

Manual Plugin Management

If you prefer to manage plugins manually, please set FLUENT_PACKAGE_SERVICE_RESTART to manual. This is useful in cases such as:

  • Pinning specific plugin versions
  • Operating in an offline environment

In this mode, automatic plugin installation and restarts are disabled. You can manually install plugins and send a SIGUSR2 signal to trigger the zero-downtime restart feature after the update.

Caution: if you use a custom unit file, need to migrate it to use this feature safely

If you use a custom unit file, such as /etc/systemd/system/fluentd.service, please remove these 2 lines.

Environment=GEM_HOME=/opt/fluentd/lib/ruby/gems/...
Environment=GEM_PATH=/opt/fluentd/lib/ruby/gems/...

We don't need GEM_HOME and GEM_PATH. They will be removed after v5.2.0, but if you use a custom unit file, you need to remove them manually. If these variables are set, the zero-downtime restart feature may not work correctly. It is because the new process inherits the original environment variables if using this feature.

Please refer to #713 for more details.

Download

Please see the download page.

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.6 on March, 2025. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

After reached EOL, td-agent v4.5.3 on Windows was released because there was a crash bug during startup on Windows. It was backported fix from fluent-package v5 as it is critical in some case. Even though this was a exceptional maintenance release, but there is no change to the fact that we already stopped maintaining td-agent v4.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Read More