Fluentd Blog

fluent-package v6.0.0 has been released

Hi users!

We're excited to announce the release of fluent-package v6.0.0 !

This is the first release of the v6 LTS series, and it will be supported until the end of 2027.

We strongly recommend upgrading to fluent-package v6 LTS!

What is Fluent Package v6 LTS?

Fluent Package v6 LTS is a long-term support (LTS), stable package of Fluentd.

The conventional Fluentd project official package, td-agent, reached its end of support at the end of 2023. Fluent Package has been officially developed and distributed by the Fluentd project since the end of August 2023 as its successor package.

The current LTS version, Fluent Package v5 LTS, will reach end of support at the end of 2025. As the next LTS version, Fluent Package v6 LTS will be supported until at least the end of 2027.

In LTS versions, we only perform bug fixes and security fixes over a pre-announced long-term period (minimum 2 years). Therefore, there are two advantages for long-term stable operations:

• Continuous updates are easier
  • You can continuously incorporate the latest bug fixes and security fixes
• You can prepare systematically for the next major update
  • Since the support period is announced in advance, you know when the next major update will occur

For detailed support schedule information, please see Scheduled support lifecycle announcement about Fluent Package v6.

Major new features and improvements

This release includes many improvements for long-term stable operations. The main improvements are as follows:

• Updated Fluentd to v1.19.0
  • Significantly improved fault tolerance and operability, including buffer evacuation.
• Zero-downtime update / restart
  • Configuration reloads and Fluentd restarts can now be performed with zero downtime.
• Refreshed platform support
  • Added support for modern platforms and discontinued support for older ones.
• OpenTelemetry support
  • Support for transferring OpenTelemetry data over HTTP/HTTPS.
• Easier updates
  • Auto-start settings and service startup command line arguments (Windows) are preserved.

Updated Fluentd to v1.19.0

While Fluent Package v5 LTS bundled Fluentd v1.16, v6 LTS moves to Fluentd v1.19. This makes the following new features available:

• Major new features and improvements in Fluentd v1.19.0
  • Improved fault tolerance and simplified recovery
  • Buffer evacuation when retry limits are exceeded
    • Data that exceeded retry limits is automatically evacuated, making it easy to resend later.
  • Enhanced buffer corruption detection
    • If buffer files are corrupted due to forced shutdown, they are automatically evacuated, improving safe restarts.
  • Expanded metrics
    • Added several new metrics (e.g., number of tracked files in in_tail) to aid monitoring.
  • Performance improvements
  • Support for Zstandard (zstd) compression
    • Compared to gzip, this can reduce disk usage and improve transfer speed.
  • Other optimizations
    • Multiple performance improvements for faster processing and reduced memory usage.

For more detailed changes since v1.16, see:

• Fluentd v1.19.0
• Fluentd v1.18.0
• Fluentd v1.17.0

Zero-downtime update / restart

Starting from v6 LTS, zero-downtime updates are available. This feature allows you to safely execute configuration file reloading and Fluentd restarts with zero downtime (Windows not supported).

For more details, see the following articles and try out this feature:

• Zero-downtime restart
• Zero-downtime update

Refreshed platform support

Fluent Package v6 supports the following platforms:

• Debian-based
  • Debian 12 (bookworm)
  • Debian 13 (trixie)
  • Ubuntu 22.04 (Jammy Jellyfish)
  • Ubuntu 24.04 (Noble Numbat)
• RHEL-based
  • RHEL 8 compatible
  • RHEL 9 compatible
  • RHEL 10 compatible
  • AmazonLinux 2023
• Windows

OpenTelemetry Support

This release supports transferring OpenTelemetry data over HTTP/HTTPS. For configuration methods and other details, please see the plugin README.

We also provide a sample for quick testing.

Easier updates

We've improved the system to preserve auto-start configuration and Windows service (fluentdwinsvc) command line options during updates. This eliminates the need for reconfiguration that was previously required during updates.

In addition, manually installed plugins still need to be reinstalled when updating an LTS version. Starting from this release, plugins will be automatically reinstalled during future updates when using the zero-downtime update feature (Windows not supported). This will make future updates easier.

Other Improvements and Fixes

• Updated from Ruby 3.2 to Ruby 3.4
• Migrated distribution CDN to fluentd.cdn.cncf.io
• Included necessary gems for Linux Capability functionality
• Disabled unnecessary linker flags (package-note-file) in RPM builds
• Added fluent-plugin-fluent-package-update-notifier plugin
  • Outputs Fluent Package version update notifications to logs (Linux environments only).
• Added fluent-plugin-obsolete-plugins plugin
  • Detects obsolete plugins at Fluentd startup and outputs to logs (Linux environments only).
• Updated fluent-plugin-systemd plugin
  • Fixed SIGABORT error.

Download

Please see the download page.

Announcement

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 reached EOL in Dec, 2023.

And fluent-package v5 will reach EOL in end of 2025.

We strongly recommend upgrading to fluent-package v6 LTS.

Follow us on X

We have been posting information about Fluentd in Japanese on @fluentd_jp. We would appreciate it if you followed our X account.

Read More

Fluentd v1.19.0 has been released

Hi users!

We have released v1.19.0 on 2025-07-30. ChangeLog is here.

This release is a new release of v1.19 series. This release includes many new features and bug fixes, with a particular focus on reducing operational and running costs and improving overall stability.

$ rm Gemfile.lock
$ bundle

Major new features and improvements

Add feature to evacuate chunk files when retry limit

In this release, we added a buffer evacuation feature to the buf_file and buf_file_single plugins. This simplifies recovery after failure.

In previous versions, when an output plugin fails to deliver data by network issue, if the retry limit was exceeded and secondary output was not configured, the data would be discarded.

In this release, buffer files are automatically evacuated when the retry limit is exceeded. Once the issue is resolved, you can restore the evacuated files to the original buffer directory and restart Fluentd to resume sending the data.

You can even restart without causing downtime to input plugins like in_udp by using the zero-downtime restart feature.

Evacuated files are stored at:

${root_dir}/buffer/${plugin_id}/

Ref. Handling Successive Failures

Note: This feature is disabled when secondary outputs or retry_forever are used. To enable buffer evacuation, remove those settings.

Improved buffer corruption detection

Buffer files may become corrupted due to events like system crashes or sudden power down. In previous versions, Fluentd could fail to handle corrupted files properly, triggering errors on every restart. You had to manually identify and delete the corrupted files.

This release improves buffer corruption detection. When corruption is detected, the file is automatically moved to a backup directory, and startup errors are suppressed.

Enhanced metrics

We have improved Fluentd's observability by enhancing its metrics capabilities:

• Input plugin metrics are now enabled by default.
  • Previously, you needed to explicitly set enable_input_metrics to true. From this release, it's collected by default. To disable, set enable_input_metrics false.
• New metrics for in_tail:
  • tracked_file_count: Number of files currently being tailed.
• New metrics for output plugin:
  • write_secondary_count: Cumulative count of records written to secondary output.
  • drop_oldest_chunk_count: Cumulative count of chunks dropped when drop_oldest_chunk is used for overflow_action.

Performance improvement: switch from yajl-ruby to json gem

Previously, we used yajl-ruby for high-performance JSON processing, such as writing buffer chunks and parsing with parser_json.

However, in recent years, the standard Ruby json library has improved performance significantly. So we changed to use the json gem as the default JSON processor.

Zstandard (zstd) compression support

We have added support for the Zstandard (zstd) compression format, which offers both high compression ratios and faster processing compared to gzip.

• The following plugins now accept compress zstd:
  • buffer
  • out_file
  • out_forward

Note: To use zstd with out_forward, the destination server should support it. You cannot use it with Fluent Bit or Fluentd versions older than v1.19.0.

TLS 1.3 support

The out_http plugin now supports TLS 1.3, offering enhanced security. By default, TLS 1.2 is used. To enable TLS 1.3, set the tls_version parameter as follows:

<match tag>
  @type http
  ...
  tls_version TLSv1_3
  ...
</match>

Other useful improvements

• in_forward plugin:
  • skip_invalid_event is now enabled by default to skip corrupted records.
• out_stdout plugin:
  • Added the use_logger parameter. Setting use_logger false forces output to STDOUT regardless of Fluentd's own logging configuration. This is especially useful in container environments like Kubernetes.
• out_file plugin:
  • Added symlink_path_use_relative to allow relative paths in symlink_path.
• System configuration:
  • Added forced_stacktrace_level to control the log level of stack traces.

Major bug fixes

Fixed memory leak in formatter_csv plugin

To improve performance, it introduced a caching mechanism in formatter_csv, but incorrectly cached threads. This caused memory leaks when used with in_exec, which spawns a new thread per log entry.

This fix is backported to Fluentd v1.16.8.

in_tail plugin: fixed incorrect behavior with encoding option

The in_tail plugin provides encoding and from_encoding options.

• Expected usage:
  • Setting only encoding: Specifying encoding information.
  • Setting both encoding and from_encoding: Converting encoding.

Due to a bug since Fluentd v0.14.12 (at least from td-agent v3), when only encoding was set, unintended encoding conversion occurred, it might corrupt log data.

In previous versions, you need to specify the same encoding for both encoding and from_encoding as workaround.

Windows: fixed bug where service doesn't stop when the supervisor is dead

On Windows, if the supervisor process exited due to configuration errors, the service could continue running incorrectly. This has been fixed so the service now stops as expected, improving error detection.

This fix is backported to Fluentd v1.16.8.

Fixed --umask Bug

Fixed a bug where the --umask setting was ignored when used together with --daemon, leading to unexpected file permissions.

Enjoy logging!

Read More

fluent-package v5.0.7 has been released

Hi users!

We have released fluent-package v5.0.7. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. Bundled Fluentd was updated to 1.16.9.

We recommend upgrading to fluent-package v5.0.7!

Changes from fluent-package v5.0.6

• Update bundled ruby to 3.2.8
• Update bundled Fluentd to v1.16.9
  • Fluentd v1.16.8 has been released
  • Fluentd v1.16.9 has been released
• Update bundled gems overall
• Update bundled openssl gem to 3.3.0 due to support FIPS
• Fix memory leaks when exception was raised frequently with fluent-plugin-elasticsearch

Update bundled openssl gem to 3.3.0 due to support FIPS

With an old gem, it caused an exception after FIPS enabled in OpenSSL. We have updated the bundled openssl gem to 3.3.0 to solve the issue.

Fix memory leaks when exception was raised frequently with fluent-plugin-elasticsearch

There were memory leaks under conditions where exceptions frequently occurred during communication between Elasticsearch and fluent-plugin-elasticsearch. The issue was fixed in fluent-plugin-elasticsearch v5.4.4 and bundled into fluent-package v5.0.7.

Download

Please see the download page.

Announcement

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.8 on September, 2025. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 reached EOL in Dec, 2023.

After reached EOL, td-agent v4.5.3 on Windows was released because there was a crash bug during startup on Windows. It was backported fix from fluent-package v5 as it is critical in some case. Even though this was a exceptional maintenance release, but there is no change to the fact that we already stopped maintaining td-agent v4.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Follow us on X

We have been posting information about Fluentd in Japanese on @fluentd_jp. We would appreciate it if you followed the X account.

Read More

Fluentd v1.16.9 has been released

Hi users!

We have released v1.16.9 on 2025-05-14. ChangeLog is here.

This release is a maintenance release of v1.16 series. In this release, we have fixed a critical bug of the previous version, v1.16.8. Please do not use v1.16.8.

This release will be bundled for upcoming fluent-package LTS version v5.0.7!

Bug Fixes

Windows: Fix bug where the service accidentally stops after starting

In the previous version v1.16.8, the Windows service (fluentdwinsvc) accidentally stops after starting without stopping the supervisor and worker processes. This is a critical bug of v1.16.8 for users who use Fluentd as a Windows service.

In this release, we have fixed this bug.

See #4954 for more information.

Enjoy logging!

Read More

Fluentd v1.16.8 has been released

Hi users!

We have released v1.16.8 on 2025-05-01. ChangeLog is here.

This release is a maintenance release of v1.16 series.

This release will be bundled for upcoming fluent-package LTS version v5.0.7!

Bug Fixes

Windows: Stop the service correctly when the supervisor is dead

In this release, we have fixed a bug that Windows service process will continue running even if the supervisor process is dead due to config error or etc.

With fluentd#4942 fixed, it becomes easier to notice config errors and other issues.

formatter_csv: Fixed memory leak

The caching mechanism introduced for performance optimization was caching threads inappropriately in formatter_csv. The in_exec plugin creates a thread each time to emit an event. Then all threads were cached in formatter_csv, causing a memory leak.

With fluentd#4920 fixed, it becomes stable.

Misc news

Windows: Add fiddle as dependency gem for Ruby 3.5

For comming Ruby 3.5, we have added a fiddle gem as dependency.

CI stabilization

We are enforcing stabilizing CI of Fluentd. It has nothing to do with Fluentd's functionality directly, but it is very important to provide stable version of Fluentd continuously.

Enjoy logging!

Read More