Fluentd Blog

Fluentd v1.20.0 has been released

Hi users!

We released v1.20.0 on October 23, 2026. ChangeLog is here.

Fluentd v1.20.0 is the first release in the v1.20 series. It enables Ruby JIT by default and adds options to limit the size of evacuated buffer files and to discover new log files sooner, along with bug fixes and performance improvements.

Notable changes

Ruby 3.2 is no longer supported

Fluentd now requires Ruby 3.3 or later. Ruby 3.3 is needed to enable Ruby JIT through RubyVM::YJIT.enable.

Ruby JIT is enabled by default

The enable_jit option in the <system> directive now defaults to true. Fluentd enables YJIT in each worker process, and it can improve the throughput.

On the other hand, JIT increases the memory usage of each worker process. If you want to keep the previous behavior, set enable_jit false as follows:

<system>
  enable_jit false
</system>

This option has no effect on Windows, because YJIT is not supported there. In that case, Fluentd just logs Ruby JIT is not available on this Ruby at the info level.

Ref. enable_jit

Fluentd no longer uses yajl-ruby

Fluentd has used the json gem as the default JSON processor since v1.19.0. This release also uses it to parse JSON streams, for example in parser_json with in_exec, and in in_forward and in_unix when they receive JSON. Fluentd itself no longer uses yajl-ruby.

There is one behavior change. When an input stream ends in the middle of a JSON document, Fluentd now logs a warning and discards the incomplete tail instead of raising ParseError. Complete JSON documents received before the truncation are processed as before.

Fluentd now depends on json gem v2.21 or later.

Note for plugin developers: Fluentd still keeps yajl-ruby as a dependency for a while, because some third-party plugins require 'yajl' without declaring it in their gemspec. If your plugin uses yajl-ruby, please add it to the dependencies of your gemspec.

Major new features and improvements

buffer: limit the total size of evacuated chunk files

The new evacuate_limit_size option in the <buffer> section limits the total size of evacuated chunk files.

Fluentd v1.19.0 added a feature to evacuate buffer chunk files when the retry limit is reached. During a long outage, this can happen again and again, and the evacuated files can fill the disk. The new limit also counts the files evacuated before. Chunks that do not fit are discarded without being evacuated.

<match app.**>
  @type forward
  @id forward_app
  <server>
    host 192.168.1.3
  </server>
  <buffer>
    @type file
    path /var/log/fluent/buf
    evacuate_limit_size 1g
  </buffer>
</match>

• The default is nil (no limit). It keeps the previous v1.19.x behavior.
• Set 0 to disable evacuation. It is the same as the behavior before v1.19.0.
• It is an approximate limit. The total size can exceed it, for example when multiple workers of the same plugin evacuate chunks at the same time.
• We recommend setting @id for the output plugin. Without @id, the evacuation directory can change when the configuration changes, and the files evacuated before are not counted.

Ref. Handling Successive Failures

in_tail: discover new files earlier with directory watching

The new opt-in enable_dir_watcher option lets in_tail discover new files sooner.

in_tail usually finds new files by periodic checks every refresh_interval (60 seconds by default). With this option, in_tail also watches the directories derived from path, and refreshes the file list when they change.

<source>
  @type tail
  path /var/log/myapp/*.log
  pos_file /var/log/fluent/myapp.pos
  tag myapp
  enable_dir_watcher true
  follow_inodes true
  <parse>
    @type none
  </parse>
</source>

• dir_watcher_limit (default: 100) sets the maximum number of directories watched at once. If there are more candidate directories, in_tail rotates the watched ones at each periodic refresh.
• Periodic refreshes still run as a fallback.
• Detection latency depends on the platform and the file system. On Linux, inotify is used where available.
• Rotated files that match path may also be found earlier. Use follow_inodes true to avoid duplicate collection after rename-based rotation.

Ref. enable_dir_watcher

in_tail: parse lines on worker threads

The new opt-in num_threads option lets in_tail parse lines on worker threads. It can reduce delays in processing other files while a large or fast-growing file is parsed.

The default 1 keeps the previous behavior, which parses lines on the event loop. Set it to 2 or more to enable worker parsing.

<source>
  @type tail
  path /var/log/app/*.log
  pos_file /var/log/fluentd/app.log.pos
  tag app
  num_threads 2
  <parse>
    @type json
  </parse>
</source>

• It does not guarantee a higher overall throughput. If all worker threads are busy, other files may still wait.
• The order of the events within a file is preserved.
• Worker parsing is available with the none, regexp, json, csv, tsv, ltsv, msgpack, apache, apache2, apache_error, nginx and syslog parsers. In other cases, such as multiline parsing or open_on_every_update true, in_tail logs a warning at startup and keeps parsing on the event loop.
• While worker parsing is active, in_monitor_agent reports worker_pending_batch_count, worker_pending_bytes and worker_parse_error_count.

Ref. num_threads

in_forward: improve interoperability with Fluent Bit v5.0.5 or later

in_forward can now receive events sent by Fluent Bit v5.0.5 or later with its default settings.

Fluent Bit v5.0.5 changed its out_forward plugin to send metadata within each event by default. The timestamp of such an event is encapsulated with the metadata as [EventTime, metadata], as defined in Forward Protocol Specification v1.5. Previous versions of in_forward could not handle this format. With the default skip_invalid_event true, these events were dropped with a skip invalid event warning.

Now in_forward extracts the timestamp and ingests the record. The metadata itself is not processed yet, and it is discarded.

System configuration: add umask option

You can now set the umask of the Fluentd process with the umask option in the <system> directive. Previously, this required the --umask command line option, which is inconvenient for services and container images.

<system>
  umask 0022
</system>

When both are given, the command line option takes precedence.

out_forward: add send_keepalive_packet option

The new send_keepalive_packet option enables TCP keepalive (SO_KEEPALIVE) on out_forward connections. The existing keepalive option only reuses connections and does not send TCP keepalive packets, so dead connections could remain in CLOSE-WAIT state.

send_keepalive_packet is disabled by default, and it requires keepalive true. Otherwise, Fluentd fails to start with a configuration error. in_forward already has a similar option.

<match app.**>
  @type forward
  keepalive true
  send_keepalive_packet true
  <server>
    host 192.168.1.3
  </server>
</match>

in_exec: add command_timeout option

The new command_timeout option limits how long an in_exec command can run. When the command runs longer than this timeout, Fluentd logs child process timed out and kills the process. The default is nil (no timeout).

Without a timeout, a hung command makes in_exec keep logging previous child process is still running. skipped., and it is hard to find the actual cause. out_exec already has the same option.

<source>
  @type exec
  command /path/to/command
  run_interval 10s
  command_timeout 30s
  tag example
  <parse>
    @type json
  </parse>
</source>

Performance improvements

• parser_csv, parser_tsv and parser_ltsv are faster and allocate fewer objects.
• Output plugins cache metadata and timekey calculation to reduce object allocations.
• formatter_out_file is faster.
• More JSON handling uses the json gem instead of yajl-ruby.
• On Windows, in_tail reads files in 64KB blocks instead of 8KB, and EventTime.now is faster.

Other improvements

• parser_syslog: accept one space between the RFC3164 priority and the header. Some syslog senders emit records in this format.
• fluent-plugin-generate: add a .gitignore file to the generated plugin template.
• The gem for Windows on Arm (aarch64-mingw-ucrt) is now available.

Major bug fixes

in_tail: improved stability

• Fixed duplicated logs when a rotated file is appended during rotate_wait with follow_inodes true.
• Fixed the remaining lines of a file never being collected when the file is rate-limited by <group> and then removed from the watch list, for example by limit_recently_modified.
• Fixed NameError when flushing buffered lines with emit_unmatched_lines true.
• Fixed errors and timeouts at shutdown when the shutdown overlaps with file discovery or file rotation.
• multiline_flush_interval is no longer affected by system clock adjustments.

buffer: fixed a chunk being flushed twice

Fixed a bug that could cause a file buffer chunk to be flushed twice, resulting in closed stream or ENOENT errors and incorrect buffer size metrics. Also fixed the stage_size metric that could briefly become negative.

http_server helper: fixed bind failures and IPv6 addresses

These fixes affect plugins using the http_server helper, such as in_monitor_agent and fluent-plugin-prometheus.

• Fixed HTTP endpoints never recovering after a bind failure. Previously, the plugin started anyway without listening on the port. Now the plugin fails to start, and the worker is restarted until the port is available. You can slow down the restart with restart_worker_interval in the <system> directive.
• Fixed URI::InvalidURIError when binding to IPv6 addresses such as :: and ::1.

supervisor: fixed graceful reload with YAML configuration

Fixed graceful reload failing with YAML configuration files, for example when triggered through the RPC endpoint /api/config.gracefulReload. Also fixed <<: !include of a file that contains a mapping. It restores the documented shared parameter syntax.

Other bug fixes

• out_file: fixed missing symlinks when symlink_path is used with chunk keys other than time. Previously, only one of the chunks with the latest timekey got a symlink.
• parser_msgpack: fixed leftover data from truncated input mixing into the next request, connection, datagram or record. It affected in_http, in_tcp, in_udp and filter_parser.
• Fixed emit_records and emit_size metrics always being 0 for input plugins with @label.
• Fixed the time of an event at the Unix epoch that could be formatted as empty, for example in out_file.

Enjoy logging!

Follow us on X

We have been posting information about Fluentd in Japanese on @fluentd_jp. We would appreciate it if you followed the X account.