After a month and 10 days from release of 0.14.1, we have been released Fluentd version 0.14.2. That version includes many feature updates and bug fixes, which solves many problems reported for 0.14.1. Thank you reporters!
However, Fluentd 0.14.2 is not stable version yet. We have some known issues, mainly for Windows environment, and some features to be added in stable Fluentd major version. These will be added in 0.14.3.
Anyway, 0.14.2 has many good things for Fluentd 0.14 users. Please try it!
Here are major changes (full ChangeLog is available here):
Core: Splitting events into some chunks
Fluentd 0.12 (, 0.14.0 and 0.14.1) stores all events in a bunch are stored in a buffer chunk, even if its bunch bytesize is bigger than
buffer_chunk_limit. In such cases,
buffer_chunk_limit is ignored. This is typically seen when Fluentd log forwarder's
buffer_chunk_limit is larger than one of log aggregator node.
This behavior may generate many larger buffer chunks than the size output plugin expects, and it may cause any unexpected troubles.
Fluentd 0.14.2 splits such large bunch of events into buffer chunks, according to configured
chunk_limit_size. There will be no more chunks larger than our expectation with this improvement, but splitting events into chunks requires a bit additional CPU time. Roughly speaking, splitting chunks consumes 45% more CPU time in worst case (see benchmark result).
The way to avoid this is to configure
buffer_chunk_limit) with the same value in all Fluentd processes, or use smaller configuration value in forwarder processes (using
forward output plugin) than aggregator processes (using
forward input plugin).
Core: compat_parameters plugin helper now supports parsers and filters
This is the last missing peace to support v0.12 style configurations in v0.14 API plugins. This plugin helper converts traditional style configurations (such like one using
format parameter in
<match> sections directly) into v0.14 style configuration (using
<parse> @type ... </parse> or
<format> @type ... </format>). #1079
Core: New RPC call to do both of flushing buffers and shutdown
A new RPC call
/api/processes.flushBuffersAndKillWorkers was added to flush buffers and then shtudown Fluentd process. #1134
parser_csv: New option to specify delimiter
parser_csv is a parser plugin to parse CSV... it was originally "Comma-Separated Values" in good old days, but now it is "Character-Separated Values". The newly added option
delimiter is to allow users to specify a delimiter (column separator) for various use cases. #1108
in_syslog: New option to receive longer syslog messages
message_length_limit option now (default is 2048). It's useful for syslog users who transfers web access logs or other large log messages via syslog protocol. #1127
Major bug fixes
- fix a bug to read non-buffer-chunk files as buffer chunks, and Fluentd process crashes #1098 #1124
- fix a bug not to load Filter plugins (even if configuration has
<filter> sections) #1118