We have released Fluentd version 0.12.19.
Here are the changes:
New features / Enhancement
- input: Standardize logging level for source configs: 743
- buffer: Add
buffer_queue_full_action parameter: 745
- parser/formatter: Use
oj when oj is installed in json parser/formatter: 748
- fluent-cat: Fix the regression of
--msgpack option: 747
buffer_queue_full_action parameter in buffers
By default, Fluentd's buffer raises
BufferQueueLimitError when buffer queue becomes full. This is fit for streaming manner, e.g.
in_forward returns error response to
out_forward can handle such error.
On the other hand, blocking action instead of raising error is sometimes useful, e.g.
out_elasticsearch for mini-batch use.
To support above 2 cases, we introduce
buffer_queue_full_action parameter in buffers. Currently, there are 2 values.
buffer_queue_full_action exception # default. existing behaviour
buffer_queue_full_action block # block input processing
If you set
buffer_queue_full_action block in buffered output configuration, input blocks emit records until buffer full is resolved.
buffer_queue_full_action but we don't recommend to use
block action first. Please consider improving destination setting to resolve
BufferQueueLimitError or use
@ERROR label for routing overflowed events to another backup destination. If you sometimes hit
BufferQueueLimitError, it means your destination capacity is not fit for your traffic.
Lastly, we are now considering back-pressure support in the future. It will resolve above problem smartly.
oj for performance improvement when it is installed
oj is very fast json library in Ruby. Fluentd uses
yajl by default but if you install
oj in your rubygems, Fluentd uses
oj instead of
yajl for better parsing / formatting performance. Using
oj is 1.5x faster in in_tail case(See PR).
In our tests, there are no big differences between
yajl. But if you hit some troubles or want to keep to use
json_parser yajl in your plugin configuration.
Lastly, v0.12.19 docker image has also been available on Docker Hub.