Fluentd Blog

Fluentd v1.12.0 has been released

Hi users!

We have released v1.12.0. ChangeLog is here.

in_tail: Support * in path with log rotation

fluentd v1.12.0 resolves the limitation for * with log rotation. follow_inodes true enables the combination of * in path with log rotation inside same directory and read_from_head true without log duplication problem.

path /path/to/*
read_from_head true
follow_inodes true  # without this parameter, log rotation may cause log duplication

in_tail: Support Linux Capability

Fluentd supports Linux capability via capng_c gem and in_tail now supports CAP_DAC_READ_SEARCH/CAP_DAC_OVERRIDE capabilities to read log files.

See official article for more details: Linux Capability

If you want more capability support in official plugins, file it on github.

Add fluent-ctl command

fluent-ctl provides same interface to control fluentd process, shutdown/restart/flush/reload.

• On Linux, fluent-ctl uses Signals
• On Windows, fluent-ctl uses Windows Event

Command example:

# Usage: fluent-ctl action PID
$ fluent-ctl shutdown 11111

Major bug fixes

• output: Prevent retry.step from being called too many times in a short time

Enjoy logging!

Read More

Fluentd v1.11.5 has been released

Hi users!

We have released v1.11.5. ChangeLog is here.

formatter: Provide newline parameter to support CRLF

This parameter is for Windows environment. Users can choose LF or CRLF newline for add_newline true case.

newline lf # or crlf

out_http: adding support for intermediate certificates

Add support for sending a client certificate chain for mutual TLS authentication in the HTTP output plugin.

Major bug fixes

• Fix a bug that windows service isn't stopped gracefuly

Enjoy logging!

Read More

Fluentd Ecosystem Survey 2020

Hey All,

Thank you for being part of the Fluent Slack Community! As part of improving Fluentd and Fluent Bit, we want to know what pains and feedback you are experiencin to better manage our future together.

We have created an anonymous survey that allows you to share this here.

Optionally, if you are interested you can add your email to give us direct feedback. Looking forward to your feedbacks!

Read More

Fluentd v1.11.3 has been released

Hi users!

We have released v1.11.3. ChangeLog is here.

log: Add ignore_same_log_interval parameter

Fluentd has ignore_repeated_log_interval parameter but this is not enough for some frequent log generation case. For example, if the plugin generates several log messages in one action, logs are not repeated:

# retry generates same error messages with ignore_repeated_log_interval
def write(chunk)
  # process 1...
  log.error "message1"
  # process 2...
  log.error "message2"
end

ignore_same_log_interval resolves these cases. With this option, logger keeps to store all log messages for validation.

<system>
  ignore_same_log_interval 60s
</system>

inexec: Add `connectmode` parameter to read stderr

in_exec doesn't read stderr and this is not good for script debug. To resolve this problem, add connect_mode parameter to read stderr logs.

<source>
  @type exec
  tag somescript
  command python /path/to/script.py
  connect_mode read_with_stderr
  <parse>
    @type none
  </parse>
</source>

Major bug fixes

• buffer: Fix calculation of timekey stats
• buffer: fix binmode usage for prevent gc

Enjoy logging!

Read More

Fluentd v1.11.2 has been released

Hi users!

We have released v1.11.2. ChangeLog is here.

Allow regular expression in filter/match tag matching

Regular expression, /pattern/, is now allowed in tag matching:

<source>
  @type forward
</source>

# For a.xxx tag
<filter a.**>
  @type record_transformer
  <record>
    new_key a-started tag record
  </record>
</filter>

# For non a.xxx tag like b.xxx/c.xxx/...
<filter /(?!a\.).*/>
  @type record_transformer
  <record>
    new_key other tag record
  </record>
</filter>

<match **>
  @type stdout
</match>

If you send following events:

% echo '{"msg":"hi"}' | fluent-cat a.foo
% echo '{"msg":"hi"}' | fluent-cat b.foo

the log shows:

2020-08-06 13:40:03.111196000 +0900 a.foo: {"msg":"hi","new_key":"a-started tag record"}
2020-08-06 13:40:09.492380000 +0900 b.foo: {"msg":"hi","new_key":"other tag record"}

Of course, simple data pipeline is important for robust log collection. If you need this feature, check your configuration/data flow first.

in_dummy is renamed in_sample

We recommend to use sample instead of dummy:

<source>
  @type sample
  sample {"hello":"world"}
</source>

# If you use fluentd v1.11.1 or earlier, use following configuration
# Fluentd v2 will remove old configuration support
<source>
  @type dummy
  dummy {"hello":"world"}
</source>

Major bug fixes

• buffer: Fix log message for chunk_limit_records case
• buffer: Fix timekey optimization for non-windows platform
• cert: Raise an error for broken certificate file
• cert: Set TLS ciphers list correcty on older OpenSSL, e.g. openssl v1.0.2

Enjoy logging!

Read More