Fluentd Blog

Fluentd v0.12.16 has been released

Hi users!

We have released Fluentd version 0.12.16. Here are the changes:

New features / Enhancement

• parser: Add null_value_pattern and null_empty_string option for ltsv, csv and tsv #657
• RPC: Implement /api/config.dump and /api/config.getDump APIs to dump in memory config #666
• Add --show-plugin-config option #663

Bug fixes

• in_http: Fix add_remote_addr and add_http_headers options not working on bulk request #673
• parser: RegexpParser#initialize should wrap only Hash configuration #647

Add null_value_pattern and null_empty_string for ltsv, csv and tsv parsers

In some real cases, TSV families have original null value pattern, e.g. -, null, NULL. Fluentd parses NULL as a string but users want to treat these values as nil. null_value_pattern is useful for such cases.

<source>
  type tail
  path /var/log/httpd/access_log
  format ltsv
  null_value_pattern ^(-|null|NULL)$
  # other parameters
</source>

By this configuration, if your logs have -, null or NULL, parser converts it to nil.

In addition, if you want to treat empty string as nil, use null_empty_string instead.

New RPC: /api/config.dump and /api/config.getDump

These RPCs dump in-memory configuration to fluentd's log or RPC response. This is useful for checking running fluentd is old or not.

Here is a configuration example:

<system>
  rpc_endpoint localhost:24220
</system>

<source>
  type forward
</source>

<match **>
  type stdout
</match>

Call /api/config.dump and get current configuration in fluentd logs.

% curl http://localhost:24220/api/config.dump
{"ok":true}

Your log has following lines

2015-09-30 19:37:51 +0900 [info]: dump in-memory config
2015-09-30 19:37:51 +0900 [info]: <ROOT>
  <system>
    rpc_endpoint localhost:24220
  </system>
  <source>
    type forward
  </source>
  <match **>
    type stdout
  </match>
</ROOT>

If you set enable_get_dump in <system> section, you can use /api/config.getDump to get configuration from RPC response. This is turn off by default for security concern.

master branch is now for v0.14

master branch becomes a development branch for v0.14 and we created v0.12 branch for maintaining v0.12 seriese. If you have v0.12 specific patch, send it to v0.12 branch instead of master. We have a plan to merge following features soon:

• Windows support #674
• Nanosecond time support #653

If we have a time, following feature may merge.

• SocketManager for sharing port between workers.

After that, we will release alpha version of v0.14 :)

Happy logging!