Fluentd v0.12.16 has been released

Hi users!

We have released Fluentd version 0.12.16. Here are the changes:

New features / Enhancement

  • parser: Add null_value_pattern and null_empty_string option for ltsv, csv and tsv #657
  • RPC: Implement /api/config.dump and /api/config.getDump APIs to dump in memory config #666
  • Add --show-plugin-config option #663

Bug fixes

  • in_http: Fix add_remote_addr and add_http_headers options not working on bulk request #673
  • parser: RegexpParser#initialize should wrap only Hash configuration #647

Add null_value_pattern and null_empty_string for ltsv, csv and tsv parsers

In some real cases, TSV families have original null value pattern, e.g. -, null, NULL. Fluentd parses NULL as a string but users want to treat these values as nil. null_value_pattern is useful for such cases.

<source>
  type tail
  path /var/log/httpd/access_log
  format ltsv
  null_value_pattern ^(-|null|NULL)$
  # other parameters
</source>

By this configuration, if your logs have -, null or NULL, parser converts it to nil.

In addition, if you want to treat empty string as nil, use null_empty_string instead.

New RPC: /api/config.dump and /api/config.getDump

These RPCs dump in-memory configuration to fluentd's log or RPC response. This is useful for checking running fluentd is old or not.

Here is a configuration example:

<system>
  rpc_endpoint localhost:24220
</system>

<source>
  type forward
</source>

<match **>
  type stdout
</match>

Call /api/config.dump and get current configuration in fluentd logs.

% curl http://localhost:24220/api/config.dump
{"ok":true}

Your log has following lines

2015-09-30 19:37:51 +0900 [info]: dump in-memory config
2015-09-30 19:37:51 +0900 [info]: <ROOT>
  <system>
    rpc_endpoint localhost:24220
  </system>
  <source>
    type forward
  </source>
  <match **>
    type stdout
  </match>
</ROOT>

If you set enable_get_dump in <system> section, you can use /api/config.getDump to get configuration from RPC response. This is turn off by default for security concern.

master branch is now for v0.14

master branch becomes a development branch for v0.14 and we created v0.12 branch for maintaining v0.12 seriese. If you have v0.12 specific patch, send it to v0.12 branch instead of master. We have a plan to merge following features soon:

  • Windows support #674
  • Nanosecond time support #653

If we have a time, following feature may merge.

  • SocketManager for sharing port between workers.

After that, we will release alpha version of v0.14 :)



Happy logging!

Subscribed to the RSS feed here.

Written by Masahiro Nakagawa

Masahiro (@repeatedly) is the main maintainer of Fluentd. He works on Fluentd development and support full-time. He is also a committer of the D programming language.


comments powered by Disqus

About Fluentd

Fluentd is an open source data collector to unify log management.

Learn

Want to learn the basics of Fluentd? Check out these pages.

Follow Us!