Fluentd Blog

Fluentd v1.6.0 has been released

Hi users!

We have released v1.6.0. ChangeLog is here. This release includes new plugin helper and some improvement.

http_server plugin helper

This plugin helper is for in_monitor_agent like plugins. Here is code example:

module Fluent::Plugin
  class FooServerInput < Input
    Fluent::Plugin.register_input('foo_server', self)
    helpers :http_server

    # ...

    def start
      super

      create_http_server(addr: @bind, port: @port, logger: log) do |serv|
        serv.get('/api/hello') { |req|
          # Return value is an Array which consists of status code, headers and body
          [200, {'Content-Type' => 'text/plain'}, 'hello get']
        }
        serv.post('/api/hello') { |req|
          [200, {'Content-Type' => 'text/plain'}, 'hello post']
        }
        # support other HTTP method
      end
    end
  end
end

This helper uses WEBrick by default. But if you install async-http gem, use async-http instead for the performance. in_monitor_agent replaced HTTP server implemenation with http_server helper.

in_monitor_agent: Add more buffer/output metrics

Since v1.6.0, monitor_agent input plugin exposes more buffer/output metrics. Here is the comparison.

# v1.6
{
  "plugin_id": "out_forward",
  "plugin_category": "output",
  "type": "forward",
  "output_plugin": true,
  "buffer_queue_length": 0,
  "buffer_timekeys": [],
  "buffer_total_queued_size": 0,
  "retry_count": 0,
  "emit_records": 0,
  "emit_count": 0,
  "write_count": 0,
  "rollback_count": 0,
  "slow_flush_count": 0,
  "flush_time_count": 0,
  "buffer_stage_length": 0,
  "buffer_stage_byte_size": 0,
  "buffer_queue_byte_size": 0,
  "buffer_available_buffer_space_ratios": 100,
  "retry": {}
}

# v1.5 or earlier
{
  "plugin_id": "out_forward",
  "plugin_category": "output",
  "type": "forward",
  "output_plugin": true,
  "buffer_queue_length": 0,
  "buffer_timekeys": [],
  "buffer_total_queued_size": 0,
  "retry_count": 0,
  "retry": {}
}

New fields are below:

• emit_records: The total number of emitted records
• emit_count: The total number of emit call in output plugin
• write_count: The total number of write/try_write call in output plugin
• rollback_count: The total number of rollback. rollback happens when write/try_write failed
• slow_flush_count: The total number of slow flush. This count will be incremented when buffer flush is longer than slow_flush_log_threshold
• flush_time_count: The total time of buffer flush in milliseconds
• buffer_stage_length: Current length of staged buffer chunks
• buffer_stage_byte_size: Current bytesize of staged buffer chunks
• buffer_queue_byte_size: Current bytesize of queued buffer chunks
• buffer_available_buffer_space_ratios: Show available space for buffer

socket/out_forward: Add connect_timeout parameter for TCP/TLS

This socket timeout is used for connection establishment. If your DNS returns wrong IP or machine, connection establishment waits a long time, e.g. 180 seconds by system default. connect_timeout parameter avoids this kind of problem.

Major bug fixes

• cert: Fix cert pattern to accept certificates which don't end with newline

Enjoy logging!