Fluentd Blog https://www.fluentd.org/blog Get the Latest Updates on Fluentd and More Fluentd v1.3.0 has been released https://www.fluentd.org/blog/fluentd-v1.3.0-has-been-released <p>Hi users!</p> <p>We have released v1.3.0. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>.</p> <h3>Output: Change thread execution control</h3> <p>This is internal change of output plugin. This change should fix unexpected error like zlib&#39;s <code>DataError</code>.</p> <h3>Output: Change default value of <code>queued_chunks_limit_size</code></h3> <p><code>queued_chunks_limit_size</code> is now same as <code>flush_thread_count</code> when <code>queued_chunks_limit_size</code> is not specified.</p> <p>Before, <code>queued_chunks_limit_size</code> is unlimited. This causes critical situation with smaller <code>flush_interval</code> when the destinaion has a problem. For example, lots of very small queued buffer are created and it causes resource limit error.</p> <p>Unlimited queued buffer is unexpected behaviour, so we changed this behaviour for safety.</p> <h3>in_syslog: octet counting support</h3> <p>We introduced <code>frame_type</code> parameter. With <code>frame_type octet_count</code> configuration, <code>in_syslog</code> accepts octet counting messages like below.</p> <pre><code>44 &lt;6&gt;Sep 10 00:00:00 localhost logger: hello! </code></pre> <h3>Major bug fixes</h3> <ul> <li>server: Fix deadlock between <code>on_writable</code> and <code>close</code> in sockets. This fixes fluentd restart issue with <code>in_forward</code>&#39;s TLS setting error.</li> </ul> <h3>Other improvements</h3> <ul> <li><code>in_http</code>: Add support for CORS preflight requests. Now you can send cross-domain AJAX requests with JSON payload to Fluentd.</li> </ul> <p>Enjoy logging!</p> 2018-11-10 <p>Hi users!</p> <p>We have released v1.3.0. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>.</p> <h3>Output: Change thread execution control</h3> <p>This is internal change of output plugin. This change should fix unexpected error like zlib&#39;s <code>DataError</code>.</p> <h3>Output: Change default value of <code>queued_chunks_limit_size</code></h3> <p><code>queued_chunks_limit_size</code> is now same as <code>flush_thread_count</code> when <code>queued_chunks_limit_size</code> is not specified.</p> <p>Before, <code>queued_chunks_limit_size</code> is unlimited. This causes critical situation with smaller <code>flush_interval</code> when the destinaion has a problem. For example, lots of very small queued buffer are created and it causes resource limit error.</p> <p>Unlimited queued buffer is unexpected behaviour, so we changed this behaviour for safety.</p> <h3>in_syslog: octet counting support</h3> <p>We introduced <code>frame_type</code> parameter. With <code>frame_type octet_count</code> configuration, <code>in_syslog</code> accepts octet counting messages like below.</p> <pre><code>44 &lt;6&gt;Sep 10 00:00:00 localhost logger: hello! </code></pre> <h3>Major bug fixes</h3> <ul> <li>server: Fix deadlock between <code>on_writable</code> and <code>close</code> in sockets. This fixes fluentd restart issue with <code>in_forward</code>&#39;s TLS setting error.</li> </ul> <h3>Other improvements</h3> <ul> <li><code>in_http</code>: Add support for CORS preflight requests. Now you can send cross-domain AJAX requests with JSON payload to Fluentd.</li> </ul> <p>Enjoy logging!</p> Fluentd v1.2.5 has been released https://www.fluentd.org/blog/fluentd-v1.2.5-has-been-released <p>Hi users!</p> <p>We have released v1.2.5. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release fixed <code>in_tail</code> resource leak.</p> <h3>Major bug fixes</h3> <ul> <li>in_tail: Fix log rotation related resource leak. If you use <code>in_tail</code> with log rotation files, we recommend to update fluentd to this version.</li> </ul> <p>Enjoy logging!</p> 2018-08-23 <p>Hi users!</p> <p>We have released v1.2.5. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release fixed <code>in_tail</code> resource leak.</p> <h3>Major bug fixes</h3> <ul> <li>in_tail: Fix log rotation related resource leak. If you use <code>in_tail</code> with log rotation files, we recommend to update fluentd to this version.</li> </ul> <p>Enjoy logging!</p> Fluentd v1.2.4 has been released https://www.fluentd.org/blog/fluentd-v1.2.4-has-been-released <p>Hi users!</p> <p>We have released v1.2.4. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is for bug fixes.</p> <h3>Major bug fixes</h3> <ul> <li><p>output: Consider timezone with larger timekey. This fixes unexpected buffer flush with <code>1d</code> timekey.</p></li> <li><p>server helper: Fix connection leak by close timing issue</p></li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p> 2018-08-01 <p>Hi users!</p> <p>We have released v1.2.4. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is for bug fixes.</p> <h3>Major bug fixes</h3> <ul> <li><p>output: Consider timezone with larger timekey. This fixes unexpected buffer flush with <code>1d</code> timekey.</p></li> <li><p>server helper: Fix connection leak by close timing issue</p></li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p> Fluentd v1.2.2 has been released https://www.fluentd.org/blog/fluentd-v1.2.2-has-been-released <p>Hi users!</p> <p>We have released v1.2.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is mainly for bug fixes.</p> <h2>filter_parser: Add <code>remove_key_name_field</code> parameter</h2> <p>With <code>reserve_data true</code>, <code>key_name</code> field is kept in the record.</p> <p>Original record:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;log&quot;:&quot;{\&quot;k2\&quot;:\&quot;v2\&quot;}&quot;} </code></pre> <p>Parsed result:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;log&quot;:&quot;{\&quot;k2\&quot;:\&quot;v2\&quot;}&quot;,&quot;k2&quot;:&quot;v2&quot;} </code></pre> <p>But we often don&#39;t need original <code>key_name</code> field when parsing is succeeded. You can remove <code>key_name</code> field with <code>remove_key_name_field true</code>. Parsed result is below:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;k2&quot;:&quot;v2&quot;} </code></pre> <h3>Major bug fixes</h3> <ul> <li>buffer: Wait for all chunks being purged before deleting <code>@queued_num</code> items</li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p> 2018-06-12 <p>Hi users!</p> <p>We have released v1.2.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is mainly for bug fixes.</p> <h2>filter_parser: Add <code>remove_key_name_field</code> parameter</h2> <p>With <code>reserve_data true</code>, <code>key_name</code> field is kept in the record.</p> <p>Original record:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;log&quot;:&quot;{\&quot;k2\&quot;:\&quot;v2\&quot;}&quot;} </code></pre> <p>Parsed result:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;log&quot;:&quot;{\&quot;k2\&quot;:\&quot;v2\&quot;}&quot;,&quot;k2&quot;:&quot;v2&quot;} </code></pre> <p>But we often don&#39;t need original <code>key_name</code> field when parsing is succeeded. You can remove <code>key_name</code> field with <code>remove_key_name_field true</code>. Parsed result is below:</p> <pre><code>{&quot;k1&quot;:&quot;v1&quot;,&quot;k2&quot;:&quot;v2&quot;} </code></pre> <h3>Major bug fixes</h3> <ul> <li>buffer: Wait for all chunks being purged before deleting <code>@queued_num</code> items</li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p> Fluentd v1.2.1 has been released https://www.fluentd.org/blog/fluentd-v1.2.1-has-been-released <p>Hi users!</p> <p>We have released v1.2.1. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is mainly for bug fixes.</p> <h2>Counter API: Add <code>wait</code> to Client</h2> <p>This is handy API for error check. Using <code>get</code>, you need to check error response from the response body. <code>wait</code> raises an exception when API call has a problem.</p> <pre><code>require &#39;fluent/plugin/filter&#39; module Fluent module Plugin class CounterFilter &lt; Filter Plugin.register_filter(&#39;counter&#39;, self) helpers :counter def start super @client = counter_client_create(scope: &#39;test&#39;) @client.establish(&#39;counter&#39;) # if init call returns an error, exception is raised begin @client.init(:name =&gt; &#39;num&#39;, type: &#39;numeric&#39;, reset_interval: 10).wait rescue Fluent::Counter::BaseError # process client specific error end end # ... end end end </code></pre> <h3>Major bug fixes</h3> <ul> <li>in<em>tcp/in</em>udp: Fix source<em>hostname</em>key to set hostname correctly</li> <li>out_file: Temporal fix for broken gzipped files with <code>gzip</code> and <code>append true</code></li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p> 2018-05-23 <p>Hi users!</p> <p>We have released v1.2.1. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is mainly for bug fixes.</p> <h2>Counter API: Add <code>wait</code> to Client</h2> <p>This is handy API for error check. Using <code>get</code>, you need to check error response from the response body. <code>wait</code> raises an exception when API call has a problem.</p> <pre><code>require &#39;fluent/plugin/filter&#39; module Fluent module Plugin class CounterFilter &lt; Filter Plugin.register_filter(&#39;counter&#39;, self) helpers :counter def start super @client = counter_client_create(scope: &#39;test&#39;) @client.establish(&#39;counter&#39;) # if init call returns an error, exception is raised begin @client.init(:name =&gt; &#39;num&#39;, type: &#39;numeric&#39;, reset_interval: 10).wait rescue Fluent::Counter::BaseError # process client specific error end end # ... end end end </code></pre> <h3>Major bug fixes</h3> <ul> <li>in<em>tcp/in</em>udp: Fix source<em>hostname</em>key to set hostname correctly</li> <li>out_file: Temporal fix for broken gzipped files with <code>gzip</code> and <code>append true</code></li> </ul> <p>Thanks for submitting bug reports and patches :)</p> <p>Enjoy logging!</p>