Fluentd Blog https://www.fluentd.org/blog Get the Latest Updates on Fluentd and More Fluentd v1.5.2 has been released https://www.fluentd.org/blog/fluentd-v1.5.2-has-been-released <p>Hi users!</p> <p>We have released v1.5.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is for fixing <code>out_forward</code> keepalive issue. If you want to use <code>keepalive</code> and <code>&lt;security&gt;</code>, update fluentd to v1.5.2.</p> <p>Enjoy logging!</p> 2019-06-14 <p>Hi users!</p> <p>We have released v1.5.2. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release is for fixing <code>out_forward</code> keepalive issue. If you want to use <code>keepalive</code> and <code>&lt;security&gt;</code>, update fluentd to v1.5.2.</p> <p>Enjoy logging!</p> Fluentd v1.5.1 has been released https://www.fluentd.org/blog/fluentd-v1.5.1-has-been-released <p>Hi users!</p> <p>We have released v1.5.1. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release includes several enhancements and bug fixes.</p> <h3>in_tail: Increase read block size to reduce IO call</h3> <p>We changed read block size from 2048 to 8192. 2048 is small for large logs. <code>in_tail</code> uses <code>IO#readpartial</code> method and the overhead of this method is bit higher. So decreasing the number of method call reduces CPU usage.</p> <h3>Major bug fixes</h3> <ul> <li>out_forward: Fix the socket handling of keepalive</li> <li>parser: Fix the use of name based timezone</li> <li>in_monitor_agent: Fix debug parameter handling</li> </ul> <p>Enjoy logging!</p> 2019-06-05 <p>Hi users!</p> <p>We have released v1.5.1. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release includes several enhancements and bug fixes.</p> <h3>in_tail: Increase read block size to reduce IO call</h3> <p>We changed read block size from 2048 to 8192. 2048 is small for large logs. <code>in_tail</code> uses <code>IO#readpartial</code> method and the overhead of this method is bit higher. So decreasing the number of method call reduces CPU usage.</p> <h3>Major bug fixes</h3> <ul> <li>out_forward: Fix the socket handling of keepalive</li> <li>parser: Fix the use of name based timezone</li> <li>in_monitor_agent: Fix debug parameter handling</li> </ul> <p>Enjoy logging!</p> Fluentd v1.5.0 has been released https://www.fluentd.org/blog/fluentd-v1.5.0-has-been-released <p>Hi users!</p> <p>We have released v1.5.0. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release includes nice features and improvements.</p> <h3>out_forward: Support keepalive feature</h3> <pre><code>&lt;source&gt; @type tail # ... other parameters &lt;/source&gt; &lt;match pattern&gt; @type forward # ... other parameters keepalive true keepalive_timeout 60s # optional. Default is no timeout &lt;/match&gt; </code></pre> <p><code>forward</code> output plugin now supports <code>keepalive</code>/<code>keepalive_timeout</code> parameters. This feature is useful to reduce the connection cost per flush. In addition, this feature is also good for load balancer situation.</p> <h3>in_http/in_syslog: Support TLS transport</h3> <p>These plugins now support <code>&lt;transport tls&gt;</code> configuration. Here is <code>syslog</code> input / rsyslog config example for TLS:</p> <pre><code># fluentd &lt;source&gt; @type syslog port 5140 bind 0.0.0.0 &lt;transport tls&gt; ca_path /etc/pki/ca.pem cert_path /etc/pki/cert.pem private_key_path /etc/pki/key.pem private_key_passphrase PASSPHRASE &lt;/transport&gt; tag system &lt;/source&gt; # rsyslog $DefaultNetstreamDriverCAFile /etc/pki/ca.pem $DefaultNetstreamDriver gtls $ActionSendStreamDriverMode 1 $ActionSendStreamDriverAuthMode anon *.* @@127.0.0.1:5140 </code></pre> <p><code>http</code> input supports same parameters with <code>&lt;transport tls&gt;</code>.</p> <h3>in_forward: Add tag/add_tag_prefix parameters</h3> <p>You can change incoming event tag in <code>forward</code> input without additional plugins.</p> <pre><code>&lt;source&gt; @type forward port 24230 tag system1.log # change tag to system1.log &lt;/source&gt; &lt;source&gt; @type forward port 24240 add_tag_prefix system2 # add prefix to incoming tag, e.g. system2.app.log &lt;/source&gt; </code></pre> <p>For event routing, we can use <code>label</code> feature but it can&#39;t modify event tag. These parameters are useful when you want to organize incoming <code>tag</code> in the aggregator.</p> <h3>Major bug fixes</h3> <ul> <li>out_forward: Don&#39;t use <code>SO_LINGER</code> on SSL/TLS WinSock.</li> <li>server: Fix recursive lock issue in TLSServer</li> </ul> <p>Enjoy logging!</p> 2019-05-19 <p>Hi users!</p> <p>We have released v1.5.0. ChangeLog is <a href="https://github.com/fluent/fluentd/blob/master/CHANGELOG.md">here</a>. This release includes nice features and improvements.</p> <h3>out_forward: Support keepalive feature</h3> <pre><code>&lt;source&gt; @type tail # ... other parameters &lt;/source&gt; &lt;match pattern&gt; @type forward # ... other parameters keepalive true keepalive_timeout 60s # optional. Default is no timeout &lt;/match&gt; </code></pre> <p><code>forward</code> output plugin now supports <code>keepalive</code>/<code>keepalive_timeout</code> parameters. This feature is useful to reduce the connection cost per flush. In addition, this feature is also good for load balancer situation.</p> <h3>in_http/in_syslog: Support TLS transport</h3> <p>These plugins now support <code>&lt;transport tls&gt;</code> configuration. Here is <code>syslog</code> input / rsyslog config example for TLS:</p> <pre><code># fluentd &lt;source&gt; @type syslog port 5140 bind 0.0.0.0 &lt;transport tls&gt; ca_path /etc/pki/ca.pem cert_path /etc/pki/cert.pem private_key_path /etc/pki/key.pem private_key_passphrase PASSPHRASE &lt;/transport&gt; tag system &lt;/source&gt; # rsyslog $DefaultNetstreamDriverCAFile /etc/pki/ca.pem $DefaultNetstreamDriver gtls $ActionSendStreamDriverMode 1 $ActionSendStreamDriverAuthMode anon *.* @@127.0.0.1:5140 </code></pre> <p><code>http</code> input supports same parameters with <code>&lt;transport tls&gt;</code>.</p> <h3>in_forward: Add tag/add_tag_prefix parameters</h3> <p>You can change incoming event tag in <code>forward</code> input without additional plugins.</p> <pre><code>&lt;source&gt; @type forward port 24230 tag system1.log # change tag to system1.log &lt;/source&gt; &lt;source&gt; @type forward port 24240 add_tag_prefix system2 # add prefix to incoming tag, e.g. system2.app.log &lt;/source&gt; </code></pre> <p>For event routing, we can use <code>label</code> feature but it can&#39;t modify event tag. These parameters are useful when you want to organize incoming <code>tag</code> in the aggregator.</p> <h3>Major bug fixes</h3> <ul> <li>out_forward: Don&#39;t use <code>SO_LINGER</code> on SSL/TLS WinSock.</li> <li>server: Fix recursive lock issue in TLSServer</li> </ul> <p>Enjoy logging!</p> Fluentd documentation is now hosted on gitbook https://www.fluentd.org/blog/fluentd-docs-renewal <p>Today, we have updated <a href="https://docs.fluentd.org">docs.fluentd.org</a> to use gitbook. This change has several merits:</p> <ul> <li>Better Look and Feel (same as fluent-bit document)</li> <li>Use popular Markdown for the content</li> <li>Reduce maintenance cost</li> </ul> <p><a href="https://github.com/fluent/fluentd-docs-gitbook">fluentd-docs-gitbook</a> is new repository and old <a href="https://github.com/fluent/fluentd-docs">fluentd-docs</a> repository is now deprecated and archived.</p> <p>If you find something wrong on docs.fluentd.org, use fluentd-docs-gitbook for issues/PRs.</p> 2019-05-17 <p>Today, we have updated <a href="https://docs.fluentd.org">docs.fluentd.org</a> to use gitbook. This change has several merits:</p> <ul> <li>Better Look and Feel (same as fluent-bit document)</li> <li>Use popular Markdown for the content</li> <li>Reduce maintenance cost</li> </ul> <p><a href="https://github.com/fluent/fluentd-docs-gitbook">fluentd-docs-gitbook</a> is new repository and old <a href="https://github.com/fluent/fluentd-docs">fluentd-docs</a> repository is now deprecated and archived.</p> <p>If you find something wrong on docs.fluentd.org, use fluentd-docs-gitbook for issues/PRs.</p> Fluentd has Graduated! https://www.fluentd.org/blog/fluentd-cncf-graduation <p>We are thrilled to announce that <a href="https://www.fluentd.org">Fluentd</a> has become the sixth <a href="https://www.cncf.io/announcement/2019/04/11/cncf-announces-fluentd-graduation">CNCF Graduated Project</a>! This is a huge recognition to the project and its community. Fluentd wide adoption by the Enterprise and contributions are in a continuous grow: we have more than 900 contributed plugins and more than 100k downloads a day!</p> <p><em>Fluentd was created in 2011 by <a href="http://github.com/frsyuki">Sadayuki “Sada” Furuhashi</a>, co-founder of <a href="https://www.treasuredata.com/">Treasure Data, Inc.</a>, as an open source data collector for building a <a href="https://www.fluentd.org/blog/unified-logging-layer">Unified Logging Layer</a>, which unifies the data collection and consumption for better use and understanding of data. In November 2016, Fluentd was accepted as CNCF’s sixth hosted project after Kubernetes, Prometheus, and OpenTracing.</em></p> <p><a href="https://www.cncf.io/announcement/2019/04/11/cncf-announces-fluentd-graduation/"> <img src="/images/blog/fluentd_graduation.jpg" width="50%" height="50%"/> </a></p> <p>As of today, <a href="https://fluentd.org">Fluentd</a> is used by major companies to solve log collection and processing at scale: Microsoft, Redhat, and Google are a few names of its known users.</p> <p>Fluentd&#39;s core maintainer, <a href="https://github.com/repeatedly">Masahiro Nakagawa</a> (well known as <a href="https://twitter.com/repeatedly">@repeatedly</a> on Twitter), said that he&#39;s thrilled to have had the opportunity to collaborate with the CNCF community and to grow its user base, and it&#39;s an honor to see Fluentd graduated. Today we celebrate this big accomplishment, and we continue looking forward to improving on upcoming Logging and data processing challenges.</p> <p><strong>Fluentd at Conferences on 2019</strong></p> <p>Fluentd team will be attending some conferences this year, save the date!</p> <ul> <li><a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019">KubeCon Europe, Barcelona 2019. May 20 - 23</a></li> <li><a href="https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2019">KubeCon China, Shanghai 2019, June 25 - 26</a></li> <li>Fluentd Meetup!, Tokyo Japan 2019, July (date to be defined)</li> <li><a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2019">KubeCon NorthAmerica, San Diego California - US, November 18 - 21</a></li> </ul> 2019-04-11 <p>We are thrilled to announce that <a href="https://www.fluentd.org">Fluentd</a> has become the sixth <a href="https://www.cncf.io/announcement/2019/04/11/cncf-announces-fluentd-graduation">CNCF Graduated Project</a>! This is a huge recognition to the project and its community. Fluentd wide adoption by the Enterprise and contributions are in a continuous grow: we have more than 900 contributed plugins and more than 100k downloads a day!</p> <p><em>Fluentd was created in 2011 by <a href="http://github.com/frsyuki">Sadayuki “Sada” Furuhashi</a>, co-founder of <a href="https://www.treasuredata.com/">Treasure Data, Inc.</a>, as an open source data collector for building a <a href="https://www.fluentd.org/blog/unified-logging-layer">Unified Logging Layer</a>, which unifies the data collection and consumption for better use and understanding of data. In November 2016, Fluentd was accepted as CNCF’s sixth hosted project after Kubernetes, Prometheus, and OpenTracing.</em></p> <p><a href="https://www.cncf.io/announcement/2019/04/11/cncf-announces-fluentd-graduation/"> <img src="/images/blog/fluentd_graduation.jpg" width="50%" height="50%"/> </a></p> <p>As of today, <a href="https://fluentd.org">Fluentd</a> is used by major companies to solve log collection and processing at scale: Microsoft, Redhat, and Google are a few names of its known users.</p> <p>Fluentd&#39;s core maintainer, <a href="https://github.com/repeatedly">Masahiro Nakagawa</a> (well known as <a href="https://twitter.com/repeatedly">@repeatedly</a> on Twitter), said that he&#39;s thrilled to have had the opportunity to collaborate with the CNCF community and to grow its user base, and it&#39;s an honor to see Fluentd graduated. Today we celebrate this big accomplishment, and we continue looking forward to improving on upcoming Logging and data processing challenges.</p> <p><strong>Fluentd at Conferences on 2019</strong></p> <p>Fluentd team will be attending some conferences this year, save the date!</p> <ul> <li><a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019">KubeCon Europe, Barcelona 2019. May 20 - 23</a></li> <li><a href="https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2019">KubeCon China, Shanghai 2019, June 25 - 26</a></li> <li>Fluentd Meetup!, Tokyo Japan 2019, July (date to be defined)</li> <li><a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2019">KubeCon NorthAmerica, San Diego California - US, November 18 - 21</a></li> </ul>