Open Source Summit Japan 2023

Hi users and developers!

We would like to share the information about an event appearance of Fluentd maintainers.

Daijiro Fukuda and Kentaro Hayashi, Fluentd Maintainers from ClearCode Inc., are going to talk at the Open Source Summit Japan on December 5th

The event has been taken thousands of attendees all over the world and one of the leading Open Source event in Japan. We are honored to take this opportunity and very excited to share about Fluentd project at the conference!

Everyone attending the event, please join us!

About the Talk

Date: December 5th 14:00-14:40
Speakers: Daijiro Fukuda and Kentaro Hayashi from ClearCode Inc.
Summary:
Fluentd, the CNCF graduated project, starts a new chapter as the OSS project. Fluentd community released the new package fluent-package as the successor of td-agent and started providing a new release cycle to fit better with needs. This talk is going to share the latest news of Fluentd, including its updates, package, rebranding and so on, with people who are using Fluentd or struggling to manage their logs. Fluentd is an open source data collector that unifies data collection and consumption. Because of its flexible plugin system and scalability, Fluentd is adopted in various systems and has become a de facto standard, especially for scale logging in distributed systems.

In the new release cycle, the package applies only bug fixes and security fixes of Fluentd over a long term, about two years. We are confident that this will meet the demands of users who want to use Fluentd stably over the long term. In this session, we will talk about (a) the history of Fluentd v1 and td-agent v4, (b) why Fluentd community decided to rebrand existing td-agent to fluent-package, (c) what the benefits and impacts of the new release cycle are, and (d) how to upgrade to fluent-package from td-agent.

Detail info: Link to event page

Overall Event Information

Name: Open Source Summit Japan 2023
Date: 2023-12-04 to 2023-12-06
Host: The Linux Foundation
Description:

Open Source Summit Japan is the leading conference in Japan connecting the open source ecosystem under one roof, providing a forum for technologists and open source industry leaders to collaborate and share information, learn about the latest in open source technologies and find out how to gain a competitive advantage by using innovative open solutions.

Open Source Summit is a conference umbrella, composed of a collection of events covering the most important technologies, topics, and issues affecting open source today.

Quote from the event web page : https://events.linuxfoundation.org/open-source-summit-japan/

Read More

fluent-package v5.0.2 has been released

Hi users!

We have released fluent-package v5.0.2 and td-agent v4.5.2. fluent-package is a stable distribution package of Fluentd.

This is a maintenance release of v5.0.x LTS series. As significant in_tail bugs (wrongly stopping tailing logs) were fixed in latest release, we recommend upgrading to fluent-package v5.0.2!

Changes from fluent-package v5.0.1

  • Update fluentd to 1.16.3 which contains significant bug fixes about in_tail. See Fluentd v1.16.3 and v1.16.2 have been released blog article about details.
  • Update plugins
    • fluent-diagtool v1.0.3. It supports fluent-package and can collect information about locally installed gems. It may help to migrate from td-agent v4 a bit. See Upgrade to fluent-package v5 for migration.
  • msi: support path which contains space or parenthesis (#589)
  • deb: fixed system user/group name in logrotate config (#592,#594)
    • It fixes a bug that unknown user error was reported during log rotation.
  • rpm: fixed to create fluentd user as system account (#596)
    • It fixes a bug that /var/lib/fluent directory was created unexpectedly. It doesn't affect the fluentd service behavior, but it is desirable one.
  • rpm: changed to keep system account after removing fluent-package. (#598)
    • In the previous versions, there was a bug that group was not cleanly removed when the package was upgraded from td-agent v4. This change makes reinstall/downgrade friendly.

About next LTS schedule

We plan to ship next LTS version of fluent-package v5.0.3 on Feb, 2024. The content of updates are still in T.B.D.

About td-agent v4.5.2

This is a exceptional maintenance release of v4.5.x series. Fluentd was updated to 1.16.3 because it contains significant bug fixes about in_tail. Note that td-agent will not be updated anymore. See Drop schedule announcement about EOL of Treasure Agent (td-agent) 4.

Download

Please see the download page.

Read More

Fluentd v1.16.3 and v1.16.2 have been released

Hi users!

We have released v1.16.3 (changelog) on 2023-11-14. We also describe the content of v1.16.2 (changelog) on 2023-07-14 here.

In these releases, in_tail issues that have been reported for years are fixed! And other some bug fixes too.

Bug Fixes

in_tail: Fix in_tail wrongly stopping tailing the current target file and causing handle leaks

This issue has been reported for years. It was looking very serious, but the cause remained unclear until recently.

At last, we found different causes when the follow_inodes option is true and when it is false (default). We fixed the problem of follow_inodes true case in v1.16.2, and fixed the problem of follow_inodes false case in v1.16.3.

Especially, in follow_inodes false case, the trouble was mainly caused by rotate_wait option. In past versions, please avoid using large value for this option. In particular, if you make it greater than refresh_interval option, it will surely cause this issue.

In Fluentd v1.15.1 (td-agent v4.4.0) or later, the following warning log is recorded when this issue occurs.

Skip update_watcher because watcher has been already updated by other inotify event

If this issue occurs, please restart Fluentd, and please update Fluentd to the latest version. fluent-package v5.0.2 will be released in this month and it contains these fixes.

See the following PRs for more information.

  • follow_inodes true case: #4208
  • follow_inodes false case: #4327

Others

Enjoy logging!

Read More

Drop schedule announcement about EOL of Treasure Agent (td-agent) 4

Hi users and developers!

We announce the dropping schedule for TD Agent 4 development.

About Treasure Agent (td-agent) 4

Since td-agent 4.0.0 was released in 2020, td-agent 4 has been maintained for a long time.

As new major version of td-agent 4 was rebranded (to fluent-package v5), and released in August 2023, we decided to stop maintaining td-agent 4.

Scheduled end of life - Dec 2023

As you know, already stepping down maintenance activity, new minor update for td-agent 4 will not be shipped anymore. (td-agent v4.5.1 will be the last release of this series, it will be shipped in this month.)

Thus, we recommend using fluent-package v5 for new deployment :)

How to migrate to Fluent Package v5

There is a good article to do it.

Follow the above instructions.

Read More

Scheduled support lifecycle announcement about Fluent Package

Hi users!

We have released fluent-package v5.0.0 (Release Candidate) and the following v5.0.1 (General Availability). fluent-package is formerly known as td-agent and a stable distribution package of Fluentd.

In this blog article, we explains why the concept of channels is introduced.

Why is LTS (Long Term Support) channel introduced?

In the previous versions, td-agent was shipped as quarterly releases in most cases. It sometimes contains fluentd minor upgrade occasionally, which means that not only bug or security fixes, but also contains new features. As a result, there was a case that the stability was affected unexpectedly.

In point view of adopting fluentd (and td-agent) for enterprise services, there was a requirement for the distribution which contains security or bug fixes only and scheduled lifecycle of it.

The LTS release is the answer to support them.

Channels for updates

There are two channels of support lifecycle.

  • Normal release
  • LTS (Long Term Support)

Here is the difference of these channels.

  • Normal release (5.x series)
    • Recommended for early adopters (eager to try new features of Fluentd)
    • Rapid release development (planned semi-annually releases, but it may vary.)
    • Fluentd will be upgraded occasionally
      • Minor upgrade will be applied. e.g. 1.16 => 1.17 and so on.
      • e.g. v5.0.1 (Fluentd v1.16.2), v5.0.2 (Fluentd v1.16.3), ... v5.1.0 (Fluentd v1.17.0)
    • The only latest version will be supported (bug fix will not be backported into older version)
  • LTS (Long Term Support, 5.0.x series)
    • Recommended for enterprise services
    • Security and bug fix release only
    • Fluentd will be upgraded only in the teeny version and will stick to 1.16.x series.
      • Thus, the version number will be 5.0.x. (5.1 series will not be shipped)
      • e.g. v5.0.1 (Fluentd v1.16.2), v5.0.2 (Fluentd v1.16.3), v5.0.3 (Fluentd v1.16.4) ... v5.0.x (Fluentd v1.16.x)
    • Next major version will be fluent-package v6.0.0, and will be shipped in 2025.

As for fluent-package v5,

  • fluent-package v5 LTS will be supported until March, 2025 (Even though Ruby 3.2 is still supported)
  • Until 5.1.0 is released, normal release channel and LTS channel are the same package.

Current releases and future plans:

Release Version Release Date End of life
v4.5.1 Aug, 2023 Dec, 2023
v5.0.1 (LTS) Aug 29, 2023 March, 2025
v6.0.0 (LTS) March, 2025? (T.B.D.) T.B.D.

Happy logging!

Read More

About Fluentd

Fluentd is an open source data collector to simplify log management.

Learn

Want to learn the basics of Fluentd? Check out these pages.

Ask the Community

Couldn't find enough information? Let's ask the community!

Ask the Experts

You need commercial-grade support from Fluentd committers and experts?

Follow Us!