Hi users!
We have released td-agent v4.3.1. td-agent is a stable distribution package of Fluentd.
Please see ChangeLog for more detail.
Please see the download page.
Although Ubuntu 22.04 LTS (Jammy Jellyfish) is already released, we don't yet provide the official td-agent package for it. An experimental package which bundles Ruby 3.1 is work in progress.
Hi users!
We have released v1.14.6. ChangeLog is here.
This release is a maintenance release of v1.14 series.
SO_LINGER
In this release, we add a new option linger_timeout
to server
plugin-helper
so that we can specify SO_LINGER
socket-option when using TCP or TLS server function of the helper.
The helper has used 0
for linger_timeout
internally before. This made the connection closing immediate,
and especially on non-Windows, this made the server send RST rather than FIN on the closing.
The main use of this option is to make the server send FIN on closing on non-Windows,
by setting a positive value such as 1
.
This new option is in transport
section, and we can specify this for plugins
using TCP or TLS server function of the helper, such as in_tcp
, in_http
,
in_syslog
, and in_forward
.
<source>
@type tcp
...
<parse>
...
</parse>
<transport tcp>
linger_timeout 1
</transport>
</source>
Note: in_forward
plugin had linger_timeout
option before, but the option is now deprecated and unified to transport
section.
See #3644 for more information.
--umask
to fluentd
command line parameterWhen we launch Fluentd with --no-supervisor
, the umask value of the environment
is not applied because Fluentd overwrites the value.
In this release, We can use --umask
option as a fluentd
command line parameter
to specify the umask value in the case of launching Fluentd with --no-supervisor
.
$ fluentd -c {config_path} --no-supervisor --umask 002
See #3671, #3679 for more information.
There were mistakes in the calculation about exponential_backoff
retry_type.
In this release, the calculation is fixed, and the count and the interval change as following.
Count
retry_max_times + 1
retries were triggered.retry_max_times
retries are triggered.Interval
When to switch to secondary
retry_max_times
+ 1 retries were mistakenly triggered,
retry_secondary_threshold
was applied to the total time to the (retry_max_times
)-th retry. So switching to
secondary occurred earlier than the correct time.retry_secondary_threshold
is correctly applied to the total retry time.The examples of retry_max_times: 10
are as follows. (Set retry_randomize: false
for testing.)
Only primary output
N-th retry | Elapsed (Before) | Elapsed (This Release) |
---|---|---|
1th | 1s | 1s |
2th | 2s | 3s |
3th | 4s | 7s |
4th | 8s | 15s |
5th | 16s | 31s |
6th | 32s | 63s |
7th | 64s | 127s |
8th | 128s | 255s |
9th | 256s | 511s |
10th | 512s | 1023s |
11th | 1024s | None |
With secondary output
N-th retry | Elapsed (Before) | Elapsed (This Release) |
---|---|---|
1th | 1s | 1s |
2th | 2s | 3s |
3th | 4s | 7s |
4th | 8s | 15s |
5th | 16s | 31s |
6th | 32s | 63s |
7th | 64s | 127s |
8th | 128s | 255s |
9th | 256s | 511s |
10th | (SEC) 409s | (SEC) 818s |
11th | (SEC) 410s | None |
See #3649 for more information.
@ERROR
labelFilter
plugins such as filter_record_transformer
can modify the contents of records
before the output.
If an error occurred in Output
plugin such that records were passed to @ERROR
label,
those modifications were not applied to those records being passed.
In this release, records reflecting those modifications are now passed to @ERROR
label.
See #3631 for more information.
rpc_endpoint
in system
config. #3641.null_value_pattern
as regexp
. #3650.CHANGELOG.md
heading styles. #3648.Enjoy logging!
Hi users!
We have released v1.14.5. ChangeLog is here.
This release is a maintenance release of v1.14 series.
in_http
: Add support for "application/x-ndjson"In this release, in_http
supports the new Content-Type "application/x-ndjson".
Its format is "ndjson", which is basically a list of JSON objects separated by "\n".
Here is an example of the format.
{"foo": "bar"}
{"buz": "hoge"}
See #3616 for more information.
RubyInstaller 3.1 has switched C-Runtime from MSVCRT to UCRT, and the ruby and
gem platform has changed to x64-mingw-ucrt
.
There was a problem in this environment that Windows-specific dependent gems
could not be installed because Fluentd for this architecture had not been released.
Beginning with this release, Fluentd for this architecture is also released and this problem is solved.
See #3613 for more information.
out_forward
: Fix hang-up issue during TLS handshakeThere was a known issue that TLS handshake takes a long time or hangs when
out_forward
uses TLS transport, even though connect_timeout
is specified.
With this release, connect_timeout
is now reliably applied to TLS transport
and able to prevent hangs.
See #3601 for more information.
retry_max_times
is 0
. #3608Enjoy logging!
Hi users!
We have released v1.14.4. ChangeLog is here.
This release is a maintenance release of v1.14 series.
in_tail
: Add max_line_size
option to skip long linesStarting from v1.14.4, in_tail
supports a new option max_line_size
that
allows to skip lines above a certain size.
<source>
@type tail
path /var/log/nginx/*.log
...
max_line_size 8MB # skip very long lines
...
</source>
This option can be used to ensure that messages coming from in_tail
is
within chunk_limit_size
, and will eliminate costly error handling on very
large messages.
BufferChunkOverflowError
handlingBeginning with this release, Fluentd will no longer raise
BufferChunkOverflowError
on an oversized event stream if each event in the
stream is smaller than chunk_limit_size
Example
es = Fluent::ArrayEventStream.new([[timestamp, {"message" => "a" * 1_000_000}],
[timestamp, {"message" => "b" * 1_000_000}],
[timestamp, {"message" => "c" * 1_000_000}]])
# Previously this could result in BufferChunkOverflowError even
# when chunk_limit_size is large enough (e.g. 1.2MB).
router.emit_stream(@tag, es)
See #3560 for more information.
out_file
: Fix inccorect append writes on macOSPrior to this release, out_file
failed to write to files if append
option was
enabled on macOS with Ruby 2.7/3.0.
This issue was due to a bug in Ruby's IO implementation. This release contains a workaround for that bug.
See #3579 for details.
Enjoy logging!
Hi users and developers!
We announce the dropping schedule for TD Agent 3 development.
Since td-agent 3.0.0 was released in 2017, td-agent 3 had been maintained for a long time.
As new major version of td-agent 4 was released in August 2020, and several updates was shipped, we decided to stop maintaining td-agent 3.
As you know, already stepping down maintenance activity, new minor update for td-agent 3 will not be shipped anymore.
Thus, we recommend to use td-agent 4, latest fluentd v1 series, for new deployment :)
There is a good article to do it.
Follow the above instructions.
Fluentd is an open source data collector to simplify log management.
2022-04-29: td-agent v4.3.1 has been released
2022-03-31: Fluentd v1.14.6 has been released
2022-02-09: Fluentd v1.14.5 has been released
2022-01-06: Fluentd v1.14.4 has been released
2021-12-14: Treasure Agent (td-agent) 3 will not be maintained anymore
Want to learn the basics of Fluentd? Check out these pages.
Couldn't find enough information? Let's ask the community!
You need commercial-grade support from Fluentd committers and experts?
©2010-2022 Fluentd Project. ALL Rights Reserved.
Fluentd is a hosted project under the Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.