Parser Plugins
Last updated
Was this helpful?
Last updated
Was this helpful?
Fluentd has nine (9) types of plugins:
This article gives an overview of the Parser Plugin.
Sometimes, the <parse>
directive for input plugins (e.g. , , and ) cannot parse the user's custom data format (for example, a context-dependent grammar that can't be parsed with a regular expression). To address such cases, Fluentd has a pluggable system that enables the user to create their own parser formats.
Write a custom format plugin. See
for more information.
From any input plugin that supports the <parse>
directive, call the custom
plugin by its name.
Here is an example to read Nginx access logs using in_tail
and parser_nginx
:
Note: When td-agent
is launched by systemd, the default user of the td-agent
process is the td-agent
user.
You must ensure that this user has read permission to the tailed /path/to/file
. For instance, on Ubuntu,
the default Nginx access file /var/log/nginx/access.log
is mode 0640
and owned by www-data:adm
. In
this case, several options are available to allow read access:
Add the td-agent
user to the adm
group, e.g. through usermod -aG
, or
If you are familiar with grok
patterns, grok-parser
plugin is useful. Use > 1.0.0
versions for fluentd
v0.14/v1.0.
If you need to parse multiple formats in one data stream, multi-format-parser
is useful.
For protocol buffers.
For Apache Avro.
Following plugins support <parse>
directive:
Use the to allow the invoking user to read the file without otherwise changing its permission bits or ownership.
If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.