mirror of
https://github.com/rsyslog/rsyslog.git
synced 2025-12-22 06:00:43 +01:00
188 lines
11 KiB
HTML
188 lines
11 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<html><head><title>Modules - rsyslog.conf</title></head>
|
|
<body>
|
|
<p>This is a part of the rsyslog.conf documentation.</p>
|
|
<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a>
|
|
<h1>Modules</h1>
|
|
<p>Rsyslog has a modular design. This enables functionality to be
|
|
dynamically loaded from modules, which may also be written by any
|
|
third party. Rsyslog itself offers all non-core functionality as
|
|
modules. Consequently, there is a growing
|
|
number of modules. Here is the entry point to their documentation and
|
|
what they do (list is currently not complete)</p>
|
|
<p>Please note that each module provides configuration
|
|
directives, which are NOT necessarily being listed below. Also
|
|
remember, that a modules configuration directive (and functionality) is
|
|
only available if it has been loaded (using $ModLoad).</p>
|
|
<p>It is relatively easy to write a rsyslog module. <b>If none of the provided
|
|
modules solve your need, you may consider writing one or have one written
|
|
for you by
|
|
<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a>
|
|
</b>(this often is a very cost-effective and efficient way of getting what you need).
|
|
<p>There exist different classes of loadable modules:
|
|
<ul>
|
|
<li><a href="rsyslog_conf_modules.html#im">Input Modules</a>
|
|
<li><a href="rsyslog_conf_modules.html#om">Output Modules</a>
|
|
<li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a>
|
|
<li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a>
|
|
<li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a>
|
|
<li><a href="rsyslog_conf_modules.html#lm">Library Modules</a>
|
|
</ul>
|
|
|
|
<a name"im"></a><h2>Input Modules</h2>
|
|
<p>Input modules are used to gather messages from various sources. They interface
|
|
to message generators.
|
|
<ul>
|
|
<li><a href="imfile.html">imfile</a> - input module for text files</li>
|
|
<li><a href="imrelp.html">imrelp</a> - RELP input module</li>
|
|
<li>imudp - udp syslog message input</li>
|
|
<li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li>
|
|
<li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li>
|
|
<li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li>
|
|
<li>immark - support for mark messages</li>
|
|
<li><a href="imklog.html">imklog</a> - kernel logging</li>
|
|
<li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li>
|
|
<li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li>
|
|
<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li>
|
|
<li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li>
|
|
</ul>
|
|
|
|
<a name"om"></a><h2>Output Modules</h2>
|
|
<p>Output modules process messages. With them, message formats can be transformed
|
|
and messages be transmitted to various different targets.
|
|
<ul>
|
|
<li><a href="omfile.html">omfile</a> - file output module</li>
|
|
<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li>
|
|
<li><a href="ompipe.html">ompipe</a> - named pipe output module</li>
|
|
<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li>
|
|
<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li>
|
|
<li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li>
|
|
<li><a href="omrelp.html">omrelp</a> - RELP output module</li>
|
|
<li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li>
|
|
<li>omgssapi - output module for GSS-enabled syslog</li>
|
|
<li><a href="ommysql.html">ommysql</a> - output module for MySQL</li>
|
|
<li>ompgsql - output module for PostgreSQL</li>
|
|
<li><a href="omlibdbi.html">omlibdbi</a> -
|
|
generic database output module (Firebird/Interbase, MS SQL, Sybase,
|
|
SQLLite, Ingres, Oracle, mSQL)</li>
|
|
<li><a href="ommail.html">ommail</a> -
|
|
permits rsyslog to alert folks by mail if something important happens</li>
|
|
<li><a href="omprog.html">omprog</a> - permits sending messages to a program for custom processing</li>
|
|
<li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li>
|
|
<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li>
|
|
<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li>
|
|
<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li>
|
|
</ul>
|
|
|
|
<a name="pm"></a><h2>Parser Modules</h2>
|
|
<p>Parser modules are used to parse message content, once the message has been
|
|
received. They can be used to process custom message formats or invalidly formatted
|
|
messages. For details, please see the <a href="messageparser.html">rsyslog
|
|
message parser documentation</a>.
|
|
<p>The current modules are currently provided as part of rsyslog:
|
|
<ul>
|
|
<li>pmrfc5424[builtin] - rsyslog.rfc5424 -
|
|
parses RFC5424-formatted messages (the new syslog standard)
|
|
<li>pmrfc3164[builtin] - rsyslog.rfc3164 -
|
|
the traditional/legacy syslog parser
|
|
<li>pmrfc3164sd - rsyslog.rfc3164sd -
|
|
a contributed module supporting RFC5424 structured data inside
|
|
RFC3164 messages (not supported by the rsyslog team)
|
|
<li><a href="pmlastmsg.html">pmlastmsg</a> - rsyslog.lastmsg -
|
|
a parser module that handles the typically malformed "last messages
|
|
repated n times" messages emitted by some syslogds.
|
|
</ul>
|
|
|
|
<a name="mm"></a><h2>Message Modification Modules</h2>
|
|
<p>Message modification modules are used to change the content of messages being processed.
|
|
They can be implemented using either the output module or the parser module interface.
|
|
From the rsyslog core's point of view, they actually are output or parser modules, it is their
|
|
implementation that makes them special.
|
|
<p>Currently, there exists only a limited set of such modules, but new ones could be written with
|
|
the methods the engine provides. They could be used, for example, to:
|
|
<ul>
|
|
<li>anonymize message content
|
|
<li>add dynamically computed content to message (fields)
|
|
</ul>
|
|
<p>Message modification modules are usually written for one specific task and thus
|
|
usually are not generic enough to be reused. However, existing module's code is
|
|
probably an excellent starting base for writing a new module. Currently, the following
|
|
modules exist inside the source tree:
|
|
<ul>
|
|
<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages.
|
|
Note that this actually is a <b>generic</b> module.
|
|
<li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside
|
|
the tag to correct the original sender system and priority of messages. Implemented via
|
|
the output module interface.
|
|
</ul>
|
|
|
|
<a name="lm"></a><h2>String Generator Modules</h2>
|
|
<p>String generator modules are used, as the name implies, to generate strings based
|
|
on the message content. They are currently tightly coupled with the template system.
|
|
Their primary use is to speed up template processing by providing a native C
|
|
interface to template generation. These modules exist since 5.5.6. To get an idea
|
|
of the potential speedup, the default file format, when generated by a string generator,
|
|
provides a roughly 5% speedup. For more complex strings, especially those that include
|
|
multiple regular expressions, the speedup may be considerably higher.
|
|
<p>String generator modules are written to a quite simple interface. However, a word of
|
|
caution is due: they access the rsyslog message object via a low-level interface.
|
|
That interface is not guaranteed yet to stay stable. So it may be necessary to
|
|
modify string generator modules if the interface changes. Obviously, we will not do that
|
|
without good reason, but it may happen.
|
|
<p>Rsyslog comes with a set of core, build-in string generators, which are used
|
|
to provide those default templates that we consider to be time-critical:
|
|
<ul>
|
|
<li>smfile - the default rsyslog file format
|
|
<li>smfwd - the default rsyslog (network) forwarding format
|
|
<li>smtradfile - the traditional syslog file format
|
|
<li>smfwd - the traditional syslog (network) forwarding format
|
|
</ul>
|
|
<p>Note that when you replace these defaults be some custom strings, you will
|
|
loose some performance (around 5%). For typical systems, this is not really relevant.
|
|
But for a high-performance systems, it may be very relevant. To solve that issue, create
|
|
a new string generator module for your custom format, starting out from one of the
|
|
default generators provided. If you can not do this yourself, you may want to
|
|
contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development
|
|
of string generators at a very low price.
|
|
<p>Note that string generator modules can be dynamically loaded. However, the default
|
|
ones provided are so important that they are build right into the executable. But this
|
|
does not need to be done that way (and it is straightforward to do it dynamic).
|
|
|
|
|
|
<a name="lm"></a><h2>Library Modules</h2>
|
|
<p>Library modules provide dynamically loadable functionality for parts of rsyslog,
|
|
most often for other loadable modules. They can not be user-configured and are loaded
|
|
automatically by some components. They are just mentioned so that error messages that
|
|
point to library moduls can be understood. No module list is provided.
|
|
|
|
<h2>Where are the modules integrated into the Message Flow?</h2>
|
|
<p>Depending on their module type, modules may access and/or modify messages at
|
|
various stages during rsyslog's processing. Note that only the "core type" (e.g. input,
|
|
output) but not any type derived from it (message modification module) specifies when
|
|
a module is called.
|
|
<p>The simplified workflow is as follows:
|
|
<p align="center">
|
|
<img src="module_workflow.png" alt"rsyslog: loadable modules and message flow">
|
|
<p>As can be seen, messages are received by input modules, then passed to one or many
|
|
parser modules, which generate the in-memory representation of the message and may
|
|
also modify the message itself. The, the internal representation is passed to
|
|
output modules, which may output a message and (with the interfaces newly introduced
|
|
in v5) may also modify messageo object content.
|
|
<p>String generator modules are not included inside this picture, because they are
|
|
not a required part of the workflow. If used, they operate "in front of" the
|
|
output modules, because they are called during template generation.
|
|
<p>Note that the actual flow is much more complex and depends a lot on queue and
|
|
filter settings. This graphic above is a high-level message flow diagram.
|
|
|
|
<p>[<a href="manual.html">manual index</a>]
|
|
[<a href="rsyslog_conf.html">rsyslog.conf</a>]
|
|
[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
|
|
<p><font size="2">This documentation is part of the
|
|
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
|
|
Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
|
|
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
|
|
version 3 or higher.</font></p>
|
|
</body>
|
|
</html>
|
|
|