rsyslog/rsyslog
1
0
Fork 0
mirror of https://github.com/rsyslog/rsyslog.git synced 2025-12-19 04:30:43 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'v7-stable' into v7-stable-tlsfix

Browse Source 
Conflicts:
	ChangeLog
	runtime/rsyslog.h
This commit is contained in:
Rainer Gerhards 2013-09-13 02:39:42 +02:00
commit 7903677bfb
204 changed files with 16895 additions and 2936 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

451
ChangeLog
View File

@ -1,4 +1,450 @@
---------------------------------------------------------------------------
Version 7.4.5 [v7.4-stable] 2013-09-??
- bugfix: segfault on startup if TLS was used but no CA cert set
- bugfix: some more build problems with newer json-c versions
Thanks to Michael Biebl for mentioning the problem.
- bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled
Thanks to Jonny Törnbom for reporting this problem
---------------------------------------------------------------------------
Version 7.4.4 [v7.4-stable] 2013-09-03
- better error messages in GuardTime signature provider
Thanks to Ahto Truu for providing the patch.
- make rsyslog use the new json-c pkgconfig file if available
Thanks to the Gentoo team for the patches.
- bugfix: imfile parameter "persistStateInterval" was unusable
due to a case typo in imfile; work-around was to use legacy config
Thanks to Brandon Murphy for reporting this bug.
- bugfix: TLV16 flag encoding error in signature files from GT provider
This fixes a problem where the TLV16 flag was improperly encoded.
Unfortunately, existing files already have the bug and may not properly
be processed. The fix uses constants from the GuardTime API lib to
prevent such problems in the future.
Thanks to Ahto Truu for providing the patch.
- bugfix: slightly malformed SMTP handling in ommail
- bugfix: segfault in omprog if no template was provided (now dflt is used)
- bugfix: segfault in ompipe if no template was provided (now dflt is used)
- bugfix: segfault in omsnmp if no template was provided (now dflt is used)
- bugfix: some omsnmp optional config params were flagged as mandatory
- bugfix: segfault in omelasticsearch when resuming queued messages
after restarting Elasticsearch
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=464
- bugfix: imtcp addtlframedelimiter could not be set to zero
Thanks to Chris Norton for alerting us.
- doc bugfix: remove no-longer existing omtemplate from developer doc
was specifically mentioned as a sample for creating new plugins
Thanks to Yannick Brosseau for alerting us of this problem.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=473
---------------------------------------------------------------------------
Version 7.4.3 [v7.4-stable] 2013-07-18
- bugfix: queue file size was not correctly processed
this could lead to using one queue file per message for sizes >2GiB
Thanks to Tomas Heinrich for the patch.
- bugfix: $QHOUR/$HHOUR were always "00" or "01"
regression some time between v5 and here
Thanks to forum user rjmcinty for reporting this bug
- bugfix: testbench tool chkseq did improperly report invalid file
This happened when permitted duplicate values existed in the very
last lines, right before end-of-file.
Thanks to Radu Gheorghe for reporting this bug.
---------------------------------------------------------------------------
Version 7.4.3 [v7.4-stable] 2013-07-18
- bugfix: memory leak if disk queues were used and json data present
- bugfix: CEE/json data was lost during disk queue operation
- bugfix: potential segfault during startup on invalid config
could happen if invalid actions were present, which could lead
to improper handling in optimizer.
- bugfix: 100% CPU utilization when DA queue became full
- bugfix: omlibdbi did not properly close connection on some errors
This happened to errors occuring in Begin/End Transaction entry
points.
- cosmetic bugfix: file name buffer was not freed on disk queue destruction
This was an extremely small one-time per run memleak, so nothing of
concern. However, it bugs under valgrind and similar memory debuggers.
- fix build on FreeBSD
Thanks to Christiano Rolim for the patch
---------------------------------------------------------------------------
Version 7.4.2 [v7.4-stable] 2013-07-04
- bugfix: in RFC5425 TLS, multiple wildcards in auth could cause segfault
- bugfix: RainerScript object required parameters were not properly
checked - this clould result to segfaults on startup if parameters
were missing.
- bugfix: double-free in omelasticsearch
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461
a security advisory for this bug is available at:
http://www.lsexperts.de/advisories/lse-2013-07-03.txt
CVE: CVE-2013-4758
PLEASE NOTE: This issue only existed if omelasticsearch was used
in a non-default configuration, where the "errorfile" parameter
was specified. Without that parameter set, the bug could not
be triggered.
Thanks to Markus Vervier and Marius Ionescu for providing a detailled
bug report. Special thanks to Markus for coordinating his security
advisory with us.
- bugfix: omrelp potential segfault at startup on invalid config parameters
- bugfix: small memory leak when $uptime property was used
- bugfix: potential segfault on rsyslog termination in imudp
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=456
- bugfix: lmsig_gt abort on invalid configuration parameters
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=448
Thanks to Risto Laanoja for the patch.
- imtcp: fix typo in "listner" parameter, which is "listener"
Currently, both names are accepted.
- solved build problems on FreeBSD
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=457
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=458
Thanks to Christiano for reproting and suggesting patches
- solved build problems on CENTOS5
---------------------------------------------------------------------------
Version 7.4.1 [v7.4-stable] 2013-06-17
- imjournal: add ratelimiting capability
The original imjournal code did not support ratelimiting at all. We
now have our own ratelimiter. This can mitigate against journal
database corruption, when the journal re-sends old data. This is a
current bug in systemd journal, but we won't outrule this to happen
in the future again. So it is better to have a safeguard in place.
By default, we permit 20,000 messages witin 10 minutes. This may
be a bit restrictive, but given the risk potential it seems reasonable.
Users requiring larger traffic flows can always adjust the value.
- bugfix: potential loop in rate limiting
if the message that tells about rate-limiting gets rate-limited itself,
it will potentially create and endless loop
- bugfix: potential segfault in imjournal if journal DB is corrupted
- bugfix: prevent a segfault in imjournal if state file is not defined
- bugfix imzmq3: potential segfault on startup
if no problem happend at startup, everything went fine
Thanks to Hongfei Cheng and Brian Knox for the patch
---------------------------------------------------------------------------
Version 7.4.0 [v7.4-stable] 2013-06-06
This starts a new stable branch based on 7.3.15 plus the following changes:
- add --enable-cached-man-pages ./configure option
permits to build rsyslog on a system where rst2man is not installed. In
that case, cached versions of the man pages are used (they were built
during "make dist", so they should be current for the version in
question.
- doc bugfix: ReadMode wrong in imfile doc, two values were swapped
Thanks to jokajak@gmail.com for mentioning this
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=450
- imjournal: no longer do periodic wakeup
- bugfix: potential hang *in debug mode* on rsyslogd termination
This ONLY affected rsyslogd if it were running with debug output
enabled.
- bugfix: $template statement with multiple spaces lead to invalid tpl name
If multiple spaces were used in front of the template name, all but one
of them became actually part of the template name. So
$template a,"..." would be name " a", and as such "a" was not
available, e.g. in
*.* /var/log/file;a
This is a legacy config problem. As it was unreported for many years,
no backport of the fix to old versions will happen.
This is a long-standing bug that was only recently reported by forum
user mc-sim.
Reference: http://kb.monitorware.com/post23448.html
- 0mq fixes; credits to Hongfei Cheng and Brian Knox
---------------------------------------------------------------------------
Version 7.3.15 [beta] 2013-05-15
- bugfix: problem in build system (especially when cross-compiling)
Thanks to Tomas Heinrich and winfried_mb2@xmsnet.nl for the patch.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=445
- bugfix: imjournal had problem with systemd journal API change
- imjournal: now obtain and include PID
- bugfix: .logsig files had tlv16 indicator bit at wrong offset
- bugfix: omrelp legacy config parameters set a timeout of zero
which lead the legacy config to be unusable.
- bugfix: segfault on startup if a disk queue was configure without file
name
Now this triggers an error message and the queue is changed to
linkedList type.
- bugfix: invalid addressing in string class (recent regression)
---------------------------------------------------------------------------
Version 7.3.14 [beta] 2013-05-06
- bugfix: some man pages were not properly installed
either rscryutil or rsgtutil man was installed, but not both
Thanks to Marius Tomaschewski for the patch.
- bugfix: potential segfault on startup when builtin module was specified
in module() statement.
Thanks to Marius Tomaschewski for reporting the bug.
- bugfix: segfault due to invalid dynafile cache handling
Accidently, the old-style cache size parameter was used when the
dynafile cache was created in a RainerScript action. If the old-style
size was lower than the one actually set, this lead to misadressing
when the size was overrun, and that could lead to all kinds of
"interesting things", often in segfaults.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=440
---------------------------------------------------------------------------
Version 7.3.13 [beta] 2013-04-29
- added omrabbitmq module (contributed, untested)
Note: this is unsupported and as such was moved immediately into the
beta version.
Thanks to Vaclav Tomec for providing this module.
- bugfix: build problem when --enable-encryption was not selected
Thanks to Michael Biebl for fixing this.
- doc bugfix: omfile parameter "VeryRobustZip" was documentas as
"VeryReliableZip"
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=437
Thanks to Thomas Doll for reporting this.
---------------------------------------------------------------------------
Version 7.3.12 [devel] 2013-04-25
- added doc for omelasticsearch
Thanks to Radu Gheorghe for the doc contribution.
- omelasticsearch: _id field support for bulk operations
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=392
Thanks to Jérôme Renard for the idea and patches.
- max number of templates for plugin use has been increased to five
- platform compatibility enhancement: solve compile issue with libgcrypt
do not use GCRY_CIPHER_MODE_AESWRAP where not available
- fix compile on Solaris
Thanks to Martin Carpenter for the patch.
- bugfix: off-by-one error in handling local FQDN name (regression)
A remporary buffer was allocated one byte too small. Did only
affect startup, not actual operations. Came up during routine tests,
and can have no effect once the engine runs. Bug was introduced in
7.3.11.
- bugfix: build problems on Solaris
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=436
- bugfix: block size limit was not properly honored
- bugfix: potential segfault in guardtime signature provider
it could segfault if an error was reported by the GuardTime API, because
an invalid free could happen then
---------------------------------------------------------------------------
Version 7.3.11 [devel] 2013-04-23
- added support for encrypting log files
- omhiredis: added support for redis pipeline support
Thanks to Brian Knox for the patch.
- bugfix: $PreserveFQDN is not properly working
Thanks to Louis Bouchard for the patch
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=426
- bugfix: imuxsock aborted due to problem in ratelimiting code
Thanks to Tomas Heinrich for the patch.
- bugfix: imuxsock aborted under some conditions
regression from ratelimiting enhancements - this was a different one
to the one Tomas Heinrich patched.
- bugfix: timestamp problems in imkmsg
---------------------------------------------------------------------------
Version 7.3.10 [devel] 2013-04-10
- added RainerScript re_extract() function
- omrelp: added support for RainerScript-based configuration
- omrelp: added ability to specify session timeout
- templates now permit substring extraction relative to end-of-string
- bugfix: failover/action suspend did not work correctly
This was experienced if the retry action took more than one second
to complete. For suspending, a cached timestamp was used, and if the
retry took longer, that timestamp was already in the past. As a
result, the action never was kept in suspended state, and as such
no failover happened. The suspend functionalit now does no longer use
the cached timestamp (should not have any performance implication, as
action suspend occurs very infrequently).
- bugfix: gnutls RFC5425 driver had some undersized buffers
Thanks to Tomas Heinrich for the patch.
- bugfix: nested if/prifilt conditions did not work properly
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=415
- bugfix: imuxsock aborted under some conditions
regression from ratelimiting enhancements
- bugfix: build problems on Solaris
Thanks to Martin Carpenter for the patches.
---------------------------------------------------------------------------
Version 7.3.9 [devel] 2013-03-27
- support for signing logs added
- imudp: now supports user-selectable inputname
- omlibdbi: now supports transaction interface
if recent enough lbdbi is present
- imuxsock: add ability to NOT create/delete sockets during startup and
shutdown
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=259
- imfile: errors persisting state file are now reported
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=292
- imfile: now detects file change when rsyslog was inactive
Previosly, this case could not be detected, so if a file was overwritten
or rotated away while rsyslog was stopped, some data was missing. This
is now detected and the new file being forwarded right from the
beginning.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=228
- updated systemd files to match current systemd source
- bugfix: imudp scheduling parameters did affect main thread, not imudp
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409
- bugfix: build problem on platforms without GLOB_NOMAGIC
- bugfix: build problems on non-Linux platforms
- bugfix: stdout/stderr were not closed on forking
but were closed when running in the forground - this was just reversed
of what it should be. This is a regression of a recent change.
---------------------------------------------------------------------------
Version 7.3.8 [devel] 2013-03-18
- imrelp: now supports listening to IPv4/v6 only instead of always both
build now requires librelp 1.0.2
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=378
- bugfix: mmanon did not build on some platforms (e.g. Ubuntu)
- bugfix: segfault in expression optimizer
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=423
- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter
To use that functionality, legacy rsyslog.conf syntax had to be used.
Also, the doc was missing information on the "ParseTrusted" set of
config directives.
- bugfix: include files got included in the wrong order
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411
This happens if an $IncludeConfig directive was done on multiple
files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf).
In that case, the order of include file processing is reversed, which
could lead to all sorts of problems.
Thanks to Nathan Stratton Treadway for his great analysis of the problem,
which made bug fixing really easy.
---------------------------------------------------------------------------
Version 7.3.7 [devel] 2013-03-12
- add support for anonymizing IPv4 addresses
- add support for writing to the Linux Journal (omjournal)
- imuxsock: add capability to ignore messages from ourselfes
This helps prevent message routing loops, and is vital to have
if omjournal is used together with traditional syslog.
- field() function now supports a string as field delimiter
- added ability to configure debug system via rsyslog.conf
- bugfix: imuxsock segfault when system log socket was used
- bugfix: mmjsonparse segfault if new-style config was used
- bugfix: script == comparison did not work properly on JSON objects
- bugfix: field() function did never return "***FIELD NOT FOUND***"
instead it returned "***ERROR in field() FUNCTION***" in that case
---------------------------------------------------------------------------
Version 7.3.6 [devel] 2013-01-28
- greatly improved speed of large-array [N]EQ RainerScript comparisons
Thanks to David Lang for a related discussion that inspired the idea
to do this with a much simpler (yet sufficient) approach than orignally
planned for.
- greatly improved speed of DNS cache for large cache sizes
- general performance improvements
- omfile: added stats counters for dynafile caches
- omfile: improved async writing, finally enabled full async write
also fixed a couple of smaller issues along that way
- impstats: added ability to write stats records to local file
and avoid going through the syslog log stream. syslog logging can now
also be turned off (see doc for details).
- bugfix: imklog issued wrong facility in error messages
...what could lead to problems in other parts of the code
- fix compile problem in imklog
- added capability to output thread-id-to-function debug info
This is a useful debug aid, but nothing of concern for regular users.
---------------------------------------------------------------------------
Version 7.3.5 [devel] 2012-12-19
- ommysql: addded batching/transaction support
- enhanced script optimizer to optimize common PRI-based comparisons
These constructs are especially used in SUSE default config files,
but also by many users (as they are more readable than the equivalent
PRI-based filter).
- omudpspoof: add support for new config system
- omudpspoof: add support for packets larger than 1472 bytes
On Ethernet, they need to be transmitted in multiple fragments. While
it is known that fragmentation can cause issues, it is the best choice
to be made in that case. Also improved debug output.
- bugfix: omudpspoof failed depending on the execution environment
The v7 engine closes fds, and closed some of libnet's fds as well, what
lead to problems (unfortunately, at least some libnet versions do not
report a proper error state but still "success"...). The order of libnet
calls has been adjusted to by in sync with what the core engine does.
- bugfix: segfault on imuxsock startup if system log socket is used
and no ratelimiting supported. Happens only during initial config
read phase, once this is over, everything works stable.
- bugfix: mmnormalize build problems
- bugfix: mmnormalize could abort rsyslog if config parameter was in error
- bugfix: no error message for invalid string template parameters
rather a malformed template was generated, and error information emitted
at runtime. However, this could be quite confusing. Note that with this
"bugfix" user experience changes: formerly, rsyslog and the affected
actions properly started up, but the actions did not produce proper
data. Now, there are startup error messages and the actions are NOT
executed (due to missing template due to template error).
- bugfix[minor]: invalid error code when mmnormalize could not access
rulebase
- bugfix(kind of): script optimizer did not work for complex boolean
expressions
- doc bugfix: corrections and improvements in mmnormalize html doc page
- bugfix: some message properties could be garbled due to race condition
This happened only on very high volume systems, if the same message was
being processed by two different actions. This was a regression caused
by the new config processor, which did no longer properly enable msg
locking in multithreaded cases. The bugfix is actually a refactoring of
the msg locking code - we no longer do unlocked operations, as the use
case for it has mostly gone away. It is potentially possible only at
very low-end systems, and there the small additional overhead of doing
the locking does not really hurt. Instead, the removal of that
capability can actually slightly improve performance in common cases,
as the code path is smaller and requires slightly less memory writes.
That probably outperforms the extra locking overhead (which in the
low-end case always happens in user space, without need for kernel
support as we can always directly aquire the lock - there is no
contention at all).
- build system cleanup (thanks to Michael Biebl for this!)
- bugfix: omelasticsearch did not properly compile on some platforms
due to missing libmath. Thanks to Michael Biebl for the fix
---------------------------------------------------------------------------
Version 7.3.4 [devel] 2012-11-23
- further (and rather drastically) improved disk queue performance
we now save one third of the IO calls
- imklog: added ParseKernelTimestamp parameter (import from 5.10.2)
Thanks to Marius Tomaschewski for the patch.
- imklog: added KeepKernelTimestamp parameter (import from 5.10.2)
Thanks to Marius Tomaschewski for the patch.
- bugfix: improper handling of backslash in string-type template()s
- bugfix: leading quote (") in string-type template() lead to thight loop
on startup
- bugfix: no error msg on invalid field option in legacy/string template
- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds
... actually, they are microseconds. So the fractional part of the
timestamp was not properly formatted. (import from 5.10.2)
Thanks to Marius Tomaschewski for the bug report and the patch idea.
---------------------------------------------------------------------------
Version 7.3.3 [devel] 2012-11-07
- improved disk queue performance
- bugfix: dynafile zip files could be corrupted
This could happen if a dynafile was destructed before the first write.
In practice, this could happen if few lines were written to a file and
it then became evicted from the dynafile cache. This would probably
look very random, because it depended on the timing in regard to
message volume and dynafile cache size.
---------------------------------------------------------------------------
Version 7.3.2 [devel] 2012-10-30
- mmnormalize: support for v6+ config interface added
- mmjsonparse: support for v6+ config interface added
---------------------------------------------------------------------------
Version 7.3.2 [devel] 2012-10-30
- totally reworked ratelimiting and "last message repeated n times"
all over rsyslog code. Each of the supported inputs now supports
linux-like ratelimiting (formerly only imuxsock did). Also, the
"last message repeated n times" is now processed at the input side
and no longer at the output side of rsyslog processing. This
provides the basis for new future additions as well as usually more
performance and a much simpler output part (which can be even further
refactored).
- imtcp: support for Linux-Type ratelimiting added
- imptcp: support for Linux-Type ratelimiting added
- imudp enhancements:
* support for input batching added (performance improvement)
* support for Linux-Type ratelimiting added
- permited action-like statements (stop, call, ...) in action lists
- bugfix: segfault on startup when modules using MSG_PASSING mode are used
- omelasticsearch: support for writing data errors to local file added
- omelasticsearch: fix check for bulk processing status response
---------------------------------------------------------------------------
Version 7.3.1 [devel] 2012-10-19
- optimized template processing performance, especially for $NOW family
of properties
- change lumberjack cookie to "@cee:" from "@cee: "
CEE originally specified the cookie with SP, whereas other lumberjack
tools used it without space. In order to keep interop with lumberjack,
we now use the cookie without space as well. I hope this can be changed
in CEE as well when it is released at a later time.
Thanks to Miloslav Trmač for pointing this out and a similiar v7 patch.
- bugfix: imuxsock and imklog truncated head of received message
This happened only under some circumstances. Thanks to Marius
Tomaschewski, Florian Piekert and Milan Bartos for their help in
solving this issue.
- bugfix: imuxsock did not properly honor $LocalHostIPIF
---------------------------------------------------------------------------
Version 7.3.0 [devel] 2012-10-09
- omlibdbi improvements, added
* support for config load phases & module() parameters
* support for default templates
* driverdirectory is now cleanly a global parameter, but can no longer
be specified as an action paramter. Note that in previous versions
this parameter was ignored in all but the first action definition
- improved omfile zip writer to increase compression
This was achieved by somewhat reducing the robustness of the zip archive.
This is controlled by the new action parameter "VeryReliableZip".
----------------------------------------------------------------------------
Version 7.2.8 [v7-stable] 2013-0?-??
- bugfix: potential segfault on startup when builtin module was specified
@ -1074,6 +1520,9 @@ expected that interfaces, even new ones, break during the initial
[ported from v4]
---------------------------------------------------------------------------
Version 5.10.2 [V5-STABLE], 201?-??-??
- bugfix: queue file size was not correctly processed
this could lead to using one queue file per message for sizes >2GiB
Thanks to Tomas Heinrich for the patch.
- updated systemd files to match current systemd source
- bugfix: spurios error messages from imuxsock about (non-error) EAGAIN
Thanks to Marius Tomaschewski for the patch.
@ -1106,7 +1555,7 @@ Version 5.10.2 [V5-STABLE], 201?-??-??
Thanks to Tomas Heinrich for the patch.
- bugfix[kind of]: omudpspoof discarded messages >1472 bytes (MTU size)
it now truncates these message, but ensures they are sent. Note that
7.2.5+ will switch to fragmented UDP messages instead (up to 64K)
7.3.5+ will switch to fragmented UDP messages instead (up to 64K)
---------------------------------------------------------------------------
Version 5.10.1 [V5-STABLE], 2012-10-17
- bugfix: imuxsock and imklog truncated head of received message

20
Makefile.am
View File

@ -71,7 +71,7 @@ EXTRA_DIST = \
contrib/gnutls/key.pem \
rsyslog.service.in
SUBDIRS = doc runtime grammar compat . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting
SUBDIRS = doc compat runtime grammar . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting
if ENABLE_RSYSLOGD
SUBDIRS += tools
@ -165,6 +165,10 @@ if ENABLE_OMZMQ3
SUBDIRS += plugins/omzmq3
endif
if ENABLE_OMRABBITMQ
SUBDIRS += plugins/omrabbitmq
endif
if ENABLE_IMZMQ3
SUBDIRS += plugins/imzmq3
endif
@ -177,6 +181,14 @@ if ENABLE_OMHDFS
SUBDIRS += plugins/omhdfs
endif
if ENABLE_OMJOURNAL
SUBDIRS += plugins/omjournal
endif
if ENABLE_IMJOURNAL
SUBDIRS += plugins/imjournal
endif
if ENABLE_ELASTICSEARCH
SUBDIRS += plugins/omelasticsearch
endif
@ -225,6 +237,10 @@ if ENABLE_MMAUDIT
SUBDIRS += plugins/mmaudit
endif
if ENABLE_MMANON
SUBDIRS += plugins/mmanon
endif
if ENABLE_ORACLE
SUBDIRS += plugins/omoracle
endif
@ -245,7 +261,6 @@ SUBDIRS += tests
# in a make distcheck is so that we detect code that accidently was not updated
# when some global update happened.
DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \
--enable-gnutls \
--enable-imfile \
--enable-snmp \
--enable-libdbi \
@ -271,6 +286,7 @@ DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \
--enable-pmsnare \
--enable-mmsnmptrapd \
--enable-elasticsearch \
--enable-valgrind \
--with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir)
# temporarily disable these checks for make distcheck 2012-09-06 rgerhards
# --enable-extended-tests \

230
action.c
View File

@ -12,11 +12,11 @@
* necessary to triple-check that everything works well in *all* modes.
* The different modes (and calling sequence) are:
*
* if set iExecEveryNthOccur > 1 || f_ReduceRepeated || iSecsExecOnceInterval
* if set iExecEveryNthOccur > 1 || iSecsExecOnceInterval
* - doSubmitToActionQComplexBatch
* - helperSubmitToActionQComplexBatch
* - doActionCallAction
* handles duplicate message processing, but in essence calls
* handles mark message reduction, but in essence calls
* - actionWriteToAction
* - qqueueEnqObj
* (now queue engine processing)
@ -98,7 +98,7 @@
#include <strings.h>
#include <time.h>
#include <errno.h>
#include <json/json.h>
#include <json.h>
#include "dirty.h"
#include "template.h"
@ -307,9 +307,6 @@ rsRetVal actionDestruct(action_t *pThis)
if(pThis->pMod != NULL)
pThis->pMod->freeInstance(pThis->pModData);
if(pThis->f_pMsg != NULL)
msgDestruct(&pThis->f_pMsg);
pthread_mutex_destroy(&pThis->mutAction);
pthread_mutex_destroy(&pThis->mutActExec);
d_free(pThis->pszName);
@ -410,16 +407,11 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams)
* mode is much faster processing (and simpler code) -- rgerhards, 2010-06-08
*/
if( pThis->iExecEveryNthOccur > 1
|| pThis->f_ReduceRepeated
|| pThis->iSecsExecOnceInterval
) {
DBGPRINTF("info: firehose mode disabled for action because "
"iExecEveryNthOccur=%d, "
"ReduceRepeated=%d, "
"iSecsExecOnceInterval=%d\n",
pThis->iExecEveryNthOccur, pThis->f_ReduceRepeated,
pThis->iSecsExecOnceInterval
);
"iExecEveryNthOccur=%d, iSecsExecOnceInterval=%d\n",
pThis->iExecEveryNthOccur, pThis->iSecsExecOnceInterval);
pThis->submitToActQ = doSubmitToActionQComplexBatch;
} else if(pThis->bWriteAllMarkMsgs == RSFALSE) {
/* nearly full-speed submission mode, default case */
@ -438,7 +430,7 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams)
CHKiRet(qqueueConstruct(&pThis->pQueue, cs.ActionQueType, 1, cs.iActionQueueSize,
(rsRetVal (*)(void*, batch_t*, int*))processBatchMain));
obj.SetName((obj_t*) pThis->pQueue, pszAName);
qqueueSetpUsr(pThis->pQueue, pThis);
qqueueSetpAction(pThis->pQueue, pThis);
if(queueParams == NULL) { /* use legacy params? */
/* ... set some properties ... */
@ -782,7 +774,6 @@ rsRetVal actionDbgPrint(action_t *pThis)
pThis->pMod->dbgPrintInstInfo(pThis->pModData);
dbgprintf("\n");
dbgprintf("\tInstance data: 0x%lx\n", (unsigned long) pThis->pModData);
dbgprintf("\tRepeatedMsgReduction: %d\n", pThis->f_ReduceRepeated);
dbgprintf("\tResume Interval: %d\n", pThis->iResumeInterval);
if(pThis->eState == ACT_STATE_SUSP) {
dbgprintf("\tresume next retry: %u, number retries: %d",
@ -809,7 +800,8 @@ rsRetVal actionDbgPrint(action_t *pThis)
/* prepare the calling parameters for doAction()
* rgerhards, 2009-05-07
*/
static rsRetVal prepareDoActionParams(action_t *pAction, batch_obj_t *pElem)
static rsRetVal
prepareDoActionParams(action_t *pAction, batch_obj_t *pElem, struct syslogTime *ttNow)
{
int i;
msg_t *pMsg;
@ -819,23 +811,23 @@ static rsRetVal prepareDoActionParams(action_t *pAction, batch_obj_t *pElem)
ASSERT(pAction != NULL);
ASSERT(pElem != NULL);
pMsg = (msg_t*) pElem->pUsrp;
pMsg = pElem->pMsg;
/* here we must loop to process all requested strings */
for(i = 0 ; i < pAction->iNumTpls ; ++i) {
switch(pAction->eParamPassing) {
case ACT_STRING_PASSING:
CHKiRet(tplToString(pAction->ppTpl[i], pMsg, &(pElem->staticActStrings[i]),
&pElem->staticLenStrings[i]));
&pElem->staticLenStrings[i], ttNow));
pElem->staticActParams[i] = pElem->staticActStrings[i];
break;
case ACT_ARRAY_PASSING:
CHKiRet(tplToArray(pAction->ppTpl[i], pMsg, (uchar***) &(pElem->staticActParams[i])));
CHKiRet(tplToArray(pAction->ppTpl[i], pMsg, (uchar***) &(pElem->staticActParams[i]), ttNow));
break;
case ACT_MSG_PASSING:
pElem->staticActParams[i] = (void*) pMsg;
break;
case ACT_JSON_PASSING:
CHKiRet(tplToJSON(pAction->ppTpl[i], pMsg, &json));
CHKiRet(tplToJSON(pAction->ppTpl[i], pMsg, &json, ttNow));
pElem->staticActParams[i] = (void*) json;
break;
default:dbgprintf("software bug/error: unknown pAction->eParamPassing %d in prepareDoActionParams\n",
@ -868,6 +860,9 @@ static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch)
ASSERT(pAction != NULL);
if(pAction->eParamPassing == ACT_STRING_PASSING || pAction->eParamPassing == ACT_MSG_PASSING)
goto done; /* we need to do nothing with these types! */
for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) {
pElem = &(pBatch->pElem[i]);
if(batchIsValidElem(pBatch, i)) {
@ -887,19 +882,6 @@ static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch)
}
}
break;
case ACT_STRING_PASSING:
case ACT_MSG_PASSING:
/* nothing to do in that case */
/* TODO ... and yet we do something ;) This is considered not
* really needed, but I was not bold enough to remove that while
* fixing the stable. It should be removed in a devel version
* soon (I really don't see a reason why we would need it).
* rgerhards, 2010-12-16
*/
for(j = 0 ; j < pAction->iNumTpls ; ++j) {
((uchar**)pElem->staticActParams)[j] = NULL;
}
break;
case ACT_JSON_PASSING:
for(j = 0 ; j < pAction->iNumTpls ; ++j) {
json_object_put((struct json_object*)
@ -907,11 +889,15 @@ static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch)
pElem->staticActParams[j] = NULL;
}
break;
case ACT_STRING_PASSING:
case ACT_MSG_PASSING:
/* can never happen, just to keep compiler happy! */
break;
}
}
}
RETiRet;
done: RETiRet;
}
@ -975,6 +961,8 @@ actionProcessMessage(action_t *pThis, msg_t *pMsg, void *actParams, int *pbShutd
ISOBJ_TYPE_assert(pMsg, msg);
CHKiRet(actionPrepare(pThis, pbShutdownImmediate));
if(pThis->pMod->mod.om.SetShutdownImmdtPtr != NULL)
pThis->pMod->mod.om.SetShutdownImmdtPtr(pThis->pModData, pbShutdownImmediate);
if(pThis->eState == ACT_STATE_ITX)
CHKiRet(actionCallDoAction(pThis, pMsg, actParams));
@ -1069,7 +1057,7 @@ tryDoAction(action_t *pAction, batch_t *pBatch, int *pnElem)
* enq side of the queue (see file header comment)! -- rgerhards, 2011-06-15
*/
if(batchIsValidElem(pBatch, i)) {
pMsg = (msg_t*) pBatch->pElem[i].pUsrp;
pMsg = pBatch->pElem[i].pMsg;
localRet = actionProcessMessage(pAction, pMsg, pBatch->pElem[i].staticActParams,
pBatch->pbShutdownImmediate);
DBGPRINTF("action %p call returned %d\n", pAction, localRet);
@ -1092,11 +1080,11 @@ tryDoAction(action_t *pAction, batch_t *pBatch, int *pnElem)
++iCommittedUpTo;
//pBatch->pElem[iCommittedUpTo++].state = BATCH_STATE_COMM;
}
pBatch->pElem[i].state = BATCH_STATE_SUB;
pBatch->eltState[i] = BATCH_STATE_SUB;
} else if(localRet == RS_RET_DEFER_COMMIT) {
pBatch->pElem[i].state = BATCH_STATE_SUB;
pBatch->eltState[i] = BATCH_STATE_SUB;
} else if(localRet == RS_RET_DISCARDMSG) {
pBatch->pElem[i].state = BATCH_STATE_DISC;
pBatch->eltState[i] = BATCH_STATE_DISC;
} else {
dbgprintf("tryDoAction: unexpected error code %d[nElem %d, Commited UpTo %d], finalizing\n",
localRet, *pnElem, iCommittedUpTo);
@ -1160,9 +1148,9 @@ submitBatch(action_t *pAction, batch_t *pBatch, int nElem)
} else if(localRet == RS_RET_ACTION_FAILED) {
/* in this case, everything not yet committed is BAD */
for(i = pBatch->iDoneUpTo ; i < wasDoneTo + nElem ; ++i) {
if( pBatch->pElem[i].state != BATCH_STATE_DISC
&& pBatch->pElem[i].state != BATCH_STATE_COMM ) {
pBatch->pElem[i].state = BATCH_STATE_BAD;
if( pBatch->eltState[i] != BATCH_STATE_DISC
&& pBatch->eltState[i] != BATCH_STATE_COMM ) {
pBatch->eltState[i] = BATCH_STATE_BAD;
pBatch->pElem[i].bPrevWasSuspended = 1;
STATSCOUNTER_INC(pAction->ctrFail, pAction->mutCtrFail);
}
@ -1228,14 +1216,18 @@ prepareBatch(action_t *pAction, batch_t *pBatch, sbool **activeSave, int *bMustR
{
int i;
batch_obj_t *pElem;
struct syslogTime ttNow;
DEFiRet;
/* indicate we have not yet read the date */
ttNow.year = 0;
pBatch->iDoneUpTo = 0;
for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) {
pElem = &(pBatch->pElem[i]);
if(batchIsValidElem(pBatch, i)) {
pElem->state = BATCH_STATE_RDY;
if(prepareDoActionParams(pAction, pElem) != RS_RET_OK) {
pBatch->eltState[i] = BATCH_STATE_RDY;
if(prepareDoActionParams(pAction, pElem, &ttNow) != RS_RET_OK) {
/* make sure we have our copy of "active" array */
if(!*bMustRestoreActivePtr) {
*activeSave = pBatch->active;
@ -1282,8 +1274,10 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate)
assert(pBatch != NULL);
pbShutdownImmdtSave = pBatch->pbShutdownImmediate;
pBatch->pbShutdownImmediate = pbShutdownImmediate;
if(pbShutdownImmediate != NULL) {
pbShutdownImmdtSave = pBatch->pbShutdownImmediate;
pBatch->pbShutdownImmediate = pbShutdownImmediate;
}
CHKiRet(prepareBatch(pAction, pBatch, &activeSave, &bMustRestoreActivePtr));
/* We now must guard the output module against execution by multiple threads. The
@ -1314,7 +1308,8 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate)
}
finalize_it:
pBatch->pbShutdownImmediate = pbShutdownImmdtSave;
if(pbShutdownImmediate != NULL)
pBatch->pbShutdownImmediate = pbShutdownImmdtSave;
RETiRet;
}
#pragma GCC diagnostic warning "-Wempty-body"
@ -1397,9 +1392,9 @@ doSubmitToActionQ(action_t *pAction, msg_t *pMsg)
STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed);
if(pAction->pQueue->qType == QUEUETYPE_DIRECT)
iRet = qqueueEnqObjDirect(pAction->pQueue, (void*) MsgAddRef(pMsg));
iRet = qqueueEnqMsgDirect(pAction->pQueue, MsgAddRef(pMsg));
else
iRet = qqueueEnqObj(pAction->pQueue, eFLOWCTL_NO_DELAY, (void*) MsgAddRef(pMsg));
iRet = qqueueEnqMsg(pAction->pQueue, eFLOWCTL_NO_DELAY, MsgAddRef(pMsg));
finalize_it:
RETiRet;
@ -1414,14 +1409,10 @@ finalize_it:
* be filtered out before calling us (what is done currently!).
*/
rsRetVal
actionWriteToAction(action_t *pAction)
actionWriteToAction(action_t *pAction, msg_t *pMsg)
{
msg_t *pMsgSave; /* to save current message pointer, necessary to restore
it in case it needs to be updated (e.g. repeated msgs) */
DEFiRet;
pMsgSave = NULL; /* indicate message poiner not saved */
/* first, we check if the action should actually be called. The action-specific
* $ActionExecOnlyEveryNthTime permits us to execute an action only every Nth
* time. So we need to check if we need to drop the (otherwise perfectly executable)
@ -1448,43 +1439,6 @@ actionWriteToAction(action_t *pAction)
}
}
/* then check if this is a regular message or the repeation of
* a previous message. If so, we need to change the message text
* to "last message repeated n times" and then go ahead and write
* it. Please note that we can not modify the message object, because
* that would update it in other selectors as well. As such, we first
* need to create a local copy of the message, which we than can update.
* rgerhards, 2007-07-10
*/
if(pAction->f_prevcount > 1) {
msg_t *pMsg;
size_t lenRepMsg;
uchar szRepMsg[1024];
if((pMsg = MsgDup(pAction->f_pMsg)) == NULL) {
/* it failed - nothing we can do against it... */
DBGPRINTF("Message duplication failed, dropping repeat message.\n");
ABORT_FINALIZE(RS_RET_ERR);
}
if(pAction->bRepMsgHasMsg == 0) { /* old format repeat message? */
lenRepMsg = snprintf((char*)szRepMsg, sizeof(szRepMsg), " last message repeated %d times",
pAction->f_prevcount);
} else {
lenRepMsg = snprintf((char*)szRepMsg, sizeof(szRepMsg), " message repeated %d times: [%.800s]",
pAction->f_prevcount, getMSG(pAction->f_pMsg));
}
/* We now need to update the other message properties. Please note that digital
* signatures inside the message are also invalidated.
*/
datetime.getCurrTime(&(pMsg->tRcvdAt), &(pMsg->ttGenTime));
memcpy(&pMsg->tTIMESTAMP, &pMsg->tRcvdAt, sizeof(struct syslogTime));
MsgReplaceMSG(pMsg, szRepMsg, lenRepMsg);
pMsgSave = pAction->f_pMsg; /* save message pointer for later restoration */
pAction->f_pMsg = pMsg; /* use the new msg (pointer will be restored below) */
}
DBGPRINTF("Called action(complex case), logging to %s\n", module.GetStateName(pAction->pMod));
/* now check if we need to drop the message because otherwise the action would be too
@ -1505,31 +1459,14 @@ actionWriteToAction(action_t *pAction)
/* we use reception time, not dequeue time - this is considered more appropriate and also faster ;)
* rgerhards, 2008-09-17 */
pAction->tLastExec = getActNow(pAction); /* re-init time flags */
pAction->f_time = pAction->f_pMsg->ttGenTime;
pAction->f_time = pMsg->ttGenTime;
/* When we reach this point, we have a valid, non-disabled action.
* So let's enqueue our message for execution. -- rgerhards, 2007-07-24
*/
iRet = doSubmitToActionQ(pAction, pAction->f_pMsg);
if(iRet == RS_RET_OK)
pAction->f_prevcount = 0; /* message processed, so we start a new cycle */
iRet = doSubmitToActionQ(pAction, pMsg);
finalize_it:
if(pMsgSave != NULL) {
/* we had saved the original message pointer. That was
* done because we needed to create a temporary one
* (most often for "message repeated n time" handling). If so,
* we need to restore the original one now, so that procesing
* can continue as normal. We also need to discard the temporary
* one, as we do not like memory leaks ;) Please note that the original
* message object will be discarded by our callers, so this is nothing
* of our business. rgerhards, 2007-07-10
*/
msgDestruct(&pAction->f_pMsg);
pAction->f_pMsg = pMsgSave; /* restore it */
}
RETiRet;
}
@ -1543,7 +1480,7 @@ doActionCallAction(action_t *pAction, batch_t *pBatch, int idxBtch)
msg_t *pMsg;
DEFiRet;
pMsg = (msg_t*)(pBatch->pElem[idxBtch].pUsrp);
pMsg = pBatch->pElem[idxBtch].pMsg;
pAction->tActNow = -1; /* we do not yet know our current time (clear prev. value) */
/* don't output marks to recently written outputs */
@ -1552,43 +1489,8 @@ doActionCallAction(action_t *pAction, batch_t *pBatch, int idxBtch)
ABORT_FINALIZE(RS_RET_OK);
}
/* suppress duplicate messages */
if ((pAction->f_ReduceRepeated == 1) && pAction->f_pMsg != NULL &&
(pMsg->msgFlags & MARK) == 0 && getMSGLen(pMsg) == getMSGLen(pAction->f_pMsg) &&
!ustrcmp(getMSG(pMsg), getMSG(pAction->f_pMsg)) &&
!strcmp(getHOSTNAME(pMsg), getHOSTNAME(pAction->f_pMsg)) &&
!strcmp(getPROCID(pMsg, LOCK_MUTEX), getPROCID(pAction->f_pMsg, LOCK_MUTEX)) &&
!strcmp(getAPPNAME(pMsg, LOCK_MUTEX), getAPPNAME(pAction->f_pMsg, LOCK_MUTEX))) {
pAction->f_prevcount++;
DBGPRINTF("msg repeated %d times, %ld sec of %d.\n",
pAction->f_prevcount, (long) getActNow(pAction) - pAction->f_time,
repeatinterval[pAction->f_repeatcount]);
/* use current message, so we have the new timestamp (means we need to discard previous one) */
msgDestruct(&pAction->f_pMsg);
pAction->f_pMsg = MsgAddRef(pMsg);
/* If domark would have logged this by now, flush it now (so we don't hold
* isolated messages), but back off so we'll flush less often in the future.
*/
if(getActNow(pAction) > REPEATTIME(pAction)) {
iRet = actionWriteToAction(pAction);
BACKOFF(pAction);
}
} else {/* new message, save it */
/* first check if we have a previous message stored
* if so, emit and then discard it first
*/
if(pAction->f_pMsg != NULL) {
if(pAction->f_prevcount > 0)
actionWriteToAction(pAction);
/* we do not care about iRet above - I think it's right but if we have
* some troubles, you know where to look at ;) -- rgerhards, 2007-08-01
*/
msgDestruct(&pAction->f_pMsg);
}
pAction->f_pMsg = MsgAddRef(pMsg);
/* call the output driver */
iRet = actionWriteToAction(pAction);
}
/* call the output driver */
iRet = actionWriteToAction(pAction, pMsg);
finalize_it:
/* we need to update the batch to handle failover processing correctly */
@ -1660,7 +1562,7 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch)
copyActive(pBatch);
for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) {
if((pBatch->pElem[i].state == BATCH_STATE_DISC) || !pBatch->active[i])
if((pBatch->eltState[i] == BATCH_STATE_DISC) || !pBatch->active[i])
continue;
if(now == 0) {
now = datetime.GetTime(NULL); /* good time call - the only one done */
@ -1670,7 +1572,7 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch)
* also faster ;) -- rgerhards, 2008-09-17 */
do {
lastAct = pAction->f_time;
if(((msg_t*)(pBatch->pElem[i].pUsrp))->msgFlags & MARK) {
if(pBatch->pElem[i].pMsg->msgFlags & MARK) {
if((now - lastAct) < MarkInterval / 2) {
pBatch->active[i] = 0;
DBGPRINTF("batch item %d: action was recently called, ignoring "
@ -1679,7 +1581,7 @@ doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch)
}
}
} while(ATOMIC_CAS_time_t(&pAction->f_time, lastAct,
((msg_t*)(pBatch->pElem[i].pUsrp))->ttGenTime, &pAction->mutCAS) == 0);
pBatch->pElem[i].pMsg->ttGenTime, &pAction->mutCAS) == 0);
if(pBatch->active[i]) {
DBGPRINTF("Called action(NotAllMark), processing batch[%d] via '%s'\n",
i, module.GetStateName(pAction->pMod));
@ -1738,7 +1640,7 @@ doQueueEnqObjDirectBatch(action_t *pAction, batch_t *pBatch)
bNeedSubmit = 1;
}
DBGPRINTF("action %p[%d]: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n",
pAction, i, batchIsValidElem(pBatch, i), pBatch->pElem[i].state,
pAction, i, batchIsValidElem(pBatch, i), pBatch->eltState[i],
pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended);
}
if(bNeedSubmit) {
@ -1778,11 +1680,11 @@ doSubmitToActionQBatch(action_t *pAction, batch_t *pBatch)
*/
for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) {
DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n",
pAction, batchIsValidElem(pBatch, i), pBatch->pElem[i].state,
pAction, batchIsValidElem(pBatch, i), pBatch->eltState[i],
pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended);
if( batchIsValidElem(pBatch, i)
&& (pAction->bExecWhenPrevSusp == 0 || pBatch->pElem[i].bPrevWasSuspended == 1)) {
doSubmitToActionQ(pAction, (msg_t*)(pBatch->pElem[i].pUsrp));
doSubmitToActionQ(pAction, pBatch->pElem[i].pMsg);
}
}
}
@ -1806,7 +1708,7 @@ helperSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch)
pAction, module.GetStateName(pAction->pMod));
for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) {
DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n",
pAction, batchIsValidElem(pBatch, i), pBatch->pElem[i].state,
pAction, batchIsValidElem(pBatch, i), pBatch->eltState[i],
pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended);
if( batchIsValidElem(pBatch, i)
&& ((pAction->bExecWhenPrevSusp == 0) || pBatch->pElem[i].bPrevWasSuspended) ) {
@ -1878,7 +1780,6 @@ actionApplyCnfParam(action_t *pAction, struct cnfparamvals *pvals)
}
/* add an Action to the current selector
* The pOMSR is freed, as it is not needed after this function.
* Note: this function pulls global data that specifies action config state.
@ -1943,7 +1844,7 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData,
&& (pAction->ppTpl[i] =
tplFind(ourConf, (char*)pTplName, strlen((char*)pTplName))) == NULL) {
snprintf(errMsg, sizeof(errMsg) / sizeof(char),
" Could not find template '%s' - action disabled\n",
" Could not find template '%s' - action disabled",
pTplName);
errno = 0;
errmsg.LogError(0, RS_RET_NOT_FOUND, "%s", errMsg);
@ -1974,13 +1875,7 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData,
pAction->pMod = pMod;
pAction->pModData = pModData;
/* now check if the module is compatible with select features */
if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK) {
pAction->f_ReduceRepeated = loadConf->globals.bReduceRepeatMsgs;
} else {
DBGPRINTF("module is incompatible with RepeatedMsgReduction - turned off\n");
pAction->f_ReduceRepeated = 0;
}
/* check if the module is compatible with select features (currently no such features exist) */
pAction->eState = ACT_STATE_RDY; /* action is enabled */
if(bSuspended)
@ -2080,13 +1975,8 @@ actionNewInst(struct nvlst *lst, action_t **ppAction)
if((iRet = addAction(&pAction, pMod, pModData, pOMSR, paramvals, queueParams,
(iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) {
/* now check if the module is compatible with select features */
if(pMod->isCompatibleWithFeature(sFEATURERepeatedMsgReduction) == RS_RET_OK)
pAction->f_ReduceRepeated = loadConf->globals.bReduceRepeatMsgs;
else {
DBGPRINTF("module is incompatible with RepeatedMsgReduction - turned off\n");
pAction->f_ReduceRepeated = 0;
}
/* check if the module is compatible with select features
* (currently no such features exist) */
pAction->eState = ACT_STATE_RDY; /* action is enabled */
loadConf->actions.nbrActions++; /* one more active action! */
}

12
action.h
View File

@ -46,9 +46,8 @@ typedef enum {
/* the following struct defines the action object data structure
*/
typedef struct action_s action_t;
struct action_s {
time_t f_time; /* used for "message repeated n times" - be careful, old, old code */
time_t f_time; /* used for "max. n messages in m seconds" processing */
time_t tActNow; /* the current time for an action execution. Initially set to -1 and
populated on an as-needed basis. This is a performance optimization. */
time_t tLastExec; /* time this action was last executed */
@ -69,9 +68,6 @@ struct action_s {
struct modInfo_s *pMod;/* pointer to output module handling this selector */
void *pModData; /* pointer to module data - content is module-specific */
sbool bRepMsgHasMsg; /* "message repeated..." has msg fragment in it (0-no, 1-yes) */
short f_ReduceRepeated;/* reduce repeated lines 0 - no, 1 - yes */
int f_prevcount; /* repetition cnt of prevline */
int f_repeatcount; /* number of "repeated" msgs */
rsRetVal (*submitToActQ)(action_t *, batch_t *);/* function submit message to action queue */
rsRetVal (*qConstruct)(struct queue_s *pThis);
enum { ACT_STRING_PASSING = 0, ACT_ARRAY_PASSING = 1, ACT_MSG_PASSING = 2,
@ -80,10 +76,6 @@ struct action_s {
int iNumTpls; /* number of array entries for template element below */
struct template **ppTpl;/* array of template to use - strings must be passed to doAction
* in this order. */
msg_t *f_pMsg; /* pointer to the message (this will replace the other vars with msg
* content later). This is preserved after the message has been
* processed - it is also used to detect duplicates.
*/
qqueue_t *pQueue; /* action queue */
pthread_mutex_t mutAction; /* primary action mutex */
pthread_mutex_t mutActExec; /* mutex to guard actual execution of doAction for single-threaded modules */
@ -104,7 +96,7 @@ rsRetVal actionDestruct(action_t *pThis);
rsRetVal actionDbgPrint(action_t *pThis);
rsRetVal actionSetGlobalResumeInterval(int iNewVal);
rsRetVal actionDoAction(action_t *pAction);
rsRetVal actionWriteToAction(action_t *pAction);
rsRetVal actionWriteToAction(action_t *pAction, msg_t *pMsg);
rsRetVal actionCallHUPHdlr(action_t *pAction);
rsRetVal actionClassInit(void);
rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, struct cnfparamvals *queueParams, int bSuspended);

2
compat/Makefile.am
View File

@ -1,6 +1,6 @@
noinst_LTLIBRARIES = compat.la
compat_la_SOURCES = getifaddrs.c
compat_la_SOURCES = getifaddrs.c ifaddrs.h
compat_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS)
compat_la_LDFLAGS = -module -avoid-version
compat_la_LIBADD = $(IMUDP_LIBS)

10
compat/getifaddrs.c
View File

@ -36,7 +36,15 @@
#include <stdlib.h>
#include <net/if.h>
#include <ifaddrs.h>
#include <libsocket_priv.h>
/* Normally this is defined in <net/if.h> but was new for Solaris 11 */
#ifndef LIFC_ENABLED
#define LIFC_ENABLED 0x20
#endif
int getallifaddrs(sa_family_t af, struct ifaddrs **ifap, int64_t flags);
int getallifs(int s, sa_family_t af, struct lifreq **lifr, int *numifs,
int64_t lifc_flags);
/*
* Create a linked list of `struct ifaddrs' structures, one for each

186
configure.ac
View File

@ -2,7 +2,7 @@
# Process this file with autoconf to produce a configure script.
AC_PREREQ(2.61)
AC_INIT([rsyslog],[7.2.7],[rsyslog@lists.adiscon.com])
AC_INIT([rsyslog],[7.4.4],[rsyslog@lists.adiscon.com])
AM_INIT_AUTOMAKE
m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
@ -33,8 +33,9 @@ PKG_PROG_PKG_CONFIG
# modules we require
PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.5)
PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0)
PKG_CHECK_MODULES([JSON_C], [json])
PKG_CHECK_MODULES([JSON_C], [json],, [
PKG_CHECK_MODULES([JSON_C], [json-c])
])
case "${host}" in
*-*-linux*)
@ -120,7 +121,10 @@ AC_TYPE_SIGNAL
AC_FUNC_STAT
AC_FUNC_STRERROR_R
AC_FUNC_VPRINTF
AC_CHECK_FUNCS([flock basename alarm clock_gettime getifaddrs gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync lseek64])
AC_CHECK_FUNCS([flock basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setsid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64])
# getifaddrs is in libc (mostly) or in libsocket (eg Solaris 11) or not defined (eg Solaris 10)
AC_SEARCH_LIBS([getifaddrs], [socket], [AC_DEFINE(HAVE_GETIFADDRS, [1], [set define])])
# the check below is probably ugly. If someone knows how to do it in a better way, please
# let me know! -- rgerhards, 2010-10-06
@ -128,7 +132,9 @@ AC_CHECK_DECL([SCM_CREDENTIALS], [AC_DEFINE(HAVE_SCM_CREDENTIALS, [1], [set defi
#include <sys/socket.h>])
AC_CHECK_DECL([SO_TIMESTAMP], [AC_DEFINE(HAVE_SO_TIMESTAMP, [1], [set define])], [], [#include <sys/types.h>
#include <sys/socket.h>])
AC_CHECK_DECL([SYS_gettid], [AC_DEFINE(HAVE_SYS_gettid, [1], [set define])], [], [#include <sys/syscall.h>])
AC_CHECK_MEMBER([struct sysinfo.uptime], [AC_DEFINE(HAVE_SYSINFO_UPTIME, [1], [set define])], [], [#include <sys/sysinfo.h>])
AC_CHECK_DECL([GLOB_NOMAGIC], [AC_DEFINE(HAVE_GLOB_NOMAGIC, [1], [set define])], [], [#include <glob.h>])
# Check for MAXHOSTNAMELEN
AC_MSG_CHECKING(for MAXHOSTNAMELEN)
@ -152,7 +158,7 @@ RS_ATOMIC_OPERATIONS
RS_ATOMIC_OPERATIONS_64BIT
# fall back to POSIX sems for atomic operations (cpu expensive)
AC_CHECK_HEADERS([semaphore.h])
AC_CHECK_HEADERS([semaphore.h sys/syscall.h])
# Additional module directories
@ -331,6 +337,20 @@ AC_ARG_ENABLE(kmsg,
)
AM_CONDITIONAL(ENABLE_IMKMSG, test x$enable_kmsg = xyes)
# imjournal
AC_ARG_ENABLE(imjournal,
[AS_HELP_STRING([--enable-imjournal],[Systemd journal message import @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_imjournal="yes" ;;
no) enable_imjournal="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-imjournal) ;;
esac],
[enable_imjournal="no"]
)
if test "x$enable_imjournal" = "xyes"; then
PKG_CHECK_MODULES([LIBSYSTEMD_JOURNAL], [libsystemd-journal >= 197])
fi
AM_CONDITIONAL(ENABLE_IMJOURNAL, test x$enable_imjournal = xyes)
# inet
AC_ARG_ENABLE(inet,
@ -664,6 +684,11 @@ if test "x$enable_libdbi" = "xyes"; then
[dbi_initialize_r],
[AC_DEFINE([HAVE_DBI_R], [1], [Define to 1 if libdbi supports the new plugin-safe interface])]
)
AC_CHECK_LIB(
[dbi],
[dbi_conn_transaction_begin],
[AC_DEFINE([HAVE_DBI_TXSUPP], [1], [Define to 1 if libdbi supports transactions])]
)
fi
AM_CONDITIONAL(ENABLE_OMLIBDBI, test x$enable_libdbi = xyes)
AC_SUBST(LIBDBI_CFLAGS)
@ -748,6 +773,40 @@ if test "x$enable_gnutls" = "xyes"; then
fi
AM_CONDITIONAL(ENABLE_GNUTLS, test x$enable_gnutls = xyes)
# libgcrypt support
AC_ARG_ENABLE(libgcrypt,
[AS_HELP_STRING([--enable-libgcrypt],[Enable log file encryption support (libgcrypt) @<:@default=yes@:>@])],
[case "${enableval}" in
yes) enable_libgcrypt="yes" ;;
no) enable_libgcrypt="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-libgcrypt) ;;
esac],
[enable_libgcrypt=yes]
)
if test "x$enable_libgcrypt" = "xyes"; then
AC_CHECK_PROG(
[HAVE_LIBGCRYPT_CONFIG],
[libgcrypt-config],
[yes],,,
)
if test "x${HAVE_LIBGCRYPT_CONFIG}" != "xyes"; then
AC_MSG_FAILURE([libgcrypt-config not found in PATH])
fi
AC_CHECK_LIB(
[gcrypt],
[gcry_cipher_open],
[LIBGCRYPT_CFLAGS="`libgcrypt-config --cflags`"
LIBGCRYPT_LIBS="`libgcrypt-config --libs`"
],
[AC_MSG_FAILURE([libgcrypt is missing])],
[`libgcrypt-config --libs --cflags`]
)
AC_DEFINE([ENABLE_LIBGCRYPT], [1], [Indicator that LIBGCRYPT is present])
fi
AM_CONDITIONAL(ENABLE_LIBGCRYPT, test x$enable_libgcrypt = xyes)
AC_SUBST(LIBGCRYPT_CFLAGS)
AC_SUBST(LIBGCRYPT_LIBS)
# support for building the rsyslogd runtime
AC_ARG_ENABLE(rsyslogrt,
@ -764,7 +823,7 @@ if test "x$enable_rsyslogrt" = "xyes"; then
RSRT_LIBS1="\$(top_builddir)/runtime/librsyslog.la"
fi
AM_CONDITIONAL(ENABLE_RSYSLOGRT, test x$enable_rsyslogrt = xyes)
RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_FLAGS)"
RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_CFLAGS)"
RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS) \$(JSON_C_LIBS)"
AC_SUBST(RSRT_CFLAGS1)
AC_SUBST(RSRT_LIBS1)
@ -860,6 +919,7 @@ AC_ARG_ENABLE(mmnormalize,
[enable_mmnormalize=no]
)
if test "x$enable_mmnormalize" = "xyes"; then
PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0)
PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1)
fi
AM_CONDITIONAL(ENABLE_MMNORMALIZE, test x$enable_mmnormalize = xyes)
@ -875,13 +935,9 @@ AC_ARG_ENABLE(mmjsonparse,
esac],
[enable_mmjsonparse=no]
)
if test "x$enable_mmjsonparse" = "xyes"; then
PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1)
fi
AM_CONDITIONAL(ENABLE_MMJSONPARSE, test x$enable_mmjsonparse = xyes)
# mmaudit
AC_ARG_ENABLE(mmaudit,
[AS_HELP_STRING([--enable-mmaudit],[Enable building mmaudit support @<:@default=no@:>@])],
@ -892,12 +948,22 @@ AC_ARG_ENABLE(mmaudit,
esac],
[enable_mmaudit=no]
)
if test "x$enable_mmaudit" = "xyes"; then
PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1)
fi
AM_CONDITIONAL(ENABLE_MMAUDIT, test x$enable_mmaudit = xyes)
# mmanon
AC_ARG_ENABLE(mmanon,
[AS_HELP_STRING([--enable-mmanon],[Enable building mmanon support @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_mmanon="yes" ;;
no) enable_mmanon="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-mmanon) ;;
esac],
[enable_mmanon=no]
)
AM_CONDITIONAL(ENABLE_MMANON, test x$enable_mmanon = xyes)
# RELP support
AC_ARG_ENABLE(relp,
[AS_HELP_STRING([--enable-relp],[Enable RELP support @<:@default=no@:>@])],
@ -909,11 +975,51 @@ AC_ARG_ENABLE(relp,
[enable_relp=no]
)
if test "x$enable_relp" = "xyes"; then
PKG_CHECK_MODULES(RELP, relp >= 1.0.1)
PKG_CHECK_MODULES(RELP, relp >= 1.0.3)
fi
AM_CONDITIONAL(ENABLE_RELP, test x$enable_relp = xyes)
# GuardTime support
AC_ARG_ENABLE(guardtime,
[AS_HELP_STRING([--enable-guardtime],[Enable log file signing support (via GuardTime) @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_guardtime="yes" ;;
no) enable_guardtime="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-guardtime) ;;
esac],
[enable_guardtime=no]
)
if test "x$enable_guardtime" = "xyes"; then
PKG_CHECK_MODULES(GUARDTIME, libgt >= 0.3.1)
fi
AM_CONDITIONAL(ENABLE_GUARDTIME, test x$enable_guardtime = xyes)
# Support using cached man file copies, to avoid the need for rst2man
# in the build environment
AC_ARG_ENABLE(cached_man_pages,
[AS_HELP_STRING([--enable-cached-man-pages],[Enable using cached versions of man files (avoid rst2man) @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_cached_man_pages="yes" ;;
no) enable_cached_man_pages="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-cached-man-pages) ;;
esac],
[enable_cached_man_pages=no]
)
if test "x$enable_cached_man_pages" = "xno"; then
# obtain path for rst2man
if test "x$enable_libgcrypt" = "xyes" || \
test "x$enable_guardtime" = "xyes"; then
AC_PATH_PROG([RST2MAN], [rst2man])
if test "x${RST2MAN}" == "x"; then
AC_MSG_FAILURE([rst2man not found in PATH])
fi
fi
fi
# RFC 3195 support
AC_ARG_ENABLE(rfc3195,
[AS_HELP_STRING([--enable-rfc3195],[Enable RFC3195 support @<:@default=no@:>@])],
@ -1063,6 +1169,21 @@ AC_ARG_ENABLE(omstdout,
)
AM_CONDITIONAL(ENABLE_OMSTDOUT, test x$enable_omstdout = xyes)
# settings for omjournal
AC_ARG_ENABLE(omjournal,
[AS_HELP_STRING([--enable-omjournal],[Compiles omjournal @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_omjournal="yes" ;;
no) enable_omjournal="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-omjournal) ;;
esac],
[enable_omjournal=no]
)
if test "x$enable_omjournal" = "xyes"; then
PKG_CHECK_MODULES([LIBSYSTEMD_JOURNAL], [libsystemd-journal >= 197])
fi
AM_CONDITIONAL(ENABLE_OMJOURNAL, test x$enable_omjournal = xyes)
# settings for pmlastmsg
AC_ARG_ENABLE(pmlastmsg,
@ -1268,6 +1389,26 @@ AM_CONDITIONAL(ENABLE_OMZMQ3, test x$enable_omzmq3 = xyes)
# END ZMQ3 SUPPORT
# BEGIN RABBITMQ OUTPUT SUPPORT
AC_ARG_ENABLE(omrabbitmq,
[AS_HELP_STRING([--enable-omrabbitmq],[Compiles omrabbitmq output module @<:@default=no@:>@])],
[case "${enableval}" in
yes) enable_omrabbitmq="yes" ;;
no) enable_omrabbitmq="no" ;;
*) AC_MSG_ERROR(bad value ${enableval} for --enable-omrabbitmq) ;;
esac],
[enable_omrabbitmq=no]
)
if test "x$enable_omrabbitmq" = "xyes"; then
PKG_CHECK_MODULES(RABBITMQ, librabbitmq >= 0.2.0)
AC_SUBST(RABBITMQ_CFLAGS)
AC_SUBST(RABBITMQ_LIBS)
fi
AM_CONDITIONAL(ENABLE_OMRABBITMQ, test x$enable_omrabbitmq = xyes)
# END RABBITMQ SUPPORT
# HIREDIS SUPPORT
AC_ARG_ENABLE(omhiredis,
@ -1298,12 +1439,14 @@ AC_CONFIG_FILES([Makefile \
plugins/im3195/Makefile \
plugins/imgssapi/Makefile \
plugins/imuxsock/Makefile \
plugins/imjournal/Makefile \
plugins/immark/Makefile \
plugins/imklog/Makefile \
plugins/imkmsg/Makefile \
plugins/omhdfs/Makefile \
plugins/omprog/Makefile \
plugins/omstdout/Makefile \
plugins/omjournal/Makefile \
plugins/pmrfc3164sd/Makefile \
plugins/pmlastmsg/Makefile \
plugins/pmcisconames/Makefile \
@ -1318,7 +1461,7 @@ AC_CONFIG_FILES([Makefile \
plugins/impstats/Makefile \
plugins/imrelp/Makefile \
plugins/imdiag/Makefile \
plugins/imzmq3/Makefile \
plugins/imzmq3/Makefile \
plugins/omtesting/Makefile \
plugins/omgssapi/Makefile \
plugins/ommysql/Makefile \
@ -1330,11 +1473,13 @@ AC_CONFIG_FILES([Makefile \
plugins/omoracle/Makefile \
plugins/omudpspoof/Makefile \
plugins/ommongodb/Makefile \
plugins/omhiredis/Makefile \
plugins/omzmq3/Makefile \
plugins/omhiredis/Makefile \
plugins/omzmq3/Makefile \
plugins/omrabbitmq/Makefile \
plugins/mmnormalize/Makefile \
plugins/mmjsonparse/Makefile \
plugins/mmaudit/Makefile \
plugins/mmanon/Makefile \
plugins/omelasticsearch/Makefile \
plugins/sm_cust_bindcdr/Makefile \
plugins/mmsnmptrapd/Makefile \
@ -1352,8 +1497,12 @@ echo " Zlib compression support enabled: $enable_zlib"
echo " rsyslog runtime will be built: $enable_rsyslogrt"
echo " rsyslogd will be built: $enable_rsyslogd"
echo " GUI components will be built: $enable_gui"
echo " cached man files will be used: $enable_cached_man_pages"
echo " Unlimited select() support enabled: $enable_unlimited_select"
echo " uuid support enabled: $enable_uuid"
echo " Log file signing support: $enable_guardtime"
echo " Log file encryption support: $enable_libgcrypt"
echo " anonymization support enabled: $enable_mmanon"
echo
echo "---{ input plugins }---"
echo " Klog functionality enabled: $enable_klog ($os_type)"
@ -1365,17 +1514,20 @@ echo " file input module enabled: $enable_imfile"
echo " Solaris input module enabled: $enable_imsolaris"
echo " periodic statistics module enabled: $enable_impstats"
echo " imzmq3 input module enabled: $enable_imzmq3"
echo " imjournal input module enabled: $enable_imjournal"
echo
echo "---{ output plugins }---"
echo " Mail support enabled: $enable_mail"
echo " omprog module will be compiled: $enable_omprog"
echo " omstdout module will be compiled: $enable_omstdout"
echo " omjournal module will be compiled: $enable_omjournal"
echo " omhdfs module will be compiled: $enable_omhdfs"
echo " omelasticsearch module will be compiled: $enable_elasticsearch"
echo " omruleset module will be compiled: $enable_omruleset"
echo " omudpspoof module will be compiled: $enable_omudpspoof"
echo " omuxsock module will be compiled: $enable_omuxsock"
echo " omzmq3 module will be compiled: $enable_omzmq3"
echo " omrabbitmq module will be compiled: $enable_omrabbitmq"
echo
echo "---{ parser modules }---"
echo " pmrfc3164sd module will be compiled: $enable_pmrfc3164sd"

20
dirty.h
View File

@ -27,26 +27,18 @@
#ifndef DIRTY_H_INCLUDED
#define DIRTY_H_INCLUDED 1
rsRetVal multiSubmitMsg(multi_submit_t *pMultiSub);
rsRetVal submitMsg(msg_t *pMsg);
rsRetVal __attribute__((deprecated)) multiSubmitMsg(multi_submit_t *pMultiSub);
rsRetVal multiSubmitMsg2(multi_submit_t *pMultiSub); /* friends only! */
rsRetVal submitMsg2(msg_t *pMsg);
rsRetVal __attribute__((deprecated)) submitMsg(msg_t *pMsg);
rsRetVal multiSubmitFlush(multi_submit_t *pMultiSub);
rsRetVal logmsgInternal(int iErr, int pri, uchar *msg, int flags);
rsRetVal parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlTypeu, prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset);
rsRetVal __attribute__((deprecated)) parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlTypeu, prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset);
rsRetVal diagGetMainMsgQSize(int *piSize); /* for imdiag */
rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName, struct cnfparamvals *queueParams);
/* Intervals at which we flush out "message repeated" messages,
* in seconds after previous message is logged. After each flush,
* we move to the next interval until we reach the largest.
* TODO: move this to action object! Only action.c and syslogd.c use it.
*/
extern int MarkInterval;
extern int repeatinterval[2];
extern qqueue_t *pMsgQueue; /* the main message queue */
extern int iConfigVerify; /* is this just a config verify run? */
extern int bHaveMainQueue;
#define MAXREPEAT ((int)((sizeof(repeatinterval) / sizeof(repeatinterval[0])) - 1))
#define REPEATTIME(f) ((f)->f_time + repeatinterval[(f)->f_repeatcount])
#define BACKOFF(f) { if (++(f)->f_repeatcount > MAXREPEAT) \
(f)->f_repeatcount = MAXREPEAT; \
}
#endif /* #ifndef DIRTY_H_INCLUDED */

6
doc/Makefile.am
View File

@ -34,11 +34,15 @@ html_files = \
ompipe.html \
omfwd.html \
omfile.html \
omjournal.html \
imjournal.html \
mmanon.html \
omusrmsg.html \
omstdout.html \
omudpspoof.html \
omruleset.html \
omsnmp.html \
sigprov_gt.html \
ommysql.html \
omoracle.html \
omlibdbi.html \
@ -70,6 +74,7 @@ html_files = \
tls_cert_client.html \
tls_cert_scenario.html \
rainerscript.html \
lookup_tables.html \
rscript_abnf.html \
rsconf1_actionexeconlywhenpreviousissuspended.html \
rsconf1_actionresumeinterval.html \
@ -115,6 +120,7 @@ html_files = \
gssapi.html \
licensing.html \
mmnormalize.html \
mmjsonparse.html \
ommail.html \
omuxsock.html \
omrelp.html \

187
doc/confsamples/normalization.conf Normal file
View File

@ -0,0 +1,187 @@
# this is a config sample for log normalization, but can
# be used as a more complex general sample.
# It is based on a plain standard rsyslog.conf for Red Hat systems.
#
# NOTE: Absolute path names for modules are used in this config
# so that we can run a different rsyslog version alongside the
# regular system-installed rsyslogd. Remove these path names
# for production environment.
#### MODULES ####
# we do not run imuxsock as we don't want to mess with the main system logger
#module(load="/home/rger/proj/rsyslog/plugins/imuxsock/.libs/imuxsock") # provides support for local system logging (e.g. via logger command)
#module(load="imklog") # provides kernel logging support (previously done by rklogd)
module(load="/home/rger/proj/rsyslog/plugins/imudp/.libs/imudp") # Provides UDP syslog reception
module(load="/home/rger/proj/rsyslog/plugins/imtcp/.libs/imtcp")
module(load="/home/rger/proj/rsyslog/plugins/mmjsonparse/.libs/mmjsonparse")
module(load="/home/rger/proj/rsyslog/plugins/mmnormalize/.libs/mmnormalize")
/* We assume to have all TCP logging (for simplicity)
* Note that we use different ports to point different sources
* to the right rule sets for normalization. While there are
* other methods (e.g. based on tag or source), using multiple
* ports is both the easiest as well as the fastest.
*/
input(type="imtcp" port="13514" Ruleset="WindowsRsyslog")
input(type="imtcp" port="13515" Ruleset="LinuxPlainText")
input(type="imtcp" port="13516" Ruleset="WindowsSnare")
#debug:
action(type="omfile" file="/home/rger/proj/rsyslog/logfile")
/* This ruleset handles structured logging.
* It is the only one ever called for remote machines
* but executed in addition to the standard action for
* the local machine. The ultimate goal is to forward
* to some Vendor's analysis tool (which digests a
* structured log format, here we use Lumberjack).
*/
template(name="lumberjack" type="string" string="%$!all-json%\n")
/* the rsyslog Windows Agent uses native Lumberjack format
* (better said: is configured to use it)
*/
ruleset(name="WindowsRsyslog") {
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
if $!id == 4634 then
set $!usr!type = "logoff";
else if $!id == 4624 then
set $!usr!type = "logon";
set $!usr!rcvdfrom = $!source;
set $!usr!rcvdat = $timereported;
set $!usr!user = $!TargetDomainName & "\\" & $!TargetUserName;
call outwriter
}
}
/* This handles clumsy snare format. Note that "#011" are
* the escape sequences for tab chars used by snare.
*/
ruleset(name="WindowsSnare") {
set $!usr!type = field($rawmsg, "#011", 6);
if $!usr!type == 4634 then {
set $!usr!type = "logoff";
set $!doProces = 1;
} else if $!usr!type == 4624 then {
set $!usr!type = "logon";
set $!doProces = 1;
} else
set $!doProces = 0;
if $!doProces == 1 then {
set $!usr!rcvdfrom = field($rawmsg, 32, 4);
set $!usr!rcvdat = field($rawmsg, "#011", 5);
/* we need to fix up the snare date */
set $!usr!rcvdat = field($!usr!rcvdat, 32, 2) & " " &
field($!usr!rcvdat, 32, 3) & " " &
field($!usr!rcvdat, 32, 4);
set $!usr!user = field($rawmsg, "#011", 8);
call outwriter
}
}
/* plain Linux log messages (here: ssh and sudo) need to be
* parsed - we use mmnormalize for fast and efficient parsing
* here.
*/
ruleset(name="LinuxPlainText") {
action(type="mmnormalize"
rulebase="/home/rger/proj/rsyslog/linux.rb" userawmsg="on")
if $parsesuccess == "OK" and $!user != "" then {
if $!type == "opened" then
set $!usr!type = "logon";
else if $!type == "closed" then
set $!usr!type = "logoff";
set $!usr!rcvdfrom = $!rcvdfrom;
set $!usr!rcvdat = $!rcvdat;
set $!usr!user = $!user;
call outwriter
}
}
/* with CSV, we the reader must receive information on the
* field names via some other method (e.g. tool configuration,
* prepending of a header to the written CSV-file). All of
* this is highly dependant on the actual CSV dialect needed.
* Below, we cover the basics.
*/
template(name="csv" type="list") {
property(name="$!usr!rcvdat" format="csv")
constant(value=",")
property(name="$!usr!rcvdfrom" format="csv")
constant(value=",")
property(name="$!usr!user" format="csv")
constant(value=",")
property(name="$!usr!type" format="csv")
constant(value="\n")
}
/* template for Lumberjack-style logging. Note that the extra
* LF at the end is just for wrinting it to file - it MUST NOT
* be included for messages intended to be sent to a remote system.
* For the latter use case, the syslog header must also be prepended,
* something we have also not done for simplicity (as we write to files).
* Note that we use a JSON-shortcut: If a tree name is specified, JSON
* for its whole subtree is generated. Thus, we only need to specify the
* $!usr top node to get everytihing we need.
*/
template(name="cee" type="string" string="@cee: %$!usr%\n")
/* this ruleset simulates forwarding to the final destination */
ruleset(name="outwriter"){
action(type="omfile"
file="/home/rger/proj/rsyslog/logfile.csv" template="csv")
action(type="omfile"
file="/home/rger/proj/rsyslog/logfile.cee" template="cee")
}
/* below is just the usual "uninteresting" stuff...
* Note that this goes into the default rule set. So
* local logging is handled "as usual" without the need
* for any extra effort.
*/
#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# Include all config files in /etc/rsyslog.d/
# commented out not to interfere with the system rsyslogd
# (just for this test configuration!)
#$IncludeConfig /etc/rsyslog.d/*.conf
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log

121
doc/cryprov_gcry.html Normal file
View File

@ -0,0 +1,121 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Language" content="en">
<title>libgcryt Log Crypto Provider (gcry)</title>
</head>
<body>
<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a>
<h1>libgcrypt Log Crypto Provider (gcry)</h1>
<p><b>Signature Provider Name:&nbsp;&nbsp;&nbsp; gt</b></p>
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Supported Since: </b>since 7.3.10
<p><b>Description</b>:</p>
<p>Provides encryption support to rsyslog.
</p>
<p><b>Configuration Parameters</b>:</p>
<p>Crypto providers are loaded by omfile, when the
provider is selected in its "cry.providerName" parameter.
Parameters for the provider are given in the omfile action instance
line.
<p>This provider creates an encryption information file with the same base name but
the extension ".encinfo" for each log file (both for fixed-name files
as well as dynafiles). Both files together form a set. So you need to
archive both in order to prove integrity.
<ul>
<li><b>cry.algo</b> &lt;Encryption Algorithm&gt;<br>
The algorithm (cipher) to be used for encryption.
The default algorithm is "AES128".
<br>Currently, the following Algorithms are supported:
<ul>
<li>3DES
<li>CAST5
<li>BLOWFISH
<li>AES128
<li>AES192
<li>AES256
<li>TWOFISH
<li>TWOFISH128
<li>ARCFOUR
<li>DES
<li>SERPENT128
<li>SERPENT192
<li>SERPENT256
<li>RFC2268_40
<li>SEED
<li>CAMELLIA128
<li>CAMELLIA192
<li>CAMELLIA256
</ul>
<br>
The actual availability of an algorithms depends on which ones
are compiled into libgcrypt. Note that some versions of libgcrypt
simply abort the process (rsyslogd in this case!) if a supported
algorithm is select but not available due to libgcrypt build
settings. There is nothing rsyslog can do against this. So in
order to avoid production downtime, always check carefully when
you change the algorithm.
</li>
<li><b>cry.mode</b> &lt;Algorithm Mode&gt;<br>
The encryption mode to be used. Default ist Cipher Block Chaining (CBC).
Note that not all encryption modes can be used together with all
algorithms.
<br>Currently, the following modes are supported:
<ul>
<li>ECB
<li>CFB
<li>CBC
<li>STREAM
<li>OFB
<li>CTR
<li>AESWRAP
</ul>
<li><b>cry.key</b> &lt;encryption key&gt;<br>
TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified
inside rsyslog.conf. This is the actual key, and as such this mode
is highly insecure. However, it can be useful for intial testing
steps. This option may be removed in the future.
</li>
<li><b>cry.keyfile</b> &lt;filename&gt;<br>
Reads the key from the specified file. The file must contain the key, only,
no headers or other meta information. Keyfiles can be generated via the
rscrytool utility.
</li>
<li><b>cry.keyprogram</b> &lt;path to program&gt;<br>
If given, the key is provided by a so-called "key program". This program
is executed and must return the key to (as well as some meta information)
via stdout. The core idea of key programs is that using this interface the
user can implement as complex (and secure) method to obtain keys as
desired, all without the need to make modifications to rsyslog.
</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
<li>currently none known
</li>
</ul>
<p><b>Samples:</b></p>
<p>This encrypts a log file. Default parameters are used, they key is
provided via a keyfile.
</p>
<textarea rows="3" cols="60">
action(type="omfile" file="/var/log/somelog"
cry.provider="gcry" keyfile="/secured/path/to/keyfile")
</textarea>
Note that the keyfile can be generated via the rscrytool utility (see its
documentation for how to actually do that).
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2013 by
<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body></html>

28
doc/debug.html
View File

@ -49,8 +49,14 @@ FileTrace=vm.c FileTrace=expr.c"</li>
<li><b>Debug</b> - if present, turns on the debug system and enables debug output
<li><b>DebugOnDemand</b> - if present, turns on the debug system but does not enable
debug output itself. You need to send SIGUSR1 to turn it on when desired.
<li><b>OutputTidToStderr</b> - if present, makes rsyslog output information about
the thread id (tid) of newly create processesto stderr. Note that not necessarily
all new threads are reported (depends on the code, e.g. of plugins). This is
only available under Linux. This usually does NOT work when privileges have
been dropped (that's not a bug, but the way it is).
<li><b>help</b> - display a very short list of commands - hopefully a life saver if you can't access the documentation...</li>
</ul>
<p>Individual options are separated by spaces.</p>
</ul>
<h3>Why Environment Variables?</h3>
<p>You may ask why we use environment variables for debug-system parameters and not
@ -70,6 +76,26 @@ rsyslog core, we get a number of data structures wrong.
<p>For these reasons, we utilize environment variables to initialize and configure
the debugging system. We understand this may be somewhat painful, but now you know
there are at least some good reasons for doing so.
<p>HOWEVER, if you have a too hard time to set debug instructions using the environment
variables, there is a cure, described in the next paragraph.
<h2>Enabling Debug via rsyslog.conf</h2>
<p>As described in the previous paragraph, enabling debug via rsyslog.conf
may not be perfect for some debugging needs, but basic debug output will work - and
that is what most often is requried. There are limited options available, but these
cover the most important use cases.
<p>Debug processing is done via legacy config statements. There currently
is no plan to move these over to the v6+ config system. Availabe settings are
<ul>
<li>$DebugFile &lt;filename&gt; - sets the debug file name
<li>$DebugLevel &lt;0|1|2&gt; - sets the respective debug level, where
0 means debug off, 1 is debug on demand activated (but debug mode off)
and 2 is full debug mode.
</ul>
<p>Note that in theory it is forbidden to specify these parameters more
than once. However, we do not enforce that and if it happens results
are undefined.
<h2>Getting debug information from a running Instance</h2>
<p>It is possible to obtain debugging information from a running instance, but this requires
some setup. We assume that the instance runs in the background, so debug output to
@ -138,7 +164,7 @@ instance of rsyslogd can be aborted by pressing ctl-c.
<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body>

31
doc/dev_oplugins.html
View File

@ -18,19 +18,10 @@ means they are primarily thought of being message sinks. In theory, however, out
plugins may aggergate other functionality, too. Nobody has taken this route so far
so if you would like to do that, it is highly suggested to post your plan on the
rsyslog mailing list, first (so that we can offer advise).
<p>The rsyslog distribution tarball contains two plugins that are extremely well
targeted for getting started:
<ul>
<li>omtemplate
<li>omstdout
</ul>
Plugin omtemplate was specifically created to provide a copy template for new output
plugins. It is bare of real functionality but has ample comments. Even if you decide
to start from another plugin (or even from scratch), be sure to read omtemplate source
and comments first. The omstdout is primarily a testing aide, but offers support for
the two different parameter-passing conventions plugins can use (plus the way to
differentiate between the two). It also is not bare of functionaly, only mostly
bare of it ;). But you can actually execute it and play with it.
<p>The rsyslog distribution tarball contains the omstdout plugin which is extremely well
targeted for getting started. Just note that this plugin itself is not meant for
production use. But it is very simplistic and so a really good starting point to
grasp the core ideas.
<p>In any case, you should also read the comments in ./runtime/module-template.h.
Output plugins are build based on a large set of code-generating macros. These
macros handle most of the plumbing needed by the interface. As long as no
@ -38,19 +29,7 @@ special callback to rsyslog is needed (it typically is not), an output plugin do
not really need to be aware that it is executed by rsyslog. As a plug-in programmer,
you can (in most cases) "code as usual". However, all macros and entry points need to be
provided and thus reading the code comments in the files mentioned is highly suggested.
<p>In short, the best idea is to start with a template. Let's assume you start by
copying omtemplate. Then, the basic steps you need to do are:
<ul>
<li>cp ./plugins/omtemplate ./plugins/your-plugin
<li>mv cd ./plugins/your-plugin
<li>vi Makefile.am, adjust to your-plugin
<li>mv omtemplate.c your-plugin.c
<li>cd ../..
<li>vi Makefile.am configure.ac
<br>search for omtemplate, copy and modify (follow comments)
</ul>
<p>Basically, this is all you need to do ... Well, except, of course, coding
your plugin ;). For testing, you need rsyslog's debugging support. Some useful
<p>For testing, you need rsyslog's debugging support. Some useful
information is given in "<a href="troubleshoot.html">troubleshooting rsyslog</a>
from the doc set.
<h2>Special Topics</h2>

22
doc/history.html
View File

@ -118,7 +118,25 @@ use case. On February, 28th rsyslog 3.12.0 was released, the first
version to contain expression support. This also meant that rsyslog
from that date on supported all syslog-ng major features, but had a
number of major features exlusive to it. With 3.12.0, I consider
rsyslog fully superior to syslog-ng (except for platform support).</p><p>Be sure to visit Rainer's <a href="http://rgerhards.blogspot.com/">syslog blog</a>
rsyslog fully superior to syslog-ng (except for platform support).</p>
<p>Following the Fedora Developer's conference in Brno <b>2012</b>, rsyslog
got very serious on implementing <b>structured logging</b> in
project Lumberjack (CEE) style. Project Lumberjack was a much broader
effort and brought closer collaboration with the syslog-ng folks, which
helped to maintain and improve interoperability. In the
<b>late winter/spring/summer 2012</b> timeframe numerous engine enhancements
were made and plugins written (among them the first "official" interfaces
to the Linux audit subsystem). At the end of the year, this culminated in the
rsyslog 7, which not only implemented Lumberjack but also was the first one
to support full condition nesting in rsyslog.conf (and a ton of other features as
well).
<p>In <b>spring 2013</b> major new security features were engineered,
namely anonymization support, as well as log file signing and
encryption capabilities.
<p>Be sure to visit Rainer's <a href="http://rgerhards.blogspot.com/">syslog blog</a>
to get some more insight into the development and futures of rsyslog and syslog in general.
Don't be shy to post to either the blog or the
<a href="http://www.rsyslog.com/PNphpBB2.phtml">rsyslog forums</a>.</p>
@ -126,4 +144,4 @@ Don't be shy to post to either the blog or the
<ul>
<li><a href="http://www.rsyslog.com/Topic4.phtml">the rsyslog change log</a></li>
</ul>
</body></html>
</body></html>

49
doc/imfile.html
View File

@ -14,7 +14,7 @@ a syslog message. A standard
text file is a file consisting of printable characters with lines
being&nbsp;delimited by LF.</p>
<p>The file is read line-by-line and any line read is passed to
rsyslog's rule engine. The rule engine applies filter conditons and
rsyslog's rule engine. The rule engine applies filter conditions and
selects which actions needs to be carried out. Empty lines are <b>not</b>
processed, as they would result in empty syslog records. They are simply
ignored.</p>
@ -49,9 +49,9 @@ releases of imfile may support per-file polling intervals, but
currently this is not the case. If multiple PollingInterval
statements are present in rsyslog.conf, only the last one is used.<br>
A short poll interval provides more rapid message forwarding, but
requires more system ressources. While it is possible, we stongly
requires more system resources. While it is possible, we stongly
recommend not to set the polling interval to 0 seconds. That will make
rsyslogd become a CPU hog, taking up considerable ressources. It is
rsyslogd become a CPU hog, taking up considerable resources. It is
supported, however, for the few very unusual situations where this
level may be needed. Even if you need quick response, 1 seconds should
be well enough. Please note that imfile keeps reading files as long as
@ -61,15 +61,15 @@ nothing is left to be processed.</li>
<p><b>Action Directives</b></p>
<ul>
<li><strong>File&nbsp;/path/to/file</strong><br>
<li><strong>(required) File&nbsp;/path/to/file</strong><br>
The file being monitored. So far, this must be an absolute name (no
macros or templates)</li>
<li><span style="font-weight: bold;">Tag
<li><span style="font-weight: bold;">(required) Tag
tag:</span><br>
The tag to be used for messages that originate from this file. If you
would like to see the colon after the tag, you need to specify it here
(as shown above).</li>
<li><span style="font-weight: bold;">StateFile
<li><span style="font-weight: bold;">(required) StateFile
&lt;name-of-state-file&gt;</span><br>
Rsyslog must keep track of which parts of the to be monitored file it
already processed. This is done in the state file. This file always is
@ -77,7 +77,9 @@ created in the rsyslog working directory (configurable via
$WorkDirectory). Be careful to use unique names for different files
being monitored. If there are duplicates, all sorts of "interesting"
things may happen. Rsyslog currently does not check if a name is
specified multiple times.</li>
specified multiple times.
Note that when $WorkDirectory is not set or set to a non-writable
location, the state file will not be generated.</li>
<li><span style="font-weight: bold;">Facility
facility</span><br>
The syslog facility to be assigned to lines read. Can be specified in
@ -92,7 +94,7 @@ textual form (e.g. "info", "warning", ...) or as numbers (e.g. 4 for
is "notice".</li>
<li><b>PersistStateInterval</b> [lines]</b><br>
Specifies how often the state file shall be written when processing the input
file. The default value is 0, which means a new state file is only written when
file. The <strong>default</strong> value is 0, which means a new state file is only written when
the monitored files is being closed (end of rsyslogd execution). Any other
value n means that the state file is written every time n file lines have
been processed. This setting can be used to guard against message duplication due
@ -101,9 +103,9 @@ performance, especially when set to a low value. Frequently writing the state
file is very time consuming.
<li><b>ReadMode</b> [mode]</b><br>
This mode should defined when having multiline messages. The value can range from 0-2 and determines the multiline detection method.
<br>0 (default) - line based (Each line is a new message)
<br>1 - indented (New log messages start at the beginning of a line. If a line starts with a space it is part of the log message before it)
<br>2 - paragraph (There is a blank line between log messages)
<br>0 (<strong>default</strong>) - line based (Each line is a new message)
<br>1 - paragraph (There is a blank line between log messages)
<br>2 - indented (New log messages start at the beginning of a line. If a line starts with a space it is part of the log message before it)
<li><b>MaxLinesAtOnce</b> [number]</b>
<br>
This is useful if multiple files need to be monitored. If set to 0, each file
@ -112,11 +114,11 @@ will be fully processed and then processing switches to the next file
[number] lines is processed in sequence for each file, and then the file is
switched. This provides a kind of mutiplexing the load of multiple files and
probably leads to a more natural distribution of events when multiple busy files
are monitored. The default is 1024.
are monitored. The <strong>default</strong> is 1024.
<li><b>MaxSubmitAtOnce</b> [number]</b>
<br>
This is an expert option. It can be used to set the maximum input batch size that
imfile can generate. The default is 1024, which is suitable for a wide range of
imfile can generate. The <strong>default</strong> is 1024, which is suitable for a wide range of
applications. Be sure to understand rsyslog message batch processing before you
modify this option. If you do not know what this doc here talks about, this is a
good indication that you should NOT modify the default.
@ -141,17 +143,17 @@ your distro puts rsyslog's config files). Note that only commands
actually needed need to be specified. The second file uses less
commands and uses defaults instead.<br>
</p>
<textarea rows="15" cols="60">module(load="folder/to/rsyslog/plugins/imfile/.libs/imfile" PollingInterval="10") #needs to be done just once
<textarea rows="15" cols="60">module(load="imfile" PollingInterval="10") #needs to be done just once
# File 1
input(type="imfile" File="/path/to/file1"
Tag="tag1"
StateFile="/var/spool/rsyslog/statefile1"
Severity="error"
Facility="local7")
Tag="tag1"
StateFile="statefile1"
Severity="error"
Facility="local7")
# File 2
input(type="imfile" File="/path/to/file2"
Tag="tag2"
StateFile="/var/spool/rsyslog/statefile2")
Tag="tag2"
StateFile="statefile2")
# ... and so on ...
#
</textarea>
@ -178,7 +180,7 @@ the current monitor. It has no parameters. If you forget this
directive, no file monitoring will take place.</li>
<li><span style="font-weight: bold;">$InputFilePollInterval
seconds</span><br>
equivalent to: PollingInterva</li>
equivalent to: PollingInterval</li>
<li><b>$InputFilePersistStateInterval</b> [lines]</b><br>
Available in 4.7.3+, 5.6.2+<br>
equivalent to: PersistStateInterval
@ -210,8 +212,7 @@ your distro puts rsyslog's config files). Note that only commands
actually needed need to be specified. The second file uses less
commands and uses defaults instead.<br>
</p>
<textarea rows="15" cols="60">$ModLoad imfile #
needs to be done just once
<textarea rows="15" cols="60">$ModLoad imfile # needs to be done just once
# File 1
$InputFileName /path/to/file1
$InputFileTag tag1:
@ -227,7 +228,7 @@ $InputRunFileMonitor
# ... and so on ...
#
# check for new lines every 10 seconds
$InputFilePollingInterval 10
$InputFilePollInterval 10
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>

111
doc/imjournal.html Normal file
View File

@ -0,0 +1,111 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<meta http-equiv="Content-Language" content="en"><title>Systemd Journal Input Module</title></head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>Systemd Journal Input Module</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; imjournal</b></p>
<p><b>Author: </b>Milan Bartos
&lt;mbartos@redhat.com&gt;</p>
<p><b>Description</b>:</p>
<p>Provides the ability to import structured log messages from systemd journal
to syslog.</p>
<p>Note that this module reads the journal database, what is considered a
relativly performance-intense operation. As such, the performance of a
configuration utilizing this
module may be notably slower then when using
<a href="imuxsock.html">imuxsock</a>. The journal provides imuxsock with a
copy of all "classical" syslog messages, however, it does not provide
structured data. If the latter is needed, imjournal must be used. Otherwise,
imjournal may be simply replaced by imuxsock.
<p>We suggest to check out our short presentation on
<a href="http://youtu.be/GTS7EuSdFKE">rsyslog journal integration</a> to
learn more details of anticipated use cases.
<p><b>Warning:</b> Some versions of systemd journal have problems with database
corruption, which leads to the journal to return the same data endlessly
in a thight loop. This results in massive message duplication inside rsyslog
probably resulting in a denial-of-service when the system ressouces get
exhausted. This can be somewhat mitigated by using proper rate-limiters, but
even then there are spikes of old data which are endlessly repeated. By default,
ratelimiting is activated and permits to process 20,000 messages within 10
minutes, what should be well enough for most use cases. If insufficient, use
the parameters described below to adjust the permitted volume.
<b>It is strongly recommended to use this plugin only if there
is hard need to do so.</b>
<p><b>Configuration Directives</b>:</p>
<p><b>Module Directives</b></p>
<ul>
<li><b>PersistStateInterval</b> number-of-messages<br>
This is a global setting. It specifies how often should the journal state be persisted.
The persists happens after each <i>number-of-messages</i>.
This option is useful for rsyslog to start reding from the last journal message it read.
<li><b>StateFile</b> /path/to/file<br>
This is a global setting. It specifies where the state file for persisting
journal state is located.
<li><b>ratelimit.interval</b> seconds (default: 600)<br>
Specifies the interval in seconds onto which rate-limiting is to be applied.
If more than ratelimit.burst messages are read during that interval, further
messages up to the end of the interval are discarded. The number of messages
discarded is emitted at the end of the interval (if there were any discards).
<br>Setting this to value zero turns off ratelimiting. Note that it is
<b>not recommended to turn of ratelimiting</b>, except that you know for
sure journal database entries will never be corrupted. Without ratelimiting,
a corrupted systemd journal database may cause a kind of denial of service (we
are stressing this point as multiple users have reported us such problems
with the journal database - information current as of June 2013).
<li><b>ratelimit.burst</b> messages (default: 20000)<br>
Specifies the maximum number of messages that can be emitted within the
ratelimit.interval interval. For futher information, see description there.
<li><b>IgnorePreviousMessages</b> [<b>off</b>/on]<br>
This option specifies whether imjournal should ignore messages currently in
journal and read only new messages. This option is only used when there is
no StateFile to avoid message loss.
</ul>
<b>Caveats/Known Bugs:</b>
<p>
<ul>
<li>As stated above, a corrupted systemd journal database can cause major
problems, depending on what the corruption results in. This is beyond the
control of the rsyslog team.
</ul>
</p>
<p><b>Sample:</b></p>
<p>
The following example shows pulling structured imjournal messages and saving them into /var/log/ceelog.
</p>
<textarea rows="11" cols="80">
module(load="imjournal" PersistStateInterval="100" StateFile="/path/to/file") #load imjournal module
module(load="mmjsonparse") #load mmjsonparse module for structured logs
template(name="CEETemplate" type="string"
string="%TIMESTAMP% %HOSTNAME% %syslogtag% @cee: %$!all-json%\n"
) #template for messages
action(type="mmjsonparse")
action(type="omfile" file="/var/log/ceelog" template="CEETemplate")
</textarea>
<p><b>Legacy Configuration Directives</b>:</p>
<ul>
<li><b>$imjournalPersistStateInterval</b><br>
Equivalent to: PersistStateInterval</li>
<li><b>$imjournalStateFile</b><br>
Equivalent to: StateFile</li>
<li><b>$imjournalRatelimitInterval</b><br>
Equivalent to: ratelimit.interval</li>
<li><b>$imjournalRatelimitBurst</b><br>
Equivalent to: ratelimit.burst</li>
<li><strong>$ImjournalIgnorePreviousMessages</strong><br>
Equivalent to: ignorePreviousMessages</li>
</ul>
</body>
</html>

96
doc/imklog.html
View File

@ -15,7 +15,10 @@
syslog engine.</p>
<p><b>Configuration Directives</b>:</p>
<ul>
<li><strong>$KLogInternalMsgFacility
<li><strong>LogPath</strong><br>
The path to the Kernel log. This value should only be changed if you really know what
you are doing.</li>
<li><strong>InternalMsgFacility
&lt;facility&gt;</strong><br>
The facility which messages internally generated by imklog will have.
imklog generates some messages of itself (e.g. on problems, startup and
@ -26,13 +29,54 @@ need to specify this configuratin directive - it is included primarily
for few limited cases where it is needed for good reason. Bottom line:
if you don't have a good idea why you should use this setting, do not
touch it.</li>
<li><span style="font-weight: bold;">$KLogPermitNonKernelFacility
[on/<span style="font-style: italic;">off</span>]<br>
</span>At least under BSD the kernel log may contain entries
<li><b>PermitNonKernelFacility [on/<i>off</i>]</b><br>
At least under BSD the kernel log may contain entries
with non-kernel facilities. This setting controls how those are
handled. The default is "off", in which case these messages are
ignored. Switch it to on to submit non-kernel messages to rsyslog
processing.<span style="font-weight: bold;"></span></li>
processing.</li>
<li><b>ParseKernelTimeStamp</b> [on/<b>off</b>]<br>
If enabled and the kernel creates a timestamp for its log messages, this timestamp will be
parsed and converted into regular message time instead to use the receive time of the kernel
message (as in 5.8.x and before). Default is to not parse the kernel timestamp, because the
clock used by the kernel to create the timestamps is not supposed to be as accurate as the
monotonic clock required to convert it. Depending on the hardware and kernel, it can result
in message time differences between kernel and system messages which occurred at same time.
<li><b>KeepKernelTimeStamp</b> [on/<b>off</b>]<br>
If enabled, this option causes to keep the [timestamp] provided by the kernel at the begin
of in each message rather than to remove it, when it could be parsed and converted into
local time for use as regular message time. Only used when <b>ParseKernelTimestamp</b> is on.
<li><b>ConsoleLogLevel</b> [<i>number</i>]
(former klogd -c option) -- sets the console log level. If specified, only messages with
up to the specified level are printed to the console. The default is -1, which means that
the current settings are not modified. To get this behavior, do not specify
ConsoleLogLevel in the configuration file. Note that this is a global parameter. Each time
it is changed, the previous definition is re-set. The one activate will be that one that is
active when imklog actually starts processing. In short words: do not specify this
directive more than once!
</ul>
<b>Caveats/Known Bugs:</b>
<p>This is obviously platform specific and requires platform
drivers.
Currently, imklog functionality is available on Linux and BSD.</p>
<p>This module is <b>not supported on Solaris</b> and not needed there.
For Solaris kernel input, use <a href="imsolaris.html">imsolaris</a>.</p>
<p><b>Sample:</b></p>
<p>The following sample pulls messages from the kernel log. All
parameters are left by default, which is usually a good idea. Please
note that loading the plugin is sufficient to activate it. No directive
is needed to start pulling kernel messages.<br>
</p>
<textarea rows="4" cols="60">module(load="imklog")
</textarea>
<p><b>Legacy Configuration Directives</b>:</p>
<ul>
<li><strong>$KLogInternalMsgFacility
&lt;facility&gt;</strong><br>
equivalent to: InternalMsgFacility</li>
<li><span style="font-weight: bold;">$KLogPermitNonKernelFacility
[on/<span style="font-style: italic;">off</span>]<br>
equivalent to: PermitNonKernelFacility</li>
<li><span style="font-weight: bold;"></span>$DebugPrintKernelSymbols
[on/<b>off</b>]<br>
Linux only, ignored on other platforms (but may be specified)</li>
@ -50,14 +94,7 @@ it except if you have a very good reason. If you have one, let us know
because otherwise new versions will no longer support it.<br>
Linux only, ignored on other platforms (but may be specified)</li>
<li><b>$klogConsoleLogLevel</b> [<i>number</i>]
(former klogd -c option) -- sets the console log level. If specified, only messages with
up to the specified level are printed to the console. The default is -1, which means that
the current settings are not modified. To get this behavior, do not specify
$klogConsoleLogLevel in the configuration file. Note that this is a global parameter. Each time
it is changed, the previous definition is re-set. The one activate will be that one that is
active when imklog actually starts processing. In short words: do not specify this
directive more than once!
<br><b>Linux only</b>, ignored on other platforms (but may be specified)</li>
<br>equivalent to: ConsoleLogLevel</li>
<li><b>$klogUseSyscallInterface</b> [on/<b>off</b>]
-- former klogd -s option<br>
Linux only, ignored on other platforms (but may be specified)</li>
@ -65,40 +102,17 @@ Linux only, ignored on other platforms (but may be specified)</li>
former klogd -2 option<br>
Linux only, ignored on other platforms (but may be specified)<br style="font-weight: bold;">
</li>
<li><b>$klogParseKernelTimestamp</b> [on/<b>off</b>]
If enabled and the kernel creates a timestamp for its log messages, this timestamp will be
parsed and converted into regular message time instead to use the receive time of the kernel
message (as in 5.8.x and before). Default is to not parse the kernel timestamp, because the
clock used by the kernel to create the timestamps is not supposed to be as accurate as the
monotonic clock required to convert it. Depending on the hardware and kernel, it can result
in message time differences between kernel and system messages which occurred at same time.
</li>
<li><b>$klogKeepKernelTimestamp</b> [on/<b>off</b>]
If enabled, this option causes to keep the [timestamp] provided by the kernel at the begin
of in each message rather than to remove it, when it could be parsed and converted into
local time for use as regular message time. Only used, when $klogParseKernelTimestamp is on.
</li>
<li><b>$klogParseKernelTimeStamp</b> [on/<b>off</b>]<br>
equivalent to: ParseKernelTimeStamp</li>
<li><b>$klogKeepKernelTimeStamp</b> [on/<b>off</b>]<br>
equivalent to: KeepKernelTimeStamp</li>
</ul>
<b>Caveats/Known Bugs:</b>
<p>This is obviously platform specific and requires platform
drivers.
Currently, imklog functionality is available on Linux and BSD.</p>
<p>This module is <b>not supported on Solaris</b> and not needed there.
For Solaris kernel input, use <a href="imsolaris.html">imsolaris</a>.</p>
<p><b>Sample:</b></p>
<p>The following sample pulls messages from the kernel log. All
parameters are left by default, which is usually a good idea. Please
note that loading the plugin is sufficient to activate it. No directive
is needed to start pulling kernel messages.<br>
</p>
<textarea rows="15" cols="60">$ModLoad imklog
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2008-2009 by <a href="http://www.gerhards.net/rainer">Rainer
Copyright &copy; 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>

100
doc/impstats.html
View File

@ -16,26 +16,66 @@ availabilty and format of counters may change and is not yet stable (so be
prepared to change your trending scripts when you upgrade to a newer rsyslog version).
<p>The set of available counters will be output as a set of syslog messages. This
output is periodic, with the interval being configurable (default is 5 minutes).
Be sure that your configuration records the counter messages (default is syslog.info).
Be sure that your configuration records the counter messages (default is syslog.=info).
Besides logging to the regular syslog stream, the module can also be configured to
write statistics data into a (local) file.
<p>Note that loading this module has impact on rsyslog performance. Depending on
settings, this impact may be noticable (for high-load environments).
<p>The rsyslog website has an updated overview of available
<a href="http://rsyslog.com/rsyslog-statistic-counter/">rsyslog statistic counters</a>.
</p>
<p><b>Configuration Directives</b>:</p>
<p><b>Module Confguration Parameters</b>:</p>
<p>This module supports module parameters, only.
<ul>
<li>$PStatInterval &lt;Seconds&gt;<br>
Sets the interval, in <b>seconds</b> at which messages are generated. Please note that the
actual interval may be a bit longer. We do not try to be precise and so the interval is
actually a sleep period which is entered after generating all messages. So the actual
interval is what is configured here plus the actual time required to generate messages.
In general, the difference should not really matter.
<li>$PStatFacility &lt;numerical facility&gt;<br>
The numerical syslog facility code to be used for generated messages. Default
is 5 (syslog).This is useful for filtering messages.</li>
<li>$PStatSeverity &lt;numerical severity&gt;<br>
The numerical syslog severity code to be used for generated messages. Default
is 6 (info).This is useful for filtering messages.</li>
<li><strong>interval </strong>[seconds] (default 300 [5minutes])<br>
Sets the interval, in <b>seconds</b> at which messages are generated. Please note that the
actual interval may be a bit longer. We do not try to be precise and so the interval is
actually a sleep period which is entered after generating all messages. So the actual
interval is what is configured here plus the actual time required to generate messages.
In general, the difference should not really matter.
<br></li>
<li><strong>facility </strong>[templateName]<br>
The numerical syslog facility code to be used for generated messages. Default
is 5 (syslog). This is useful for filtering messages.
<br></li>
<li><strong>severity </strong>[templateName]<br>
The numerical syslog severity code to be used for generated messages. Default
is 6 (info).This is useful for filtering messages.
<br></li>
<li><strong>format </strong>[json/cee/<b>legacy</b>](rsyslog v6.3.8+ only)<br>
Specifies the format of emitted stats messages. The default of "legacy" is
compatible with pre v6-rsyslog. The other options provide support for
structured formats (note the "cee" is actually "project lumberack" logging).
<br></li>
<li><strong>log.syslog </strong>[<b>on</b>/off] - available since 7.3.6<br>
This is a boolean setting specifying if data should be sent
to the usual syslog stream. This is useful if custom formatting
or more elaborate processing is desired. However, output is placed
under the same restrictions as regular syslog data, especially in
regard to the queue position (stats data may sit for an extended
period of time in queues if they are full).<br></li>
<li><strong>log.file </strong>[file name] - available since 7.3.6<br>
If specified, statistics data is written the specified file. For
robustness, this should be a local file. The file format cannot be
customized, it consists of a date header, followed by a colon,
followed by the actual statistics record, all on one line. Only
very limited error handling is done, so if things go wrong stats
records will probably be lost. Logging to file an be a useful
alternative if for some reasons (e.g. full queues) the regular
syslog stream method shall not be used solely. Note that turning
on file logging does NOT turn of syslog logging. If that is desired
log.syslog="off" must be explicitely set.
<br></li>
</ul>
<p><b>Legacx Configuration Directives</b>:</p>
A limited set of parameters can also be set via the legacy configuration
syntax. Note that this is intended as an upward compatibilit layer, so
newer features are intentionally <b>not</b> available via legacy directives.
<ul>
<li>$PStatInterval &lt;Seconds&gt; - same as the "interval" parameter.
<li>$PStatFacility &lt;numerical facility&gt; - same as the "facility" parameter.
<li>$PStatSeverity &lt;numerical severity&gt; - same as the "severity" parameter.
<li>$PStatJSON &lt;on/<b>off</b>&gt; (rsyslog v6.3.8+ only)<br>
If set to on, stats messages are emitted as structured cee-enhanced syslog. If
set to off, legacy format is used (which is compatible with pre v6-rsyslog).
@ -45,23 +85,45 @@ set to off, legacy format is used (which is compatible with pre v6-rsyslog).
<ul>
<li>This module MUST be loaded right at the top of rsyslog.conf, otherwise
stats may not get turned on in all places.</li>
<li>experimental code</li>
</ul>
<p><b>Sample:</b></p>
<p><b>Samples:</b></p>
<p>This activates the module and records messages to /var/log/rsyslog-stats in 10 minute intervals:<br>
</p>
<textarea rows="8" cols="60">$ModLoad impstats
<textarea rows="5" cols="60">module(load="impstats" interval="600" severity="7")
# to actually gather the data:
syslog.=debug /var/log/rsyslog-stats
</textarea>
<p><b>Legacy Sample:</b></p>
<p>This activates the module and records messages to /var/log/rsyslog-stats in 10 minute intervals:</p>
<textarea rows="6" cols="60">$ModLoad impstats
$PStatInterval 600
$PStatSeverity 7
syslog.debug /var/log/rsyslog-stats
syslog.=debug /var/log/rsyslog-stats
</textarea>
<p>In the next sample, the default interval of 5 minutes is used. However, this time
stats data is NOT emitted to the syslog stream but to a local file instead.
<p>
<textarea rows="3" cols="70">module(load="impstats" interval="600" severity="7"
log.syslog="off" /* need to turn log stream logging off! */
log.file="/path/to/local/stats.log")
</textarea>
<p>And finally, we log to both the regular syslog log stream as well as a file.
Within the log stream, we forward the data records to another server:
<p>
<textarea rows="4" cols="70">module(load="impstats" interval="600" severity="7"
log.file="/path/to/local/stats.log")
syslog.=debug @central.example.net
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2010 by <a href="http://www.gerhards.net/rainer">Rainer
Copyright &copy; 2013 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>

25
doc/imptcp.html
View File

@ -13,11 +13,9 @@
<p><b>Description</b>:</p>
<p>Provides the ability to receive syslog messages via plain TCP syslog.
This is a specialised input plugin tailored for high performance on Linux. It will
probably not run on any other platform. Also, it does no provide TLS services.
probably not run on any other platform. Also, it does not provide TLS services.
Encryption can be provided by using <a href="rsyslog_stunnel.html">stunnel</a>.
<p>This module has no limit on the number of listeners and sessions that can be used.
<p>Multiple receivers may be configured by
specifying $InputPTCPServerRun multiple times.
</p>
<p><b>Configuration Directives</b>:</p>
@ -33,7 +31,10 @@ globaly to all inputs defined by the module.
Number of helper worker threads to process incoming messages. These
threads are utilized to pull data off the network. On a busy system, additional
helper threads (but not more than there are CPUs/Cores) can help improving
performance. The default value is two.
performance. The default value is two, which means there
is a default thread count of three (the main input thread plus two
helpers).
No more than 16 threads can be set (if tried to, rsyslog always resorts to 16).
</ul>
<p><b>Input Parameters</b>:</p>
<p>These parameters can be used with the "input()" statement. They apply to the
@ -95,6 +96,13 @@ the message was received from.
Binds specified ruleset to next server defined.
<li><b>Address</b> &lt;name&gt;<br>
On multi-homed machines, specifies to which local address the listerner should be bound.
<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting
interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number
of seconds (5 recommended) to activate rate-limiting.
</li>
<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting
burst in number of messages. Default is 10,000.
</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
@ -130,11 +138,8 @@ Equivalent to: Port </li>
Equivalent to: Name </li>
<li>$InputPTCPServerBindRuleset &lt;name&gt;<br>
Equivalent to: Ruleset </li>
<li>$InputPTCPHelperThreads &lt;number&gt;<br>
Number of helper worker threads to process incoming messages. These
threads are utilized to pull data off the network. On a busy system, additional
helper threads (but not more than there are CPUs/Cores) can help improving
performance. The default value is two.
<li>$InputPTCPServerHelperThreads &lt;number&gt;<br>
Equivalent to: threads </li>
<li>$InputPTCPServerListenIP &lt;name&gt;<br>
Equivalent to: Address </li>
</ul>
@ -154,7 +159,7 @@ $InputPTCPServerRun 514
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2010-2012 by <a href="http://www.gerhards.net/rainer">Rainer
Copyright &copy; 2010-2013 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>

8
doc/imrelp.html
View File

@ -30,14 +30,12 @@ Clients send messages to the RELP server via omrelp.</p>
<p><b>Configuration Directives</b>:</p>
<ul>
<li><b>Ruleset</b> &lt;name&gt;</br>
Binds the specified ruleset to all RELP listeners.
<li><b>Port</b> &lt;port&gt;<br>
Starts a RELP server on selected port</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
<li>see description</li>
<li>ruleset can only be bound via legacy configuration format</li>
<li>To obtain the remote system's IP address, you need to have at least
librelp 1.0.0 installed. Versions below it return the hostname instead
of the IP address.</li>
@ -47,14 +45,14 @@ not specific ones. This is due to a currently existing limitation in librelp.
<p><b>Sample:</b></p>
<p>This sets up a RELP server on port 20514.<br>
</p>
<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imrelp/.libs/imrelp") # needs to be done just once
<textarea rows="15" cols="60">module(load="imrelp") # needs to be done just once
input(type="imrelp" port="20514")
</textarea>
<p><b>Legacy Configuration Directives</b>:</p>
<ul>
<li>InputRELPServerBindRuleset &lt;name&gt; (available in 6.3.6+)</br>
equivalent to: RuleSet
Binds the specified ruleset to all RELP listeners.
<li>InputRELPServerRun &lt;port&gt;<br>
equivalent to: Port</li>
</ul>

11
doc/imtcp.html
View File

@ -17,10 +17,6 @@
Encryption is natively provided by selecting the approprioate network stream driver and
can also be provided by using <a href="rsyslog_stunnel.html">stunnel</a>
(an alternative is the use the <a href="imgssapi.html">imgssapi</a> module).</p>
<p>Multiple receivers may be configured by specifying
$InputTCPServerRun multiple times. This is available since version 4.3.1, earlier
versions do NOT support it.
</p>
<p><b>Configuration Directives</b>:</p>
<p><b>Global Directives</b>:</p>
@ -100,6 +96,13 @@ activated. This is the default and should be left unchanged until you know
very well what you do. It may be useful to turn it off, if you know this framing
is not used and some senders emit multi-line messages into the message stream.
</li>
<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting
interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number
of seconds (5 recommended) to activate rate-limiting.
</li>
<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting
burst in number of messages. Default is 10,000.
</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>

69
doc/imudp.html
View File

@ -18,8 +18,8 @@
multiple input actions.
</p>
<p><b>Configuration Directives</b>:</p>
<p><b>Global Directives</b>:</p>
<p><b>Configuration Parameters</b>:</p>
<p><b>Module Parameters</b>:</p>
<ul>
<li><b>TimeRequery</b> &lt;nbr-of-times&gt;<br>
this is a performance
@ -37,7 +37,7 @@ processing under Linux (and thus reduce chance of packet loss).
<li><b>SchedulingPriority</b> &lt;number&gt;<br>
Scheduling priority to use.
</ul>
<p><b>Action Directives</b>:</p>
<p><b>Input Parameters</b>:</p>
<ul>
<li><b>Address</b> &lt;IP&gt;<br>
local IP address (or name) the UDP listens should bind to</li>
@ -47,6 +47,34 @@ default 514, start UDP server on this port. Either a single port can be specifie
<br>Array of ports: Port=["514","515","10514","..."]</li>
<li><b>Ruleset</b> &lt;ruleset&gt;<br>
Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li>
<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting
interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number
of seconds (5 recommended) to activate rate-limiting.
</li>
<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting
burst in number of messages. Default is 10,000.
</li>
<li><b>InputName</b> [name] - (available since 7.3.9) specifies the value of
the inputname. In older versions, this was always "imudp" for all listeners,
which still i the default.
Starting with 7.3.9 it can be set to different values for each listener.
Note that when a single input statement defines multipe listner ports, the
inputname will be the same for all of them. If you want to differentiate in that
case, use "InputName.AppendPort" to make them unique.
Note that the "InputName" parameter can be an empty string. In that case, the
corresponding inputname property will obviously also be the empty string. This
is primarily meant to be used togehter with "InputName.AppendPort" to set the
inputname equal to the port.
</li>
<li><b>InputName.AppendPort</b> [on/<b>off</b>] - (available since 7.3.9)
appends the port the the inputname. Note that when no inputname is specified,
the default of "imudp" is used and the port is appended to that default. So,
for example, a listner port of 514 in that case will lead to an inputname
of "imudp514". The ability to append a port is most useful when multiple ports
are defined for a single input and each of the inputnames shall be unique.
Note that there currently is no differentiation between IPv4/v6 listeners on
the same port.
</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
@ -56,13 +84,33 @@ privilege drop. This may be worked around by using a sufficiently-privileged
user account.
</li>
</ul>
<p><b>Sample:</b></p>
<p><b>Samples:</b></p>
<p>This sets up an UPD server on port 514:<br>
</p>
<textarea rows="15" cols="60">module(load="imudp") # needs to be done just once
<textarea rows="3" cols="60">module(load="imudp") # needs to be done just once
input(type="imudp" port="514")
</textarea>
<p>In the next example, we set up three listeners at ports 10514, 10515 and 10516
and assign a listner name of "udp" to it, followed by the port number:
</p>
<textarea rows="4" cols="60">module(load="imudp")
input(type="imudp" port=["10514","10515","10516"]
inputname="udp" inputname.appendPort="on")
</textarea>
<p>The next example is almost equal to the previous one, but
now the inputname property will just be set to the port number.
So if a message was received on port 10515, the input name will be
"10515" in this example whereas it was "udp10515" in the previous one.
Note that to do that we set the inputname to the empty string.
</p>
<textarea rows="4" cols="60">module(load="imudp")
input(type="imudp" port=["10514","10515","10516"]
inputname="" inputname.appendPort="on")
</textarea>
<p><b>Legacy Configuration Directives</b>:</p>
<p>Multiple receivers may be configured by specifying
$UDPServerRun multiple times.
@ -81,23 +129,20 @@ equivalent to: SchedulingPolicy
<li>$IMUDPSchedulingPriority &lt;number&gt; Available since 4.7.4+, 5.7.3+, 6.1.3+.<br>
equivalent to: SchedulingPriority
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
<li>currently none known</li>
</ul>
<p><b>Sample:</b></p>
<p>This sets up an UPD server on port 514:<br>
</p>
<textarea rows="15" cols="60">$ModLoad imudp # needs to be done just once
<textarea rows="3" cols="60">$ModLoad imudp # needs to be done just once
$UDPServerRun 514
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2009 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
Copyright &copy; 2009-2013 by
<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body></html>

27
doc/imuxsock.html
View File

@ -65,6 +65,12 @@ you must turn it on (via SysSock.Annotate and Annotate).
<li><b>SysSock.IgnoreTimestamp</b> [<b>on</b>/off]<br>
Ignore timestamps included in the messages, applies to messages received via the system log socket.
</li>
<li><b>SysSock.IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br>
Ignores messages that originated from the same instance of rsyslogd. There usually
is no reason to receive messages from ourselfs. This setting is vital
when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a>
module documentation for a more in-depth description.
</li>
<li><b>SysSock.Use</b> (imuxsock) [on/<b>off</b>]
do NOT listen for the local log socket. This is most useful if you run multiple
instances of rsyslogd where only one shall handle the system log socket.
@ -95,13 +101,25 @@ properties for the system log socket.</li>
<li><b>SysSock.ParseTrusted</b> &lt;on/<b>off</b>&gt; if Annotation is turned on, create
JSON/lumberjack properties out of the trusted properties (which can be accessed
via RainerScript JSON Variables, e.g. "$!pid") instead of adding them to the message.
properties for the system log socket.</li>
</li>
<li><b>SysSock.Unlink</b> &lt;<b>on</b>/off&gt; (available since 7.3.9)<br>
if turned on (default), the system socket is unlinked and re-created when
opened and also unlinked when finally closed. Note that this setting has
no effect when running under systemd control (because systemd handles
the socket).
</li>
</ul>
<p><b>Input Instance Parameters</b></p>
<ul>
<li><b>IgnoreTimestamp</b> [<b>on</b>/off]
<br>Ignore timestamps included in the message. Applies to the next socket being added.</li>
<li><b>IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br>
Ignore messages that originated from the same instance of rsyslogd. There usually
is no reason to receive messages from ourselfs. This setting is vital
when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a>
module documentation for a more in-depth description.
</li>
<li><b>FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied
to the next socket.</li>
<li><b>RateLimit.Interval</b> [number] - specifies the rate-limiting
@ -154,6 +172,13 @@ that the local hostname can be overridden in cases where that is desired.</li>
properties for the non-system log socket in question.</li>
<li><b>ParseTrusted</b> &lt;on/<b>off</b>&gt; equivalent to the SysSock.ParseTrusted module
parameter, but applies to the input that is being defined.
<li><b>Unlink</b> &lt;<b>on</b>/off&gt; (available since 7.3.9)<br>
if turned on (default), the socket is unlinked and re-created when
opened and also unlinked when finally closed. Set it to off if you
handle socket creation yourself. Note that handling socket creation
oneself has the advantage that a limited amount of messages may be
queued by the OS if rsyslog is not running.
</li>
</ul>
<b>Caveats/Known Bugs:</b><br>

205
doc/lookup_tables.html Normal file
View File

@ -0,0 +1,205 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<title>Lookup Tables</title>
</head>
<body>
<h1>Lookup Tables</h1>
<p><b><font color="red">NOTE: this is</font> proposed functionality, which is
<font color="red">NOT YET IMPLEMENTED</font>!</b>
<p><b>Lookup tables</a> are a powerful construct
to obtain "class" information based on message content (e.g. to build
log file names for different server types, departments or remote
offices).</b>
<p>The base idea is to use a message variable as an index into a table which then
returns another value. For example, $fromhost-ip could be used as an index, with
the table value representing the type of server or the department or remote office
it is located in. A main point with lookup tables is that the lookup is very fast.
So while lookup tables can be emulated with if-elseif constructs, they are generally
much faster. Also, it is possible to reload lookup tables during rsyslog runtime without
the need for a full restart.
<p>The lookup tables itself exists in a separate configuration file (one per table). This
file is loaded on rsyslog startup and when a reload is requested.
<p>There are different types of lookup tables:
<ul>
<li><b>string</b> - the value to be looked up is an arbitrary string. Only exact
some strings match.
<li><b>array</b> - the value to be looked up is an integer number from a consequtive set.
The set does not need to start at zero or one, but there must be no number missing. So, for example
5,6,7,8,9 would be a valid set of index values, while 1,2,4,5 would not be (due to missing
2).
A match happens if the requested number is present.
<li><b>sparseArray</b> - the value to be looked up is an integer value, but there may
be gaps inside the set of values (usually there are large gaps). A typical use case would
be the matching of IPv4 address information. A match happens on the first value that is
less than or equal to the requested value.
</ul>
<p>Note that index integer numbers are represented by unsigned 32 bits.
<p>Lookup tables can be access via the lookup() built-in function. The core idea is to
set a local variable to the lookup result and later on use that local variable in templates.
<p>More details on usage now follow.
<h2>Lookup Table File Format</h2>
<p>Lookup table files contain a single JSON object. This object contains of a header and a
table part.
<h3>Header</h3>
<p>The header is the top-level json. It has paramters "version", "nomatch", and "type".
The version parameter
must be given and must always be one for this version of rsyslog. The nomatch
parameter is optional. If specified, it contains the value to be used if lookup()
is provided an index value for which no entry exists. The default for
"nomatch" is the empty string. Type specifies the type of lookup to be done.
<h3>Table</h3>
This must be an array of elements, even if only a single value exists (for obvious
reasons, we do not expect this to occur often). Each array element must contain two
fields "index" and "value".
<h3>Example</h3>
<p>This is a sample of how an ip-to-office mapping may look like:
<pre>
{ "version":1, "nomatch":"unk", "type":"string",
"table":[ {"index":"10.0.1.1", "value":"A" },
{"index":"10.0.1.2", "value":"A" },
{"index":"10.0.1.3", "value":"A" },
{"index":"10.0.2.1", "value":"B" },
{"index":"10.0.2.2", "value":"B" },
{"index":"10.0.2.3", "value":"B" }
]
}
</pre>
Note: if a different IP comes in, the value "unk"
is returend thanks to the nomatch parameter in
the first line.
<p>
<h2>RainerScript Statements</h2>
<h3>lookup_table() Object</h3>
<p>This statement defines and intially loads a lookup table. Its format is
as follows:
<pre>
lookup_table(name="name" file="/path/to/file" reloadOnHUP="on|off")
</pre>
<h4>Parameters</h4>
<ul>
<li><b>name</b> (mandatory)<br>
Defines the name of lookup table for further reference
inside the configuration. Names must be unique. Note that
it is possible, though not advisible, to have different
names for the same file.
<li><b>file</b> (mandatory)<br>
Specifies the full path for the lookup table file. This file
must be readable for the user rsyslog is run under (important
when dropping privileges). It must point to a valid lookup
table file as described above.
<li><b>reloadOnHUP</b> (optional, default "on")<br>
Specifies if the table shall automatically be reloaded
as part of HUP processing. For static tables, the
default is "off" and specifying "on" triggers an
error message. Note that the default of "on" may be
somewhat suboptimal performance-wise, but probably
is what the user intuitively expects. Turn it off
if you know that you do not need the automatic
reload capability.
</ul>
<h3>lookup() Function</h3>
<p>This function is used to actually do the table lookup. Format:
<pre>
lookup_table("name", indexvalue)
</pre>
<h4>Parameters</h4>
<ul>
<li><b>return value</b><br>
The function returns the string that is associated with the
given indexvalue. If the indexvalue is not present inside the
lookup table, the "nomatch" string is returned (or an empty string
if it is not defined).
<li><b>name</b> (constant string)<br>
The lookup table to be used. Note that this must be specificed as a
constant. In theory, variable table names could be made possible, but
their runtime behaviour is not as good as for static names, and we do
not (yet) see good use cases where dynamic table names could be useful.
<li><b>indexvalue</b> (expression)<br>
The value to be looked up. While this is an arbitrary RainerScript expression,
it's final value is always converted to a string in order to conduct
the lookup. For example, "lookup(table, 3+4)" would be exactly the same
as "lookup(table, "7")". In most cases, indexvalue will probably be
a single variable, but it could also be the result of all RainerScript-supported
expression types (like string concatenation or substring extraction).
Valid samples are "lookup(name, $fromhost-ip &amp; $hostname)" or
"lookup(name, substr($fromhost-ip, 0, 5))" as well as of course the
usual "lookup(table, $fromhost-ip)".
</ul>
<h3>load_lookup_table Statement</h3>
<p><b>Note: in the final implementation, this MAY be implemented as an action.
This is a low-level decesion that must be made during the detail development
process. Parameters and semantics will remain the same of this happens.</b>
<p>This statement is used to reload a lookup table. It will fail if
the table is static. While this statement is executed, lookups to this table
are temporarily blocked. So for large tables, there may be a slight performance
hit during the load phase. It is assume that always a triggering condition
is used to load the table.
<pre>
load_lookup_table(name="name" errOnFail="on|off" valueOnFail="value")
</pre>
<h4>Parameters</h4>
<ul>
<li><b>name</b> (string)<br>
The lookup table to be used.
<li><b>errOnFail</b> (boolean, default "on")<br>
Specifies whether or not an error message is to be emitted if
there are any problems reloading the lookup table.
<li><b>valueOnFail</b> (optional, string)<br>
This parameter affects processing if the lookup table cannot
be loaded for some reason: If the parameter is not present,
the previous table will be kept in use. If the parameter is
given, the previous table will no longer be used, and instead
an empty table be with nomath=valueOnFail be generated. In short,
that means when the parameter is set and the reload fails,
all matches will always return what is specified in valueOnFail.
</ul>
<h3>Usage example</h3>
<p>For clarity, we show only those parts of rsyslog.conf that affect
lookup tables. We use the remote office example that an example lookup
table file is given above for.
<pre>
lookup_table(name="ip2office" file="/path/to/ipoffice.lu"
reloadOnHUP="off")
template(name="depfile" type="string"
string="/var/log/%$usr.dep%/messages")
set $usr.dep = lookup("ip2office", $fromhost-ip);
action(type="omfile" dynfile="depfile")
# support for reload "commands"
if $fromhost-ip == "10.0.1.123"
and $msg contains "reload office lookup table"
then
load_lookup_table(name="ip2office" errOnFail="on")
</pre>
<p>Note: for performance reasons, it makes sense to put the reload command into
a dedicated ruleset, bound to a specific listener - which than should also
be sufficiently secured, e.g. via TLS mutual auth.
<h2>Implementation Details</h2>
<p>The lookup table functionality is implemented via highly efficient algorithms.
The string lookup is based on a parse tree and has O(1) time complexity. The array
lookup is also O(1). In case of sparseArray, we have O(log n).
<p>To preserve space and, more important, increase cache hit performance, equal
data values are only stored once, no matter how often a lookup index points to them.
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body>
</html>

8
doc/manual.html
View File

@ -13,13 +13,13 @@ It is quite compatible to stock sysklogd and can be used as a drop-in
replacement. Its <a href="features.html">
advanced features</a> make it suitable for enterprise-class, <a href="rsyslog_tls.html">encryption protected syslog</a>
relay chains while at the same time being very easy to setup for the
novice user. And as we know what enterprise users really need, there is
also <a href="http://www.rsyslog.com/professional-services">professional
rsyslog support</a> available directly from the source!</p>
novice user. And as we know what enterprise users really need, there are
also <a href="http://www.rsyslog.com/professional-services"> rsyslog
professional services</a> available directly from the source!</p>
<p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a>
to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the
project goals.</p>
<p><b>This documentation is for version 7.2.7 (v7.2-stable branch) of rsyslog.</b>
<p><b>This documentation is for version 7.4.4 (v7.4-stable branch) of rsyslog.</b>
Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b>
to obtain current version information and project status.
</p><p><b>If you like rsyslog, you might

119
doc/mmanon.html Normal file
View File

@ -0,0 +1,119 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<meta http-equiv="Content-Language" content="en">
<title>IP Address Anonimization Module (mmanon)</title></head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>IP Address Anonimization Module (mmanon)</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; mmanon</b></p>
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Available since</b>: 7.3.7</p>
<p><b>Description</b>:</p>
<p>The mmanon module permits to anonymize IP addresses. It is a message
modification module that actually changes the IP address inside the message,
so after calling mmanon, the original message can no longer be obtained.
Note that anonymization will break digital signatures on the message, if
they exist.
<p><i>How are IP-Addresses defined?</i>
<p>We assume that an IP address consists of four octets in dotted notation,
where each of the octets has a value between 0 and 255, inclusively. After
the last octet, there must be either a space or a colon. So, for example,
"1.2.3.4 Test" and "1.2.3.4:514 Test" are detected as containing valid IP
addresses, whereas this is not the case for "1.2.300.4 Test" or
"1.2.3.4-Test". The message text may contain multiple addresses. If so,
each of them is anonimized (according to the same rules).
<b>Important:</b> We may change the set of acceptable characters after
the last octet in the future, if there are good reasons to do so.
<p>&nbsp;</p>
<p><b>Module Configuration Parameters</b>:</p>
<p>Currently none.
<p>&nbsp;</p>
<p><b>Action Confguration Parameters</b>:</p>
<ul>
<li><b>mode</b> - default "rewrite"<br>
There exists the "simple" and "rewrite" mode. In simple mode, only octets
as whole can be anonymized and the length of the message is never changed.
This means that when the last three octets of the address 10.1.12.123 are
anonymized, the result will be 10.0.00.000. This means that the length of the
original octets is still visible and may be used to draw some privacy-evasive
conclusions. This mode is slightly faster than "overwrite" mode, and this
may matter in high throughput environments.<br>
The default "rewrite" mode will do full anonymization of any number of bits
and it will also normlize the address, so that no information about the
original IP address is available. So in the above example, 10.1.12.123 would
be anonymized to 10.0.0.0.
<li><b>ipv4.bits</b> - default 16<br>
This set the number of bits that should be anonymized (bits are from the
right, so lower bits are anonymized first). This setting permits to save
network information while still anonymizing user-specific data. The more
bits you discard, the better the anonymization obviously is. The default
of 16 bits reflects what German data privacy rules consider as being
sufficinetly anonymized. We assume, this can also be used as a rough
but conservative guideline for other countries.<br>
Note: when in simple mode, only bits on a byte boundary can be specified.
As such, any value other than 8, 16, 24 or 32 is invalid. If an invalid
value is given, it is rounded to the next byte boundary (so we favor stronger
anonymization in that case). For example, a bit value of 12 will become 16 in
simple mode (an error message is also emitted).
<li><b>replacementChar</b> - default "x"<br>
In simple mode, this sets the character
that the to-be-anonymized part of the IP address is to be overwritten
with. In rewrite mode, this parameter is <b>not permitted</b>, as in
this case we need not necessarily rewrite full octets. As such, the anonymized
part is always zero-filled and replacementChar is of no use. If it is
specified, an error message is emitted and the parameter ignored.
</ul>
<p><b>Caveats/Known Bugs:</b>
<ul>
<li><b>only IPv4</b> is supported
</ul>
<p><b>Samples:</b></p>
<p>In this snippet, we write one file without anonymization and another one
with the message anonymized. Note that once mmanon has run, access to the
original message is no longer possible (execept if stored in user
variables before anonymization).
<p><textarea rows="5" cols="60">module(load="mmanon")
action(type="omfile" file="/path/to/non-anon.log")
action(type="mmanon")
action(type="omfile" file="/path/to/anon.log")
</textarea>
<p>This next snippet is almost identical to the first one, but
here we anonymize the full IPv4 address. Note that by
modifying the number of bits, you can anonymize different parts
of the address. Keep in mind that in simple mode (used here), the bit values
must match IP address bytes, so for IPv4 only the values 8, 16, 24 and
32 are valid. Also, in this example the replacement is done
via asterisks instead of lower-case "x"-letters. Also keep in mind that
"replacementChar" can only be set in simple mode.
<p><textarea rows="5" cols="60">module(load="mmanon")
action(type="omfile" file="/path/to/non-anon.log")
action(type="mmanon" ipv4.bits="32" mode="simple" replacementChar="*")
action(type="omfile" file="/path/to/anon.log")
</textarea>
<p>The next snippet is also based on the first one, but anonimzes an
"odd" number of bits, 12. The value of 12 is used by some folks as a
compromise between keeping privacy and still permiting to gain some
more in-depth insight from log files. Note that anonymizing 12 bits
may be insufficient to fulfill legal requirements (if such exist).
<p><textarea rows="5" cols="60">module(load="mmanon")
action(type="omfile" file="/path/to/non-anon.log")
action(type="mmanon" ipv4.bits="12")
action(type="omfile" file="/path/to/anon.log")
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual
index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>
</body></html>

45
doc/mmjsonparse.html Normal file
View File

@ -0,0 +1,45 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<title>CEE/lumberjack JSON support Module (mmjsonparse)</title>
</head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>Log Message Normalization Module</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; mmjsonparse</b></p>
<p><b>Available since: </b>6.6.0+
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Description</b>:</p>
<p>This module provides support for parsing structured log messages
that follow the CEE/lumberjack spec. The so-called "CEE cookie" is checked
and, if present, the JSON-encoded structured message content is parsed.
The properties are than available as original message properties.
</p>
<p><b>Action specific Configuration Directives</b>:</p>
<p>currently none
<ul>
<p><b>Legacy Configuration Directives</b>:</p>
<p>none
<b>Caveats/Known Bugs:</b>
<p>None known at this time.
</ul>
<p><b>Sample:</b></p>
<p>This activates the module and applies normalization to all messages:<br>
</p>
<textarea rows="2" cols="60">module(load="mmjsonparse")
action(type="mmjsonparse")
</textarea>
<p>The same in legacy format:</p>
<textarea rows="2" cols="60">$ModLoad mmjsonparse
*.* :mmjsonparse:
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2012 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body></html>

46
doc/mmnormalize.html
View File

@ -11,37 +11,53 @@
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Description</b>:</p>
<p>This module provides the capability to normalize log messages via
<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to libee, unstructured text,
<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to liblognorm, unstructured text,
like usually found in log messages, can very quickly be parsed and put into
a normal form. This is done so quickly, that it usually should be possible
a normal form. This is done so quickly, that it should be possible
to normalize events in realtime.
<p>This module is implemented via the output module interface. That means that
<p>This module is implemented via the output module interface. This means that
mmnormalize should be called just like an action. After it has been called,
the normalized message properties are avaialable and can be access. These properties
are called the "CEE" properties, because liblognorm creates a format that is
inspired by the CEE approach.
the normalized message properties are avaialable and can be accessed. These properties
are called the "CEE/lumberjack" properties, because liblognorm creates a format that is
inspired by the CEE/lumberjack approach.
<p><b>Please note:</b> CEE/lumberjack properties are different from regular properties.
They have always "$!" prepended to the property name given in the rulebase. Such a
property needs to be called with <b>%$!propertyname%</b>.
<p>Note that mmnormalize should only be called once on each message. Behaviour is
undifined if multiple calls to mmnormalize happen for the same message.
undefined if multiple calls to mmnormalize happen for the same message.
</p>
<p><b>Configuration Directives</b>:</p>
<p><b>Action Parameters</b>:</p>
<ul>
<li>$mmnormalizeRuleBase &lt;rulebase-file&gt;<br>
Specifies which rulebase file is to use. This file is loaded. If there are
<li><b>ruleBase</b> [word]<br>
Specifies which rulebase file is to use. If there are
multiple mmnormalize instances, each one can use a different file. However,
a single instance can use only a single file. This parameter MUST be given,
because normalization can only happen based on a rulebase.
<li>$mmnormalizeUseRawMsg &lt;on/off&gt;<br>
because normalization can only happen based on a rulebase. It is recommended
that an absolute path name is given. Information on how to create the rulebase
can be found in the <a href="http://www.liblognorm.com/files/manual/index.html">liblognorm manual</a>.
<li><b>useRawMsg</b> [boolean]<br>
Specifies if the raw message should be used for normalization (on) or just the
MSG part of the message (off). Default is "off".
</ul>
<p><b>Legacy Configuration Directives</b>:</p>
<ul>
<li>$mmnormalizeRuleBase &lt;rulebase-file&gt; - equivalent to the "ruleBase"
parameter.
<li>$mmnormalizeUseRawMsg &lt;on/off&gt; - equivalent to the "useRawMsg"
parameter.
</ul>
<b>Caveats/Known Bugs:</b>
<p>None known at this time.
</ul>
<p><b>Sample:</b></p>
<p>This activates the module and applies normalization to all messages:<br>
</p>
<textarea rows="8" cols="60">$ModLoad mmnormalize
$mmnormalizeRuleBase rulebase.rb
<textarea rows="2" cols="60">module(load="mmnormalize")
action(type="mmnormalize" ruleBase="/path/to/rulebase.rb")
</textarea>
<p>The same in legacy format:</p>
<textarea rows="3" cols="60">$ModLoad mmnormalize
$mmnormalizeRuleBase /path/to/rulebase.rb
*.* :mmnormalize:
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
@ -49,7 +65,7 @@ $mmnormalizeRuleBase rulebase.rb
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2010 by <a href="http://www.gerhards.net/rainer">Rainer
Copyright &copy; 2010-2012 by <a href="http://www.gerhards.net/rainer">Rainer
Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>

2
doc/multi_ruleset.html
View File

@ -191,7 +191,7 @@ ruleset(name="test1"){
}
# and now define listners bound to the relevant ruleset
# and now define listeners bound to the relevant ruleset
input(type="imptcp" port="10514" ruleset="remote10514")
input(type="imptcp" port="10515" ruleset="remote10515")
input(type="imptcp" port="10516" ruleset="remote10516")

2
doc/multi_ruleset_legacy_format.html
View File

@ -168,7 +168,7 @@ mail.* /var/log/mail10516
# being written to the remote10516 file - as usual...
*.* /var/log/remote10516
# and now define listners bound to the relevant ruleset
# and now define listeners bound to the relevant ruleset
$InputTCPServerBindRuleset remote10514
$InputTCPServerRun 10514

177
doc/omelasticsearch.html Normal file
View File

@ -0,0 +1,177 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="en" http-equiv="Content-Language" />
<title>Elasticsearch Output Module</title>
</head>
<body>
<p>
<a href="rsyslog_conf_modules.html">back</a></p>
<h1>
Elasticsearch Output Module</h1>
<p>
<b>Module Name:&nbsp;&nbsp;&nbsp; omelasticsearch</b></p>
<p>
<b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p>
<b>Available since: </b>6.4.0+</p>
<p>
<b>Description</b>:</p>
<p>
This module provides native support for logging to <a href="http://www.elasticsearch.org/">Elasticsearch</a>.</p>
<p>
<b>Action Parameters</b>:</p>
<ul>
<li>
<b>server</b><br />
Host name or IP address of the Elasticsearch server. Defaults to &quot;localhost&quot;</li>
<li>
<b>serverport</b><br />
HTTP port to connect to Elasticsearch. Defaults to 9200</li>
<li>
<b>searchIndex</b><br />
<a href="http://www.elasticsearch.org/guide/appendix/glossary.html#index">Elasticsearch index</a> to send your logs to. Defaults to &quot;system&quot;</li>
<li>
<b>dynSearchIndex </b>&lt;on/<b>off</b>&gt;<br />
Whether the string provided for <strong>searchIndex</strong> should be taken as a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a>. Defaults to &quot;off&quot;, which means the index name will be taken literally. Otherwise, it will look for a template with that name, and the resulting string will be the index name. For example, let&#39;s assume you define a template named &quot;date-days&quot; containing &quot;%timereported:1:10:date-rfc3339%&quot;. Then, with dynSearchIndex=&quot;on&quot;, if you say searchIndex=&quot;date-days&quot;, each log will be sent to and index named after the first 10 characters of the timestamp, like &quot;2013-03-22&quot;.</li>
<li>
<b>searchType</b><br />
<a href="http://www.elasticsearch.org/guide/appendix/glossary.html#type">Elasticsearch type</a> to send your index to. Defaults to &quot;events&quot;</li>
<li>
<b>dynSearchType</b> &lt;on/<strong>off</strong>&gt;<br />
Like <strong>dynSearchIndex</strong>, it allows you to specify a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a> for <strong>searchType</strong>, instead of a static string.</li>
<li>
<strong>asyncrepl </strong>&lt;on/<strong>off</strong>&gt;<br />
By default, an indexing operation returns after all <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#replica_shard">replica shards</a> have indexed the document. With asyncrepl=&quot;on&quot; it will return after it was indexed on the <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#primary_shard">primary shard</a> only - thus trading some consistency for speed.</li>
<li>
<strong>timeout</strong><br />
How long Elasticsearch will wait for a primary shard to be available for indexing your log before sending back an error. Defaults to &quot;1m&quot;.</li>
<li>
<strong>template</strong><br />
This is the JSON document that will be indexed in Elasticsearch. The resulting string needs to be a valid JSON, otherwise Elasticsearch will return an error. Defaults to:</li>
</ul>
<pre>
$template JSONDefault, &quot;{\&quot;message\&quot;:\&quot;%msg:::json%\&quot;,\&quot;fromhost\&quot;:\&quot;%HOSTNAME:::json%\&quot;,\&quot;facility\&quot;:\&quot;%syslogfacility-text%\&quot;,\&quot;priority\&quot;:\&quot;%syslogpriority-text%\&quot;,\&quot;timereported\&quot;:\&quot;%timereported:::date-rfc3339%\&quot;,\&quot;timegenerated\&quot;:\&quot;%timegenerated:::date-rfc3339%\&quot;}&quot;
</pre>
<p>
Which will produce this sort of documents (pretty-printed here for readability):</p>
<ul>
</ul>
<pre>
{
&nbsp;&nbsp;&nbsp; &quot;message&quot;: &quot; this is a test message&quot;,
&nbsp;&nbsp;&nbsp; &quot;fromhost&quot;: &quot;test-host&quot;,
&nbsp;&nbsp;&nbsp; &quot;facility&quot;: &quot;user&quot;,
&nbsp;&nbsp;&nbsp; &quot;priority&quot;: &quot;info&quot;,
&nbsp;&nbsp;&nbsp; &quot;timereported&quot;: &quot;2013-03-12T18:05:01.344864+02:00&quot;,
&nbsp;&nbsp;&nbsp; &quot;timegenerated&quot;: &quot;2013-03-12T18:05:01.344864+02:00&quot;
}</pre>
<ul>
<li>
<strong>bulkmode </strong>&lt;on/<strong>off</strong>&gt;<br />
The default &quot;off&quot; setting means logs are shipped one by one. Each in its own HTTP request, using the <a href="http://www.elasticsearch.org/guide/reference/api/index_.html">Index API</a>. Set it to &quot;on&quot; and it will use Elasticsearch&#39;s <a href="http://www.elasticsearch.org/guide/reference/api/bulk.html">Bulk API</a> to send multiple logs in the same request. The maximum number of logs sent in a single bulk request depends on your queue settings - usually limited by the <a href="http://www.rsyslog.com/doc/node35.html">dequeue batch size</a>. More information about queues can be found <a href="http://www.rsyslog.com/doc/node32.html">here</a>.</li>
<li>
<strong>parent</strong><br />
Specifying a string here will index your logs with that string the parent ID of those logs. Please note that you need to define the <a href="http://www.elasticsearch.org/guide/reference/mapping/parent-field.html">parent field</a> in your <a href="http://www.elasticsearch.org/guide/reference/mapping/">mapping</a> for that to work. By default, logs are indexed without a parent.</li>
<li>
<strong>dynParent </strong>&lt;on/<strong>off</strong>&gt;<br />
Using the same parent for all the logs sent in the same action is quite unlikely. So you&#39;d probably want to turn this &quot;on&quot; and specify a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a> that will provide meaningful parent IDs for your logs.</li>
<li>
<strong>uid</strong><br />
If you have basic HTTP authentication deployed (eg: through the <a href="https://github.com/Asquera/elasticsearch-http-basic">elasticsearch-basic plugin</a>), you can specify your user-name here.</li>
<li>
<strong>pwd</strong><br />
Password for basic authentication.</li>
</ul>
<p>
<b>Samples:</b></p>
<p>
The following sample does the following:</p>
<ul>
<li>
loads the omelasticsearch module</li>
<li>
outputs all logs to Elasticsearch using the default settings</li>
</ul>
<pre>
module(load=&quot;omelasticsearch&quot;)
*.* action(type=&quot;omelasticsearch&quot;)</pre>
<p>
The following sample does the following:</p>
<ul>
<li>
loads the omelasticsearch module</li>
<li>
defines a template that will make the JSON contain the following properties (more info about what properties you can use <a href="http://www.rsyslog.com/doc/property_replacer.html">here</a>):
<ul>
<li>
RFC-3339 timestamp when the event was generated</li>
<li>
the message part of the event</li>
<li>
hostname of the system that generated the message</li>
<li>
severity of the event, as a string</li>
<li>
facility, as a string</li>
<li>
the tag of the event</li>
</ul>
</li>
<li>
outputs to Elasticsearch with the following settings
<ul>
<li>
host name of the server is myserver.local</li>
<li>
port is 9200</li>
<li>
JSON docs will look as defined in the template above</li>
<li>
index will be &quot;test-index&quot;</li>
<li>
type will be &quot;test-type&quot;</li>
<li>
activate bulk mode. For that to work effectively, we use an in-memory queue that can hold up to 5000 events. The maximum bulk size will be 300</li>
<li>
retry indefinitely if the HTTP request failed (eg: if the target server is down)</li>
</ul>
</li>
</ul>
<pre>
module(load=&quot;omelasticsearch&quot;)
template(name=&quot;testTemplate&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; type=&quot;list&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; option.json=&quot;on&quot;) {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;{&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;timestamp\&quot;:\&quot;&quot;)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; property(name=&quot;timereported&quot; dateFormat=&quot;rfc3339&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;,\&quot;message\&quot;:\&quot;&quot;)&nbsp;&nbsp;&nbsp;&nbsp; property(name=&quot;msg&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;,\&quot;host\&quot;:\&quot;&quot;)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; property(name=&quot;hostname&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;,\&quot;severity\&quot;:\&quot;&quot;)&nbsp;&nbsp;&nbsp; property(name=&quot;syslogseverity-text&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;,\&quot;facility\&quot;:\&quot;&quot;)&nbsp;&nbsp;&nbsp; property(name=&quot;syslogfacility-text&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;,\&quot;syslogtag\&quot;:\&quot;&quot;)&nbsp;&nbsp; property(name=&quot;syslogtag&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; constant(value=&quot;\&quot;}&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
*.* action(type=&quot;omelasticsearch&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; server=&quot;myserver.local&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; serverport=&quot;9200&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; template=&quot;testTemplate&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; searchIndex=&quot;test-index&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; searchType=&quot;test-type&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; bulkmode=&quot;on&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; queue.type=&quot;linkedlist&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; queue.size=&quot;5000&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; queue.dequeuebatchsize=&quot;300&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; action.resumeretrycount=&quot;-1&quot;)</pre>
<p>
&nbsp;</p>
<pre>
</pre>
<p>
[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p>
<font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br />
Copyright &copy; 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the ASL 2.0.</font></p>
</body>
</html>

121
doc/omfile.html
View File

@ -13,14 +13,14 @@
<p>The omfile plug-in provides the core functionality of writing messages to files residing inside the local file system (which may actually be remote if methods like NFS are used). Both files named with static names as well files with names based on message content are supported by this module. It is a built-in module that does not need to be loaded. </p>
<p>&nbsp;</p>
<p><b>Global Configuration Directives</b>:</p>
<p><b>Module Parameters</b>:</p>
<ul>
<li><strong>Template </strong>[templateName]<br>
sets a new default template for file actions.<br></li>
</ul>
<p>&nbsp;</p>
<p><b>Action specific Configuration Directives</b>:</p>
<p><b>Action Parameters</b>:</p>
<ul>
<li><strong>DynaFileCacheSize </strong>(not mandatory, default will be used)<br>
Defines a template to be used for the output. <br></li><br>
@ -28,6 +28,16 @@
<li><strong>ZipLevel </strong>0..9 [default 0]<br>
if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br>
<li><b>VeryRobustZip</b> [<b>on</b>/off] (v7.3.0+) - if ZipLevel is greater 0,
then this setting controls if extra headers are written to make the resulting file
extra hardened against malfunction. If set to off, data appended to previously unclean
closed files may not be accessible without extra tools.
Note that this risk is usually expected to be bearable, and thus "off" is the default mode.
The extra headers considerably
degrade compression, files with this option set to "on" may be four to five times as
large as files processed in "off" mode.
</li><br>
<li><strong>FlushInterval </strong>(not mandatory, default will be used)<br>
Defines a template to be used for the output. <br></li><br>
@ -47,10 +57,10 @@
Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
<li><strong>FileOwner </strong><br>
Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
Set the file owner for files newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
<li><strong>FileGroup </strong><br>
Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
Set the group for files newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
<li><strong>DirCreateMode </strong>[defaul 0700]<br>
This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br>
@ -59,7 +69,7 @@
The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br>
<li><strong>FailOnCHOwnFailure </strong>on/off [default on]<br>
This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br>
This option modifies behaviour of file creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br>
<li><strong>CreateDirs </strong>on/off [default on]<br>
create directories on an as-needed basis<br></li><br>
@ -73,77 +83,106 @@
<li><strong>DynaFile </strong><br>
For each message, the file name is generated based on the given template. Then, this file is opened. As with the ``file'' property, data is appended if the file already exists. If the file does not exist, a new file is created. A cache of recent files is kept. Note that this cache can consume quite some memory (especially if large buffer sizes are used). Files are kept open as long as they stay inside the cache. Currently, files are only evicted from the cache when there is need to do so (due to insufficient cache size). To force-close (and evict) a dynafile from cache, send a HUP signal to rsyslogd. <br></li><br>
<li><b>Sig.Provider </b>[ProviderName]<br>
Selects a signature provider for log signing. Currently,
there only is one provider called
"<a href="sigprov_gt.html">gt</a>".<br></li><br>
<li><b>Cry.Provider </b>[ProviderName]<br>
Selects a crypto provider for log encryption. Currently,
there only is one provider called
"<a href="cryprov_gcry.html">gcry</a>".<br></li><br>
<li><strong>Template </strong>[templateName]<br>
sets a new default template for file actions.<br></li><br>
</ul>
<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul>
<p><b>Caveats/Known Bugs:</b></p>
<ul>
<li>One needs to be careful with log rotation if signatures and/or encryption
are being used. These create side-files, which form a set and must be kept
together.
<br>
For signatures, the ".sigstate" file must NOT be rotated away if
signature chains are to be build across multiple files. This is because
.sigstate contains just global information for the whole file set. However,
all other files need to be rotated together. The proper sequence is to
<ol>
<li> move all files inside the file set
<li> only AFTER this is completely done, HUP rsyslog
</ol>
This sequence will ensure that all files inside the set are atomically
closed and in sync. HUPing only after a subset of files have been moved
results in inconsistencies and will most probably render the file set
unusable.
</li>
</ul>
<p><b>Sample:</b></p>
<p>The following command writes all syslog messages into a file.</p>
<textarea rows="5" cols="60">Module (path="builtin:omfile")
<textarea rows="5" cols="60">Module (load="builtin:omfile")
*.* action(type="omfile"
DirCreateMode="0700"
FileCreateMode="0644"
File="/var/log/messages")
DirCreateMode="0700"
FileCreateMode="0644"
File="/var/log/messages")
</textarea>
<br><br>
<p><b>Legacy Configuration Directives</b>:</p>
<ul>
<li><strong>$DynaFileCacheSize </strong>(not mandatory, default will be used)<br>
Defines a template to be used for the output. <br></li><br>
<li><strong>$DynaFileCacheSize </strong><br>
equivalent to the "dynaFileCacheSize" parameter<br></li><br>
<li><strong>$OMFileZipLevel </strong>0..9 [default 0]<br>
if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br>
<li><strong>$OMFileZipLevel </strong><br>
equivalent to the "zipLevel" parameter<br></li><br>
<li><strong>$OMFileFlushInterval </strong>(not mandatory, default will be used)<br>
Defines a template to be used for the output. <br></li><br>
<li><strong>$OMFileFlushInterval </strong><br>
equivalent to the "flushInterval" parameter<br></li><br>
<li><strong>$OMFileASyncWriting </strong>on/off [default off]<br>
if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br>
<li><strong>$OMFileASyncWriting </strong><br>
equivalent to the "asyncWriting" parameter<br></li><br>
<li><strong>$OMFileFlushOnTXEnd </strong>on/off [default on]<br>
Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br>
<li><strong>$OMFileFlushOnTXEnd </strong><br>
equivalent to the "flushOnTXEnd" parameter<br></li><br>
<li><strong>$OMFileIOBufferSize </strong>&lt;size_nbr&gt;, default 4k<br>
size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br>
<li><strong>$OMFileIOBufferSize </strong><br>
equivalent to the "IOBufferSize" parameter<br></li><br>
<li><strong>$DirOwner </strong><br>
Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
equivalent to the "dirOwner" parameter<br></li><br>
<li><strong>$DirGroup </strong><br>
Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
equivalent to the "dirGroup" parameter<br></li><br>
<li><strong>$FileOwner </strong><br>
Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
equivalent to the "fileOwner" parameter<br></li><br>
<li><strong>$FileGroup </strong><br>
Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br>
equivalent to the "fileGroup" parameter<br></li><br>
<li><strong>$DirCreateMode </strong>[defaul 0700]<br>
This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br>
<li><strong>$DirCreateMode </strong><br>
equivalent to the "dirCreateMode" parameter<br></li><br>
<li><strong>$FileCreateMode </strong>[default 0644]<br>
The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br>
<li><strong>$FileCreateMode </strong><br>
equivalent to the "fileCreateMode" parameter<br></li><br>
<li><strong>$FailOnCHOwnFailure </strong>on/off [default on]<br>
This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br>
<li><strong>$FailOnCHOwnFailure </strong><br>
equivalent to the "failOnChOwnFailure" parameter<br></li><br>
<li><strong>$F$OMFileForceCHOwn </strong><br>
force ownership change for all files<br></li><br>
equivalent to the "ForceChOwn" parameter<br></li><br>
<li><strong>$CreateDirs </strong>on/off [default on]<br>
create directories on an as-needed basis<br></li><br>
<li><strong>$CreateDirs </strong><br>
equivalent to the "createDirs" parameter<br></li><br>
<li><strong>$ActionFileEnableSync </strong>on/off [default off]<br>
enables file syncing capability of omfile.<br></li><br>
<li><strong>$ActionFileEnableSync </strong><br>
equivalent to the "enableSync" parameter<br></li><br>
<li><strong>$ActionFileDefaultTemplate </strong>[templateName]<br>
sets a new default template for file actions.<br></li><br>
<li><strong>$ActionFileDefaultTemplate </strong><br>
equivalent to the "template" module parameter<br></li><br>
<li><strong>$ResetConfigVariables </strong><br>
Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br>
Resets all configuration variables to their default value.<br></li><br>
</ul>
@ -160,7 +199,7 @@ $FileCreateMode 0644
index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>

4
doc/omfwd.html
View File

@ -16,7 +16,7 @@
<p><b>Global Configuration Directives</b>:</p>
<ul>
<li><strong>Template </strong>[templateName]<br>
sets a new default template for file actions.<br></li>
sets a non-standard default template for this module.<br></li>
</ul>
<p>&nbsp;</p>
@ -59,7 +59,7 @@
<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul>
<p><b>Sample:</b></p>
<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p>
<textarea rows="5" cols="60">Module (path="builtin:omfwd")
<textarea rows="5" cols="60">Module (load="builtin:omfwd")
*.* action(type="omfwd"
Target="192.168.2.11"
Port="10514"

86
doc/omjournal.html Normal file
View File

@ -0,0 +1,86 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<meta http-equiv="Content-Language" content="en">
<title>Linux Journal Output Module (omjournal)</title></head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>Linux Journal Output Module (omjournal)</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; omjournal</b></p>
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Available since</b>: 7.3.7</p>
<p><b>Description</b>:</p>
<p>The omjournal output module provides an interface to the Linux journal.
It is meant to be used in those cases where the Linux journal is being used
as the sole system log database. With omjournal, messages from various
sources (e.g. files and remote devices) can also be written to the journal
and processed by its tools.
<p>A typical use case we had on our mind is a SOHO environment, where the
user wants to include syslog data obtained from the local router to be
part of the journal data.
<p>We suggest to check out our short presentation on
<a href="http://youtu.be/GTS7EuSdFKE">rsyslog journal integration</a> to
learn more details of anticipated use cases.
<p>&nbsp;</p>
<p><b>Module Configuration Parameters</b>:</p>
<p>Currently none.
<p>&nbsp;</p>
<p><b>Action Confguration Parameters</b>:</p>
<p>Currently none.
<p><b>Caveats/Known Bugs:</b>
<ul>
<li>One needs to be careful that no message routing loop is created. The
systemd journal forwards messages it receives to the traditional syslog
system (if present). That means rsyslog will receive the same message that
it just wrote as new input on imuxsock. If not handled specially and assuming
all messages be written to the journal, the message would be emitted to the
journal again and a deadly loop is started.
<p>To prevent that, imuxsock by default does not accept messages originating
from its own process ID, aka it ignores messages from the current instance of
rsyslogd. However, this setting can be changed, and if so the problem may occur.
</ul>
<p><b>Sample:</b></p>
<p>We assume we have a DSL router inside the network and would like to
receive its syslog message into the journal. Note that this configuration can be
used without havoing any other syslog functionality at all (most importantly, there
is no need to write any file to /var/log!). We assume syslog over UDP, as this
is the most probable choice for the SOHO environment that this use case reflects.
To log to syslog data to the journal, add the following snippet to rsyslog.conf:
<textarea rows="20" cols="60">/* first, we make sure all necessary
* modules are present:
*/
module(load="imudp") # input module for UDP syslog
module(load="omjournal") # output module for journal
/* then, define the actual server that listens to the
* router. Note that 514 is the default port for UDP
* syslog and that we use a dedicated ruleset to
* avoid mixing messages with the local log stream
* (if there is any).
*/
input(type="imudp" port="514" ruleset="writeToJournal")
/* inside that ruleset, we just write data to the journal: */
ruleset(name="writeToJournal") {
action(type="omjournal")
}
</textarea>
<p>Note that this can be your sole rsyslog.conf if you do not use rsyslog
for anything else than receving the router syslog messages.
<p>If you do not receive messages, <b>you probably need to enable inbound UDP
syslog traffic in your firewall</b>.
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual
index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>
</body></html>

46
doc/omlibdbi.html
View File

@ -54,7 +54,23 @@ dlopen()ed plugin (as omlibdbi is). So in short, you probably save you
a lot of headache if you make sure you have at least libdbi version
0.8.3 on your system.
</p>
<p><b>Action Parameters</b>:</p>
<p><b>Module Parameters</b></p>
<ul>
<li><b>template</b><br>
The default template to use. This template is used when no template is
explicitely specified in the action() statement.
<li><b>driverdirectory</b><br>
Path to the libdbi drivers. Usually,
you do not need to set it. If you installed libdbi-drivers at a
non-standard location, you may need to specify the directory here. If
you are unsure, do <b>not</b> use this configuration directive.
Usually, everything works just fine.
Note that this was an action() paramter in rsyslog versions below 7.3.0.
However, only the first action's driverdirectory parameter was actually used.
This has been cleaned up in 7.3.0, where this now is a module paramter.
</li>
</ul>
<p><b>Action Parameters</b></p>
<ul>
<li><b>server</b><br>Name or address of the MySQL server
<li><b>db</b><br>Database to use
@ -68,24 +84,18 @@ writiting "mysql" (suggest to use ommysql instead), "firebird" (Firbird
and InterBase), "ingres", "msql", "Oracle", "sqlite", "sqlite3",
"freetds" (for Microsoft SQL and Sybase) and "pgsql" (suggest to use
ompgsql instead).</li>
<li><b>driverdirectory</b><br>
Path to the libdbi drivers. Usually,
you do not need to set it. If you installed libdbi-drivers at a
non-standard location, you may need to specify the directory here. If
you are unsure, do <b>not</b> use this configuration directive.
Usually, everything works just fine.</li>
</ul>
<p><b>Legacy (pre-v6) Configuration Directives</b>:</p>
<p>It is strongly recommended NOT to use legacy format.
<ul>
<li><b>$ActionLibdbiDriverDirectory /path/to/dbd/drivers</b>
<li><i>$ActionLibdbiDriverDirectory /path/to/dbd/drivers</i>
- like the driverdirectory action parameter.
<li><strong>$ActionLibdbiDriver drivername</strong><br> - like the drivername action parameter.
<li><span style="font-weight: bold;">$ActionLibdbiHost hostname</span> - like the server action parameter
The host to connect to.</li>
<li><b>$ActionLibdbiUserName user</b> - like the uid action parameter
<li><b>$ActionlibdbiPassword</b> - like the pwd action parameter
<li><b>$ActionlibdbiDBName db</b> - like the db action parameter
<li><b>selector line: :omlibdbi:<i>;template</i></b><br>
<li><i>$ActionLibdbiDriver drivername</i> - like the drivername action parameter
<li><i>$ActionLibdbiHost hostname</i> - like the server action parameter
<li><i>$ActionLibdbiUserName user</i> - like the uid action parameter
<li><i>$ActionlibdbiPassword</i> - like the pwd action parameter
<li><i>$ActionlibdbiDBName db</i> - like the db action parameter
<li><i>selector line: :omlibdbi:<code>;template</code></i><br>
executes the recently configured omlibdbi action. The ;template part is
optional. If no template is provided, a default template is used (which
is currently optimized for MySQL - sorry, folks...)</li>
@ -114,14 +124,14 @@ database "syslog_db" on mysqlsever.example.com. The server is MySQL and
being accessed under the account of "user" with password "pwd" (if you
have empty passwords, just remove the $ActionLibdbiPassword line).<br>
</p>
<textarea rows="5" cols="60">$ModLoad omlibdbi
<textarea rows="5" cols="60">module(load="omlibdbi")
*.* action(type="omlibdbi" driver="mysql"
server="mysqlserver.example.com" db="syslog_db"
uid="user" pwd="pwd"
</textarea>
<p><b>Sample:</b></p>
<p><b>Legacy Sample:</b></p>
<p>The same as above, but in legacy config format (pre rsyslog-v6):
<textarea rows="10" cols="60">$ModLoad omlibdbi
<textarea rows="8" cols="60">$ModLoad omlibdbi
$ActionLibdbiDriver mysql
$ActionLibdbiHost mysqlserver.example.com
$ActionLibdbiUserName user

60
doc/ommongodb.html Normal file
View File

@ -0,0 +1,60 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<meta http-equiv="Content-Language" content="en">
<title>MongoDB Output Module</title>
</head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>MongoDB Output Module</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; ommongodb</b></p>
<p><b>Author: </b>Rainer Gerhards
&lt;rgerhards@adiscon.com&gt;</p>
<p><b>Description</b>:</p>
<p>This module provides native support for logging to MongoDB.
</p>
<p><b>Action Parameters</b>:</p>
<ul>
<li><b>server</b><br>Name or address of the MongoDB server
<li><b>serverport</b><br>Permits to select
a non-standard port for the MongoDB server. The default is 0, which means the
system default port is used. There is no need to specify this parameter unless
you know the server is running on a non-standard listen port.
<li><b>db</b><br>Database to use
<li><b>collection</b><br>Collection to use
<li><b>uid</b><br>logon userid used to connect to server. Must have proper permissions.
<li><b>pwd</b><br>the user's password
<li><b>template</b><br>Template to use when submitting messages.
</ul>
<p>Note rsyslog contains a canned default template to write to the MongoDB. It
will be used automatically if no other template is specified to be used. This template is:
<p>
<textarea rows="5" cols="80">template(name="BSON" type="string" string="\"sys\" : \"%hostname%\", \"time\" : \"%timereported:::rfc3339%\", \"time_rcvd\" : \"%timegenerated:::rfc3339%\", \"msg\" : \"%msg%\", \"syslog_fac\" : \"%syslogfacility%\", \"syslog_sever\" : \"%syslogseverity%\", \"syslog_tag\" : \"%syslogtag%\", \"procid\" : \"%programname%\", \"pid\" : \"%procid%\", \"level\" : \"%syslogpriority-text%\"")
</textarea>
<p>This creates the BSON document needed for MongoDB if no template is specified. The default
schema is aligned to CEE and project lumberjack. As such, the field names are standard
lumberjack field names, and <b>not</b>
<a href="property_replacer.html">rsyslog property names</a>. When specifying templates, be sure
to use rsyslog property names as given in the table. If you would like to use lumberjack-based
field names inside MongoDB (which probably is useful depending on the use case), you need to
select fields names based on the lumberjack schema.
If you just want to use a subset of the fields, but with lumberjack names, you can look up the
mapping in the default template. For example, the lumberjack field "level" contains the rsyslog
property "syslogpriority-text".
<p><b>Sample:</b></p>
<p>The following sample writes all syslog messages to the
database "syslog" and into the collection "log" on mongosever.example.com. The server is
being accessed under the account of "user" with password "pwd".
</p>
<textarea rows="5" cols="80">module(load="ommongodb")
*.* action(type="ommongodb" server="mongoserver.example.com" db="syslog" collection="log" uid="user" pwd="pwd")
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the ASL 2.0.</font></p>
</body></html>

41
doc/omrelp.html
View File

@ -16,10 +16,34 @@ RELP protocol. For RELP's advantages over plain tcp syslog, please see
the documentation for <a href="imrelp.html">imrelp</a>
(the server counterpart).&nbsp;</p>
<span style="font-weight: bold;">Setup</span>
<p>Please note the <a href="http://www.librelp.com">librelp</a>
<p>Please note that <a href="http://www.librelp.com">librelp</a>
is required for imrelp (it provides the core relp protocol
implementation).</p>
<p><b>Configuration Directives</b>:</p>
<p><b>Action Configuration Parameters</b>:</p>
<p>This module supports RainerScript configuration starting with
rsyslog 7.3.10. For older versions, legacy configuration directives
must be used.
<ul>
<li><b>target </b>(mandatory)<br>
The target server to connect to.
</li>
<li><b>template </b>(not mandatory, default "RSYSLOG_ForwardFormat")<br>
Defines the template to be used for the output.
</li>
<li><b>timeout </b>(not mandatory, default 90)<br>
Timeout for relp sessions. If set too low, valid sessions
may be considered dead and tried to recover.
</li>
</ul>
<p><b>Sample:</b></p>
<p>The following sample sends all messages to the central server
"centralserv" at port 2514 (note that that server must run imrelp on
port 2514).
</p>
<textarea rows="3" cols="60">module(load="omrelp")
action(type="omrelp" target="centralserv" port="2514")
</textarea>
<p><b>Legacy Configuration Directives</b>:</p>
<p>This module uses old-style action configuration to keep
consistent with the forwarding rule. So far, no additional
configuration directives can be specified. To send a message via RELP,
@ -33,18 +57,15 @@ use</p>
<b>Caveats/Known Bugs:</b>
<p>See <a href="imrelp.html">imrelp</a>,
which documents them.&nbsp;</p>
<p><b>Sample:</b></p>
<p><b>Legacy Sample:</b></p>
<p>The following sample sends all messages to the central server
"centralserv" at port 2514 (note that that server must run imrelp on
port 2514). Rsyslog's high-precision timestamp format is used, thus the
special "RSYSLOG_ForwardFormat" (case sensitive!) template is used.<br>
port 2514).
</p>
<textarea rows="15" cols="60">$ModLoad omrelp
# forward messages to the remote server "myserv" on
# port 2514
*.* :omrelp:centralserv:2514;RSYSLOG_ForwardFormat
<textarea rows="3" cols="60">$ModLoad omrelp
*.* :omrelp:centralserv:2514
</textarea>
Note: to use IPv6 addresses, encode them in [::1] format.
<p>Note: to use IPv6 addresses, encode them in [::1] format.
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the

173
doc/omudpspoof.html
View File

@ -7,46 +7,161 @@
<h1>UDP spoofing output module (omudpspoof)</h1>
<p><b>Module Name:&nbsp;&nbsp;&nbsp; omstdout</b></p>
<p><b>Author: </b>David Lang &lt;david@lang.hm&gt; and Rainer Gerhards
&lt;rgerhards@adiscon.com&gt;</p>
<p><b>Available Since</b>: 5.1.3</p>
<p><b>Authors: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;
and David Lang &lt;david@lang.hm&gt;
</p>
<p><b>Available Since</b>: 5.1.3 / v7 config since 7.2.5</p>
<p><b>Description</b>:</p>
<p>This module is similar to the regular UDP forwarder, but permits to
spoof the sender address. Also, it enables to circle through a number of
source ports.
<p><b>Configuration Directives</b>:</p>
<p><b>Important:</b> This module requires root priveleges for its low-level
socket access. As such, the <b>module will not work if rsyslog is configured to
drop privileges</b>.
<p><b>load() Parameters</b>:</p>
<ul>
<li><b>$ActionOMOMUDPSpoofSourceNameTemplate</b> &lt;templatename&gt;<br>
This is the name of the template that contains a
numerical IP address that is to be used as the source system IP address.
While it may often be a constant value, it can be generated as usual via the
property replacer, as long as it is a valid IPv4 address. If not specified, the
build-in default template RSYSLOG_omudpspoofDfltSourceTpl is used. This template is defined
as follows:<br>
$template RSYSLOG_omudpspoofDfltSourceTpl,"%fromhost-ip%"<br>
So in essence, the default template spoofs the address of the system the message
was received from. This is considered the most important use case.
<li><b>$ActionOMUDPSpoofTargetHost</b> &lt;hostname&gt;<br>
Host that the messages shall be sent to.
<li><b>$ActionOMUDPSpoofTargetPort</b> &lt;port&gt;<br>
Remote port that the messages shall be sent to.
<li><b>$ActionOMUDPSpoofDefaultTemplate</b> &lt;templatename&gt;<br>
This setting instructs omudpspoof to use a template different from the
default template for all of its actions that do not have a template specified
explicitely.
<li><b>$ActionOMUDPSpoofSourcePortStart</b> &lt;number&gt;<br>
Specifies the start value for circeling the source ports. Must be less than or
equal to the end value. Default is 32000.
<li><b>$ActionOMUDPSpoofSourcePortEnd</b> &lt;number&gt;<br>
Specifies the ending value for circeling the source ports. Must be less than or
equal to the start value. Default is 42000.
<li><strong>Template </strong>[templateName]<br>
sets a non-standard default template for this module.<br></li>
</ul>
<p>&nbsp;</p>
<p><b>action() parameters</b>:</p>
<ul>
<li><strong>Target </strong>string<br>
Name or IP-Address of the system that shall receive messages. Any resolvable name is fine. <br></li><br>
<li><strong>Port </strong>[Integer, Default 514]<br>
Name or numerical value of port to use when connecting to target. <br></li><br>
<li><b>Template</b>[Word]<br>
Template to use as message text.
<br></li><br>
<li><strong>SourceTemplate </strong>[Word]<br>
This is the name of the template that contains a
numerical IP address that is to be used as the source system IP address.
While it may often be a constant value, it can be generated as usual via the
property replacer, as long as it is a valid IPv4 address. If not specified, the
build-in default template RSYSLOG_omudpspoofDfltSourceTpl is used. This template is defined
as follows:<br>
template(name="RSYSLOG_omudpspoofDfltSourceTpl" type="string" string="%fromhost-ip%")<br>
So in essence, the default template spoofs the address of the system the message
was received from. This is considered the most important use case.
<br></li><br>
<li><b>SourcePortStart</b>[Word]<br>
Specifies the start value for circeling the source ports. Must be less than or
equal to the end value. Default is 32000.
<br></li><br>
<li><b>SourcePortEnd</b>[Word]<br>
Specifies the ending value for circeling the source ports. Must be less than or
equal to the start value. Default is 42000.
<br></li><br>
<li><b>mtu</b>[Integer, default 1500]<br>
Maximum MTU supported by the network. Default respects Ethernet and must
usually not be adjusted. Setting a too-high MTU can lead to message loss,
too low to excess message fragmentation. Change only if you really know what
you are doing. This is always given in number of bytes.
<br></li><br>
</ul>
<p><b>pre-v7 Configuration Directives</b>:</p>
<ul>
<li><b>$ActionOMOMUDPSpoofSourceNameTemplate</b> &lt;templatename&gt;
- equivalent to the "sourceTemplate" parameter.
<li><b>$ActionOMUDPSpoofTargetHost</b> &lt;hostname&gt; - equivalent to the "target" parameter.
<li><b>$ActionOMUDPSpoofTargetPort</b> &lt;port&gt; - equivalent to the "target" parameter.
<li><b>$ActionOMUDPSpoofDefaultTemplate</b> &lt;templatename&gt;
- equivalent to the "template" load() parameter.
<li><b>$ActionOMUDPSpoofSourcePortStart</b> &lt;number&gt;
- equivalent to the "SourcePortStart" parameter.
<li><b>$ActionOMUDPSpoofSourcePortEnd</b> &lt;number&gt;
- equivalent to the "SourcePortEnd" parameter.
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
<li><b>IPv6</b> is currently not supported. If you need this capability, please let us
know via the rsyslog mailing list.
<li>Versions shipped prior to rsyslog 7.2.5 do not support message sizes over 1472 bytes (more
pricesely: over the network-supported MTU). Starting with 7.2.5, those messages will be
fragmented, up to a total upper limit of 64K (induced by UDP). Message sizes over
64K will be truncated. For older versions, messages over 1472 may be totally discarded
or truncated, depending on version and environment.
</ul>
<p><b>Sample:</b></p>
<p><b>Config Samples</b></p>
<p>The following sample forwards all syslog messages in standard form to the
remote server server.example.com. The original sender's address is used. We do not
care about the source port. This example is considered the typical use case for
omudpspoof.
</p>
<textarea rows="3" cols="80">module(load="omudpspoof")
action(type="omudpspoof" target="server.example.com")
</textarea>
<p>The following sample forwards all syslog messages in unmodified form to the
remote server server.example.com. The sender address 192.0.2.1 with fixed
source port 514 is used.
</p>
<textarea rows="7" cols="80">module(load="omudpspoof")
template(name="spoofaddr" type="string" string="192.0.2.1")
template(name="spooftemplate" type="string" string="%rawmsg%")
action(type="omudpspoof" target="server.example.com"
sourcetemplate="spoofaddr" template="spooftemplate"
sourceport.start="514" sourceport.end="514)
</textarea>
<p>The following sample is exatly like the previous, but it specifies a larger size
MTU. If, for example, the envrionment supports Jumbo Ethernet frames, increasing the
MTU is useful as it reduces packet fragmentation, which most often is the source of
problems. Note that setting the MTU to a value larger than the local-attached network
supports will lead to send errors and loss of message. So use with care!
</p>
<textarea rows="8" cols="80">module(load="omudpspoof")
template(name="spoofaddr" type="string" string="192.0.2.1")
template(name="spooftemplate" type="string" string="%rawmsg%")
action(type="omudpspoof" target="server.example.com"
sourcetemplate="spoofaddr" template="spooftemplate"
sourceport.start="514" sourceport.end="514
mtu="8000")
</textarea>
<p>Of course, the action can be combined with any type of filter, for
example a tradition PRI filter:</p>
<textarea rows="8" cols="80">module(load="omudpspoof")
template(name="spoofaddr" type="string" string="192.0.2.1")
template(name="spooftemplate" type="string" string="%rawmsg%")
local0.* action(type="omudpspoof" target="server.example.com"
sourcetemplate="spoofaddr" template="spooftemplate"
sourceport.start="514" sourceport.end="514
mtu="8000")
</textarea>
<p>... or any complex expression-based filter:</p>
<textarea rows="8" cols="80">module(load="omudpspoof")
template(name="spoofaddr" type="string" string="192.0.2.1")
template(name="spooftemplate" type="string" string="%rawmsg%")
if prifilt("local0.*") and $msg contains "error" then
action(type="omudpspoof" target="server.example.com"
sourcetemplate="spoofaddr" template="spooftemplate"
sourceport.start="514" sourceport.end="514
mtu="8000")
</textarea>
<p>and of course it can also be combined with as many other actions
as one likes:</p>
<textarea rows="11" cols="80">module(load="omudpspoof")
template(name="spoofaddr" type="string" string="192.0.2.1")
template(name="spooftemplate" type="string" string="%rawmsg%")
if prifilt("local0.*") and $msg contains "error" then {
action(type="omudpspoof" target="server.example.com"
sourcetemplate="spoofaddr" template="spooftemplate"
sourceport.start="514" sourceport.end="514
mtu="8000")
action(type="omfile" file="/var/log/somelog")
stop # or whatever...
}
</textarea>
<p><b>Legacy Sample (pre-v7):</b></p>
<p>The following sample forwards all syslog messages in standard form to the
remote server server.example.com. The original sender's address is used. We do not
care about the source port. This example is considered the typical use case for

6
doc/property_replacer.html
View File

@ -413,6 +413,12 @@ option when forwarding to remote hosts - they may treat the date as invalid
<td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td>
</tr>
<tr>
<td>pos-end-relative</td>
<td>the from and to position is relative to the end of the string
instead of the usual start of string. (available since rsyslog v7.3.10)
</td>
</tr>
<tr>
<td><b>ControlCharacters</b></td>
<td>Option values for how to process control characters</td>
</tr>

30
doc/rainerscript.html
View File

@ -51,6 +51,11 @@ of a and b should be tested as "a &lt;&gt; b". The "not" operator
should be reserved to cases where it actually is needed to form a
complex boolean expression. In those cases, parenthesis are highly
recommended.
<h2>Lookup Tables</h2>
<p><a href="lookup_tables.html">Lookup tables</a> are a powerful construct
to obtain "class" information based on message content (e.g. to build
log file names for different server types, departments or remote
offices).
<h2>Functions</h2>
<p>RainerScript supports a currently quite limited set of functions:
<ul>
@ -61,11 +66,30 @@ variable, if it exists. Returns an empty string if it does not exist.
<li>cstr(expr) - converts expr to a string value
<li>cnum(expr) - converts expr to a number (integer)
<li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise
<li>re_extract(expr, re, match, submatch, no-found) - extracts
data from a string (property) via a regular expression match.
POSIX ERE regular expressions are used. The variable "match" contains
the number of the match to use. This permits to pick up more than the
first expression match. Submatch is the submatch to match (max 50 supported).
The "no-found" parameter specifies which string is to be returned in case when
the regular expression is not found. Note that match and submatch start with
zero. It currently is not possible to extract more than one submatch with
a single call.
<li>field(str, delim, matchnbr) - returns a field-based substring. str is the string
to search, delim is the numerical ascii value of the field delimiter (so that
non-printable characters can by specified) and matchnbr is the match to search
to search, delim is the delimiter and matchnbr is the match to search
for (the first match starts at 1). This works similar as the field based
property-replacer option.
Versions prior to 7.3.7 only support a single character as delimiter character.
Starting with version 7.3.7, a full string can be used as delimiter. If a single
character is being used as delimiter, delim is the numerical ascii value of the
field delimiter character (so that non-printable characters can by specified). If a
string is used as delmiter, a multi-character string (e.g. "#011") is to be
specified. Samples:<br>
set $!usr!field = field($msg, 32, 3); -- the third field, delimited by space<br>
set $!usr!field = field($msg, "#011", 3); -- the third field, delmited by "#011"<br>
Note that when a single character is specified as string [field($msg, ",", 3)] a
string-based extraction is done, which is more performance intense than the
equivalent single-character [field($msg, 44 ,3)] extraction.
<li>prifilt(constant) - mimics a traditional PRI-based filter (like "*.*" or
"mail.info"). The traditional filter string must be given as a <b>constant string</b>.
Dynamic string evaluation is not permitted (for performance reasons).
@ -80,7 +104,7 @@ if $msg contains getenv('TRIGGERVAR') then /path/to/errfile
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body></html>

2
doc/rsconf1_rulesetcreatemainqueue.html
View File

@ -58,7 +58,7 @@ mail.* /var/log/mail10516
# being written to the remote10516 file - as usual...
*.* /var/log/remote10516
# and now define listners bound to the relevant ruleset
# and now define listeners bound to the relevant ruleset
$InputTCPServerBindRuleset remote10514
$InputTCPServerRun 10514

2
doc/rsconf1_rulesetparser.html
View File

@ -9,7 +9,7 @@
<p><b>Type:</b> ruleset-specific configuration directive</p>
<p><b>Parameter Values:</b> string</p>
<p><b>Available since:</b> 5.3.4+</p>
<p><b>Default:</b> rsyslog.rfc5424 followed by rsyslog.rfc5425</p>
<p><b>Default:</b> rsyslog.rfc5424 followed by rsyslog.rfc3164</p>
<p><b>Description:</b></p>
<p>
This directive permits to specify which

2
doc/rsyslog_conf_basic_structure.html
View File

@ -80,7 +80,7 @@ A ruleset can be "bound" (assigned) to a specific input. In the analogy, this me
a message comes in via that input, the "program" (ruleset) bound to it will be executed
(but not any other!).
<p>There is detail documentation available for
<a href="multi_ruleset">rsyslog rulesets</a>.
<a href="multi_ruleset.html">rsyslog rulesets</a>.
<p>For quick reference, rulesets are defined as follows:
<pre>
ruleset(name="rulesetname") {

190
doc/rsyslog_conf_global.html
View File

@ -17,12 +17,12 @@ appear as implementation progresses.
many parameter settings modify queue parameters. If in doubt, use the
default, it is usually well-chosen and applicable in most cases.</p>
<ul>
<li><a href="rsconf1_abortonuncleanconfig.html">$AbortOnUncleanConfig</a> - abort startup if there is
<li><a href="rsconf1_abortonuncleanconfig.html"><b>$AbortOnUncleanConfig</b></a> - abort startup if there is
any issue with the config file</li>
<li><a href="rsconf1_actionexeconlywhenpreviousissuspended.html">$ActionExecOnlyWhenPreviousIsSuspended</a></li>
<li>$ActionName &lt;a_single_word&gt; - used primarily for documentation, e.g. when
<li><b>$ActionName</b> &lt;a_single_word&gt; - used primarily for documentation, e.g. when
generating a configuration graph. Available sice 4.3.1.
<li>$ActionExecOnlyOnceEveryInterval &lt;seconds&gt; -
<li><b>$ActionExecOnlyOnceEveryInterval</b> &lt;seconds&gt; -
execute action only if the last execute is at last
&lt;seconds&gt; seconds in the past (more info in <a href="ommail.html">ommail</a>,
but may be used with any action)</li>
@ -46,60 +46,60 @@ The question is if this is desired behavior? Or should the rule only be
triggered if the messages occur within an e.g. 20 minute window? If the
later is the case, you need a
<br>
$ActionExecOnlyEveryNthTimeTimeout 1200
<b>$ActionExecOnlyEveryNthTimeTimeout 1200</b>
<br>
This directive will timeout previous messages seen if they are older
than 20 minutes. In the example above, the count would now be always 1
and consequently no rule would ever be triggered.
<li><a href="omfile.html">$ActionFileDefaultTemplate</a> [templateName] - sets a new default template for file actions</li>
<li><a href="omfile.html">$ActionFileEnableSync</a> [on/<span style="font-weight: bold;">off</span>] - enables file
<li><a href="omfile.html"><b>$ActionFileDefaultTemplate</b></a> [templateName] - sets a new default template for file actions</li>
<li><a href="omfile.html"><b>$ActionFileEnableSync</b></a> [on/<span style="font-weight: bold;">off</span>] - enables file
syncing capability of omfile</li>
<li><a href="omfwd.html">$ActionForwardDefaultTemplate</a> [templateName] - sets a new
<li><a href="omfwd.html"><b>$ActionForwardDefaultTemplate</b></a> [templateName] - sets a new
default template for UDP and plain TCP forwarding action</li>
<li>$ActionGSSForwardDefaultTemplate [templateName] - sets a
<li><b>$ActionGSSForwardDefaultTemplate</b> [templateName] - sets a
new default template for GSS-API forwarding action</li>
<li>$ActionQueueCheckpointInterval &lt;number&gt;</li>
<li>$ActionQueueDequeueBatchSize &lt;number&gt; [default 16]</li>
<li>$ActionQueueDequeueSlowdown &lt;number&gt; [number
<li><b>$ActionQueueCheckpointInterval</b> &lt;number&gt;</li>
<li><b>$ActionQueueDequeueBatchSize</b> &lt;number&gt; [default 16]</li>
<li><b>$ActionQueueDequeueSlowdown</b> &lt;number&gt; [number
is timeout in <i> micro</i>seconds (1000000us is 1sec!),
default 0 (no delay). Simple rate-limiting!]</li>
<li>$ActionQueueDiscardMark &lt;number&gt; [default
<li><b>$ActionQueueDiscardMark</b> &lt;number&gt; [default
9750]</li>
<li>$ActionQueueDiscardSeverity &lt;number&gt;
<li><b>$ActionQueueDiscardSeverity</b> &lt;number&gt;
[*numerical* severity! default 8 (nothing discarded)]</li>
<li>$ActionQueueFileName &lt;name&gt;</li>
<li>$ActionQueueHighWaterMark &lt;number&gt; [default
<li><b>$ActionQueueFileName</b> &lt;name&gt;</li>
<li><b>$ActionQueueHighWaterMark</b> &lt;number&gt; [default
8000]</li>
<li>$ActionQueueImmediateShutdown [on/<b>off</b>]</li>
<li>$ActionQueueSize &lt;number&gt;</li>
<li>$ActionQueueLowWaterMark &lt;number&gt; [default
<li><b>$ActionQueueImmediateShutdown</b> [on/<b>off</b>]</li>
<li><b>$ActionQueueSize</b> &lt;number&gt;</li>
<li><b>$ActionQueueLowWaterMark</b> &lt;number&gt; [default
2000]</li>
<li>$ActionQueueMaxFileSize &lt;size_nbr&gt;, default 1m</li>
<li>$ActionQueueTimeoutActionCompletion &lt;number&gt;
<li><b>$ActionQueueMaxFileSize</b> &lt;size_nbr&gt;, default 1m</li>
<li><b>$ActionQueueTimeoutActionCompletion</b> &lt;number&gt;
[number is timeout in ms (1000ms is 1sec!), default 1000, 0 means
immediate!]</li>
<li>$ActionQueueTimeoutEnqueue &lt;number&gt; [number
<li><b>$ActionQueueTimeoutEnqueue</b> &lt;number&gt; [number
is timeout in ms (1000ms is 1sec!), default 2000, 0 means indefinite]</li>
<li>$ActionQueueTimeoutShutdown &lt;number&gt; [number
<li><b>$ActionQueueTimeoutShutdown</b> &lt;number&gt; [number
is timeout in ms (1000ms is 1sec!), default 0 (indefinite)]</li>
<li>$ActionQueueWorkerTimeoutThreadShutdown
<li><b>$ActionQueueWorkerTimeoutThreadShutdown</b>
&lt;number&gt; [number is timeout in ms (1000ms is 1sec!),
default 60000 (1 minute)]</li>
<li>$ActionQueueType [FixedArray/LinkedList/<b>Direct</b>/Disk]</li>
<li>$ActionQueueSaveOnShutdown&nbsp; [on/<b>off</b>]
<li><b>$ActionQueueType</b> [FixedArray/LinkedList/<b>Direct</b>/Disk]</li>
<li><b>$ActionQueueSaveOnShutdown&nbsp;</b> [on/<b>off</b>]
</li>
<li>$ActionQueueWorkerThreads &lt;number&gt;, num worker threads, default 1, recommended 1</li>
<li>$ActionQueueWorkerThreadMinumumMessages &lt;number&gt;, default 100</li>
<li><a href="rsconf1_actionresumeinterval.html">$ActionResumeInterval</a></li>
<li>$ActionResumeRetryCount &lt;number&gt; [default 0, -1 means eternal]</li>
<li><a href="omfwd.html">$ActionSendResendLastMsgOnReconnect</a> &lt;[on/<b>off</b>]&gt; specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication.
<li><a href="omfwd.html">$ActionSendStreamDriver</a> &lt;driver basename&gt; just like $DefaultNetstreamDriver, but for the specific action</li>
<li><a href="omfwd.html">$ActionSendStreamDriverMode</a> &lt;mode&gt;, default 0, mode to use with the stream driver (driver-specific)</li>
<li><a href="omfwd.html">$ActionSendStreamDriverAuthMode</a> &lt;mode&gt;,&nbsp; authentication mode to use with the stream driver. Note that this directive requires TLS
<li><b>$ActionQueueWorkerThreads</b> &lt;number&gt;, num worker threads, default 1, recommended 1</li>
<li><b>$ActionQueueWorkerThreadMinumumMessages</b> &lt;number&gt;, default 100</li>
<li><a href="rsconf1_actionresumeinterval.html"><b>$ActionResumeInterval</b></a></li>
<li><b>$ActionResumeRetryCount</b> &lt;number&gt; [default 0, -1 means eternal]</li>
<li><a href="omfwd.html"><b>$ActionSendResendLastMsgOnReconnect</b></a> &lt;[on/<b>off</b>]&gt; specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication.
<li><a href="omfwd.html"><b>$ActionSendStreamDriver</b></a> &lt;driver basename&gt; just like $DefaultNetstreamDriver, but for the specific action</li>
<li><a href="omfwd.html"><b>$ActionSendStreamDriverMode</b></a> &lt;mode&gt;, default 0, mode to use with the stream driver (driver-specific)</li>
<li><a href="omfwd.html"><b>$ActionSendStreamDriverAuthMode</b></a> &lt;mode&gt;,&nbsp; authentication mode to use with the stream driver. Note that this directive requires TLS
netstream drivers. For all others, it will be ignored.
(driver-specific)</li>
<li><a href="omfwd.html">$ActionSendStreamDriverPermittedPeer</a> &lt;ID&gt;,&nbsp; accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS
<li><a href="omfwd.html"><b>$ActionSendStreamDriverPermittedPeer</b></a> &lt;ID&gt;,&nbsp; accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS
netstream drivers. For all others, it will be ignored.
(driver-specific) -<span style="font-weight: bold;"> directive may go away</span>!</li>
<li><a href="omfwd.html"><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send
@ -120,40 +120,40 @@ heartbeat. Note that this option auto-resets to &quot;off&quot;, so if you inten
actions, it must be specified in front off <b>all</b> selector lines that should provide this
functionality.
</li>
<li><a href="rsconf1_allowedsender.html">$AllowedSender</a></li>
<li><a href="rsconf1_controlcharacterescapeprefix.html">$ControlCharacterEscapePrefix</a></li>
<li><a href="rsconf1_debugprintcfsyslinehandlerlist.html">$DebugPrintCFSyslineHandlerList</a></li>
<li><a href="rsconf1_allowedsender.html"><b>$AllowedSender</b></a></li>
<li><a href="rsconf1_controlcharacterescapeprefix.html"><b>$ControlCharacterEscapePrefix</b></a></li>
<li><a href="rsconf1_debugprintcfsyslinehandlerlist.html"><b>$DebugPrintCFSyslineHandlerList</b></a></li>
<li><a href="rsconf1_debugprintmodulelist.html">$DebugPrintModuleList</a></li>
<li><a href="rsconf1_debugprinttemplatelist.html">$DebugPrintTemplateList</a></li>
<li>$DefaultNetstreamDriver &lt;drivername&gt;, the default <a href="netstream.html">network stream driver</a> to use. Defaults to&nbsp;ptcp.$DefaultNetstreamDriverCAFile &lt;/path/to/cafile.pem&gt;</li>
<li>$DefaultNetstreamDriverCertFile &lt;/path/to/certfile.pem&gt;</li>
<li>$DefaultNetstreamDriverKeyFile &lt;/path/to/keyfile.pem&gt;</li>
<li><a href="rsconf1_debugprintmodulelist.html"><b>$DebugPrintModuleList</b></a></li>
<li><a href="rsconf1_debugprinttemplatelist.html"><b>$DebugPrintTemplateList</b></a></li>
<li><b>$DefaultNetstreamDriver</b> &lt;drivername&gt;, the default <a href="netstream.html">network stream driver</a> to use. Defaults to&nbsp;ptcp.$DefaultNetstreamDriverCAFile &lt;/path/to/cafile.pem&gt;</li>
<li><b>$DefaultNetstreamDriverCertFile</b> &lt;/path/to/certfile.pem&gt;</li>
<li><b>$DefaultNetstreamDriverKeyFile</b> &lt;/path/to/keyfile.pem&gt;</li>
<li><b>$DefaultRuleset</b> <i>name</i> - changes the default ruleset for unbound inputs to
the provided <i>name</i> (the default default ruleset is named
&quot;RSYSLOG_DefaultRuleset&quot;). It is advised to also read
our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li>
<li><a href="omfile.html"><b>$CreateDirs</b></a> [<b>on</b>/off] - create directories on an as-needed basis</li>
<li><a href="omfile.html">$DirCreateMode</a></li>
<li><a href="omfile.html">$DirGroup</a></li>
<li><a href="omfile.html">$DirOwner</a></li>
<li><a href="rsconf1_dropmsgswithmaliciousdnsptrrecords.html">$DropMsgsWithMaliciousDnsPTRRecords</a></li>
<li><a href="rsconf1_droptrailinglfonreception.html">$DropTrailingLFOnReception</a></li>
<li><a href="omfile.html">$DynaFileCacheSize</a></li>
<li><a href="rsconf1_escape8bitcharsonreceive.html">$Escape8BitCharactersOnReceive</a></li>
<li><a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a></li>
<li><a href="omfile.html"><b>$DirCreateMode</b></a></li>
<li><a href="omfile.html"><b>$DirGroup</b></a></li>
<li><a href="omfile.html"><b>$DirOwner</b></a></li>
<li><a href="rsconf1_dropmsgswithmaliciousdnsptrrecords.html"><b>$DropMsgsWithMaliciousDnsPTRRecords</b></a></li>
<li><a href="rsconf1_droptrailinglfonreception.html"><b>$DropTrailingLFOnReception</b></a></li>
<li><a href="omfile.html"><b>$DynaFileCacheSize</b></a></li>
<li><a href="rsconf1_escape8bitcharsonreceive.html"><b>$Escape8BitCharactersOnReceive</b></a></li>
<li><a href="rsconf1_escapecontrolcharactersonreceive.html"><b>$EscapeControlCharactersOnReceive</b></a></li>
<li><b>$EscapeControlCharactersOnReceive</b> [<b>on</b>|off] - escape USASCII HT character</li>
<li>$SpaceLFOnReceive [on/<b>off</b>] - instructs rsyslogd to replace LF with spaces during message reception (sysklogd compatibility aid)</li>
<li>$ErrorMessagesToStderr [<b>on</b>|off] - direct rsyslogd error message to stderr (in addition to other targets)</li>
<li><a href="omfile.html">$FailOnChownFailure</a></li>
<li><a href="omfile.html">$FileCreateMode</a></li>
<li><a href="omfile.html">$FileGroup</a></li>
<li><a href="omfile.html">$FileOwner</a></li>
<li><a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a></li>
<li><a href="rsconf1_gssforwardservicename.html">$GssForwardServiceName</a></li>
<li><a href="rsconf1_gsslistenservicename.html">$GssListenServiceName</a></li>
<li><a href="rsconf1_gssmode.html">$GssMode</a></li>
<li><a href="rsconf1_includeconfig.html">$IncludeConfig</a></li><li>MainMsgQueueCheckpointInterval &lt;number&gt;</li>
<li><b>$SpaceLFOnReceive</b> [on/<b>off</b>] - instructs rsyslogd to replace LF with spaces during message reception (sysklogd compatibility aid)</li>
<li><b>$ErrorMessagesToStderr</b> [<b>on</b>|off] - direct rsyslogd error message to stderr (in addition to other targets)</li>
<li><a href="omfile.html"><b>$FailOnChownFailure</b></a></li>
<li><a href="omfile.html"><b>$FileCreateMode</b></a></li>
<li><a href="omfile.html"><b>$FileGroup</b></a></li>
<li><a href="omfile.html"><b>$FileOwner</b></a></li>
<li><a href="rsconf1_generateconfiggraph.html"><b>$GenerateConfigGraph</b></a></li>
<li><a href="rsconf1_gssforwardservicename.html"><b>$GssForwardServiceName</b></a></li>
<li><a href="rsconf1_gsslistenservicename.html"><b>$GssListenServiceName</b></a></li>
<li><a href="rsconf1_gssmode.html"><b>$GssMode</b></a></li>
<li><a href="rsconf1_includeconfig.html"><b>$IncludeConfig</b></a></li><li>MainMsgQueueCheckpointInterval &lt;number&gt;</li>
<li><b>$LocalHostName</b> [name] - this directive permits to overwrite the system
hostname with the one specified in the directive. If the directive is given
multiple times, all but the last one will be ignored. Please note that startup
@ -166,39 +166,39 @@ This information might be needed by some log analyzers. If set to off, no such
status messages are logged, what may be useful for other scenarios.
[available since 4.7.0 and 5.3.0]
<li><b>$MainMsgQueueDequeueBatchSize</b> &lt;number&gt; [default 32]</li>
<li>$MainMsgQueueDequeueSlowdown &lt;number&gt; [number
<li><b>$MainMsgQueueDequeueSlowdown</b> &lt;number&gt; [number
is timeout in <i> micro</i>seconds (1000000us is 1sec!),
default 0 (no delay). Simple rate-limiting!]</li>
<li>$MainMsgQueueDiscardMark &lt;number&gt; [default 9750]</li>
<li>$MainMsgQueueDiscardSeverity &lt;severity&gt;
<li><b>$MainMsgQueueDiscardMark</b> &lt;number&gt; [default 9750]</li>
<li><b>$MainMsgQueueDiscardSeverity</b> &lt;severity&gt;
[either a textual or numerical severity! default 4 (warning)]</li>
<li>$MainMsgQueueFileName &lt;name&gt;</li>
<li>$MainMsgQueueHighWaterMark &lt;number&gt; [default
<li><b>$MainMsgQueueFileName</b> &lt;name&gt;</li>
<li><b>$MainMsgQueueHighWaterMark</b> &lt;number&gt; [default
8000]</li>
<li>$MainMsgQueueImmediateShutdown [on/<b>off</b>]</li>
<li><a href="rsconf1_mainmsgqueuesize.html">$MainMsgQueueSize</a></li>
<li>$MainMsgQueueLowWaterMark &lt;number&gt; [default
<li><b>$MainMsgQueueImmediateShutdown</b> [on/<b>off</b>]</li>
<li><a href="rsconf1_mainmsgqueuesize.html"><b>$MainMsgQueueSize</b></a></li>
<li><b>$MainMsgQueueLowWaterMark</b> &lt;number&gt; [default
2000]</li>
<li>$MainMsgQueueMaxFileSize &lt;size_nbr&gt;, default
<li><b>$MainMsgQueueMaxFileSize</b> &lt;size_nbr&gt;, default
1m</li>
<li>$MainMsgQueueTimeoutActionCompletion
<li><b>$MainMsgQueueTimeoutActionCompletion</b>
&lt;number&gt; [number is timeout in ms (1000ms is 1sec!),
default
1000, 0 means immediate!]</li>
<li>$MainMsgQueueTimeoutEnqueue &lt;number&gt; [number
<li><b>$MainMsgQueueTimeoutEnqueue</b> &lt;number&gt; [number
is timeout in ms (1000ms is 1sec!), default 2000, 0 means indefinite]</li>
<li>$MainMsgQueueTimeoutShutdown &lt;number&gt; [number
<li><b>$MainMsgQueueTimeoutShutdown</b> &lt;number&gt; [number
is timeout in ms (1000ms is 1sec!), default 0 (indefinite)]</li>
<li>$MainMsgQueueWorkerTimeoutThreadShutdown
<li><b>$MainMsgQueueWorkerTimeoutThreadShutdown</b>
&lt;number&gt; [number is timeout in ms (1000ms is 1sec!),
default 60000 (1 minute)]</li>
<li>$MainMsgQueueType [<b>FixedArray</b>/LinkedList/Direct/Disk]</li>
<li>$MainMsgQueueSaveOnShutdown&nbsp; [on/<b>off</b>]
<li><b>$MainMsgQueueType</b> [<b>FixedArray</b>/LinkedList/Direct/Disk]</li>
<li><b>$MainMsgQueueSaveOnShutdown&nbsp;</b> [on/<b>off</b>]
</li>
<li>$MainMsgQueueWorkerThreads &lt;number&gt;, num
<li><b>$MainMsgQueueWorkerThreads</b> &lt;number&gt;, num
worker threads, default 1, recommended 1</li>
<li>$MainMsgQueueWorkerThreadMinumumMessages &lt;number&gt;, default 100</li>
<li><a href="rsconf1_markmessageperiod.html">$MarkMessagePeriod</a> (immark)</li>
<li><b>$MainMsgQueueWorkerThreadMinumumMessages</b> &lt;number&gt;, default 100</li>
<li><a href="rsconf1_markmessageperiod.html"><b>$MarkMessagePeriod</b></a> (immark)</li>
<li><b><i>$MaxMessageSize</i></b> &lt;size_nbr&gt;, default 2k - allows to specify maximum supported message size
(both for sending and receiving). The default
should be sufficient for almost all cases. Do not set this below 1k, as it would cause
@ -221,9 +221,9 @@ instead of UDP (plain TCP syslog, RELP). This resolves the UDP stack size restri
<br>Note that 2k, the current default, is the smallest size that must be
supported in order to be compliant to the upcoming new syslog RFC series.
</li>
<li><a href="rsconf1_maxopenfiles.html">$MaxOpenFiles</a></li>
<li><a href="rsconf1_moddir.html">$ModDir</a></li>
<li><a href="rsconf1_modload.html">$ModLoad</a></li>
<li><a href="rsconf1_maxopenfiles.html"><b>$MaxOpenFiles</b></a></li>
<li><a href="rsconf1_moddir.html"><b>$ModDir</b></a></li>
<li><a href="rsconf1_modload.html"><b>$ModLoad</b></a></li>
<li><a href="omfile.html"><b>$OMFileAsyncWriting</b></a> [on/<b>off</b>], if turned on, the files will be written
in asynchronous mode via a separate thread. In that case, double buffers will be used so
that one buffer can be filled while the other buffer is being written. Note that in order
@ -246,15 +246,15 @@ error recovery thus can handle write errors without data loss. Note that this op
severely reduces the effect of zip compression and should be switched to off
for that use case. Note that the default -on- is primarily an aid to preserve
the traditional syslogd behaviour.</li>
<li><a href="omfile.html">$omfileForceChown</a> - force ownership change for all files</li>
<li><a href="omfile.html"><b>$omfileForceChown</b></a> - force ownership change for all files</li>
<li><b>$RepeatedMsgContainsOriginalMsg</b> [on/<b>off</b>] - "last message repeated n times" messages, if generated,
have a different format that contains the message that is being repeated.
Note that only the first "n" characters are included, with n to be at least 80 characters, most
probably more (this may change from version to version, thus no specific limit is given). The bottom
line is that n is large enough to get a good idea which message was repeated but it is not necessarily
large enough for the whole message. (Introduced with 4.1.5). Once set, it affects all following actions.</li>
<li><a href="rsconf1_repeatedmsgreduction.html">$RepeatedMsgReduction</a></li>
<li><a href="rsconf1_resetconfigvariables.html">$ResetConfigVariables</a></li>
<li><a href="rsconf1_repeatedmsgreduction.html"><b>$RepeatedMsgReduction</b></a></li>
<li><a href="rsconf1_resetconfigvariables.html"><b>$ResetConfigVariables</b></a></li>
<li><b>$Ruleset</b> <i>name</i> - starts a new ruleset or switches back to one already defined.
All following actions belong to that new rule set.
the <i>name</i> does not yet exist, it is created. To switch back to rsyslog's
@ -268,17 +268,17 @@ a specific (list of) message parsers to be used with the ruleset.
<li><b>$OptimizeForUniprocessor</b> [on/<b>off</b>] - turns on optimizatons which lead to better
performance on uniprocessors. If you run on multicore-machiens, turning this off lessens CPU load. The
default may change as uniprocessor systems become less common. [available since 4.1.0]</li>
<li>$PreserveFQDN [on/<b>off</b>) - if set to off (legacy default to remain compatible
<li><b>$PreserveFQDN</b> [on/<b>off</b>) - if set to off (legacy default to remain compatible
to sysklogd), the domain part from a name that is within the same domain as the receiving
system is stripped. If set to on, full names are always used.</li>
<li>$WorkDirectory &lt;name&gt; (directory for spool and other work files.
<li><b>$WorkDirectory</b> &lt;name&gt; (directory for spool and other work files.
Do <b>not</b> use trailing slashes)</li>
<li>$UDPServerAddress &lt;IP&gt; (imudp) -- local IP
<li><b>$UDPServerAddress</b> &lt;IP&gt; (imudp) -- local IP
address (or name) the UDP listens should bind to</li>
<li>$UDPServerRun &lt;port&gt; (imudp) -- former
<li><b>$UDPServerRun</b> &lt;port&gt; (imudp) -- former
-r&lt;port&gt; option, default 514, start UDP server on this
port, "*" means all addresses</li>
<li>$UDPServerTimeRequery &lt;nbr-of-times&gt; (imudp) -- this is a performance
<li><b>$UDPServerTimeRequery</b> &lt;nbr-of-times&gt; (imudp) -- this is a performance
optimization. Getting the system time is very costly. With this setting, imudp can
be instructed to obtain the precise time only once every n-times. This logic is
only activated if messages come in at a very fast rate, so doing less frequent
@ -286,10 +286,10 @@ time calls should usually be acceptable. The default value is two, because we ha
seen that even without optimization the kernel often returns twice the identical time.
You can set this value as high as you like, but do so at your own risk. The higher
the value, the less precise the timestamp.
<li><a href="droppriv.html">$PrivDropToGroup</a></li>
<li><a href="droppriv.html">$PrivDropToGroupID</a></li>
<li><a href="droppriv.html">$PrivDropToUser</a></li>
<li><a href="droppriv.html">$PrivDropToUserID</a></li>
<li><a href="droppriv.html"><b>$PrivDropToGroup</b></a></li>
<li><a href="droppriv.html"><b>$PrivDropToGroupID</b></a></li>
<li><a href="droppriv.html"><b>$PrivDropToUser</b></a></li>
<li><a href="droppriv.html"><b>$PrivDropToUserID</b></a></li>
<li><b>$Sleep</b> &lt;seconds&gt; - puts the rsyslog main thread to sleep for the specified
number of seconds immediately when the directive is encountered. You should have a
good reason for using this directive!</li>
@ -306,7 +306,7 @@ rsyslog.conf</b>. Otherwise, if error messages are triggered before this directi
is processed, rsyslog will fix the local host IP to "127.0.0.1", what than can
not be reset.
</li>
<li><a href="rsconf1_umask.html">$UMASK</a></li>
<li><a href="rsconf1_umask.html"><b>$UMASK</b></a></li>
</ul>
<p><b>Where &lt;size_nbr&gt; or integers are specified above,</b>
modifiers can be used after the number part. For example, 1k means

16
doc/rsyslog_conf_modules.html
View File

@ -45,6 +45,7 @@ to message generators.
<li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li>
<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li>
<li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li>
<li><a href="imjournal.html">imjournal</a> - Linux journal inuput module</li>
</ul>
<a name"om"></a><h2>Output Modules</h2>
@ -53,6 +54,7 @@ and messages be transmitted to various different targets.
<ul>
<li><a href="omfile.html">omfile</a> - file output module</li>
<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li>
<li><a href="omjournal.html">omjournal</a> - Linux journal output module</li>
<li><a href="ompipe.html">ompipe</a> - named pipe output module</li>
<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li>
<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li>
@ -72,6 +74,8 @@ permits rsyslog to alert folks by mail if something important happens</li>
<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li>
<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li>
<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li>
<li><a href="ommongodb.html">ommongodb</a> - output module for MongoDB</li>
<li><a href="omelasticsearch.html">omelasticsearch</a> - output module for ElasticSearch</li>
</ul>
<a name="pm"></a><h2>Parser Modules</h2>
@ -99,18 +103,18 @@ They can be implemented using either the output module or the parser module inte
From the rsyslog core's point of view, they actually are output or parser modules, it is their
implementation that makes them special.
<p>Currently, there exists only a limited set of such modules, but new ones could be written with
the methods the engine provides. They could be used, for example, to:
<ul>
<li>anonymize message content
<li>add dynamically computed content to message (fields)
</ul>
the methods the engine provides. They could be used, for example, to
add dynamically computed content to message (fields).
<p>Message modification modules are usually written for one specific task and thus
usually are not generic enough to be reused. However, existing module's code is
probably an excellent starting base for writing a new module. Currently, the following
modules exist inside the source tree:
<ul>
<li><a href="mmanon.html">mmanon</a> - used to anonymize log messages.
<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages.
Note that this actually is a <b>generic</b> module.
<li><a href="mmjsonparse.html">mmjsonparse</a> - used to interpret CEE/lumberjack
enabled structured log messages.
<li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside
the tag to correct the original sender system and priority of messages. Implemented via
the output module interface.
@ -179,7 +183,7 @@ filter settings. This graphic above is a high-level message flow diagram.
[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright &copy; 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
Copyright &copy; 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>
</body>

2
doc/rsyslog_conf_templates.html
View File

@ -134,6 +134,8 @@ csv-data is generated, "json", which formats proper json content (but without a
header) and "jsonf", which formats as a complete json field.
<li>position.from - obtain substring starting from this position (1 is the first position)
<li>position.to - obtain substring up to this position
<li>position.relativeToEnd - the from and to position is relative to the end of the string
instead of the usual start of string. (available since rsyslog v7.3.10)
<li>field.number - obtain this field match
<li>field.delimiter - decimal value of delimiter character for field extraction
<li>regex.expression - expression to use

17
doc/rsyslog_packages.html
View File

@ -12,20 +12,29 @@ like to maintain a package for a new distribution, please mail me at
appreciated. While I create the core daemon, the package maintainers are really
filling it with life, making it available to the average user. I am very
grateful for that!</p>
<p>This list has last been updated on 2008-07-11 by
<p>This list has last been updated on 2013-07-25 by
<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>.
New packages may appear at any time, so be sure to check this page whenever you
need a new one.</p>
<ul>
<li><b>Ubuntu</b> (maintained by Adiscon)
<ul>
<li><a href="http://www.rsyslog.com/ubuntu-repository/">http://www.rsyslog.com/ubuntu-repository/</a>
</ul>
<li><b>RHEL/CentOS</b> (maintained by Adiscon)
<ul>
<li><a href="http://www.rsyslog.com/rhelcentos-rpms/">http://www.rsyslog.com/rhelcentos-rpms/</a>
</ul>
<li><b>BSD</b> (maintained by infofarmer)
<ul>
<li><a href="http://www.freshports.org/sysutils/rsyslog/"> http://www.freshports.org/sysutils/rsyslog/</a>
<li><a href="http://www.freshports.org/sysutils/rsyslog/">http://www.freshports.org/sysutils/rsyslog/</a>
</ul>
<li><b>CentOS 4.3</b> (maintained by James Bergamin)
<ul>
<li><a href="http://www.se-community.com/~james/rsyslog/">
http://www.se-community.com/~james/rsyslog/</a>
<li><a href="http://www.se-community.com/~james/rsyslog/">http://www.se-community.com/~james/rsyslog/</a>
</ul>
<li><b>Debian</b> (maintained by Michael Biebl)

100
doc/sigprov_gt.html Normal file
View File

@ -0,0 +1,100 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Language" content="en">
<title>GuardTime Log Signature Provider (gt)</title>
</head>
<body>
<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a>
<h1>GuardTime Log Signature Provider (gt)</h1>
<p><b>Signature Provider Name:&nbsp;&nbsp;&nbsp; gt</b></p>
<p><b>Author: </b>Rainer Gerhards &lt;rgerhards@adiscon.com&gt;</p>
<p><b>Supported Since: </b>since 7.3.9
<p><b>Description</b>:</p>
<p>Provides the ability to sign syslog messages via the
GuardTime signature services.
</p>
<p><b>Configuration Parameters</b>:</p>
<p>Signature providers are loaded by omfile, when the
provider is selected in its "sig.providerName" parameter.
Parameters for the provider are given in the omfile action instance
line.
<p>This provider creates a signature file with the same base name but
the extension ".gtsig" for each log file (both for fixed-name files
as well as dynafiles). Both files together form a set. So you need to
archive both in order to prove integrity.
<ul>
<li><b>sig.hashFunction</b> &lt;Hash Algorithm&gt;<br>
The following hash algorithms are currently supported:
<ul>
<li>SHA1
<li>RIPEMD-160
<li>SHA2-224
<li>SHA2-256
<li>SHA2-384
<li>SHA2-512
</ul>
</li>
<li><b>sig.timestampService</b> &lt;timestamper URL&gt;<br>
This provides the URL of the timestamper service. If not selected,
a default server is selected. This may not necessarily be a good
one for your region.
</li>
<li><b>sig.block.sizeLimit</b> &lt;nbr-records&gt;<br>
The maximum number of records inside a single signature block. By
default, there is no size limit, so the signature is only written
on file closure. Note that a signature request typically takes between
one and two seconds. So signing to frequently is probably not a good
idea.
</li>
<li><b>sig.keepRecordHashes</b> &lt;on/<b>off</b>&gt;<br>
Controls if record hashes are written to the .gtsig file. This
enhances the ability to spot the location of a signature breach,
but costs considerable disk space (65 bytes for each log record
for SHA2-512 hashes, for example).
</li>
<li><b>sig.keepTreeHashes</b> &lt;on/<b>off</b>&gt;<br>
Controls if tree (intermediate) hashes are written to the .gtsig file. This
enhances the ability to spot the location of a signature breach,
but costs considerable disk space (a bit mire than the amount
sig.keepRecordHashes requries). Note that both Tree and Record
hashes can be kept inside the signature file.
</li>
</ul>
<b>Caveats/Known Bugs:</b>
<ul>
<li>currently none known
</li>
</ul>
<p><b>Samples:</b></p>
<p>This writes a log file with it's associated signature file. Default
parameters are used.
</p>
<textarea rows="3" cols="60">
action(type="omfile" file="/var/log/somelog"
sig.provider="gt")
</textarea>
<p>In the next sample, we use the more secure SHA2-512 hash function,
sign every 10,000 records and Tree and Record hashes are kept.
<textarea rows="3" cols="60">
action(type="omfile" file="/var/log/somelog"
sig.provider="gt" sig.hashfunction="SHA2-512"
sig.block.sizelimit="10000"
sig.keepTreeHashes="on" sig.keepRecordHashes="on")
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>]
[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a>
project.<br>
Copyright &copy; 2013 by
<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>.
Released under the GNU GPL version 3 or higher.</font></p>
</body></html>

47
doc/v7compatibility.html
View File

@ -82,6 +82,53 @@ errors and error messages. Starting with 7.2.1, this has been reduced to 10
successive failures. This still gives the plugin a chance to recover. In extreme
cases, a plugin may now enter suspend mode where it previously did not do so.
In practice, we do NOT expect that.
<h1>Notes for the 7.3/7.4 branch</h1>
<h2>"last message repeated n times" Processing</h2>
<p>This processing has been optimized and moved to the input side. This results
in usually far better performance and also de-couples different sources
from the same
processing. It is now also integrated in to the more generic rate-limiting
processing.
<h3>User-Noticable Changes</h3>
The code works almost as before, with two exceptions:
<ul>
<li>The supression amount can be different, as the new algorithm
precisely check's a single source, and while that source is being
read. The previous algorithm worked on a set of mixed messages
from multiple sources.
<li>The previous algorithm wrote a "last message repeated n times" message
at least every 60 seconds. For performance reasons, we do no longer do
this but write this message only when a new message arrives or rsyslog
is shut down.
</ul>
<p>Note that the new algorithms needs support from input modules. If old
modules which do not have the necessary support are used, duplicate
messages will most probably not be detected. Upgrading the module code is
simple, and all rsyslog-provided plugins support the new method, so this
should not be a real problem (crafting a solution would result in rather
complex code - for a case that most probably would never happen).
<h3>Performance Implications</h3>
<p>In general, the new method enables far faster output procesing. However, it
needs to be noted that the "last message repeated n" processing needs parsed
messages in order to detect duplicated. Consequently, if it is enabled the
parser step cannot be deferred to the main queue processing thread and
thus must be done during input processing. The changes workload distribution
and may have (good or bad) effect on the overall performance. If you have
a very high performance installation, it is suggested to check the performance
profile before deploying the new version. Note: for high-performance
environments it is highly recommended NOT to use "last message repeated n times"
processing but rather the other (more efficient) rate-limiting methods. These
also do NOT require the parsing step to be done during input processing.
<h2>Stricter string-template Processing</h2>
<p>Previously, no error message for invalid string template parameters
was generated.
Rather a malformed template was generated, and error information emitted
at runtime. However, this could be quite confusing. Note that the new code
changes user experience: formerly, rsyslog and the affected
actions properly started up, but the actions did not produce proper
data. Now, there are startup error messages and the actions are NOT
executed (due to missing template due to template error).
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>

7
grammar/grammar.y
View File

@ -148,7 +148,6 @@ value: STRING { $$ = nvlstNewStr($1); }
script: stmt { $$ = $1; }
| script stmt { $$ = scriptAddStmt($1, $2); }
stmt: actlst { $$ = $1; }
| STOP { $$ = cnfstmtNew(S_STOP); }
| IF expr THEN block { $$ = cnfstmtNew(S_IF);
$$->d.s_if.expr = $2;
$$->d.s_if.t_then = $4;
@ -161,14 +160,16 @@ stmt: actlst { $$ = $1; }
| UNSET VAR ';' { $$ = cnfstmtNewUnset($2); }
| PRIFILT block { $$ = cnfstmtNewPRIFILT($1, $2); }
| PROPFILT block { $$ = cnfstmtNewPROPFILT($1, $2); }
| CALL NAME { $$ = cnfstmtNewCall($2); }
| CONTINUE { $$ = cnfstmtNewContinue(); }
block: stmt { $$ = $1; }
| '{' script '}' { $$ = $2; }
actlst: s_act { $$ = $1; }
| actlst '&' s_act { $$ = scriptAddStmt($1, $3); }
/* s_act are actions and action-like statements */
s_act: BEGIN_ACTION nvlst ENDOBJ { $$ = cnfstmtNewAct($2); }
| LEGACY_ACTION { $$ = cnfstmtNewLegaAct($1); }
| STOP { $$ = cnfstmtNew(S_STOP); }
| CALL NAME { $$ = cnfstmtNewCall($2); }
| CONTINUE { $$ = cnfstmtNewContinue(); }
expr: expr AND expr { $$ = cnfexprNew(AND, $1, $3); }
| expr OR expr { $$ = cnfexprNew(OR, $1, $3); }
| NOT expr { $$ = cnfexprNew(NOT, NULL, $2); }

2
grammar/lexer.l
View File

@ -88,7 +88,9 @@ extern int yydebug;
/* somehow, I need these prototype even though the headers are
* included. I guess that's some autotools magic I don't understand...
*/
#if !defined(__FreeBSD__)
int fileno(FILE *stream);
#endif
%}

1
grammar/parserif.h
View File

@ -16,7 +16,6 @@ extern int yylineno;
*/
void cnfDoObj(struct cnfobj *o);
void cnfDoScript(struct cnfstmt *script);
void cnfDoRuleset(struct cnfstmt *script);
void cnfDoCfsysline(char *ln);
void cnfDoBSDTag(char *ln);
void cnfDoBSDHost(char *ln);

742
grammar/rainerscript.c
View File

@ -2,7 +2,7 @@
*
* Module begun 2011-07-01 by Rainer Gerhards
*
* Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH.
* Copyright 2011-2013 Rainer Gerhards and Adiscon GmbH.
*
* This file is part of the rsyslog runtime library.
*
@ -50,9 +50,69 @@
DEFobjCurrIf(obj)
DEFobjCurrIf(regexp)
void cnfexprOptimize(struct cnfexpr *expr);
struct cnfexpr* cnfexprOptimize(struct cnfexpr *expr);
static void cnfstmtOptimizePRIFilt(struct cnfstmt *stmt);
static void cnfarrayPrint(struct cnfarray *ar, int indent);
struct cnffunc * cnffuncNew_prifilt(int fac);
/* debug support: convert token to a human-readable string. Note that
* this function only supports a single thread due to a static buffer.
* This is deemed a solid solution, as it is intended to be used during
* startup, only.
* NOTE: This function MUST be updated if new tokens are defined in the
* grammar.
*/
char *
tokenToString(int token)
{
char *tokstr;
static char tokbuf[512];
switch(token) {
case NAME: tokstr = "NAME"; break;
case FUNC: tokstr = "FUNC"; break;
case BEGINOBJ: tokstr ="BEGINOBJ"; break;
case ENDOBJ: tokstr ="ENDOBJ"; break;
case BEGIN_ACTION: tokstr ="BEGIN_ACTION"; break;
case BEGIN_PROPERTY: tokstr ="BEGIN_PROPERTY"; break;
case BEGIN_CONSTANT: tokstr ="BEGIN_CONSTANT"; break;
case BEGIN_TPL: tokstr ="BEGIN_TPL"; break;
case BEGIN_RULESET: tokstr ="BEGIN_RULESET"; break;
case STOP: tokstr ="STOP"; break;
case SET: tokstr ="SET"; break;
case UNSET: tokstr ="UNSET"; break;
case CONTINUE: tokstr ="CONTINUE"; break;
case CALL: tokstr ="CALL"; break;
case LEGACY_ACTION: tokstr ="LEGACY_ACTION"; break;
case LEGACY_RULESET: tokstr ="LEGACY_RULESET"; break;
case PRIFILT: tokstr ="PRIFILT"; break;
case PROPFILT: tokstr ="PROPFILT"; break;
case IF: tokstr ="IF"; break;
case THEN: tokstr ="THEN"; break;
case ELSE: tokstr ="ELSE"; break;
case OR: tokstr ="OR"; break;
case AND: tokstr ="AND"; break;
case NOT: tokstr ="NOT"; break;
case VAR: tokstr ="VAR"; break;
case STRING: tokstr ="STRING"; break;
case NUMBER: tokstr ="NUMBER"; break;
case CMP_EQ: tokstr ="CMP_EQ"; break;
case CMP_NE: tokstr ="CMP_NE"; break;
case CMP_LE: tokstr ="CMP_LE"; break;
case CMP_GE: tokstr ="CMP_GE"; break;
case CMP_LT: tokstr ="CMP_LT"; break;
case CMP_GT: tokstr ="CMP_GT"; break;
case CMP_CONTAINS: tokstr ="CMP_CONTAINS"; break;
case CMP_CONTAINSI: tokstr ="CMP_CONTAINSI"; break;
case CMP_STARTSWITH: tokstr ="CMP_STARTSWITH"; break;
case CMP_STARTSWITHI: tokstr ="CMP_STARTSWITHI"; break;
case UMINUS: tokstr ="UMINUS"; break;
default: snprintf(tokbuf, sizeof(tokbuf), "%c[%d]", token, token);
tokstr = tokbuf; break;
}
return tokstr;
}
char*
getFIOPName(unsigned iFIOP)
@ -84,6 +144,97 @@ getFIOPName(unsigned iFIOP)
return pRet;
}
static void
prifiltInvert(struct funcData_prifilt *prifilt)
{
int i;
for(i = 0 ; i < LOG_NFACILITIES+1 ; ++i) {
prifilt->pmask[i] = ~prifilt->pmask[i];
}
}
/* set prifilt so that it matches for some severities, sev is its numerical
* value. Mode is one of the compop tokens CMP_EQ, CMP_LT, CMP_LE, CMP_GT,
* CMP_GE, CMP_NE.
*/
static void
prifiltSetSeverity(struct funcData_prifilt *prifilt, int sev, int mode)
{
static int lessthanmasks[] = { 0x00, 0x01, 0x03, 0x07, 0x0f, 0x1f, 0x3f, 0x7f, 0xff };
int i;
for(i = 0 ; i < LOG_NFACILITIES+1 ; ++i) {
if(mode == CMP_EQ || mode == CMP_NE)
prifilt->pmask[i] = 1 << sev;
else if(mode == CMP_LT)
prifilt->pmask[i] = lessthanmasks[sev];
else if(mode == CMP_LE)
prifilt->pmask[i] = lessthanmasks[sev+1];
else if(mode == CMP_GT)
prifilt->pmask[i] = ~lessthanmasks[sev+1];
else if(mode == CMP_GE)
prifilt->pmask[i] = ~lessthanmasks[sev];
else
DBGPRINTF("prifiltSetSeverity: program error, invalid mode %s\n",
tokenToString(mode));
}
if(mode == CMP_NE)
prifiltInvert(prifilt);
}
/* set prifilt so that it matches for some facilities, fac is its numerical
* value. Mode is one of the compop tokens CMP_EQ, CMP_LT, CMP_LE, CMP_GT,
* CMP_GE, CMP_NE. For the given facilities, all severities are enabled.
* NOTE: fac MUST be in the range 0..24 (not multiplied by 8)!
*/
static void
prifiltSetFacility(struct funcData_prifilt *prifilt, int fac, int mode)
{
int i;
memset(prifilt->pmask, 0, sizeof(prifilt->pmask));
switch(mode) {
case CMP_EQ:
prifilt->pmask[fac] = TABLE_ALLPRI;
break;
case CMP_NE:
prifilt->pmask[fac] = TABLE_ALLPRI;
prifiltInvert(prifilt);
break;
case CMP_LT:
for(i = 0 ; i < fac ; ++i)
prifilt->pmask[i] = TABLE_ALLPRI;
break;
case CMP_LE:
for(i = 0 ; i < fac+1 ; ++i)
prifilt->pmask[i] = TABLE_ALLPRI;
break;
case CMP_GE:
for(i = fac ; i < LOG_NFACILITIES+1 ; ++i)
prifilt->pmask[i] = TABLE_ALLPRI;
break;
case CMP_GT:
for(i = fac+1 ; i < LOG_NFACILITIES+1 ; ++i)
prifilt->pmask[i] = TABLE_ALLPRI;
break;
default:break;
}
}
/* combine a prifilt with AND/OR (the respective token values are
* used to keep things simple).
*/
static void
prifiltCombine(struct funcData_prifilt *prifilt, struct funcData_prifilt *prifilt2, int mode)
{
int i;
for(i = 0 ; i < LOG_NFACILITIES+1 ; ++i) {
if(mode == AND)
prifilt->pmask[i] = prifilt->pmask[i] & prifilt2->pmask[i];
else
prifilt->pmask[i] = prifilt->pmask[i] | prifilt2->pmask[i];
}
}
void
readConfFile(FILE *fp, es_str_t **str)
@ -136,6 +287,14 @@ readConfFile(FILE *fp, es_str_t **str)
es_addChar(str, '\0');
}
/* comparison function for qsort() and bsearch() string array compare */
static int
qs_arrcmp(const void *s1, const void *s2)
{
return es_strcmp(*((es_str_t**)s1), *((es_str_t**)s2));
}
struct objlst*
objlstNew(struct cnfobj *o)
{
@ -275,8 +434,8 @@ nvlstPrint(struct nvlst *lst)
dbgprintf("\tname: '%s', value '%s'\n", name, value);
free(value);
break;
default:dbgprintf("nvlstPrint: unknown type '%c' [%d]\n",
lst->val.datatype, lst->val.datatype);
default:dbgprintf("nvlstPrint: unknown type '%s'\n",
tokenToString(lst->val.datatype));
break;
}
free(name);
@ -695,7 +854,7 @@ nvlstGetParam(struct nvlst *valnode, struct cnfparamdescr *param,
r = doGetInt(valnode, param, val);
break;
case eCmdHdlrNonNegInt:
r = doGetPositiveInt(valnode, param, val);
r = doGetNonNegInt(valnode, param, val);
break;
case eCmdHdlrPositiveInt:
r = doGetPositiveInt(valnode, param, val);
@ -779,8 +938,14 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params,
for(i = 0 ; i < params->nParams ; ++i) {
param = params->descr + i;
if((valnode = nvlstFindNameCStr(lst, param->name)) == NULL)
if((valnode = nvlstFindNameCStr(lst, param->name)) == NULL) {
if(param->flags & CNFPARAM_REQUIRED) {
parser_errmsg("parameter '%s' required but not specified - "
"fix config", param->name);
bInError = 1;
}
continue;
}
if(vals[i].bUsed) {
parser_errmsg("parameter '%s' specified more than once - "
"one instance is ignored. Fix config", param->name);
@ -791,7 +956,6 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params,
}
}
if(bInError) {
if(bValsWasNULL)
cnfparamvalsDestruct(vals, params);
@ -978,8 +1142,8 @@ var2CString(struct var *r, int *bMustFree)
return cstr;
}
rsRetVal
doExtractField(uchar *str, uchar delim, int matchnbr, uchar **resstr)
static rsRetVal
doExtractFieldByChar(uchar *str, uchar delim, int matchnbr, uchar **resstr)
{
int iCurrFld;
int iLen;
@ -1016,8 +1180,6 @@ doExtractField(uchar *str, uchar delim, int matchnbr, uchar **resstr)
/* now copy */
memcpy(pBuf, pFld, iLen);
pBuf[iLen] = '\0'; /* terminate it */
if(*(pFldEnd+1) != '\0')
++pFldEnd; /* OK, skip again over delimiter char */
*resstr = pBuf;
} else {
ABORT_FINALIZE(RS_RET_FIELD_NOT_FOUND);
@ -1026,6 +1188,141 @@ finalize_it:
RETiRet;
}
static rsRetVal
doExtractFieldByStr(uchar *str, char *delim, rs_size_t lenDelim, int matchnbr, uchar **resstr)
{
int iCurrFld;
int iLen;
uchar *pBuf;
uchar *pFld;
uchar *pFldEnd;
DEFiRet;
/* first, skip to the field in question */
iCurrFld = 1;
pFld = str;
while(pFld != NULL && iCurrFld < matchnbr) {
if((pFld = (uchar*) strstr((char*)pFld, delim)) != NULL) {
pFld += lenDelim;
++iCurrFld;
}
}
dbgprintf("field() field requested %d, field found %d\n", matchnbr, iCurrFld);
if(iCurrFld == matchnbr) {
/* field found, now extract it */
/* first of all, we need to find the end */
pFldEnd = (uchar*) strstr((char*)pFld, delim);
if(pFldEnd == NULL) {
iLen = strlen((char*) pFld);
} else { /* found delmiter! Note that pFldEnd *is* already on
* the first delmi char, we don't need that. */
iLen = pFldEnd - pFld;
}
/* we got our end pointer, now do the copy */
CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(char)));
/* now copy */
memcpy(pBuf, pFld, iLen);
pBuf[iLen] = '\0'; /* terminate it */
*resstr = pBuf;
} else {
ABORT_FINALIZE(RS_RET_FIELD_NOT_FOUND);
}
finalize_it:
RETiRet;
}
static inline void
doFunc_re_extract(struct cnffunc *func, struct var *ret, void* usrptr)
{
size_t submatchnbr;
short matchnbr;
regmatch_t pmatch[50];
int bMustFree;
es_str_t *estr;
char *str;
struct var r[CNFFUNC_MAX_ARGS];
int iLenBuf;
unsigned iOffs;
short iTry = 0;
uchar bFound = 0;
iOffs = 0;
sbool bHadNoMatch = 0;
cnfexprEval(func->expr[0], &r[0], usrptr);
/* search string is already part of the compiled regex, so we don't
* need it here!
*/
cnfexprEval(func->expr[2], &r[2], usrptr);
cnfexprEval(func->expr[3], &r[3], usrptr);
str = (char*) var2CString(&r[0], &bMustFree);
matchnbr = (short) var2Number(&r[2], NULL);
submatchnbr = (size_t) var2Number(&r[3], NULL);
if(submatchnbr > sizeof(pmatch)/sizeof(regmatch_t)) {
DBGPRINTF("re_extract() submatch %d is too large\n", submatchnbr);
bHadNoMatch = 1;
goto finalize_it;
}
/* first see if we find a match, iterating through the series of
* potential matches over the string.
*/
while(!bFound) {
int iREstat;
iREstat = regexp.regexec(func->funcdata, (char*)(str + iOffs),
submatchnbr+1, pmatch, 0);
dbgprintf("re_extract: regexec return is %d\n", iREstat);
if(iREstat == 0) {
if(pmatch[0].rm_so == -1) {
dbgprintf("oops ... start offset of successful regexec is -1\n");
break;
}
if(iTry == matchnbr) {
bFound = 1;
} else {
dbgprintf("re_extract: regex found at offset %d, new offset %d, tries %d\n",
iOffs, (int) (iOffs + pmatch[0].rm_eo), iTry);
iOffs += pmatch[0].rm_eo;
++iTry;
}
} else {
break;
}
}
dbgprintf("re_extract: regex: end search, found %d\n", bFound);
if(!bFound) {
bHadNoMatch = 1;
goto finalize_it;
} else {
/* Match- but did it match the one we wanted? */
/* we got no match! */
if(pmatch[submatchnbr].rm_so == -1) {
bHadNoMatch = 1;
goto finalize_it;
}
/* OK, we have a usable match - we now need to malloc pB */
iLenBuf = pmatch[submatchnbr].rm_eo - pmatch[submatchnbr].rm_so;
estr = es_newStrFromBuf(str + iOffs + pmatch[submatchnbr].rm_so,
iLenBuf);
}
if(bMustFree) free(str);
if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr);
if(r[2].datatype == 'S') es_deleteStr(r[2].d.estr);
if(r[3].datatype == 'S') es_deleteStr(r[3].d.estr);
finalize_it:
if(bHadNoMatch) {
cnfexprEval(func->expr[4], &r[4], usrptr);
estr = var2String(&r[4], &bMustFree);
if(r[4].datatype == 'S') es_deleteStr(r[4].d.estr);
}
ret->datatype = 'S';
ret->d.estr = estr;
return;
}
/* Perform a function call. This has been moved out of cnfExprEval in order
* to keep the code small and easier to maintain.
*/
@ -1129,18 +1426,29 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr)
if(bMustFree) free(str);
if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr);
break;
case CNFFUNC_RE_EXTRACT:
doFunc_re_extract(func, ret, usrptr);
break;
case CNFFUNC_FIELD:
cnfexprEval(func->expr[0], &r[0], usrptr);
cnfexprEval(func->expr[1], &r[1], usrptr);
cnfexprEval(func->expr[2], &r[2], usrptr);
str = (char*) var2CString(&r[0], &bMustFree);
delim = var2Number(&r[1], NULL);
matchnbr = var2Number(&r[2], NULL);
localRet = doExtractField((uchar*)str, (char) delim, matchnbr, &resStr);
if(r[1].datatype == 'S') {
char *delimstr;
delimstr = (char*) es_str2cstr(r[1].d.estr, NULL);
localRet = doExtractFieldByStr((uchar*)str, delimstr, es_strlen(r[1].d.estr),
matchnbr, &resStr);
free(delimstr);
} else {
delim = var2Number(&r[1], NULL);
localRet = doExtractFieldByChar((uchar*)str, (char) delim, matchnbr, &resStr);
}
if(localRet == RS_RET_OK) {
ret->d.estr = es_newStrFromCStr((char*)resStr, strlen((char*)resStr));
free(resStr);
} else if(localRet == RS_RET_OK) {
} else if(localRet == RS_RET_FIELD_NOT_FOUND) {
ret->d.estr = es_newStrFromCStr("***FIELD NOT FOUND***",
sizeof("***FIELD NOT FOUND***")-1);
} else {
@ -1208,26 +1516,29 @@ evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop)
{
int i;
int r = 0;
for(i = 0 ; (r == 0) && (i < ar->nmemb) ; ++i) {
switch(cmpop) {
case CMP_EQ:
r = es_strcmp(estr_l, ar->arr[i]) == 0;
break;
case CMP_NE:
r = es_strcmp(estr_l, ar->arr[i]) != 0;
break;
case CMP_STARTSWITH:
r = es_strncmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0;
break;
case CMP_STARTSWITHI:
r = es_strncasecmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0;
break;
case CMP_CONTAINS:
r = es_strContains(estr_l, ar->arr[i]) != -1;
break;
case CMP_CONTAINSI:
r = es_strCaseContains(estr_l, ar->arr[i]) != -1;
break;
es_str_t **res;
if(cmpop == CMP_EQ) {
res = bsearch(&estr_l, ar->arr, ar->nmemb, sizeof(es_str_t*), qs_arrcmp);
r = res != NULL;
} else if(cmpop == CMP_NE) {
res = bsearch(&estr_l, ar->arr, ar->nmemb, sizeof(es_str_t*), qs_arrcmp);
r = res == NULL;
} else {
for(i = 0 ; (r == 0) && (i < ar->nmemb) ; ++i) {
switch(cmpop) {
case CMP_STARTSWITH:
r = es_strncmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0;
break;
case CMP_STARTSWITHI:
r = es_strncasecmp(estr_l, ar->arr[i], es_strlen(ar->arr[i])) == 0;
break;
case CMP_CONTAINS:
r = es_strContains(estr_l, ar->arr[i]) != -1;
break;
case CMP_CONTAINSI:
r = es_strCaseContains(estr_l, ar->arr[i]) != -1;
break;
}
}
}
return r;
@ -1285,9 +1596,7 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr)
int bMustFree, bMustFree2;
long long n_r, n_l;
DBGPRINTF("eval expr %p, type '%c'[%s](%u)\n", expr, expr->nodetype,
tokenval2str(expr->nodetype), expr->nodetype);
dbgprintf("eval expr %p, type '%s'\n", expr, tokenToString(expr->nodetype));
switch(expr->nodetype) {
/* note: comparison operations are extremely similar. The code can be copyied, only
* places flagged with "CMP" need to be changed.
@ -1697,6 +2006,7 @@ cnffuncDestruct(struct cnffunc *func)
/* some functions require special destruction */
switch(func->fID) {
case CNFFUNC_RE_MATCH:
case CNFFUNC_RE_EXTRACT:
if(func->funcdata != NULL)
regexp.regfree(func->funcdata);
break;
@ -1712,7 +2022,13 @@ void
cnfexprDestruct(struct cnfexpr *expr)
{
dbgprintf("cnfexprDestruct expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype);
if(expr == NULL) {
/* this is valid and can happen during optimizer run! */
DBGPRINTF("cnfexprDestruct got NULL ptr - valid, so doing nothing\n");
return;
}
DBGPRINTF("cnfexprDestruct expr %p, type '%s'\n", expr, tokenToString(expr->nodetype));
switch(expr->nodetype) {
case CMP_NE:
case CMP_EQ:
@ -1962,7 +2278,8 @@ cnfstmtPrintOnly(struct cnfstmt *stmt, int indent, sbool subtree)
free(cstr);
break;
case S_ACT:
doIndent(indent); dbgprintf("ACTION %p [%s]\n", stmt->d.act, stmt->printable);
doIndent(indent); dbgprintf("ACTION %p [%s:%s]\n", stmt->d.act,
modGetName(stmt->d.act->pMod), stmt->printable);
break;
case S_IF:
doIndent(indent); dbgprintf("IF\n");
@ -2130,59 +2447,69 @@ cnfstmtNew(unsigned s_type)
return cnfstmt;
}
void cnfstmtDestructLst(struct cnfstmt *root);
/* delete a single stmt */
static void
cnfstmtDestruct(struct cnfstmt *stmt)
{
switch(stmt->nodetype) {
case S_NOP:
case S_STOP:
break;
case S_CALL:
es_deleteStr(stmt->d.s_call.name);
break;
case S_ACT:
actionDestruct(stmt->d.act);
break;
case S_IF:
cnfexprDestruct(stmt->d.s_if.expr);
if(stmt->d.s_if.t_then != NULL) {
cnfstmtDestructLst(stmt->d.s_if.t_then);
}
if(stmt->d.s_if.t_else != NULL) {
cnfstmtDestructLst(stmt->d.s_if.t_else);
}
break;
case S_SET:
free(stmt->d.s_set.varname);
cnfexprDestruct(stmt->d.s_set.expr);
break;
case S_UNSET:
free(stmt->d.s_set.varname);
break;
case S_PRIFILT:
cnfstmtDestructLst(stmt->d.s_prifilt.t_then);
cnfstmtDestructLst(stmt->d.s_prifilt.t_else);
break;
case S_PROPFILT:
if(stmt->d.s_propfilt.propName != NULL)
es_deleteStr(stmt->d.s_propfilt.propName);
if(stmt->d.s_propfilt.regex_cache != NULL)
rsCStrRegexDestruct(&stmt->d.s_propfilt.regex_cache);
if(stmt->d.s_propfilt.pCSCompValue != NULL)
cstrDestruct(&stmt->d.s_propfilt.pCSCompValue);
cnfstmtDestructLst(stmt->d.s_propfilt.t_then);
break;
default:
dbgprintf("error: unknown stmt type during destruct %u\n",
(unsigned) stmt->nodetype);
break;
}
free(stmt->printable);
free(stmt);
}
/* delete a stmt and all others following it */
void
cnfstmtDestruct(struct cnfstmt *root)
cnfstmtDestructLst(struct cnfstmt *root)
{
struct cnfstmt *stmt, *todel;
for(stmt = root ; stmt != NULL ; ) {
switch(stmt->nodetype) {
case S_NOP:
case S_STOP:
break;
case S_CALL:
es_deleteStr(stmt->d.s_call.name);
break;
case S_ACT:
actionDestruct(stmt->d.act);
break;
case S_IF:
cnfexprDestruct(stmt->d.s_if.expr);
if(stmt->d.s_if.t_then != NULL) {
cnfstmtDestruct(stmt->d.s_if.t_then);
}
if(stmt->d.s_if.t_else != NULL) {
cnfstmtDestruct(stmt->d.s_if.t_else);
}
break;
case S_SET:
free(stmt->d.s_set.varname);
cnfexprDestruct(stmt->d.s_set.expr);
break;
case S_UNSET:
free(stmt->d.s_set.varname);
break;
case S_PRIFILT:
cnfstmtDestruct(stmt->d.s_prifilt.t_then);
cnfstmtDestruct(stmt->d.s_prifilt.t_else);
break;
case S_PROPFILT:
if(stmt->d.s_propfilt.propName != NULL)
es_deleteStr(stmt->d.s_propfilt.propName);
if(stmt->d.s_propfilt.regex_cache != NULL)
rsCStrRegexDestruct(&stmt->d.s_propfilt.regex_cache);
if(stmt->d.s_propfilt.pCSCompValue != NULL)
cstrDestruct(&stmt->d.s_propfilt.pCSCompValue);
cnfstmtDestruct(stmt->d.s_propfilt.t_then);
break;
default:
dbgprintf("error: unknown stmt type during destruct %u\n",
(unsigned) stmt->nodetype);
break;
}
free(stmt->printable);
todel = stmt;
stmt = stmt->next;
free(todel);
cnfstmtDestruct(todel);
}
}
@ -2395,14 +2722,155 @@ constFoldConcat(struct cnfexpr *expr)
}
/* optimize comparisons with syslog severity/facility. This is a special
* handler as the numerical values also support GT, LT, etc ops.
*/
static inline struct cnfexpr*
cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr)
{
struct cnffunc *func;
if(!strcmp("$syslogseverity", ((struct cnfvar*)expr->l)->name)) {
if(expr->r->nodetype == 'N') {
int sev = (int) ((struct cnfnumval*)expr->r)->val;
if(sev >= 0 && sev <= 7) {
DBGPRINTF("optimizer: change comparison OP to FUNC prifilt()\n");
func = cnffuncNew_prifilt(0); /* fac is irrelevant, set below... */
prifiltSetSeverity(func->funcdata, sev, expr->nodetype);
cnfexprDestruct(expr);
expr = (struct cnfexpr*) func;
} else {
parser_errmsg("invalid syslogseverity %d, expression will always "
"evaluate to FALSE", sev);
}
}
} else if(!strcmp("$syslogfacility", ((struct cnfvar*)expr->l)->name)) {
if(expr->r->nodetype == 'N') {
int fac = (int) ((struct cnfnumval*)expr->r)->val;
if(fac >= 0 && fac <= 24) {
DBGPRINTF("optimizer: change comparison OP to FUNC prifilt()\n");
func = cnffuncNew_prifilt(0); /* fac is irrelevant, set below... */
prifiltSetFacility(func->funcdata, fac, expr->nodetype);
cnfexprDestruct(expr);
expr = (struct cnfexpr*) func;
} else {
parser_errmsg("invalid syslogfacility %d, expression will always "
"evaluate to FALSE", fac);
}
}
}
return expr;
}
/* optimize a comparison with a variable as left-hand operand
* NOTE: Currently support CMP_EQ, CMP_NE only and code NEEDS
* TO BE CHANGED fgr other comparisons!
*/
static inline struct cnfexpr*
cnfexprOptimize_CMP_var(struct cnfexpr *expr)
{
struct cnffunc *func;
if(!strcmp("$syslogfacility-text", ((struct cnfvar*)expr->l)->name)) {
if(expr->r->nodetype == 'S') {
char *cstr = es_str2cstr(((struct cnfstringval*)expr->r)->estr, NULL);
int fac = decodeSyslogName((uchar*)cstr, syslogFacNames);
if(fac == -1) {
parser_errmsg("invalid facility '%s', expression will always "
"evaluate to FALSE", cstr);
} else {
/* we can acutally optimize! */
DBGPRINTF("optimizer: change comparison OP to FUNC prifilt()\n");
func = cnffuncNew_prifilt(fac);
if(expr->nodetype == CMP_NE)
prifiltInvert(func->funcdata);
cnfexprDestruct(expr);
expr = (struct cnfexpr*) func;
}
free(cstr);
}
} else if(!strcmp("$syslogseverity-text", ((struct cnfvar*)expr->l)->name)) {
if(expr->r->nodetype == 'S') {
char *cstr = es_str2cstr(((struct cnfstringval*)expr->r)->estr, NULL);
int sev = decodeSyslogName((uchar*)cstr, syslogPriNames);
if(sev == -1) {
parser_errmsg("invalid syslogseverity '%s', expression will always "
"evaluate to FALSE", cstr);
} else {
/* we can acutally optimize! */
DBGPRINTF("optimizer: change comparison OP to FUNC prifilt()\n");
func = cnffuncNew_prifilt(0);
prifiltSetSeverity(func->funcdata, sev, expr->nodetype);
cnfexprDestruct(expr);
expr = (struct cnfexpr*) func;
}
free(cstr);
}
} else {
expr = cnfexprOptimize_CMP_severity_facility(expr);
}
return expr;
}
static inline struct cnfexpr*
cnfexprOptimize_NOT(struct cnfexpr *expr)
{
struct cnffunc *func;
if(expr->r->nodetype == 'F') {
func = (struct cnffunc *)expr->r;
if(func->fID == CNFFUNC_PRIFILT) {
DBGPRINTF("optimize NOT prifilt() to inverted prifilt()\n");
expr->r = NULL;
cnfexprDestruct(expr);
prifiltInvert(func->funcdata);
expr = (struct cnfexpr*) func;
}
}
return expr;
}
static inline struct cnfexpr*
cnfexprOptimize_AND_OR(struct cnfexpr *expr)
{
struct cnffunc *funcl, *funcr;
if(expr->l->nodetype == 'F') {
if(expr->r->nodetype == 'F') {
funcl = (struct cnffunc *)expr->l;
funcr = (struct cnffunc *)expr->r;
if(funcl->fID == CNFFUNC_PRIFILT && funcr->fID == CNFFUNC_PRIFILT) {
DBGPRINTF("optimize combine AND/OR prifilt()\n");
expr->l = NULL;
prifiltCombine(funcl->funcdata, funcr->funcdata, expr->nodetype);
cnfexprDestruct(expr);
expr = (struct cnfexpr*) funcl;
}
}
}
return expr;
}
/* optimize array for EQ/NEQ comparisons. We sort the array in
* this case so that we can apply binary search later on.
*/
static inline void
cnfexprOptimize_CMPEQ_arr(struct cnfarray *arr)
{
DBGPRINTF("optimizer: sorting array for CMP_EQ/NEQ comparison\n");
qsort(arr->arr, arr->nmemb, sizeof(es_str_t*), qs_arrcmp);
}
/* (recursively) optimize an expression */
void
struct cnfexpr*
cnfexprOptimize(struct cnfexpr *expr)
{
long long ln, rn;
struct cnfexpr *exprswap;
dbgprintf("optimize expr %p, type '%c'(%u)\n", expr, expr->nodetype, expr->nodetype);
dbgprintf("optimize expr %p, type '%s'\n", expr, tokenToString(expr->nodetype));
switch(expr->nodetype) {
case '&':
constFoldConcat(expr);
@ -2439,6 +2907,8 @@ cnfexprOptimize(struct cnfexpr *expr)
break;
case CMP_NE:
case CMP_EQ:
expr->l = cnfexprOptimize(expr->l);
expr->r = cnfexprOptimize(expr->r);
if(expr->l->nodetype == 'A') {
if(expr->r->nodetype == 'A') {
parser_errmsg("warning: '==' or '<>' "
@ -2450,10 +2920,41 @@ cnfexprOptimize(struct cnfexpr *expr)
expr->r = exprswap;
}
}
default:/* nodetype we cannot optimize */
if(expr->l->nodetype == 'V') {
expr = cnfexprOptimize_CMP_var(expr);
} else if(expr->r->nodetype == 'A') {
cnfexprOptimize_CMPEQ_arr((struct cnfarray *)expr->r);
}
break;
case CMP_LE:
case CMP_GE:
case CMP_LT:
case CMP_GT:
expr->l = cnfexprOptimize(expr->l);
expr->r = cnfexprOptimize(expr->r);
expr = cnfexprOptimize_CMP_severity_facility(expr);
break;
case CMP_CONTAINS:
case CMP_CONTAINSI:
case CMP_STARTSWITH:
case CMP_STARTSWITHI:
expr->l = cnfexprOptimize(expr->l);
expr->r = cnfexprOptimize(expr->r);
break;
case AND:
case OR:
expr->l = cnfexprOptimize(expr->l);
expr->r = cnfexprOptimize(expr->r);
expr = cnfexprOptimize_AND_OR(expr);
break;
case NOT:
expr->r = cnfexprOptimize(expr->r);
expr = cnfexprOptimize_NOT(expr);
break;
default:/* nodetypes we cannot optimize */
break;
}
return expr;
}
/* removes NOPs from a statement list and returns the
@ -2496,8 +2997,7 @@ cnfstmtOptimizeIf(struct cnfstmt *stmt)
struct cnffunc *func;
struct funcData_prifilt *prifilt;
expr = stmt->d.s_if.expr;
cnfexprOptimize(expr);
expr = stmt->d.s_if.expr = cnfexprOptimize(stmt->d.s_if.expr);
stmt->d.s_if.t_then = removeNOPs(stmt->d.s_if.t_then);
stmt->d.s_if.t_else = removeNOPs(stmt->d.s_if.t_else);
cnfstmtOptimize(stmt->d.s_if.t_then);
@ -2515,8 +3015,11 @@ cnfstmtOptimizeIf(struct cnfstmt *stmt)
sizeof(prifilt->pmask));
stmt->d.s_prifilt.t_then = t_then;
stmt->d.s_prifilt.t_else = t_else;
stmt->printable = (uchar*)
es_str2cstr(((struct cnfstringval*)func->expr[0])->estr, NULL);
if(func->nParams == 0)
stmt->printable = (uchar*)strdup("[Optimizer Result]");
else
stmt->printable = (uchar*)
es_str2cstr(((struct cnfstringval*)func->expr[0])->estr, NULL);
cnfexprDestruct(expr);
cnfstmtOptimizePRIFilt(stmt);
}
@ -2557,7 +3060,7 @@ cnfstmtOptimizePRIFilt(struct cnfstmt *stmt)
DBGPRINTF("optimizer: removing always-true PRIFILT %p\n", stmt);
if(stmt->d.s_prifilt.t_else != NULL) {
parser_errmsg("error: always-true PRI filter has else part!\n");
cnfstmtDestruct(stmt->d.s_prifilt.t_else);
cnfstmtDestructLst(stmt->d.s_prifilt.t_else);
}
free(stmt->printable);
stmt->printable = NULL;
@ -2610,7 +3113,6 @@ cnfstmtOptimize(struct cnfstmt *root)
struct cnfstmt *stmt;
if(root == NULL) goto done;
for(stmt = root ; stmt != NULL ; stmt = stmt->next) {
dbgprintf("RRRR: stmtOptimize: stmt %p, nodetype %u\n", stmt, stmt->nodetype);
switch(stmt->nodetype) {
case S_IF:
cnfstmtOptimizeIf(stmt);
@ -2623,7 +3125,7 @@ dbgprintf("RRRR: stmtOptimize: stmt %p, nodetype %u\n", stmt, stmt->nodetype);
cnfstmtOptimize(stmt->d.s_propfilt.t_then);
break;
case S_SET:
cnfexprOptimize(stmt->d.s_set.expr);
stmt->d.s_set.expr = cnfexprOptimize(stmt->d.s_set.expr);
break;
case S_ACT:
cnfstmtOptimizeAct(stmt);
@ -2710,6 +3212,13 @@ funcName2ID(es_str_t *fname, unsigned short nParams)
return CNFFUNC_INVALID;
}
return CNFFUNC_RE_MATCH;
} else if(!es_strbufcmp(fname, (unsigned char*)"re_extract", sizeof("re_extract") - 1)) {
if(nParams != 5) {
parser_errmsg("number of parameters for re_extract() must be five "
"but is %d.", nParams);
return CNFFUNC_INVALID;
}
return CNFFUNC_RE_EXTRACT;
} else if(!es_strbufcmp(fname, (unsigned char*)"field", sizeof("field") - 1)) {
if(nParams != 3) {
parser_errmsg("number of parameters for field() must be three "
@ -2740,7 +3249,7 @@ initFunc_re_match(struct cnffunc *func)
func->funcdata = NULL;
if(func->expr[1]->nodetype != 'S') {
parser_errmsg("param 2 of re_match() must be a constant string");
parser_errmsg("param 2 of re_match/extract() must be a constant string");
FINALIZE;
}
@ -2787,6 +3296,7 @@ finalize_it:
RETiRet;
}
struct cnffunc *
cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst)
{
@ -2817,6 +3327,7 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst)
/* some functions require special initialization */
switch(func->fID) {
case CNFFUNC_RE_MATCH:
case CNFFUNC_RE_EXTRACT:
/* need to compile the regexp in param 2, so this MUST be a constant */
initFunc_re_match(func);
break;
@ -2829,6 +3340,27 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst)
return func;
}
/* A special function to create a prifilt() expression during optimization
* phase.
*/
struct cnffunc *
cnffuncNew_prifilt(int fac)
{
struct cnffunc* func;
if((func = malloc(sizeof(struct cnffunc))) != NULL) {
func->nodetype = 'F';
func->fname = es_newStrFromCStr("prifilt", sizeof("prifilt")-1);
func->nParams = 0;
func->fID = CNFFUNC_PRIFILT;
func->funcdata = calloc(1, sizeof(struct funcData_prifilt));
((struct funcData_prifilt *)func->funcdata)->pmask[fac >> 3] = TABLE_ALLPRI;
}
return func;
}
/* returns 0 if everything is OK and config parsing shall continue,
* and 1 if things are so wrong that config parsing shall be aborted.
*/
@ -2856,12 +3388,16 @@ cnfDoInclude(char *name)
/* Use GLOB_MARK to append a trailing slash for directories. */
/* Use GLOB_NOMAGIC to detect wildcards that match nothing. */
result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles);
#ifdef HAVE_GLOB_NOMAGIC
/* Silently ignore wildcards that match nothing */
result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles);
if(result == GLOB_NOMATCH) {
return 0;
}
#else
result = glob(finalName, GLOB_MARK, NULL, &cfgFiles);
if(result == GLOB_NOMATCH && containsGlobWildcard(finalName)) {
#endif /* HAVE_GLOB_NOMAGIC */
return 0;
}
if(result == GLOB_NOSPACE || result == GLOB_ABORTED) {
char errStr[1024];
@ -2924,6 +3460,8 @@ void
cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk)
{
int i;
if(paramvals == NULL)
return;
for(i = 0 ; i < blk->nParams ; ++i) {
if(paramvals[i].bUsed) {
varDelete(&paramvals[i].val);

3
grammar/rainerscript.h
View File

@ -226,6 +226,7 @@ enum cnffuncid {
CNFFUNC_CSTR,
CNFFUNC_CNUM,
CNFFUNC_RE_MATCH,
CNFFUNC_RE_EXTRACT,
CNFFUNC_FIELD,
CNFFUNC_PRIFILT
};
@ -330,7 +331,7 @@ struct cnfstmt * cnfstmtNewSet(char *var, struct cnfexpr *expr);
struct cnfstmt * cnfstmtNewUnset(char *var);
struct cnfstmt * cnfstmtNewCall(es_str_t *name);
struct cnfstmt * cnfstmtNewContinue(void);
void cnfstmtDestruct(struct cnfstmt *root);
void cnfstmtDestructLst(struct cnfstmt *root);
void cnfstmtOptimize(struct cnfstmt *root);
struct cnfarray* cnfarrayNew(es_str_t *val);
struct cnfarray* cnfarrayDup(struct cnfarray *old);

18
platform/redhat/rsyslog.conf
View File

@ -1,11 +1,13 @@
# rsyslog configuration file (for Red Hat-based systems)
# note that most of this config file uses old-style format,
# because it is well-known AND quite suitable for simple cases
# like we have with the default config. For more advanced
# things, RainerScript configuration is suggested.
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
/* rsyslog configuration file (for Red Hat-based systems)
* note that most of this config file uses old-style format,
* because it is well-known AND quite suitable for simple cases
* like we have with the default config. For more advanced
* things, RainerScript configuration is suggested.
*
* For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
* or latest version online at http://www.rsyslog.com/doc/rsyslog_conf.html
* If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
*/
#### MODULES ####

10
plugins/imdiag/imdiag.c
View File

@ -53,6 +53,7 @@
#include "srUtils.h"
#include "msg.h"
#include "datetime.h"
#include "ratelimit.h"
#include "net.h" /* for permittedPeers, may be removed when this is removed */
MODULE_TYPE_INPUT
@ -199,7 +200,7 @@ finalize_it:
/* actually submit a message to the rsyslog core
*/
static rsRetVal
doInjectMsg(int iNum)
doInjectMsg(int iNum, ratelimit_t *ratelimiter)
{
uchar szMsg[1024];
msg_t *pMsg;
@ -219,7 +220,7 @@ doInjectMsg(int iNum)
pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME;
MsgSetRcvFrom(pMsg, pRcvDummy);
CHKiRet(MsgSetRcvFromIP(pMsg, pRcvIPDummy));
CHKiRet(submitMsg(pMsg));
CHKiRet(ratelimitAddMsg(ratelimiter, NULL, pMsg));
finalize_it:
RETiRet;
@ -237,6 +238,7 @@ injectMsg(uchar *pszCmd, tcps_sess_t *pSess)
int iFrom;
int nMsgs;
int i;
ratelimit_t *ratelimit;
DEFiRet;
/* we do not check errors here! */
@ -244,13 +246,15 @@ injectMsg(uchar *pszCmd, tcps_sess_t *pSess)
iFrom = atoi((char*)wordBuf);
getFirstWord(&pszCmd, wordBuf, sizeof(wordBuf)/sizeof(uchar), TO_LOWERCASE);
nMsgs = atoi((char*)wordBuf);
ratelimitNew(&ratelimit, "imdiag", "injectmsg");
for(i = 0 ; i < nMsgs ; ++i) {
doInjectMsg(i + iFrom);
doInjectMsg(i + iFrom, ratelimit);
}
CHKiRet(sendResponse(pSess, "%d messages injected\n", nMsgs));
DBGPRINTF("imdiag: %d messages injected\n", nMsgs);
ratelimitDestruct(ratelimit);
finalize_it:
RETiRet;

31
plugins/imfile/imfile.c
View File

@ -48,6 +48,7 @@
#include "prop.h"
#include "stringbuf.h"
#include "ruleset.h"
#include "ratelimit.h"
MODULE_TYPE_INPUT /* must be present for input modules, do not remove */
MODULE_TYPE_NOKEEP
@ -82,6 +83,7 @@ typedef struct fileInfo_s {
strm_t *pStrm; /* its stream (NULL if not assigned) */
int readMode; /* which mode to use in ReadMulteLine call? */
ruleset_t *pRuleset; /* ruleset to bind listener to (use system default if unspecified) */
ratelimit_t *ratelimiter;
multi_submit_t multiSub;
} fileInfo_t;
@ -189,9 +191,7 @@ static rsRetVal enqLine(fileInfo_t *pInfo, cstr_t *cstrLine)
pMsg->iFacility = LOG_FAC(pInfo->iFacility);
pMsg->iSeverity = LOG_PRI(pInfo->iSeverity);
MsgSetRuleset(pMsg, pInfo->pRuleset);
pInfo->multiSub.ppMsgs[pInfo->multiSub.nElem++] = pMsg;
if(pInfo->multiSub.nElem == pInfo->multiSub.maxElem)
CHKiRet(multiSubmitMsg(&pInfo->multiSub));
ratelimitAddMsg(pInfo->ratelimiter, &pInfo->multiSub, pMsg);
finalize_it:
RETiRet;
}
@ -235,6 +235,7 @@ openFile(fileInfo_t *pThis)
/* read back in the object */
CHKiRet(obj.Deserialize(&pThis->pStrm, (uchar*) "strm", psSF, NULL, pThis));
strm.CheckFileChange(pThis->pStrm);
CHKiRet(strm.SeekCurrOffs(pThis->pStrm));
/* note: we do not delete the state file, so that the last position remains
@ -246,6 +247,8 @@ finalize_it:
strm.Destruct(&psSF);
if(iRet != RS_RET_OK) {
if(pThis->pStrm != NULL)
strm.Destruct(&pThis->pStrm);
CHKiRet(strm.Construct(&pThis->pStrm));
CHKiRet(strm.SettOperationsMode(pThis->pStrm, STREAMMODE_READ));
CHKiRet(strm.SetsType(pThis->pStrm, STREAMTYPE_FILE_MONITOR));
@ -304,10 +307,7 @@ static rsRetVal pollFile(fileInfo_t *pThis, int *pbHadFileData)
}
finalize_it:
if(pThis->multiSub.nElem > 0) {
/* submit everything that was not yet submitted */
CHKiRet(multiSubmitMsg(&pThis->multiSub));
}
multiSubmitFlush(&pThis->multiSub);
pthread_cleanup_pop(0);
if(pCStr != NULL) {
@ -415,6 +415,7 @@ addListner(instanceConf_t *inst)
pThis->lenTag = ustrlen(pThis->pszTag);
pThis->pszStateFile = (uchar*) strdup((char*) inst->pszStateFile);
CHKiRet(ratelimitNew(&pThis->ratelimiter, "imfile", (char*)inst->pszFileName));
CHKmalloc(pThis->multiSub.ppMsgs = MALLOC(inst->nMultiSub * sizeof(msg_t*)));
pThis->multiSub.maxElem = inst->nMultiSub;
pThis->multiSub.nElem = 0;
@ -448,8 +449,6 @@ CODESTARTnewInpInst
pvals = nvlstGetParams(lst, &inppblk, NULL);
if(pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS,
"imfile: required parameter are missing\n");
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
@ -479,7 +478,7 @@ CODESTARTnewInpInst
inst->readMode = pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "maxlinesatonce")) {
inst->maxLinesAtOnce = pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "persistStateInterval")) {
} else if(!strcmp(inppblk.descr[i].name, "persiststateinterval")) {
inst->iPersistStateInterval = pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "maxsubmitatonce")) {
inst->nMultiSub = pvals[i].val.d.n;
@ -595,7 +594,7 @@ CODESTARTactivateCnf
for(inst = runModConf->root ; inst != NULL ; inst = inst->next) {
addListner(inst);
}
/* if we could not set up any listners, there is no point in running... */
/* if we could not set up any listeners, there is no point in running... */
if(iFilPtr == 0) {
errmsg.LogError(0, NO_ERRCODE, "imfile: no file monitors could be started, "
"input not activated.\n");
@ -738,12 +737,20 @@ persistStrmState(fileInfo_t *pInfo)
CHKiRet(strm.ConstructFinalize(psSF));
CHKiRet(strm.Serialize(pInfo->pStrm, psSF));
CHKiRet(strm.Flush(psSF));
CHKiRet(strm.Destruct(&psSF));
finalize_it:
if(psSF != NULL)
strm.Destruct(&psSF);
if(iRet != RS_RET_OK) {
errmsg.LogError(0, iRet, "imfile: could not persist state "
"file %s - data may be repeated on next "
"startup. Is WorkDirectory set?",
pInfo->pszStateFile);
}
RETiRet;
}
@ -765,6 +772,8 @@ CODESTARTafterRun
persistStrmState(&files[i]);
strm.Destruct(&(files[i].pStrm));
}
ratelimitDestruct(files[i].ratelimiter);
free(files[i].multiSub.ppMsgs);
free(files[i].pszFileName);
free(files[i].pszTag);
free(files[i].pszStateFile);

21
plugins/imgssapi/imgssapi.c
View File

@ -72,6 +72,7 @@ MODULE_CNFNAME("imgssapi")
/* some forward definitions - they may go away when we no longer include imtcp.c */
static rsRetVal addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal);
static rsRetVal actGSSListener(uchar *port);
static int TCPSessGSSInit(void);
static void TCPSessGSSClose(tcps_sess_t* pSess);
static rsRetVal TCPSessGSSRecv(tcps_sess_t *pSess, void *buf, size_t buf_len, ssize_t *);
@ -90,6 +91,7 @@ DEFobjCurrIf(glbl)
static tcpsrv_t *pOurTcpsrv = NULL; /* our TCP server(listener) TODO: change for multiple instances */
static gss_cred_id_t gss_server_creds = GSS_C_NO_CREDENTIAL;
static uchar *srvPort;
/* our usr structure for the tcpsrv object */
typedef struct gsssrv_s {
@ -315,6 +317,16 @@ finalize_it:
static rsRetVal
addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal)
{
DEFiRet;
srvPort = pNewVal;
RETiRet;
}
static rsRetVal
actGSSListener(uchar *port)
{
DEFiRet;
gsssrv_t *pGSrv;
@ -340,7 +352,7 @@ addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal)
CHKiRet(tcpsrv.SetCBOnRegularClose(pOurTcpsrv, onRegularClose));
CHKiRet(tcpsrv.SetCBOnErrClose(pOurTcpsrv, onErrClose));
CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, UCHAR_CONSTANT("imgssapi")));
tcpsrv.configureTCPListen(pOurTcpsrv, pNewVal, 1);
tcpsrv.configureTCPListen(pOurTcpsrv, port, 1);
CHKiRet(tcpsrv.ConstructFinalize(pOurTcpsrv));
}
@ -676,6 +688,11 @@ ENDfreeCnf
*/
BEGINrunInput
CODESTARTrunInput
/* This will fail if the priviledges are dropped. Should be
* moved to the '*activateCnfPrePrivDrop' section eventually.
*/
actGSSListener(srvPort);
iRet = tcpsrv.Run(pOurTcpsrv);
ENDrunInput
@ -683,7 +700,7 @@ ENDrunInput
/* initialize and return if will run or not */
BEGINwillRun
CODESTARTwillRun
if(pOurTcpsrv == NULL)
if(srvPort == NULL)
ABORT_FINALIZE(RS_RET_NO_RUN);
net.PrintAllowedSenders(2); /* TCP */

7
plugins/imjournal/Makefile.am Normal file
View File

@ -0,0 +1,7 @@
pkglib_LTLIBRARIES = imjournal.la
imjournal_la_SOURCES = imjournal.c imjournal.h
imjournal_la_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) $(LIBSYSTEMD_JOURNAL_CFLAGS)
#imjournal_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) $(LIBSYSTEMD_JOURNAL_CFLAGS)
imjournal_la_LDFLAGS = -module -avoid-version
imjournal_la_LIBADD = $(LIBSYSTEMD_JOURNAL_LIBS)

717
plugins/imjournal/imjournal.c Executable file
View File

@ -0,0 +1,717 @@
/* The systemd journal import module
*
* To test under Linux:
* emmit log message into systemd journal
*
* Copyright (C) 2008-2013 Adiscon GmbH
*
* This file is part of rsyslog.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* -or-
* see COPYING.ASL20 in the source distribution
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "config.h"
#include "rsyslog.h"
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <stdarg.h>
#include <ctype.h>
#include <stdlib.h>
#include <time.h>
#include <sys/poll.h>
#include <sys/socket.h>
#include <errno.h>
#include <systemd/sd-journal.h>
#include "dirty.h"
#include "cfsysline.h"
#include "obj.h"
#include "msg.h"
#include "module-template.h"
#include "datetime.h"
#include "imjournal.h"
#include "net.h"
#include "glbl.h"
#include "prop.h"
#include "errmsg.h"
#include "srUtils.h"
#include "unicode-helper.h"
#include "ratelimit.h"
MODULE_TYPE_INPUT
MODULE_TYPE_NOKEEP
MODULE_CNFNAME("imjournal")
/* Module static data */
DEF_IMOD_STATIC_DATA
DEFobjCurrIf(datetime)
DEFobjCurrIf(glbl)
DEFobjCurrIf(prop)
DEFobjCurrIf(net)
DEFobjCurrIf(errmsg)
static struct configSettings_s {
char *stateFile;
int iPersistStateInterval;
int ratelimitInterval;
int ratelimitBurst;
int bIgnorePrevious;
} cs;
/* module-global parameters */
static struct cnfparamdescr modpdescr[] = {
{ "statefile", eCmdHdlrGetWord, 0 },
{ "ratelimit.interval", eCmdHdlrInt, 0 },
{ "ratelimit.burst", eCmdHdlrInt, 0 },
{ "persiststateinterval", eCmdHdlrInt, 0 },
{ "ignorepreviousmessages", eCmdHdlrBinary, 0 }
};
static struct cnfparamblk modpblk =
{ CNFPARAMBLK_VERSION,
sizeof(modpdescr)/sizeof(struct cnfparamdescr),
modpdescr
};
#define DFLT_persiststateinterval 10
static int bLegacyCnfModGlobalsPermitted = 1;/* are legacy module-global config parameters permitted? */
static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */
static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 */
static ratelimit_t *ratelimiter = NULL;
static sd_journal *j;
/* enqueue the the journal message into the message queue.
* The provided msg string is not freed - thus must be done
* by the caller.
*/
static rsRetVal
enqMsg(uchar *msg, uchar *pszTag, int iFacility, int iSeverity, struct timeval *tp, struct json_object *json)
{
struct syslogTime st;
msg_t *pMsg;
DEFiRet;
assert(msg != NULL);
assert(pszTag != NULL);
if(tp == NULL) {
CHKiRet(msgConstruct(&pMsg));
} else {
datetime.timeval2syslogTime(tp, &st);
CHKiRet(msgConstructWithTime(&pMsg, &st, tp->tv_sec));
}
MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY);
MsgSetInputName(pMsg, pInputName);
MsgSetRawMsgWOSize(pMsg, (char*)msg);
MsgSetMSGoffs(pMsg, 0); /* we do not have a header... */
MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp());
MsgSetRcvFromIP(pMsg, pLocalHostIP);
MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName()));
MsgSetTAG(pMsg, pszTag, ustrlen(pszTag));
pMsg->iFacility = iFacility;
pMsg->iSeverity = iSeverity;
if(json != NULL) {
msgAddJSON(pMsg, (uchar*)"!", json);
}
CHKiRet(ratelimitAddMsg(ratelimiter, NULL, pMsg));
finalize_it:
RETiRet;
}
/* Read journal log while data are available, each read() reads one
* record of printk buffer.
*/
static rsRetVal
readjournal() {
DEFiRet;
struct timeval tv;
uint64_t timestamp;
struct json_object *json = NULL;
int r;
/* Information from messages */
char *message;
char *sys_pid;
char *sys_iden;
char *sys_iden_help;
const void *get;
const void *pidget;
char *parse;
char *get2;
size_t length;
size_t pidlength;
const void *equal_sign;
struct json_object *jval;
char *data;
char *name;
size_t l;
long prefixlen = 0;
int priority = 0;
int facility = 0;
/* Get message text */
if (sd_journal_get_data(j, "MESSAGE", &get, &length) < 0) {
logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar *)"log message from journal doesn't have MESSAGE", 0);
iRet = RS_RET_OK;
goto ret;
}
message = strndup(get+8, length-8);
if (message == NULL) {
iRet = RS_RET_OUT_OF_MEMORY;
goto ret;
}
/* Get message priority */
if (sd_journal_get_data(j, "PRIORITY", &get, &length) >= 0) {
get2 = strndup(get, length);
priority = ((char *)get2)[9] - '0';
free (get2);
}
/* Get syslog facility */
if (sd_journal_get_data(j, "SYSLOG_FACILITY", &get, &length) >= 0) {
get2 = strndup(get, length);
char f = ((char *)get2)[16];
if (f >= '0' && f <= '9') {
facility += f - '0';
}
f = ((char *)get2)[17];
if (f >= '0' && f <= '9') {
facility *= 10;
facility += (f - '0');
}
free (get2);
} else {
/* message is missing facility -> internal systemd journal msg, drop */
iRet = RS_RET_OK;
goto free_message;
}
/* Get message identifier, client pid and add ':' */
if (sd_journal_get_data(j, "SYSLOG_IDENTIFIER", &get, &length) >= 0) {
sys_iden = strndup(get+18, length-18);
} else {
sys_iden = strdup("journal");
}
if (sys_iden == NULL) {
iRet = RS_RET_OUT_OF_MEMORY;
goto free_message;
}
if (sd_journal_get_data(j, "SYSLOG_PID", &pidget, &pidlength) >= 0) {
sys_pid = strndup(pidget+11, pidlength-11);
if (sys_pid == NULL) {
iRet = RS_RET_OUT_OF_MEMORY;
free (sys_iden);
goto free_message;
}
} else {
sys_pid = NULL;
}
if (sys_pid) {
r = asprintf(&sys_iden_help, "%s[%s]:", sys_iden, sys_pid);
} else {
r = asprintf(&sys_iden_help, "%s:", sys_iden);
}
free (sys_iden);
free (sys_pid);
if (-1 == r) {
iRet = RS_RET_OUT_OF_MEMORY;
goto finalize_it;
}
json = json_object_new_object();
SD_JOURNAL_FOREACH_DATA(j, get, l) {
/* locate equal sign, this is always present */
equal_sign = memchr(get, '=', l);
/* ... but we know better than to trust the specs */
if (equal_sign == NULL) {
errmsg.LogError(0, RS_RET_ERR, "SD_JOURNAL_FOREACH_DATA()"
"returned a malformed field (has no '='): '%s'", get);
continue; /* skip the entry */
}
/* get length of journal data prefix */
prefixlen = ((char *)equal_sign - (char *)get);
/* translate name fields to lumberjack names */
parse = (char *)get;
switch (*parse)
{
case '_':
++parse;
if (*parse == 'P') {
if (!strncmp(parse+1, "ID=", 4)) {
name = strdup("pid");
} else {
name = strndup(get, prefixlen);
}
} else if (*parse == 'G') {
if (!strncmp(parse+1, "ID=", 4)) {
name = strdup("gid");
} else {
name = strndup(get, prefixlen);
}
} else if (*parse == 'U') {
if (!strncmp(parse+1, "ID=", 4)) {
name = strdup("uid");
} else {
name = strndup(get, prefixlen);
}
} else if (*parse == 'E') {
if (!strncmp(parse+1, "XE=", 4)) {
name = strdup("exe");
} else {
name = strndup(get, prefixlen);
}
} else if (*parse == 'C') {
parse++;
if (*parse == 'O') {
if (!strncmp(parse+1, "MM=", 4)) {
name = strdup("appname");
} else {
name = strndup(get, prefixlen);
}
} else if (*parse == 'M') {
if (!strncmp(parse+1, "DLINE=", 7)) {
name = strdup("cmd");
} else {
name = strndup(get, prefixlen);
}
} else {
name = strndup(get, prefixlen);
}
} else {
name = strndup(get, prefixlen);
}
break;
default:
name = strndup(get, prefixlen);
break;
}
if (name == NULL) {
iRet = RS_RET_OUT_OF_MEMORY;
goto ret;
}
prefixlen++; /* remove '=' */
data = strndup(get + prefixlen, l - prefixlen);
if (data == NULL) {
iRet = RS_RET_OUT_OF_MEMORY;
free (name);
goto ret;
}
/* and save them to json object */
jval = json_object_new_string((char *)data);
json_object_object_add(json, name, jval);
free (data);
free (name);
}
/* calculate timestamp */
if (sd_journal_get_realtime_usec(j, &timestamp) >= 0) {
tv.tv_sec = timestamp / 1000000;
tv.tv_usec = timestamp % 1000000;
}
/* submit message */
enqMsg((uchar *)message, (uchar *) sys_iden_help, facility, priority, &tv, json);
finalize_it:
free(sys_iden_help);
free_message:
free(message);
ret:
RETiRet;
}
/* This function gets journal cursor and saves it into state file
*/
static rsRetVal
persistJournalState () {
DEFiRet;
FILE *sf; /* state file */
char *cursor;
int ret = 0;
/* On success, sd_journal_get_cursor() returns 1 in systemd
197 or older and 0 in systemd 198 or newer */
if ((ret = sd_journal_get_cursor(j, &cursor)) >= 0) {
if ((sf = fopen(cs.stateFile, "wb")) != NULL) {
if (fprintf(sf, "%s", cursor) < 0) {
iRet = RS_RET_IO_ERROR;
}
fclose(sf);
free(cursor);
} else {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_FOPEN_FAILURE, "fopen() failed: "
"'%s', path: '%s'\n", errStr, cs.stateFile);
iRet = RS_RET_FOPEN_FAILURE;
}
} else {
char errStr[256];
rs_strerror_r(-(ret), errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR, "sd_journal_get_cursor() failed: '%s'\n", errStr);
iRet = RS_RET_ERR;
}
RETiRet;
}
/* Polls the journal for new messages. Similar to sd_journal_wait()
* except for the special handling of EINTR.
*/
static rsRetVal
pollJournal()
{
DEFiRet;
struct pollfd pollfd;
int r;
pollfd.fd = sd_journal_get_fd(j);
pollfd.events = sd_journal_get_events(j);
r = poll(&pollfd, 1, -1);
if (r == -1) {
if (errno == EINTR) {
/* EINTR is also received during termination
* so return now to check the term state.
*/
ABORT_FINALIZE(RS_RET_OK);
} else {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR,
"poll() failed: '%s'", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
}
assert(r == 1);
r = sd_journal_process(j);
if (r < 0) {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR,
"sd_journal_process() failed: '%s'", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
finalize_it:
RETiRet;
}
/* This function loads a journal cursor from the state file.
*/
static rsRetVal
loadJournalState()
{
DEFiRet;
if (cs.stateFile[0] != '/') {
char *new_stateFile;
if (-1 == asprintf(&new_stateFile, "%s/%s", (char *)glbl.GetWorkDir(), cs.stateFile)) {
errmsg.LogError(0, RS_RET_OUT_OF_MEMORY, "imjournal: asprintf failed\n");
ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
}
free (cs.stateFile);
cs.stateFile = new_stateFile;
}
/* if state file exists, set cursor to appropriate position */
if (access(cs.stateFile, F_OK|R_OK) != -1) {
FILE *r_sf;
if ((r_sf = fopen(cs.stateFile, "rb")) != NULL) {
char readCursor[128 + 1];
if (fscanf(r_sf, "%128s\n", readCursor) != EOF) {
if (sd_journal_seek_cursor(j, readCursor) != 0) {
errmsg.LogError(0, RS_RET_ERR, "imjournal: "
"couldn't seek to cursor `%s'\n", readCursor);
iRet = RS_RET_ERR;
goto finalize_it;
}
sd_journal_next(j);
} else {
errmsg.LogError(0, RS_RET_IO_ERROR, "imjournal: "
"fscanf on state file `%s' failed\n", cs.stateFile);
iRet = RS_RET_IO_ERROR;
goto finalize_it;
}
fclose(r_sf);
} else {
errmsg.LogError(0, RS_RET_FOPEN_FAILURE, "imjournal: "
"open on state file `%s' failed\n", cs.stateFile);
}
} else {
/* when IgnorePrevious, seek to the end of journal */
if (cs.bIgnorePrevious) {
if (sd_journal_seek_tail(j) < 0) {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR,
"sd_journal_seek_tail() failed: '%s'", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
if (sd_journal_previous(j) < 0) {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR,
"sd_journal_previous() failed: '%s'", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
}
}
finalize_it:
RETiRet;
}
BEGINrunInput
CODESTARTrunInput
CHKiRet(ratelimitNew(&ratelimiter, "imjournal", NULL));
dbgprintf("imjournal: ratelimiting burst %d, interval %d\n", cs.ratelimitBurst,
cs.ratelimitInterval);
ratelimitSetLinuxLike(ratelimiter, cs.ratelimitInterval, cs.ratelimitBurst);
ratelimitSetNoTimeCache(ratelimiter);
if (cs.stateFile) {
CHKiRet(loadJournalState());
}
/* this is an endless loop - it is terminated when the thread is
* signalled to do so. This, however, is handled by the framework.
*/
while (glbl.GetGlobalInputTermState() == 0) {
int count = 0, r;
r = sd_journal_next(j);
if (r < 0) {
char errStr[256];
rs_strerror_r(errno, errStr, sizeof(errStr));
errmsg.LogError(0, RS_RET_ERR,
"sd_journal_next() failed: '%s'", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
if (r == 0) {
/* No new messages, wait for activity. */
CHKiRet(pollJournal());
continue;
}
CHKiRet(readjournal());
if (cs.stateFile) { /* can't persist without a state file */
/* TODO: This could use some finer metric. */
count++;
if (count == cs.iPersistStateInterval) {
count = 0;
persistJournalState();
}
}
}
finalize_it:
ENDrunInput
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
bLegacyCnfModGlobalsPermitted = 1;
cs.iPersistStateInterval = DFLT_persiststateinterval;
cs.stateFile = NULL;
cs.ratelimitBurst = 20000;
cs.ratelimitInterval = 600;
ENDbeginCnfLoad
BEGINendCnfLoad
CODESTARTendCnfLoad
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
ENDfreeCnf
/* open journal */
BEGINwillRun
CODESTARTwillRun
int ret;
ret = sd_journal_open(&j, SD_JOURNAL_LOCAL_ONLY);
if (ret < 0) {
iRet = RS_RET_IO_ERROR;
}
ENDwillRun
/* close journal */
BEGINafterRun
CODESTARTafterRun
if (cs.stateFile) { /* can't persist without a state file */
persistJournalState();
}
sd_journal_close(j);
ratelimitDestruct(ratelimiter);
ENDafterRun
BEGINmodExit
CODESTARTmodExit
if(pInputName != NULL)
prop.Destruct(&pInputName);
if(pLocalHostIP != NULL)
prop.Destruct(&pLocalHostIP);
/* release objects we used */
objRelease(glbl, CORE_COMPONENT);
objRelease(net, CORE_COMPONENT);
objRelease(datetime, CORE_COMPONENT);
objRelease(prop, CORE_COMPONENT);
objRelease(errmsg, CORE_COMPONENT);
ENDmodExit
BEGINsetModCnf
struct cnfparamvals *pvals = NULL;
int i;
CODESTARTsetModCnf
pvals = nvlstGetParams(lst, &modpblk, NULL);
if (pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module "
"config parameters [module(...)]");
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
if (Debug) {
dbgprintf("module (global) param blk for imjournal:\n");
cnfparamsPrint(&modpblk, pvals);
}
for (i = 0 ; i < modpblk.nParams ; ++i) {
if (!pvals[i].bUsed)
continue;
if (!strcmp(modpblk.descr[i].name, "persiststateinterval")) {
cs.iPersistStateInterval = (int) pvals[i].val.d.n;
} else if (!strcmp(modpblk.descr[i].name, "statefile")) {
cs.stateFile = (char *)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(modpblk.descr[i].name, "ratelimit.burst")) {
cs.ratelimitBurst = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "ratelimit.interval")) {
cs.ratelimitInterval = (int) pvals[i].val.d.n;
} else if (!strcmp(modpblk.descr[i].name, "ignorepreviousmessages")) {
cs.bIgnorePrevious = (int) pvals[i].val.d.n;
} else {
dbgprintf("imjournal: program error, non-handled "
"param '%s' in beginCnfLoad\n", modpblk.descr[i].name);
}
}
finalize_it:
if (pvals != NULL)
cnfparamvalsDestruct(pvals, &modpblk);
ENDsetModCnf
BEGINisCompatibleWithFeature
CODESTARTisCompatibleWithFeature
if(eFeat == sFEATURENonCancelInputTermination)
iRet = RS_RET_OK;
ENDisCompatibleWithFeature
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_IMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
ENDqueryEtryPt
BEGINmodInit()
CODESTARTmodInit
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
CHKiRet(objUse(datetime, CORE_COMPONENT));
CHKiRet(objUse(glbl, CORE_COMPONENT));
CHKiRet(objUse(prop, CORE_COMPONENT));
CHKiRet(objUse(net, CORE_COMPONENT));
CHKiRet(objUse(errmsg, CORE_COMPONENT));
/* we need to create the inputName property (only once during our lifetime) */
CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imjournal"), sizeof("imjournal") - 1));
CHKiRet(prop.CreateStringProp(&pLocalHostIP, UCHAR_CONSTANT("127.0.0.1"), sizeof("127.0.0.1") - 1));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalpersiststateinterval", 0, eCmdHdlrInt,
NULL, &cs.iPersistStateInterval, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalratelimitinterval", 0, eCmdHdlrInt,
NULL, &cs.ratelimitInterval, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalratelimitburst", 0, eCmdHdlrInt,
NULL, &cs.ratelimitBurst, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalstatefile", 0, eCmdHdlrGetWord,
NULL, &cs.stateFile, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalignorepreviousmessages", 0, eCmdHdlrBinary,
NULL, &cs.bIgnorePrevious, STD_LOADABLE_MODULE_ID));
ENDmodInit
/* vim:set ai:
*/

36
plugins/imjournal/imjournal.h Normal file
View File

@ -0,0 +1,36 @@
/* imjournal.h
* These are the definitions for the journal messages import module
*
* Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH.
*
* This file is part of rsyslog.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* -or-
* see COPYING.ASL20 in the source distribution
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef IMJOURNAL_H_INCLUDED
#define IMJOURNAL_H_INCLUDED 1
#include "rsyslog.h"
#include "dirty.h"
#include <systemd/sd-journal.h>
struct modConfData_s {
};
rsRetVal Syslog(int priority, uchar *msg, struct timeval *tp, struct json_object *json);
#endif /* #ifndef IMJOURNAL_H_INCLUDED */
/* vi:set ai:
*/

5
plugins/imklog/bsd.c
View File

@ -58,9 +58,6 @@ static int fklog = -1; /* kernel log fd */
#ifdef OS_LINUX
/* submit a message to imklog Syslog() API. In this function, we check if
* a kernel timestamp is present and, if so, extract and strip it.
* Note: this is an extra processing step. We should revisit the whole
* idea in v6 and remove all that old stuff that we do not longer need
* (like symbol resolution). <-- TODO
* Note that this is heavily Linux specific and thus is not compiled or
* used for BSD.
* Special thanks to Lennart Poettering for suggesting on how to convert
@ -175,7 +172,7 @@ klogWillRun(modConfData_t *pModConf)
fklog = open((char*)GetPath(pModConf), O_RDONLY, 0);
if (fklog < 0) {
imklogLogIntMsg(RS_RET_ERR_OPEN_KLOG, "imklog: cannot open kernel log(%s): %s.",
imklogLogIntMsg(LOG_ERR, "imklog: cannot open kernel log(%s): %s.",
GetPath(pModConf), rs_strerror_r(errno, errmsg, sizeof(errmsg)));
ABORT_FINALIZE(RS_RET_ERR_OPEN_KLOG);
}

8
plugins/imklog/imklog.c
View File

@ -95,7 +95,7 @@ static struct cnfparamdescr modpdescr[] = {
{ "permitnonkernelfacility", eCmdHdlrBinary, 0 },
{ "consoleloglevel", eCmdHdlrInt, 0 },
{ "parsekerneltimestamp", eCmdHdlrBinary, 0 },
{ "keepkerneltimestamp", eCmdHdlrBinary, 0 },
{ "keepkerneltimestamp", eCmdHdlrBinary, 0 },
{ "internalmsgfacility", eCmdHdlrFacility, 0 }
};
static struct cnfparamblk modpblk =
@ -105,7 +105,7 @@ static struct cnfparamblk modpblk =
};
static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */
static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 */
static prop_t *pLocalHostIP = NULL;
static inline void
initConfigSettings(void)
@ -150,7 +150,8 @@ enqMsg(uchar *msg, uchar* pszTag, int iFacility, int iSeverity, struct timeval *
MsgSetTAG(pMsg, pszTag, ustrlen(pszTag));
pMsg->iFacility = iFacility;
pMsg->iSeverity = iSeverity;
CHKiRet(submitMsg(pMsg));
/* note: we do NOT use rate-limiting, as the kernel itself does rate-limiting */
CHKiRet(submitMsg2(pMsg));
finalize_it:
RETiRet;
@ -294,6 +295,7 @@ CODESTARTbeginCnfLoad
pModConf->bParseKernelStamp = 0;
pModConf->bKeepKernelStamp = 0;
pModConf->console_log_level = -1;
pModConf->bKeepKernelStamp = 0;
pModConf->iFacilIntMsg = klogFacilIntMsg();
loadModConf->configSetViaV2Method = 0;
bLegacyCnfModGlobalsPermitted = 1;

2
plugins/imklog/imklog.h
View File

@ -35,9 +35,9 @@ struct modConfData_s {
int iFacilIntMsg;
uchar *pszPath;
int console_log_level;
sbool bPermitNonKernel;
sbool bParseKernelStamp;
sbool bKeepKernelStamp;
sbool bPermitNonKernel;
sbool configSetViaV2Method;
};

48
plugins/imkmsg/kmsg.c
View File

@ -32,10 +32,9 @@
#include <errno.h>
#include <string.h>
#include <ctype.h>
#ifdef OS_LINUX
#include <sys/klog.h>
#endif
#include <json/json.h>
#include <sys/sysinfo.h>
#include <json.h>
#include "rsyslog.h"
#include "srUtils.h"
@ -58,9 +57,8 @@ submitSyslog(uchar *buf)
{
long offs = 0;
struct timeval tv;
long int timestamp = 0;
struct timespec monotonic;
struct timespec realtime;
struct sysinfo info;
unsigned long int timestamp = 0;
char name[1024];
char value[1024];
char msg[1024];
@ -87,12 +85,12 @@ submitSyslog(uchar *buf)
/* get timestamp */
for (; isdigit(*buf); buf++) {
timestamp += (timestamp * 10) + (*buf - '0');
timestamp = (timestamp * 10) + (*buf - '0');
}
while (*buf != ';') {
buf++; /* skip everything till the first ; */
}
}
buf++; /* skip ; */
/* get message */
@ -131,10 +129,24 @@ submitSyslog(uchar *buf)
}
/* calculate timestamp */
clock_gettime(CLOCK_MONOTONIC, &monotonic);
clock_gettime(CLOCK_REALTIME, &realtime);
tv.tv_sec = realtime.tv_sec + ((timestamp / 1000000l) - monotonic.tv_sec);
tv.tv_usec = (realtime.tv_nsec + ((timestamp / 1000000000l) - monotonic.tv_nsec)) / 1000;
sysinfo(&info);
gettimeofday(&tv, NULL);
/* get boot time */
tv.tv_sec -= info.uptime;
tv.tv_sec += timestamp / 1000000;
tv.tv_usec += timestamp % 1000000;
while (tv.tv_usec < 0) {
tv.tv_sec--;
tv.tv_usec += 1000000;
}
while (tv.tv_usec >= 1000000) {
tv.tv_sec++;
tv.tv_usec -= 1000000;
}
Syslog(priority, (uchar *)msg, &tv, json);
}
@ -146,7 +158,6 @@ rsRetVal
klogWillRun(modConfData_t *pModConf)
{
char errmsg[2048];
int r;
DEFiRet;
fklog = open(_PATH_KLOG, O_RDONLY, 0);
@ -156,17 +167,6 @@ klogWillRun(modConfData_t *pModConf)
ABORT_FINALIZE(RS_RET_ERR_OPEN_KLOG);
}
/* Set level of kernel console messaging.. */
if(pModConf->console_log_level != -1) {
r = klogctl(8, NULL, pModConf->console_log_level);
if(r != 0) {
imkmsgLogIntMsg(LOG_WARNING, "imkmsg: cannot set console log level: %s",
rs_strerror_r(errno, errmsg, sizeof(errmsg)));
/* make sure we do not try to re-set! */
pModConf->console_log_level = -1;
}
}
finalize_it:
RETiRet;
}

83
plugins/impstats/impstats.c
View File

@ -27,6 +27,12 @@
#include <signal.h>
#include <string.h>
#include <pthread.h>
#include <fcntl.h>
#include <sys/uio.h>
#if defined(__FreeBSD__)
#include <sys/stat.h>
#endif
#include "dirty.h"
#include "cfsysline.h"
#include "module-template.h"
@ -63,11 +69,14 @@ typedef struct configSettings_s {
} configSettings_t;
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
rsconf_t *pConf; /* our overall config object */
int iStatsInterval;
int iFacility;
int iSeverity;
int logfd; /* fd if logging to file, or -1 if closed */
statsFmtType_t statsFmt;
sbool bLogToSyslog;
char *logfile;
sbool configSetViaV2Method;
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
@ -82,6 +91,8 @@ static struct cnfparamdescr modpdescr[] = {
{ "interval", eCmdHdlrInt, 0 },
{ "facility", eCmdHdlrInt, 0 },
{ "severity", eCmdHdlrInt, 0 },
{ "log.syslog", eCmdHdlrBinary, 0 },
{ "log.file", eCmdHdlrGetWord, 0 },
{ "format", eCmdHdlrGetWord, 0 }
};
static struct cnfparamblk modpblk =
@ -120,7 +131,7 @@ initConfigSettings(void)
/* actually submit a message to the rsyslog core
*/
static inline rsRetVal
static inline void
doSubmitMsg(uchar *line)
{
msg_t *pMsg;
@ -138,11 +149,55 @@ doSubmitMsg(uchar *line)
pMsg->iSeverity = runModConf->iSeverity;
pMsg->msgFlags = 0;
submitMsg(pMsg);
/* we do not use rate-limiting, as the stats message always need to be emitted */
submitMsg2(pMsg);
DBGPRINTF("impstats: submit [%d,%d] msg '%s'\n", runModConf->iFacility,
runModConf->iSeverity, line);
finalize_it:
RETiRet;
return;
}
/* log stats message to file; limited error handling done */
static inline void
doLogToFile(cstr_t *cstr)
{
struct iovec iov[4];
ssize_t nwritten;
ssize_t nexpect;
time_t t;
char timebuf[32];
if(cstrLen(cstr) == 0)
goto done;
if(runModConf->logfd == -1) {
runModConf->logfd = open(runModConf->logfile, O_WRONLY|O_CREAT|O_APPEND|O_CLOEXEC, S_IRUSR|S_IWUSR);
if(runModConf->logfd == -1) {
dbgprintf("error opening stats file %s\n", runModConf->logfile);
goto done;
}
}
time(&t);
iov[0].iov_base = ctime_r(&t, timebuf);
iov[0].iov_len = nexpect = strlen(iov[0].iov_base) - 1; /* -1: strip \n */
iov[1].iov_base = ": ";
iov[1].iov_len = 2;
nexpect += 2;
iov[2].iov_base = rsCStrGetSzStrNoNULL(cstr);
iov[2].iov_len = (size_t) cstrLen(cstr);
nexpect += cstrLen(cstr);
iov[3].iov_base = "\n";
iov[3].iov_len = 1;
nexpect++;
nwritten = writev(runModConf->logfd, iov, 4);
if(nwritten != nexpect) {
dbgprintf("error writing stats file %s, nwritten %lld, expected %lld\n",
runModConf->logfile, (long long) nwritten, (long long) nexpect);
}
done: return;
}
@ -153,7 +208,10 @@ static rsRetVal
doStatsLine(void __attribute__((unused)) *usrptr, cstr_t *cstr)
{
DEFiRet;
doSubmitMsg(rsCStrGetSzStrNoNULL(cstr));
if(runModConf->bLogToSyslog)
doSubmitMsg(rsCStrGetSzStrNoNULL(cstr));
if(runModConf->logfile != NULL)
doLogToFile(cstr);
RETiRet;
}
@ -178,6 +236,9 @@ CODESTARTbeginCnfLoad
loadModConf->iFacility = DEFAULT_FACILITY;
loadModConf->iSeverity = DEFAULT_SEVERITY;
loadModConf->statsFmt = statsFmt_Legacy;
loadModConf->logfd = -1;
loadModConf->logfile = NULL;
loadModConf->bLogToSyslog = 1;
bLegacyCnfModGlobalsPermitted = 1;
/* init legacy config vars */
initConfigSettings();
@ -210,6 +271,10 @@ CODESTARTsetModCnf
loadModConf->iFacility = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "severity")) {
loadModConf->iSeverity = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "log.syslog")) {
loadModConf->bLogToSyslog = (sbool) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "log.file")) {
loadModConf->logfile = es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(modpblk.descr[i].name, "format")) {
mode = es_str2cstr(pvals[i].val.d.estr, NULL);
if(!strcasecmp(mode, "json")) {
@ -270,7 +335,9 @@ BEGINactivateCnf
rsRetVal localRet;
CODESTARTactivateCnf
runModConf = pModConf;
DBGPRINTF("impstats: stats interval %d seconds\n", runModConf->iStatsInterval);
DBGPRINTF("impstats: stats interval %d seconds, logToSyslog %d, logFile %s\n",
runModConf->iStatsInterval, runModConf->bLogToSyslog,
runModConf->logfile == NULL ? "deactivated" : (char*)runModConf->logfile);
localRet = statsobj.EnableStats();
if(localRet != RS_RET_OK) {
errmsg.LogError(0, localRet, "impstats: error enabling statistics gathering");
@ -282,6 +349,9 @@ ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
if(runModConf->logfd != -1)
close(runModConf->logfd);
free(runModConf->logfile);
ENDfreeCnf
@ -297,6 +367,7 @@ CODESTARTrunInput
if(glbl.GetGlobalInputTermState() == 1)
break; /* terminate input! */
DBGPRINTF("impstats: woke up, generating messages\n");
generateStatsMsgs();
}
ENDrunInput

41
plugins/imptcp/imptcp.c
View File

@ -67,6 +67,7 @@
#include "ruleset.h"
#include "msg.h"
#include "statsobj.h"
#include "ratelimit.h"
#include "net.h" /* for permittedPeers, may be removed when this is removed */
/* the define is from tcpsrv.h, we need to find a new (but easier!!!) abstraction layer some time ... */
@ -121,6 +122,8 @@ struct instanceConf_s {
uchar *pszBindRuleset; /* name of ruleset to bind to */
uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */
ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */
int ratelimitInterval;
int ratelimitBurst;
struct instanceConf_s *next;
};
@ -158,6 +161,8 @@ static struct cnfparamdescr inppdescr[] = {
{ "keepalive.time", eCmdHdlrInt, 0 },
{ "keepalive.interval", eCmdHdlrInt, 0 },
{ "addtlframedelimiter", eCmdHdlrInt, 0 },
{ "ratelimit.interval", eCmdHdlrInt, 0 },
{ "ratelimit.burst", eCmdHdlrInt, 0 }
};
static struct cnfparamblk inppblk =
{ CNFPARAMBLK_VERSION,
@ -195,6 +200,7 @@ struct ptcpsrv_s {
sbool bKeepAlive; /* support keep-alive packets */
sbool bEmitMsgOnClose;
sbool bSuppOctetFram;
ratelimit_t *ratelimiter;
};
/* the ptcp session object. Describes a single active session.
@ -295,6 +301,7 @@ destructSess(ptcpsess_t *pSess)
static void
destructSrv(ptcpsrv_t *pSrv)
{
ratelimitDestruct(pSrv->ratelimiter);
prop.Destruct(&pSrv->pInputName);
pthread_mutex_destroy(&pSrv->mutSessLst);
free(pSrv->pszInputName);
@ -679,14 +686,7 @@ doSubmitMsg(ptcpsess_t *pThis, struct syslogTime *stTime, time_t ttGenTime, mult
MsgSetRuleset(pMsg, pSrv->pRuleset);
STATSCOUNTER_INC(pThis->pLstn->ctrSubmit, pThis->pLstn->mutCtrSubmit);
if(pMultiSub == NULL) {
CHKiRet(submitMsg(pMsg));
} else {
pMultiSub->ppMsgs[pMultiSub->nElem++] = pMsg;
if(pMultiSub->nElem == pMultiSub->maxElem)
CHKiRet(multiSubmitMsg(pMultiSub));
}
ratelimitAddMsg(pSrv->ratelimiter, pMultiSub, pMsg);
finalize_it:
/* reset status variables */
@ -805,12 +805,11 @@ processDataRcvd(ptcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG
* we have just received a bunch of data! -- rgerhards, 2009-06-16
* EXTRACT from tcps_sess.c
*/
#define NUM_MULTISUB 1024
static rsRetVal
DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen)
{
multi_submit_t multiSub;
msg_t *pMsgs[NUM_MULTISUB];
msg_t *pMsgs[CONF_NUM_MULTISUB];
struct syslogTime stTime;
time_t ttGenTime;
char *pEnd;
@ -821,7 +820,7 @@ DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen)
datetime.getCurrTime(&stTime, &ttGenTime);
multiSub.ppMsgs = pMsgs;
multiSub.maxElem = NUM_MULTISUB;
multiSub.maxElem = CONF_NUM_MULTISUB;
multiSub.nElem = 0;
/* We now copy the message to the session buffer. */
@ -831,15 +830,11 @@ DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen)
CHKiRet(processDataRcvd(pThis, *pData++, &stTime, ttGenTime, &multiSub));
}
if(multiSub.nElem > 0) {
/* submit anything that was not yet submitted */
CHKiRet(multiSubmitMsg(&multiSub));
}
iRet = multiSubmitFlush(&multiSub);
finalize_it:
RETiRet;
}
#undef NUM_MULTISUB
/****************************************** --END-- TCP SUPPORT FUNCTIONS ***********************************/
@ -1051,6 +1046,8 @@ createInstance(instanceConf_t **pinst)
inst->bEmitMsgOnClose = 0;
inst->iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER;
inst->pBindRuleset = NULL;
inst->ratelimitBurst = 10000; /* arbitrary high limit */
inst->ratelimitInterval = 0; /* off */
/* node created, let's add to config */
if(loadModConf->tail == NULL) {
@ -1130,6 +1127,9 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst)
pSrv->iKeepAliveProbes = inst->iKeepAliveProbes;
pSrv->iKeepAliveTime = inst->iKeepAliveTime;
pSrv->bEmitMsgOnClose = inst->bEmitMsgOnClose;
CHKiRet(ratelimitNew(&pSrv->ratelimiter, "imtcp", (char*)inst->pszBindPort));
ratelimitSetLinuxLike(pSrv->ratelimiter, inst->ratelimitInterval, inst->ratelimitBurst);
ratelimitSetThreadSafe(pSrv->ratelimiter);
CHKmalloc(pSrv->port = ustrdup(inst->pszBindPort));
pSrv->iAddtlFrameDelim = inst->iAddtlFrameDelim;
if(inst->pszBindAddr == NULL)
@ -1419,10 +1419,7 @@ BEGINnewInpInst
CODESTARTnewInpInst
DBGPRINTF("newInpInst (imptcp)\n");
pvals = nvlstGetParams(lst, &inppblk, NULL);
if(pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS,
"imptcp: required parameter are missing\n");
if((pvals = nvlstGetParams(lst, &inppblk, NULL)) == NULL) {
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
@ -1458,6 +1455,10 @@ CODESTARTnewInpInst
inst->iAddtlFrameDelim = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "notifyonconnectionclose")) {
inst->bEmitMsgOnClose = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.burst")) {
inst->ratelimitBurst = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) {
inst->ratelimitInterval = (int) pvals[i].val.d.n;
} else {
dbgprintf("imptcp: program error, non-handled "
"param '%s'\n", inppblk.descr[i].name);

25
plugins/imrelp/imrelp.c
View File

@ -113,11 +113,29 @@ static struct cnfparamblk inppblk =
* we will only see the hostname (twice). -- rgerhards, 2009-10-14
*/
static relpRetVal
onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *pMsg, size_t lenMsg)
onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *msg, size_t lenMsg)
{
prop_t *pProp = NULL;
msg_t *pMsg;
DEFiRet;
parseAndSubmitMessage(pHostname, pIP, pMsg, lenMsg, PARSE_HOSTNAME,
eFLOWCTL_LIGHT_DELAY, pInputName, NULL, 0, runModConf->pBindRuleset);
CHKiRet(msgConstruct(&pMsg));
MsgSetInputName(pMsg, pInputName);
MsgSetRawMsg(pMsg, (char*)msg, lenMsg);
MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY);
MsgSetRuleset(pMsg, runModConf->pBindRuleset);
pMsg->msgFlags = PARSE_HOSTNAME | NEEDS_PARSING;
/* TODO: optimize this, we can store it inside the session, requires
* changes to librelp --> next librelp iteration?. rgerhards, 2012-10-29
*/
MsgSetRcvFromStr(pMsg, pHostname, ustrlen(pHostname), &pProp);
CHKiRet(prop.Destruct(&pProp));
CHKiRet(MsgSetRcvFromIPStr(pMsg, pIP, ustrlen(pIP), &pProp));
CHKiRet(prop.Destruct(&pProp));
CHKiRet(submitMsg2(pMsg));
finalize_it:
RETiRet;
}
@ -190,6 +208,7 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst)
if(pRelpEngine == NULL) {
CHKiRet(relpEngineConstruct(&pRelpEngine));
CHKiRet(relpEngineSetDbgprint(pRelpEngine, dbgprintf));
CHKiRet(relpEngineSetFamily(pRelpEngine, glbl.GetDefPFFamily()));
CHKiRet(relpEngineSetEnableCmd(pRelpEngine, (uchar*) "syslog", eRelpCmdState_Required));
CHKiRet(relpEngineSetSyslogRcv(pRelpEngine, onSyslogRcv));
if (!glbl.GetDisableDNS()) {

31
plugins/imtcp/imtcp.c
View File

@ -105,6 +105,8 @@ struct instanceConf_s {
uchar *pszBindRuleset; /* name of ruleset to bind to */
ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */
uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */
int ratelimitInterval;
int ratelimitBurst;
int bSuppOctetFram;
struct instanceConf_s *next;
};
@ -136,9 +138,10 @@ static struct cnfparamdescr modpdescr[] = {
{ "disablelfdelimiter", eCmdHdlrBinary, 0 },
{ "octetcountedframing", eCmdHdlrBinary, 0 },
{ "notifyonconnectionclose", eCmdHdlrBinary, 0 },
{ "addtlframedelimiter", eCmdHdlrPositiveInt, 0 },
{ "addtlframedelimiter", eCmdHdlrNonNegInt, 0 },
{ "maxsessions", eCmdHdlrPositiveInt, 0 },
{ "maxlistners", eCmdHdlrPositiveInt, 0 },
{ "maxlisteners", eCmdHdlrPositiveInt, 0 },
{ "streamdriver.mode", eCmdHdlrPositiveInt, 0 },
{ "streamdriver.authmode", eCmdHdlrString, 0 },
{ "permittedpeer", eCmdHdlrArray, 0 },
@ -155,7 +158,9 @@ static struct cnfparamdescr inppdescr[] = {
{ "port", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */
{ "name", eCmdHdlrString, 0 },
{ "ruleset", eCmdHdlrString, 0 },
{ "supportOctetCountedFraming", eCmdHdlrBinary, 0 }
{ "supportOctetCountedFraming", eCmdHdlrBinary, 0 },
{ "ratelimit.interval", eCmdHdlrInt, 0 },
{ "ratelimit.burst", eCmdHdlrInt, 0 }
};
static struct cnfparamblk inppblk =
{ CNFPARAMBLK_VERSION,
@ -251,6 +256,8 @@ createInstance(instanceConf_t **pinst)
inst->pszBindRuleset = NULL;
inst->pszInputName = NULL;
inst->bSuppOctetFram = 1;
inst->ratelimitInterval = 0;
inst->ratelimitBurst = 10000;
/* node created, let's add to config */
if(loadModConf->tail == NULL) {
@ -334,6 +341,7 @@ addListner(modConfData_t *modConf, instanceConf_t *inst)
CHKiRet(tcpsrv.SetRuleset(pOurTcpsrv, inst->pBindRuleset));
CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, inst->pszInputName == NULL ?
UCHAR_CONSTANT("imtcp") : inst->pszInputName));
CHKiRet(tcpsrv.SetLinuxLikeRatelimiters(pOurTcpsrv, inst->ratelimitInterval, inst->ratelimitBurst));
tcpsrv.configureTCPListen(pOurTcpsrv, inst->pszBindPort, inst->bSuppOctetFram);
finalize_it:
@ -376,6 +384,10 @@ CODESTARTnewInpInst
inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(inppblk.descr[i].name, "supportOctetCountedFraming")) {
inst->bSuppOctetFram = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.burst")) {
inst->ratelimitBurst = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) {
inst->ratelimitInterval = (int) pvals[i].val.d.n;
} else {
dbgprintf("imtcp: program error, non-handled "
"param '%s'\n", inppblk.descr[i].name);
@ -442,7 +454,8 @@ CODESTARTsetModCnf
loadModConf->iAddtlFrameDelim = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "maxsessions")) {
loadModConf->iTCPSessMax = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "maxlistners")) {
} else if(!strcmp(modpblk.descr[i].name, "maxlisteners") ||
!strcmp(modpblk.descr[i].name, "maxlistners")) { /* keep old name for a while */
loadModConf->iTCPLstnMax = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "keepalive")) {
loadModConf->bKeepAlive = (int) pvals[i].val.d.n;
@ -487,10 +500,10 @@ CODESTARTendCnfLoad
loadModConf->pszStrmDrvrAuthMode = NULL;
} else {
loadModConf->pszStrmDrvrAuthMode = cs.pszStrmDrvrAuthMode;
cs.pszStrmDrvrAuthMode = NULL;
}
}
if((cs.pszStrmDrvrAuthMode == NULL) || (cs.pszStrmDrvrAuthMode[0] == '\0'))
free(cs.pszStrmDrvrAuthMode);
free(cs.pszStrmDrvrAuthMode);
cs.pszStrmDrvrAuthMode = NULL;
loadModConf = NULL; /* done loading */
@ -550,6 +563,7 @@ ENDactivateCnf
BEGINfreeCnf
instanceConf_t *inst, *del;
CODESTARTfreeCnf
free(pModConf->pszStrmDrvrAuthMode);
if(pModConf->permittedPeers != NULL) {
cnfarrayContentDestruct(pModConf->permittedPeers);
free(pModConf->permittedPeers);
@ -580,7 +594,9 @@ ENDwillRun
BEGINafterRun
CODESTARTafterRun
/* do cleanup here */
if(pOurTcpsrv != NULL)
iRet = tcpsrv.Destruct(&pOurTcpsrv);
net.clearAllowedSenders(UCHAR_CONSTANT("TCP"));
ENDafterRun
@ -594,9 +610,6 @@ ENDisCompatibleWithFeature
BEGINmodExit
CODESTARTmodExit
if(pOurTcpsrv != NULL)
iRet = tcpsrv.Destruct(&pOurTcpsrv);
if(pPermPeersRoot != NULL) {
net.DestructPermittedPeers(&pPermPeersRoot);
}

90
plugins/imudp/imudp.c
View File

@ -4,8 +4,6 @@
* NOTE: read comments in module-template.h to understand how this file
* works!
*
* File begun on 2007-12-21 by RGerhards (extracted from syslogd.c)
*
* Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH.
*
* This file is part of rsyslog.
@ -53,6 +51,7 @@
#include "prop.h"
#include "ruleset.h"
#include "statsobj.h"
#include "ratelimit.h"
#include "unicode-helper.h"
MODULE_TYPE_INPUT
@ -76,7 +75,9 @@ static struct lstn_s {
struct lstn_s *next;
int sock; /* socket */
ruleset_t *pRuleset; /* bound ruleset */
prop_t *pInputName;
statsobj_t *stats; /* listener stats */
ratelimit_t *ratelimiter;
STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit)
} *lcnfRoot = NULL, *lcnfLast = NULL;
@ -91,7 +92,6 @@ static uchar *pRcvBuf = NULL; /* receive buffer (for a single packet). We use a
* it so that we can check available memory in willRun() and request
* termination if we can not get it. -- rgerhards, 2007-12-27
*/
static prop_t *pInputName = NULL; /* our inputName currently is always "imudp", and this will hold it */
#define TIME_REQUERY_DFLT 2
#define SCHED_PRIO_UNSET -12345678 /* a value that indicates that the scheduling priority has not been set */
@ -108,8 +108,12 @@ struct instanceConf_s {
uchar *pszBindAddr; /* IP to bind socket to */
uchar *pszBindPort; /* Port to bind socket to */
uchar *pszBindRuleset; /* name of ruleset to bind to */
uchar *inputname;
ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */
int ratelimitInterval;
int ratelimitBurst;
struct instanceConf_s *next;
sbool bAppendPortToInpname;
};
struct modConfData_s {
@ -139,8 +143,12 @@ static struct cnfparamblk modpblk =
/* input instance parameters */
static struct cnfparamdescr inppdescr[] = {
{ "port", eCmdHdlrArray, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */
{ "inputname", eCmdHdlrGetWord, 0 },
{ "inputname.appendport", eCmdHdlrBinary, 0 },
{ "address", eCmdHdlrString, 0 },
{ "ruleset", eCmdHdlrString, 0 }
{ "ruleset", eCmdHdlrString, 0 },
{ "ratelimit.interval", eCmdHdlrInt, 0 },
{ "ratelimit.burst", eCmdHdlrInt, 0 }
};
static struct cnfparamblk inppblk =
{ CNFPARAMBLK_VERSION,
@ -165,6 +173,10 @@ createInstance(instanceConf_t **pinst)
inst->pszBindPort = NULL;
inst->pszBindAddr = NULL;
inst->pszBindRuleset = NULL;
inst->inputname = NULL;
inst->bAppendPortToInpname = 0;
inst->ratelimitBurst = 10000; /* arbitrary high limit */
inst->ratelimitInterval = 0; /* off */
/* node created, let's add to config */
if(loadModConf->tail == NULL) {
@ -223,7 +235,8 @@ addListner(instanceConf_t *inst)
struct lstn_s *newlcnfinfo;
uchar *bindName;
uchar *port;
uchar statname[64];
uchar dispname[64], inpnameBuf[128];
uchar *inputname;
/* check which address to bind to. We could do this more compact, but have not
* done so in order to make the code more readable. -- rgerhards, 2007-12-27
@ -248,11 +261,29 @@ addListner(instanceConf_t *inst)
newlcnfinfo->next = NULL;
newlcnfinfo->sock = newSocks[iSrc];
newlcnfinfo->pRuleset = inst->pBindRuleset;
snprintf((char*)dispname, sizeof(dispname), "imudp(%s:%s)", bindName, port);
dispname[sizeof(dispname)-1] = '\0'; /* just to be on the save side... */
CHKiRet(ratelimitNew(&newlcnfinfo->ratelimiter, (char*)dispname, NULL));
if(inst->inputname == NULL) {
inputname = (uchar*)"imudp";
} else {
inputname = inst->inputname;
}
if(inst->bAppendPortToInpname) {
snprintf((char*)inpnameBuf, sizeof(inpnameBuf), "%s%s",
inputname, port);
inpnameBuf[sizeof(inpnameBuf)-1] = '\0';
inputname = inpnameBuf;
}
CHKiRet(prop.Construct(&newlcnfinfo->pInputName));
CHKiRet(prop.SetString(newlcnfinfo->pInputName,
inputname, ustrlen(inputname)));
CHKiRet(prop.ConstructFinalize(newlcnfinfo->pInputName));
ratelimitSetLinuxLike(newlcnfinfo->ratelimiter, inst->ratelimitInterval,
inst->ratelimitBurst);
/* support statistics gathering */
CHKiRet(statsobj.Construct(&(newlcnfinfo->stats)));
snprintf((char*)statname, sizeof(statname), "imudp(%s:%s)", bindName, port);
statname[sizeof(statname)-1] = '\0'; /* just to be on the save side... */
CHKiRet(statsobj.SetName(newlcnfinfo->stats, statname));
CHKiRet(statsobj.SetName(newlcnfinfo->stats, dispname));
STATSCOUNTER_INIT(newlcnfinfo->ctrSubmit, newlcnfinfo->mutCtrSubmit);
CHKiRet(statsobj.AddCounter(newlcnfinfo->stats, UCHAR_CONSTANT("submitted"),
ctrType_IntCtr, &(newlcnfinfo->ctrSubmit)));
@ -304,7 +335,6 @@ std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, insta
static inline rsRetVal
processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *frominetPrev, int *pbIsPermitted)
{
DEFiRet;
int iNbrTimeUsed;
time_t ttGenTime;
struct syslogTime stTime;
@ -314,9 +344,15 @@ processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *f
msg_t *pMsg;
prop_t *propFromHost = NULL;
prop_t *propFromHostIP = NULL;
multi_submit_t multiSub;
msg_t *pMsgs[CONF_NUM_MULTISUB];
char errStr[1024];
DEFiRet;
assert(pThrd != NULL);
multiSub.ppMsgs = pMsgs;
multiSub.maxElem = CONF_NUM_MULTISUB;
multiSub.nElem = 0;
iNbrTimeUsed = 0;
while(1) { /* loop is terminated if we have a bad receive, done below in the body */
if(pThrd->bShallStop == RSTRUE)
@ -367,7 +403,7 @@ processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *f
*pbIsPermitted = 1; /* no check -> everything permitted */
}
DBGPRINTF("recv(%d,%d),acl:%d,msg:%s\n", lstn->sock, (int) lenRcvBuf, *pbIsPermitted, pRcvBuf);
DBGPRINTF("imudp:recv(%d,%d),acl:%d,msg:%s\n", lstn->sock, (int) lenRcvBuf, *pbIsPermitted, pRcvBuf);
if(*pbIsPermitted != 0) {
if((runModConf->iTimeRequery == 0) || (iNbrTimeUsed++ % runModConf->iTimeRequery) == 0) {
@ -376,19 +412,22 @@ processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *f
/* we now create our own message object and submit it to the queue */
CHKiRet(msgConstructWithTime(&pMsg, &stTime, ttGenTime));
MsgSetRawMsg(pMsg, (char*)pRcvBuf, lenRcvBuf);
MsgSetInputName(pMsg, pInputName);
MsgSetInputName(pMsg, lstn->pInputName);
MsgSetRuleset(pMsg, lstn->pRuleset);
MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY);
pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME | NEEDS_DNSRESOL;
if(*pbIsPermitted == 2)
pMsg->msgFlags |= NEEDS_ACLCHK_U; /* request ACL check after resolution */
CHKiRet(msgSetFromSockinfo(pMsg, &frominet));
CHKiRet(submitMsg(pMsg));
CHKiRet(ratelimitAddMsg(lstn->ratelimiter, &multiSub, pMsg));
STATSCOUNTER_INC(lstn->ctrSubmit, lstn->mutCtrSubmit);
}
}
finalize_it:
multiSubmitFlush(&multiSub);
if(propFromHost != NULL)
prop.Destruct(&propFromHost);
if(propFromHostIP != NULL)
@ -678,10 +717,18 @@ createListner(es_str_t *port, struct cnfparamvals *pvals)
continue;
if(!strcmp(inppblk.descr[i].name, "port")) {
continue; /* array, handled by caller */
} else if(!strcmp(inppblk.descr[i].name, "inputname")) {
inst->inputname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(inppblk.descr[i].name, "inputname.appendport")) {
inst->bAppendPortToInpname = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "address")) {
inst->pszBindAddr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(inppblk.descr[i].name, "ruleset")) {
inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.burst")) {
inst->ratelimitBurst = (int) pvals[i].val.d.n;
} else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) {
inst->ratelimitInterval = (int) pvals[i].val.d.n;
} else {
dbgprintf("imudp: program error, non-handled "
"param '%s'\n", inppblk.descr[i].name);
@ -699,10 +746,7 @@ BEGINnewInpInst
CODESTARTnewInpInst
DBGPRINTF("newInpInst (imudp)\n");
pvals = nvlstGetParams(lst, &inppblk, NULL);
if(pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS,
"imudp: required parameter are missing\n");
if((pvals = nvlstGetParams(lst, &inppblk, NULL)) == NULL) {
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
if(Debug) {
@ -825,7 +869,7 @@ CODESTARTactivateCnfPrePrivDrop
for(inst = runModConf->root ; inst != NULL ; inst = inst->next) {
addListner(inst);
}
/* if we could not set up any listners, there is no point in running... */
/* if we could not set up any listeners, there is no point in running... */
if(lcnfRoot == NULL) {
errmsg.LogError(0, NO_ERRCODE, "imudp: no listeners could be started, "
"input not activated.\n");
@ -851,7 +895,7 @@ CODESTARTfreeCnf
for(inst = pModConf->root ; inst != NULL ; ) {
free(inst->pszBindPort);
free(inst->pszBindAddr);
free(inst->pBindRuleset);
free(inst->inputname);
del = inst;
inst = inst->next;
free(del);
@ -892,7 +936,9 @@ CODESTARTafterRun
net.clearAllowedSenders((uchar*)"UDP");
for(lstn = lcnfRoot ; lstn != NULL ; ) {
statsobj.Destruct(&(lstn->stats));
ratelimitDestruct(lstn->ratelimiter);
close(lstn->sock);
prop.Destruct(&lstn->pInputName);
lstnDel = lstn;
lstn = lstn->next;
free(lstnDel);
@ -907,9 +953,6 @@ ENDafterRun
BEGINmodExit
CODESTARTmodExit
if(pInputName != NULL)
prop.Destruct(&pInputName);
/* release what we no longer need */
objRelease(errmsg, CORE_COMPONENT);
objRelease(glbl, CORE_COMPONENT);
@ -964,11 +1007,6 @@ CODEmodInit_QueryRegCFSLineHdlr
CHKiRet(objUse(ruleset, CORE_COMPONENT));
CHKiRet(objUse(net, LM_NET_FILENAME));
/* we need to create the inputName property (only once during our lifetime) */
CHKiRet(prop.Construct(&pInputName));
CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imudp"), sizeof("imudp") - 1));
CHKiRet(prop.ConstructFinalize(pInputName));
/* register config file handlers */
CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputudpserverbindruleset", 0, eCmdHdlrGetWord,
NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID));

205
plugins/imuxsock/imuxsock.c
View File

@ -6,7 +6,7 @@
*
* File begun on 2007-12-20 by RGerhards (extracted from syslogd.c)
*
* Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH.
* Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH.
*
* This file is part of rsyslog.
*
@ -55,6 +55,7 @@
#include "statsobj.h"
#include "datetime.h"
#include "hashtable.h"
#include "ratelimit.h"
MODULE_TYPE_INPUT
MODULE_TYPE_NOKEEP
@ -105,15 +106,6 @@ STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit)
STATSCOUNTER_DEF(ctrLostRatelimit, mutCtrLostRatelimit)
STATSCOUNTER_DEF(ctrNumRatelimiters, mutCtrNumRatelimiters)
struct rs_ratelimit_state {
unsigned short interval;
unsigned short burst;
unsigned done;
unsigned missed;
time_t begin;
};
typedef struct rs_ratelimit_state rs_ratelimit_state_t;
/* a very simple "hash function" for process IDs - we simply use the
* pid itself: it is quite expected that all pids may log some time, but
@ -143,6 +135,7 @@ typedef struct lstn_s {
int flowCtl; /* flow control settings for this socket */
int ratelimitInterval;
int ratelimitBurst;
ratelimit_t *dflt_ratelimiter;/*ratelimiter to apply if none else is to be used */
intTiny ratelimitSev; /* severity level (and below) for which rate-limiting shall apply */
struct hashtable *ht; /* our hashtable for rate-limiting */
sbool bParseHost; /* should parser parse host name? read-only after startup */
@ -151,7 +144,9 @@ typedef struct lstn_s {
sbool bAnnotate; /* annotate events with trusted properties */
sbool bParseTrusted; /* parse trusted properties */
sbool bWritePid; /* write original PID into tag */
sbool bDiscardOwnMsgs; /* discard messages that originated from ourselves */
sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */
sbool bUnlink; /* unlink&re-create socket at start and end of processing */
} lstn_t;
static lstn_t listeners[MAXFUNIX];
@ -206,6 +201,8 @@ struct instanceConf_s {
int ratelimitSeverity;
int bAnnotate; /* annotate trusted properties */
int bParseTrusted; /* parse trusted properties */
sbool bDiscardOwnMsgs; /* discard messages that originated from our own pid? */
sbool bUnlink;
struct instanceConf_s *next;
};
@ -223,7 +220,9 @@ struct modConfData_s {
sbool bOmitLocalLogging;
sbool bWritePidSysSock;
sbool bUseSysTimeStamp;
sbool bDiscardOwnMsgs;
sbool configSetViaV2Method;
sbool bUnlink;
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */
@ -232,7 +231,9 @@ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current lo
static struct cnfparamdescr modpdescr[] = {
{ "syssock.use", eCmdHdlrBinary, 0 },
{ "syssock.name", eCmdHdlrGetWord, 0 },
{ "syssock.unlink", eCmdHdlrBinary, 0 },
{ "syssock.ignoretimestamp", eCmdHdlrBinary, 0 },
{ "syssock.ignoreownmessages", eCmdHdlrBinary, 0 },
{ "syssock.flowcontrol", eCmdHdlrBinary, 0 },
{ "syssock.usesystimestamp", eCmdHdlrBinary, 0 },
{ "syssock.annotate", eCmdHdlrBinary, 0 },
@ -251,8 +252,10 @@ static struct cnfparamblk modpblk =
/* input instance parameters */
static struct cnfparamdescr inppdescr[] = {
{ "socket", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: addunixlistensocket */
{ "unlink", eCmdHdlrBinary, 0 },
{ "createpath", eCmdHdlrBinary, 0 },
{ "parsetrusted", eCmdHdlrBinary, 0 },
{ "ignoreownmessages", eCmdHdlrBinary, 0 },
{ "hostname", eCmdHdlrString, 0 },
{ "ignoretimestamp", eCmdHdlrBinary, 0 },
{ "flowcontrol", eCmdHdlrBinary, 0 },
@ -272,74 +275,9 @@ static struct cnfparamblk inppblk =
/* we do not use this, because we do not bind to a ruleset so far
* enable when this is changed: #include "im-helper.h" */ /* must be included AFTER the type definitions! */
static void
initRatelimitState(struct rs_ratelimit_state *rs, unsigned short interval, unsigned short burst)
{
rs->interval = interval;
rs->burst = burst;
rs->done = 0;
rs->missed = 0;
rs->begin = 0;
}
static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */
/* ratelimiting support, modelled after the linux kernel
* returns 1 if message is within rate limit and shall be
* processed, 0 otherwise.
* This implementation is NOT THREAD-SAFE and must not
* be called concurrently.
*/
static inline int
withinRatelimit(struct rs_ratelimit_state *rs, time_t tt, pid_t pid)
{
int ret;
uchar msgbuf[1024];
if(rs->interval == 0) {
ret = 1;
goto finalize_it;
}
assert(rs->burst != 0);
if(rs->begin == 0)
rs->begin = tt;
/* resume if we go out of out time window */
if(tt > rs->begin + rs->interval) {
if(rs->missed) {
snprintf((char*)msgbuf, sizeof(msgbuf),
"imuxsock lost %u messages from pid %lu due to rate-limiting",
rs->missed, (unsigned long) pid);
logmsgInternal(RS_RET_RATE_LIMITED, LOG_SYSLOG|LOG_INFO, msgbuf, 0);
rs->missed = 0;
}
rs->begin = 0;
rs->done = 0;
}
/* do actual limit check */
if(rs->burst > rs->done) {
rs->done++;
ret = 1;
} else {
if(rs->missed == 0) {
snprintf((char*)msgbuf, sizeof(msgbuf),
"imuxsock begins to drop messages from pid %lu due to rate-limiting",
(unsigned long) pid);
logmsgInternal(RS_RET_RATE_LIMITED, LOG_SYSLOG|LOG_INFO, msgbuf, 0);
}
rs->missed++;
ret = 0;
}
finalize_it:
return ret;
}
/* create input instance, set default paramters, and
* add it to the list of instances.
*/
@ -352,7 +290,7 @@ createInstance(instanceConf_t **pinst)
inst->sockName = NULL;
inst->pLogHostName = NULL;
inst->ratelimitInterval = DFLT_ratelimitInterval;
inst->ratelimitBurst = DFLT_ratelimitSeverity;
inst->ratelimitBurst = DFLT_ratelimitBurst;
inst->ratelimitSeverity = DFLT_ratelimitSeverity;
inst->bUseFlowCtl = 0;
inst->bIgnoreTimestamp = 1;
@ -361,6 +299,8 @@ createInstance(instanceConf_t **pinst)
inst->bWritePid = 0;
inst->bAnnotate = 0;
inst->bParseTrusted = 0;
inst->bDiscardOwnMsgs = 1;
inst->bUnlink = 1;
inst->next = NULL;
/* node created, let's add to config */
@ -446,7 +386,8 @@ addListner(instanceConf_t *inst)
CHKiRet(prop.ConstructFinalize(listeners[nfd].hostName));
}
if(inst->ratelimitInterval > 0) {
if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) {
if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn,
(void(*)(void*))ratelimitDestruct)) == NULL) {
/* in this case, we simply turn off rate-limiting */
DBGPRINTF("imuxsock: turning off rate limiting because we could not "
"create hash table\n");
@ -460,11 +401,19 @@ addListner(instanceConf_t *inst)
listeners[nfd].flags = inst->bIgnoreTimestamp ? IGNDATE : NOFLAG;
listeners[nfd].bCreatePath = inst->bCreatePath;
listeners[nfd].sockName = ustrdup(inst->sockName);
listeners[nfd].bUseCreds = (inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0;
listeners[nfd].bUseCreds = (inst->bDiscardOwnMsgs || inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0;
listeners[nfd].bAnnotate = inst->bAnnotate;
listeners[nfd].bParseTrusted = inst->bParseTrusted;
listeners[nfd].bDiscardOwnMsgs = inst->bDiscardOwnMsgs;
listeners[nfd].bUnlink = inst->bUnlink;
listeners[nfd].bWritePid = inst->bWritePid;
listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp;
CHKiRet(ratelimitNew(&listeners[nfd].dflt_ratelimiter, "imuxsock", NULL));
ratelimitSetLinuxLike(listeners[nfd].dflt_ratelimiter,
listeners[nfd].ratelimitInterval,
listeners[nfd].ratelimitBurst);
ratelimitSetSeverity(listeners[nfd].dflt_ratelimiter,
listeners[nfd].ratelimitSev);
nfd++;
} else {
errmsg.LogError(0, NO_ERRCODE, "Out of unix socket name descriptors, ignoring %s\n",
@ -476,7 +425,7 @@ finalize_it:
}
/* discard all log sockets except for "socket" 0. Data for it comes from
/* discard/Destruct all log sockets except for "socket" 0. Data for it comes from
* the constant memory pool - and if not, it is freeed via some other pointer.
*/
static rsRetVal discardLogSockets(void)
@ -494,6 +443,7 @@ static rsRetVal discardLogSockets(void)
if(listeners[i].ht != NULL) {
hashtable_destroy(listeners[i].ht, 1); /* 1 => free all values automatically */
}
ratelimitDestruct(listeners[i].dflt_ratelimiter);
}
return RS_RET_OK;
@ -508,7 +458,8 @@ createLogSocket(lstn_t *pLstn)
struct sockaddr_un sunx;
DEFiRet;
unlink((char*)pLstn->sockName);
if(pLstn->bUnlink)
unlink((char*)pLstn->sockName);
memset(&sunx, 0, sizeof(sunx));
sunx.sun_family = AF_UNIX;
if(pLstn->bCreatePath) {
@ -605,19 +556,26 @@ finalize_it:
* listener (the latter being a performance enhancement).
*/
static inline rsRetVal
findRatelimiter(lstn_t *pLstn, struct ucred *cred, rs_ratelimit_state_t **prl)
findRatelimiter(lstn_t *pLstn, struct ucred *cred, ratelimit_t **prl)
{
rs_ratelimit_state_t *rl;
ratelimit_t *rl;
int r;
pid_t *keybuf;
char pidbuf[256];
DEFiRet;
if(cred == NULL)
FINALIZE;
#if 0 // TODO: check deactivated?
if(pLstn->ratelimitInterval == 0) {
*prl = NULL;
FINALIZE;
}
#endif
if(pLstn->ht == NULL) {
*prl = NULL;
FINALIZE;
}
rl = hashtable_search(pLstn->ht, &cred->pid);
if(rl == NULL) {
@ -625,10 +583,14 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, rs_ratelimit_state_t **prl)
DBGPRINTF("imuxsock: no ratelimiter for pid %lu, creating one\n",
(unsigned long) cred->pid);
STATSCOUNTER_INC(ctrNumRatelimiters, mutCtrNumRatelimiters);
CHKmalloc(rl = malloc(sizeof(rs_ratelimit_state_t)));
snprintf(pidbuf, sizeof(pidbuf), "pid %lu",
(unsigned long) cred->pid);
pidbuf[sizeof(pidbuf)-1] = '\0'; /* to be on safe side */
CHKiRet(ratelimitNew(&rl, "imuxsock", pidbuf));
ratelimitSetLinuxLike(rl, pLstn->ratelimitInterval, pLstn->ratelimitBurst);
ratelimitSetSeverity(rl, pLstn->ratelimitSev);
CHKmalloc(keybuf = malloc(sizeof(pid_t)));
*keybuf = cred->pid;
initRatelimitState(rl, pLstn->ratelimitInterval, pLstn->ratelimitBurst);
r = hashtable_insert(pLstn->ht, keybuf, rl);
if(r == 0)
ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
@ -637,6 +599,8 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, rs_ratelimit_state_t **prl)
*prl = rl;
finalize_it:
if(*prl == NULL)
*prl = pLstn->dflt_ratelimiter;
RETiRet;
}
@ -763,28 +727,6 @@ copyescaped(uchar *dstbuf, uchar *inbuf, int inlen)
}
#if 0
/* Creates new field to be added to event
* used for SystemLogParseTrusted parsing
*/
struct ee_field *
createNewField(char *fieldname, char *value, int lenValue) {
es_str_t *newStr;
struct ee_value *newVal;
struct ee_field *newField;
newStr = es_newStrFromBuf(value, (es_size_t) lenValue);
newVal = ee_newValue(ctxee);
ee_setStrValue(newVal, newStr);
newField = ee_newFieldFromNV(ctxee, fieldname, newVal);
return newField;
}
#endif
/* submit received message to the queue engine
* We now parse the message according to expected format so that we
* can also mangle it if necessary.
@ -803,8 +745,8 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim
uchar bufParseTAG[CONF_TAG_MAXSIZE];
struct syslogTime st;
time_t tt;
rs_ratelimit_state_t *ratelimiter = NULL;
int lenProp;
ratelimit_t *ratelimiter = NULL;
uchar propBuf[1024];
uchar msgbuf[8192];
uchar *pmsgbuf;
@ -813,6 +755,11 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim
struct json_object *json = NULL, *jval;
DEFiRet;
if(pLstn->bDiscardOwnMsgs && cred != NULL && cred->pid == glblGetOurPid()) {
DBGPRINTF("imuxsock: discarding message from our own pid\n");
FINALIZE;
}
/* TODO: handle format errors?? */
/* we need to parse the pri first, because we need the severity for
* rate-limiting as well.
@ -831,10 +778,7 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim
facil = LOG_FAC(pri);
sever = LOG_PRI(pri);
if(sever >= pLstn->ratelimitSev) {
/* note: if cred == NULL, then ratelimiter == NULL as well! */
findRatelimiter(pLstn, cred, &ratelimiter); /* ignore error, better so than others... */
}
findRatelimiter(pLstn, cred, &ratelimiter); /* ignore error, better so than others... */
if(ts == NULL) {
datetime.getCurrTime(&st, &tt);
@ -843,10 +787,12 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim
tt = ts->tv_sec;
}
#if 0 // TODO: think about stats counters (or wait for request...?)
if(ratelimiter != NULL && !withinRatelimit(ratelimiter, tt, cred->pid)) {
STATSCOUNTER_INC(ctrLostRatelimit, mutCtrLostRatelimit);
FINALIZE;
}
#endif
/* created trusted properties */
if(cred != NULL && pLstn->bAnnotate) {
@ -981,8 +927,7 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim
MsgSetRcvFrom(pMsg, pLstn->hostName == NULL ? glbl.GetLocalHostNameProp() : pLstn->hostName);
CHKiRet(MsgSetRcvFromIP(pMsg, pLocalHostIP));
CHKiRet(submitMsg(pMsg));
ratelimitAddMsg(ratelimiter, NULL, pMsg);
STATSCOUNTER_INC(ctrSubmit, mutCtrSubmit);
finalize_it:
RETiRet;
@ -1117,13 +1062,20 @@ activateListeners()
listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock;
listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock;
listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock;
listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock) ? 1 : 0;
listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock || runModConf->bDiscardOwnMsgs) ? 1 : 0;
listeners[0].bWritePid = runModConf->bWritePidSysSock;
listeners[0].bAnnotate = runModConf->bAnnotateSysSock;
listeners[0].bParseTrusted = runModConf->bParseTrusted;
listeners[0].bDiscardOwnMsgs = runModConf->bDiscardOwnMsgs;
listeners[0].bUnlink = runModConf->bUnlink;
listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp;
listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG;
listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY;
CHKiRet(ratelimitNew(&listeners[0].dflt_ratelimiter, "imuxsock", NULL));
ratelimitSetLinuxLike(listeners[0].dflt_ratelimiter,
listeners[0].ratelimitInterval,
listeners[0].ratelimitBurst);
ratelimitSetSeverity(listeners[0].dflt_ratelimiter,listeners[0].ratelimitSev);
sd_fds = sd_listen_fds(0);
if(sd_fds < 0) {
@ -1165,6 +1117,8 @@ CODESTARTbeginCnfLoad
pModConf->bWritePidSysSock = 0;
pModConf->bAnnotateSysSock = 0;
pModConf->bParseTrusted = 0;
pModConf->bDiscardOwnMsgs = 1;
pModConf->bUnlink = 1;
pModConf->ratelimitIntervalSysSock = DFLT_ratelimitInterval;
pModConf->ratelimitBurstSysSock = DFLT_ratelimitBurst;
pModConf->ratelimitSeveritySysSock = DFLT_ratelimitSeverity;
@ -1199,6 +1153,10 @@ CODESTARTsetModCnf
loadModConf->pLogSockName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(modpblk.descr[i].name, "syssock.ignoretimestamp")) {
loadModConf->bIgnoreTimestamp = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "syssock.ignoreownmessages")) {
loadModConf->bDiscardOwnMsgs = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "syssock.unlink")) {
loadModConf->bUnlink = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "syssock.flowcontrol")) {
loadModConf->bUseFlowCtl = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "syssock.usesystimestamp")) {
@ -1261,6 +1219,10 @@ CODESTARTnewInpInst
inst->bCreatePath = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "parsetrusted")) {
inst->bParseTrusted = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "ignoreownmessages")) {
inst->bDiscardOwnMsgs = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "unlink")) {
inst->bUnlink = (int) pvals[i].val.d.n;
} else if(!strcmp(modpblk.descr[i].name, "hostname")) {
inst->pLogHostName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(modpblk.descr[i].name, "ignoretimestamp")) {
@ -1440,8 +1402,10 @@ CODESTARTafterRun
listeners[i].fd < SD_LISTEN_FDS_START + sd_fds)
continue;
DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName);
unlink((char*) listeners[i].sockName);
if(listeners[i].bUnlink) {
DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName);
unlink((char*) listeners[i].sockName);
}
}
discardLogSockets();
@ -1553,8 +1517,17 @@ CODEmodInit_QueryRegCFSLineHdlr
listeners[0].bUseCreds = 0;
listeners[0].bAnnotate = 0;
listeners[0].bParseTrusted = 0;
listeners[0].bDiscardOwnMsgs = 1;
listeners[0].bUnlink = 1;
listeners[0].bCreatePath = 0;
listeners[0].bUseSysTimeStamp = 1;
if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn,
(void(*)(void*))ratelimitDestruct)) == NULL) {
/* in this case, we simply turn off rate-limiting */
DBGPRINTF("imuxsock: turning off rate limiting for system socket "
"because we could not create hash table\n");
listeners[0].ratelimitInterval = 0;
}
/* initialize socket names */
for(i = 1 ; i < MAXFUNIX ; ++i) {

59
plugins/imzmq3/README
View File

@ -1,24 +1,59 @@
ZeroMQ 3.x Input Plugin
Building this plugin:
Requires libzmq and libczmq. First, install libzmq from the HEAD on github:
http://github.com/zeromq/libzmq. You can clone the repository, build, then
install it. The directions for doing so are there in the readme. Then, do
the same for libczmq: http://github.com/zeromq/czmq. At some point, the 3.1
version of libzmq will be released, and a supporting version of libczmq.
At that time, you could simply download and install the tarballs instead of
using git to clone the repositories. Those tarballs (when available) can
be found at http://download.zeromq.org. As of this writing (5/31/2012), the
most recent version of czmq (1.1.0) and libzmq (3.1.0-beta) will not compile
properly.
Requires libzmq and libczmq. First, download the tarballs of both libzmq
and its supporting libczmq from http://download.zeromq.org. As of this
writing (04/23/2013), the most recent versions of libzmq and czmq are
3.2.2 and 1.3.2 respectively. Configure, build, and then install both libs.
Imzmq3 allows you to push data into rsyslog from a zeromq socket. The example
below binds a SUB socket to port 7172, and then any messages with the topic
"foo" will be pushed into rsyslog.
Please note:
This plugin only supports the newer (v7) config format. Legacy config support
was removed.
Example Rsyslog.conf snippet:
-------------------------------------------------------------------------------
$InputZmq3ServerRun action=BIND,type=SUB,description=tcp://*:7172,subscribe=foo
module(load="imzmq3" ioThreads="1")
input(type="imzmq3" action="CONNECT" socktype="SUB" description="tcp://*:7172" subscribe="foo,bar")
-------------------------------------------------------------------------------
Note you can specify multiple subscriptions with a comma-delimited list, with
no spaces between values.
The only global parameter for this plugin is ioThreads, which is optional and
probably best left to the zmq default unless you know exactly what you are
doing.
The instance-level parameters are:
Required
description
subscribe (required if the sockType is SUB)
Optional
sockType (defaults to SUB)
action (defaults to BIND
sndHWM
rcvHWM
identity
sndBuf
rcvBuf
linger
backlog
sndTimeout
rcvTimeout
maxMsgSize
rate
recoveryIVL
multicastHops
reconnectIVL
reconnectIVLMax
ipv4Only
affinity
These all correspond to zmq optional settings. Except where noted, the defaults
are the zmq defaults if not set. See http://api.zeromq.org/3-2:zmq-setsockopt
for info on these.

787
plugins/imzmq3/imzmq3.c
View File

File diff suppressed because it is too large Load Diff

8
plugins/mmanon/Makefile.am Normal file
View File

@ -0,0 +1,8 @@
pkglib_LTLIBRARIES = mmanon.la
mmanon_la_SOURCES = mmanon.c
mmanon_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS)
mmanon_la_LDFLAGS = -module -avoid-version
mmanon_la_LIBADD =
EXTRA_DIST =

401
plugins/mmanon/mmanon.c Normal file
View File

@ -0,0 +1,401 @@
/* mmanon.c
* anonnymize IP addresses inside the syslog message part
*
* Copyright 2013 Adiscon GmbH.
*
* This file is part of rsyslog.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* -or-
* see COPYING.ASL20 in the source distribution
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "config.h"
#include "rsyslog.h"
#include <stdio.h>
#include <stdarg.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <signal.h>
#include <errno.h>
#include <unistd.h>
#include <stdint.h>
#include "conf.h"
#include "syslogd-types.h"
#include "srUtils.h"
#include "template.h"
#include "module-template.h"
#include "errmsg.h"
MODULE_TYPE_OUTPUT
MODULE_TYPE_NOKEEP
MODULE_CNFNAME("mmanon")
DEFobjCurrIf(errmsg);
DEF_OMOD_STATIC_DATA
/* config variables */
/* precomputed table of IPv4 anonymization masks */
static const uint32_t ipv4masks[33] = {
0xffffffff, 0xfffffffe, 0xfffffffc, 0xfffffff8,
0xfffffff0, 0xffffffe0, 0xffffffc0, 0xffffff80,
0xffffff00, 0xfffffe00, 0xfffffc00, 0xfffff800,
0xfffff000, 0xffffe000, 0xffffc000, 0xffff8000,
0xffff0000, 0xfffe0000, 0xfffc0000, 0xfff80000,
0xfff00000, 0xffe00000, 0xffc00000, 0xff800000,
0xff000000, 0xfe000000, 0xfc000000, 0xf8000000,
0xf0000000, 0xe0000000, 0xc0000000, 0x80000000,
0x00000000
};
/* define operation modes we have */
#define SIMPLE_MODE 0 /* just overwrite */
#define REWRITE_MODE 1 /* rewrite IP address, canoninized */
typedef struct _instanceData {
char replChar;
int8_t mode;
struct {
int8_t bits;
} ipv4;
} instanceData;
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */
/* tables for interfacing with the v6 config system */
/* action (instance) parameters */
static struct cnfparamdescr actpdescr[] = {
{ "mode", eCmdHdlrGetWord, 0 },
{ "replacementchar", eCmdHdlrGetChar, 0 },
{ "ipv4.bits", eCmdHdlrInt, 0 },
};
static struct cnfparamblk actpblk =
{ CNFPARAMBLK_VERSION,
sizeof(actpdescr)/sizeof(struct cnfparamdescr),
actpdescr
};
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
loadModConf = pModConf;
pModConf->pConf = pConf;
ENDbeginCnfLoad
BEGINendCnfLoad
CODESTARTendCnfLoad
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
runModConf = pModConf;
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
ENDfreeCnf
BEGINcreateInstance
CODESTARTcreateInstance
ENDcreateInstance
BEGINisCompatibleWithFeature
CODESTARTisCompatibleWithFeature
ENDisCompatibleWithFeature
BEGINfreeInstance
CODESTARTfreeInstance
ENDfreeInstance
static inline void
setInstParamDefaults(instanceData *pData)
{
pData->mode = REWRITE_MODE;
pData->replChar = 'x';
pData->ipv4.bits = 16;
}
BEGINnewActInst
struct cnfparamvals *pvals;
int i;
sbool bHadBitsErr;
CODESTARTnewActInst
DBGPRINTF("newActInst (mmanon)\n");
if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) {
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
CODE_STD_STRING_REQUESTnewActInst(1)
CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG));
CHKiRet(createInstance(&pData));
setInstParamDefaults(pData);
for(i = 0 ; i < actpblk.nParams ; ++i) {
if(!pvals[i].bUsed)
continue;
if(!strcmp(actpblk.descr[i].name, "mode")) {
if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"simple",
sizeof("simple")-1)) {
pData->mode = SIMPLE_MODE;
} else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"rewrite",
sizeof("rewrite")-1)) {
pData->mode = REWRITE_MODE;
} else {
char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL);
errmsg.LogError(0, RS_RET_INVLD_MODE,
"mmanon: invalid anonymization mode '%s' - ignored",
cstr);
free(cstr);
}
pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0];
} else if(!strcmp(actpblk.descr[i].name, "replacementchar")) {
pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0];
} else if(!strcmp(actpblk.descr[i].name, "ipv4.bits")) {
pData->ipv4.bits = (int8_t) pvals[i].val.d.n;
} else {
dbgprintf("mmanon: program error, non-handled "
"param '%s'\n", actpblk.descr[i].name);
}
}
if(pData->mode == SIMPLE_MODE) {
bHadBitsErr = 0;
if(pData->ipv4.bits < 8) {
pData->ipv4.bits = 8;
bHadBitsErr = 1;
} else if(pData->ipv4.bits < 16) {
pData->ipv4.bits = 16;
bHadBitsErr = 1;
} else if(pData->ipv4.bits < 24) {
pData->ipv4.bits = 24;
bHadBitsErr = 1;
} else if(pData->ipv4.bits != 32) {
pData->ipv4.bits = 32;
bHadBitsErr = 1;
}
if(bHadBitsErr)
errmsg.LogError(0, RS_RET_INVLD_ANON_BITS,
"mmanon: invalid number of ipv4 bits "
"in simple mode, corrected to %d",
pData->ipv4.bits);
} else { /* REWRITE_MODE */
if(pData->ipv4.bits < 1 || pData->ipv4.bits > 32) {
pData->ipv4.bits = 32;
errmsg.LogError(0, RS_RET_INVLD_ANON_BITS,
"mmanon: invalid number of ipv4 bits "
"in rewrite mode, corrected to %d",
pData->ipv4.bits);
}
if(pData->replChar != 'x') {
errmsg.LogError(0, RS_RET_REPLCHAR_IGNORED,
"mmanon: replacementChar parameter is ignored "
"in rewrite mode");
}
}
CODE_STD_FINALIZERnewActInst
cnfparamvalsDestruct(pvals, &actpblk);
ENDnewActInst
BEGINdbgPrintInstInfo
CODESTARTdbgPrintInstInfo
ENDdbgPrintInstInfo
BEGINtryResume
CODESTARTtryResume
ENDtryResume
static int
getnum(uchar *msg, int lenMsg, int *idx)
{
int num = 0;
int i = *idx;
while(i < lenMsg && msg[i] >= '0' && msg[i] <= '9') {
num = num * 10 + msg[i] - '0';
++i;
}
*idx = i;
return num;
}
/* write an IP address octet to the output position */
static int
writeOctet(uchar *msg, int idx, int *nxtidx, uint8_t octet)
{
if(octet > 99) {
msg[idx++] = '0' + octet / 100;
octet = octet % 100;
}
if(octet > 9) {
msg[idx++] = '0' + octet / 10;
octet = octet % 10;
}
msg[idx++] = '0' + octet;
if(nxtidx != NULL) {
if(idx + 1 != *nxtidx) {
/* we got shorter, fix it! */
msg[idx] = '.';
*nxtidx = idx + 1;
}
}
return idx;
}
/* currently works for IPv4 only! */
void
anonip(instanceData *pData, uchar *msg, int *pLenMsg, int *idx)
{
int i = *idx;
int octet;
uint32_t ipv4addr;
int ipstart[4];
int j;
int endpos;
int lenMsg = *pLenMsg;
while(i < lenMsg && (msg[i] <= '0' || msg[i] >= '9')) {
++i; /* skip to first number */
}
if(i >= lenMsg)
goto done;
/* got digit, let's see if ip */
ipstart[0] = i;
octet = getnum(msg, lenMsg, &i);
if(octet > 255 || msg[i] != '.') goto done;
ipv4addr = octet << 24;
++i;
ipstart[1] = i;
octet = getnum(msg, lenMsg, &i);
if(octet > 255 || msg[i] != '.') goto done;
ipv4addr |= octet << 16;
++i;
ipstart[2] = i;
octet = getnum(msg, lenMsg, &i);
if(octet > 255 || msg[i] != '.') goto done;
ipv4addr |= octet << 8;
++i;
ipstart[3] = i;
octet = getnum(msg, lenMsg, &i);
if(octet > 255 || !(msg[i] == ' ' || msg[i] == ':')) goto done;
ipv4addr |= octet;
/* OK, we now found an ip address */
if(pData->mode == SIMPLE_MODE) {
if(pData->ipv4.bits == 8)
j = ipstart[3];
else if(pData->ipv4.bits == 16)
j = ipstart[2];
else if(pData->ipv4.bits == 24)
j = ipstart[1];
else /* due to our checks, this *must* be 32 */
j = ipstart[0];
while(j < i) {
if(msg[j] != '.')
msg[j] = pData->replChar;
++j;
}
} else { /* REWRITE_MODE */
ipv4addr &= ipv4masks[pData->ipv4.bits];
if(pData->ipv4.bits > 24)
writeOctet(msg, ipstart[0], &(ipstart[1]), ipv4addr >> 24);
if(pData->ipv4.bits > 16)
writeOctet(msg, ipstart[1], &(ipstart[2]), (ipv4addr >> 16) & 0xff);
if(pData->ipv4.bits > 8)
writeOctet(msg, ipstart[2], &(ipstart[3]), (ipv4addr >> 8) & 0xff);
endpos = writeOctet(msg, ipstart[3], NULL, ipv4addr & 0xff);
/* if we had truncation, we need to shrink the msg */
dbgprintf("existing i %d, endpos %d\n", i, endpos);
if(i - endpos > 0) {
*pLenMsg = lenMsg - (i - endpos);
memmove(msg+endpos, msg+i, lenMsg - i + 1);
}
}
done: *idx = i;
return;
}
BEGINdoAction
msg_t *pMsg;
uchar *msg;
int lenMsg;
int i;
CODESTARTdoAction
pMsg = (msg_t*) ppString[0];
lenMsg = getMSGLen(pMsg);
msg = getMSG(pMsg);
for(i = 0 ; i < lenMsg ; ++i) {
anonip(pData, msg, &lenMsg, &i);
}
if(lenMsg != getMSGLen(pMsg))
setMSGLen(pMsg, lenMsg);
ENDdoAction
BEGINparseSelectorAct
CODESTARTparseSelectorAct
CODE_STD_STRING_REQUESTparseSelectorAct(1)
if(strncmp((char*) p, ":mmanon:", sizeof(":mmanon:") - 1)) {
errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED,
"mmanon supports only v6+ config format, use: "
"action(type=\"mmanon\" ...)");
}
ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED);
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
BEGINmodExit
CODESTARTmodExit
objRelease(errmsg, CORE_COMPONENT);
ENDmodExit
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
ENDqueryEtryPt
BEGINmodInit()
CODESTARTmodInit
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
DBGPRINTF("mmanon: module compiled with rsyslog version %s.\n", VERSION);
CHKiRet(objUse(errmsg, CORE_COMPONENT));
ENDmodInit

4
plugins/mmaudit/Makefile.am
View File

@ -1,8 +1,8 @@
pkglib_LTLIBRARIES = mmaudit.la
mmaudit_la_SOURCES = mmaudit.c
mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS)
mmaudit_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS)
mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS)
mmaudit_la_LDFLAGS = -module -avoid-version
mmaudit_la_LIBADD =
EXTRA_DIST =

3
plugins/mmaudit/mmaudit.c
View File

@ -43,8 +43,7 @@
#include <errno.h>
#include <unistd.h>
#include <ctype.h>
#include <libestr.h>
#include <libee/libee.h>
#include <json.h>
#include "conf.h"
#include "syslogd-types.h"
#include "template.h"

4
plugins/mmjsonparse/Makefile.am
View File

@ -1,8 +1,8 @@
pkglib_LTLIBRARIES = mmjsonparse.la
mmjsonparse_la_SOURCES = mmjsonparse.c
mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS)
mmjsonparse_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS)
mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS)
mmjsonparse_la_LDFLAGS = -module -avoid-version
mmjsonparse_la_LIBADD =
EXTRA_DIST =

77
plugins/mmjsonparse/mmjsonparse.c
View File

@ -35,8 +35,7 @@
#include <errno.h>
#include <unistd.h>
#include <ctype.h>
#include <libestr.h>
#include <json/json.h>
#include <json.h>
#include "conf.h"
#include "syslogd-types.h"
#include "template.h"
@ -62,15 +61,46 @@ typedef struct _instanceData {
struct json_tokener *tokener;
} instanceData;
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */
BEGINinitConfVars /* (re)set config variables to default values */
CODESTARTinitConfVars
resetConfigVariables(NULL, NULL);
ENDinitConfVars
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
loadModConf = pModConf;
pModConf->pConf = pConf;
ENDbeginCnfLoad
BEGINendCnfLoad
CODESTARTendCnfLoad
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
runModConf = pModConf;
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
ENDfreeCnf
BEGINcreateInstance
CODESTARTcreateInstance
pData->tokener = json_tokener_new();
if(pData->tokener == NULL) {
errmsg.LogError(0, RS_RET_ERR, "error: could not create json "
"tokener, cannot activate action");
ABORT_FINALIZE(RS_RET_ERR);
}
finalize_it:
ENDcreateInstance
@ -88,7 +118,7 @@ ENDfreeInstance
BEGINdbgPrintInstInfo
CODESTARTdbgPrintInstInfo
dbgprintf("mmjsonparse\n");
DBGPRINTF("mmjsonparse\n");
ENDdbgPrintInstInfo
@ -104,7 +134,8 @@ processJSON(instanceData *pData, msg_t *pMsg, char *buf, size_t lenBuf)
const char *errMsg;
DEFiRet;
dbgprintf("mmjsonparse: toParse: '%s'\n", buf);
assert(pData->tokener != NULL);
DBGPRINTF("mmjsonparse: toParse: '%s'\n", buf);
json_tokener_reset(pData->tokener);
json = json_tokener_parse_ex(pData->tokener, buf, lenBuf);
@ -123,7 +154,7 @@ processJSON(instanceData *pData, msg_t *pMsg, char *buf, size_t lenBuf)
else if(!json_object_is_type(json, json_type_object))
errMsg = "JSON value is not an object";
if(errMsg != NULL) {
dbgprintf("mmjsonparse: Error parsing JSON '%s': %s\n",
DBGPRINTF("mmjsonparse: Error parsing JSON '%s': %s\n",
buf, errMsg);
}
}
@ -177,6 +208,22 @@ finalize_it:
MsgSetParseSuccess(pMsg, bSuccess);
ENDdoAction
BEGINnewActInst
CODESTARTnewActInst
/* Note: we currently do not have any parameters, so we do not need
* the lst ptr. However, we will most probably need params in the
* future.
*/
DBGPRINTF("newActInst (mmjsonparse)\n");
CODE_STD_STRING_REQUESTnewActInst(1)
CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG));
CHKiRet(createInstance(&pData));
/*setInstParamDefaults(pData);*/
CODE_STD_FINALIZERnewActInst
/* cnfparamvalsDestruct(pvals, &actpblk);*/
ENDnewActInst
BEGINparseSelectorAct
CODESTARTparseSelectorAct
@ -197,14 +244,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
* the format specified (if any) is always ignored.
*/
CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat"));
/* finally build the instance */
pData->tokener = json_tokener_new();
if(pData->tokener == NULL) {
errmsg.LogError(0, RS_RET_ERR, "error: could not create json "
"tokener, cannot activate action");
ABORT_FINALIZE(RS_RET_ERR);
}
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
@ -218,7 +257,8 @@ ENDmodExit
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
ENDqueryEtryPt
@ -238,7 +278,6 @@ BEGINmodInit()
unsigned long opts;
int bMsgPassingSupported;
CODESTARTmodInit
INITLegCnfVars
*ipIFVersProvided = CURR_MOD_IF_VERSION;
/* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr

182
plugins/mmnormalize/mmnormalize.c
View File

@ -40,7 +40,7 @@
#include <unistd.h>
#include <libestr.h>
#include <libee/libee.h>
#include <json/json.h>
#include <json.h>
#include <liblognorm.h>
#include "conf.h"
#include "syslogd-types.h"
@ -65,6 +65,7 @@ DEF_OMOD_STATIC_DATA
typedef struct _instanceData {
sbool bUseRawMsg; /**< use %rawmsg% instead of %msg% */
uchar *rulebase; /**< name of rulebase to use */
ln_ctx ctxln; /**< context to be used for liblognorm */
ee_ctx ctxee; /**< context to be used for libee */
} instanceData;
@ -75,6 +76,58 @@ typedef struct configSettings_s {
} configSettings_t;
static configSettings_t cs;
/* tables for interfacing with the v6 config system */
/* action (instance) parameters */
static struct cnfparamdescr actpdescr[] = {
{ "rulebase", eCmdHdlrGetWord, 1 },
{ "userawmsg", eCmdHdlrBinary, 0 }
};
static struct cnfparamblk actpblk =
{ CNFPARAMBLK_VERSION,
sizeof(actpdescr)/sizeof(struct cnfparamdescr),
actpdescr
};
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */
/* to be called to build the libee part of the instance ONCE ALL PARAMETERS ARE CORRECT
* (and set within pData!).
*/
static rsRetVal
buildInstance(instanceData *pData)
{
DEFiRet;
if((pData->ctxee = ee_initCtx()) == NULL) {
errmsg.LogError(0, RS_RET_ERR_LIBEE_INIT, "error: could not initialize libee "
"ctx, cannot activate action");
ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT);
}
if((pData->ctxln = ln_initCtx()) == NULL) {
errmsg.LogError(0, RS_RET_ERR_LIBLOGNORM_INIT, "error: could not initialize "
"liblognorm ctx, cannot activate action");
ee_exitCtx(pData->ctxee);
ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_INIT);
}
ln_setEECtx(pData->ctxln, pData->ctxee);
if(ln_loadSamples(pData->ctxln, (char*) pData->rulebase) != 0) {
errmsg.LogError(0, RS_RET_NO_RULEBASE, "error: normalization rulebase '%s' "
"could not be loaded cannot activate action", cs.rulebase);
ee_exitCtx(pData->ctxee);
ln_exitCtx(pData->ctxln);
ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_SAMPDB_LOAD);
}
finalize_it:
RETiRet;
}
BEGINinitConfVars /* (re)set config variables to default values */
CODESTARTinitConfVars
resetConfigVariables(NULL, NULL);
@ -86,6 +139,35 @@ CODESTARTcreateInstance
ENDcreateInstance
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
loadModConf = pModConf;
pModConf->pConf = pConf;
ENDbeginCnfLoad
BEGINendCnfLoad
CODESTARTendCnfLoad
loadModConf = NULL; /* done loading */
/* free legacy config vars */
free(cs.rulebase);
cs.rulebase = NULL;
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
runModConf = pModConf;
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
ENDfreeCnf
BEGINisCompatibleWithFeature
CODESTARTisCompatibleWithFeature
ENDisCompatibleWithFeature
@ -93,6 +175,7 @@ ENDisCompatibleWithFeature
BEGINfreeInstance
CODESTARTfreeInstance
free(pData->rulebase);
ee_exitCtx(pData->ctxee);
ln_exitCtx(pData->ctxln);
ENDfreeInstance
@ -141,7 +224,7 @@ CODESTARTdoAction
es_deleteStr(str);
/* reformat to our json data struct */
// TODO: this is all extremly ineffcient!
/* TODO: this is all extremly ineffcient! */
ee_fmtEventToJSON(event, &str);
cstrJSON = es_str2cstr(str, NULL);
dbgprintf("mmnormalize generated: %s\n", cstrJSON);
@ -156,6 +239,59 @@ CODESTARTdoAction
ENDdoAction
static inline void
setInstParamDefaults(instanceData *pData)
{
pData->rulebase = NULL;
pData->bUseRawMsg = 0;
}
BEGINnewActInst
struct cnfparamvals *pvals;
int i;
int bDestructPValsOnExit;
CODESTARTnewActInst
DBGPRINTF("newActInst (mmnormalize)\n");
bDestructPValsOnExit = 0;
pvals = nvlstGetParams(lst, &actpblk, NULL);
if(pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "mmnormalize: error reading "
"config parameters");
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
bDestructPValsOnExit = 1;
if(Debug) {
dbgprintf("action param blk in mmnormalize:\n");
cnfparamsPrint(&actpblk, pvals);
}
CHKiRet(createInstance(&pData));
setInstParamDefaults(pData);
for(i = 0 ; i < actpblk.nParams ; ++i) {
if(!pvals[i].bUsed)
continue;
if(!strcmp(actpblk.descr[i].name, "rulebase")) {
pData->rulebase = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "userawmsg")) {
pData->bUseRawMsg = (int) pvals[i].val.d.n;
} else {
DBGPRINTF("mmnormalize: program error, non-handled "
"param '%s'\n", actpblk.descr[i].name);
}
}
CODE_STD_STRING_REQUESTnewActInst(1)
CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG));
iRet = buildInstance(pData);
CODE_STD_FINALIZERnewActInst
if(bDestructPValsOnExit)
cnfparamvalsDestruct(pvals, &actpblk);
ENDnewActInst
BEGINparseSelectorAct
CODESTARTparseSelectorAct
CODE_STD_STRING_REQUESTparseSelectorAct(1)
@ -165,15 +301,21 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
}
if(cs.rulebase == NULL) {
errmsg.LogError(0, RS_RET_NO_RULESET, "error: no normalization rulebase was specified, use "
errmsg.LogError(0, RS_RET_NO_RULEBASE, "error: no normalization rulebase was specified, use "
"$MMNormalizeSampleDB directive first!");
ABORT_FINALIZE(RS_RET_NO_RULESET);
ABORT_FINALIZE(RS_RET_NO_RULEBASE);
}
/* ok, if we reach this point, we have something for us */
p += sizeof(":mmnormalize:") - 1; /* eat indicator sequence (-1 because of '\0'!) */
CHKiRet(createInstance(&pData));
pData->rulebase = cs.rulebase;
pData->bUseRawMsg = cs.bUseRawMsg;
/* all config vars auto-reset! */
cs.bUseRawMsg = 0;
cs.rulebase = NULL; /* we used it up! */
/* check if a non-standard template is to be applied */
if(*(p-1) == ';')
--p;
@ -181,34 +323,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
* the format specified (if any) is always ignored.
*/
CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_TPL_AS_MSG, (uchar*) "RSYSLOG_FileFormat"));
/* finally build the instance */
if((pData->ctxee = ee_initCtx()) == NULL) {
errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize libee ctx, cannot "
"activate action");
ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT);
}
if((pData->ctxln = ln_initCtx()) == NULL) {
errmsg.LogError(0, RS_RET_NO_RULESET, "error: could not initialize liblognorm ctx, cannot "
"activate action");
ee_exitCtx(pData->ctxee);
ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_INIT);
}
ln_setEECtx(pData->ctxln, pData->ctxee);
if(ln_loadSamples(pData->ctxln, (char*) cs.rulebase) != 0) {
errmsg.LogError(0, RS_RET_NO_RULESET, "error: normalization rulebase '%s' could not be loaded "
"cannot activate action", cs.rulebase);
ee_exitCtx(pData->ctxee);
ln_exitCtx(pData->ctxln);
ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_SAMPDB_LOAD);
}
pData->bUseRawMsg = cs.bUseRawMsg;
/* all config vars auto-reset! */
cs.bUseRawMsg = 0;
free(cs.rulebase);
cs.rulebase = NULL;
CHKiRet(buildInstance(pData));
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
@ -222,7 +337,8 @@ ENDmodExit
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
ENDqueryEtryPt

3
plugins/omelasticsearch/Makefile.am
View File

@ -1,6 +1,7 @@
pkglib_LTLIBRARIES = omelasticsearch.la
omelasticsearch_la_SOURCES = omelasticsearch.c
# TODO: replace cJSON
omelasticsearch_la_SOURCES = omelasticsearch.c cJSON/cjson.c cJSON/cjson.h
omelasticsearch_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS)
omelasticsearch_la_LDFLAGS = -module -avoid-version
omelasticsearch_la_LIBADD = $(CURL_LIBS) $(LIBM)

17
plugins/omelasticsearch/README Normal file
View File

@ -0,0 +1,17 @@
How to produce an error:
========================
It's quite easy to get 400, if you put a wrong mapping to your
index. That would be easy to reproduce in "normal" omelasticsearch usage
conditions, by only altering the ES configuration:
1. Make your index first. Let's call it "testindex":
$ curl -XPUT localhost:9200/testindex/
2. Put your mapping for a search type called "mytype", where you specify
that date property should be an integer:
$ curl -XPUT localhost:9200/testindex/mytype/_mapping -d '{"mytype":{"properties": {"timegenerated":{"type":"integer"}}}}'
3. Now try to insert something where date is not an integer:
$ curl -XPOST localhost:9200/testindex/mytype/ -d '{"timegenerated":"bla"}'
{"error":"MapperParsingException[Failed to parse [date]]; nested: NumberFormatException[For input string: \"bla\"]; ","status":400}

247
plugins/omelasticsearch/cJSON/README Normal file
View File

@ -0,0 +1,247 @@
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
Welcome to cJSON.
cJSON aims to be the dumbest possible parser that you can get your job done with.
It's a single file of C, and a single header file.
JSON is described best here: http://www.json.org/
It's like XML, but fat-free. You use it to move data around, store things, or just
generally represent your program's state.
First up, how do I build?
Add cJSON.c to your project, and put cJSON.h somewhere in the header search path.
For example, to build the test app:
gcc cJSON.c test.c -o test -lm
./test
As a library, cJSON exists to take away as much legwork as it can, but not get in your way.
As a point of pragmatism (i.e. ignoring the truth), I'm going to say that you can use it
in one of two modes: Auto and Manual. Let's have a quick run-through.
I lifted some JSON from this page: http://www.json.org/fatfree.html
That page inspired me to write cJSON, which is a parser that tries to share the same
philosophy as JSON itself. Simple, dumb, out of the way.
Some JSON:
{
"name": "Jack (\"Bee\") Nimble",
"format": {
"type": "rect",
"width": 1920,
"height": 1080,
"interlace": false,
"frame rate": 24
}
}
Assume that you got this from a file, a webserver, or magic JSON elves, whatever,
you have a char * to it. Everything is a cJSON struct.
Get it parsed:
cJSON *root = cJSON_Parse(my_json_string);
This is an object. We're in C. We don't have objects. But we do have structs.
What's the framerate?
cJSON *format = cJSON_GetObjectItem(root,"format");
int framerate = cJSON_GetObjectItem(format,"frame rate")->valueint;
Want to change the framerate?
cJSON_GetObjectItem(format,"frame rate")->valueint=25;
Back to disk?
char *rendered=cJSON_Print(root);
Finished? Delete the root (this takes care of everything else).
cJSON_Delete(root);
That's AUTO mode. If you're going to use Auto mode, you really ought to check pointers
before you dereference them. If you want to see how you'd build this struct in code?
cJSON *root,*fmt;
root=cJSON_CreateObject();
cJSON_AddItemToObject(root, "name", cJSON_CreateString("Jack (\"Bee\") Nimble"));
cJSON_AddItemToObject(root, "format", fmt=cJSON_CreateObject());
cJSON_AddStringToObject(fmt,"type", "rect");
cJSON_AddNumberToObject(fmt,"width", 1920);
cJSON_AddNumberToObject(fmt,"height", 1080);
cJSON_AddFalseToObject (fmt,"interlace");
cJSON_AddNumberToObject(fmt,"frame rate", 24);
Hopefully we can agree that's not a lot of code? There's no overhead, no unnecessary setup.
Look at test.c for a bunch of nice examples, mostly all ripped off the json.org site, and
a few from elsewhere.
What about manual mode? First up you need some detail.
Let's cover how the cJSON objects represent the JSON data.
cJSON doesn't distinguish arrays from objects in handling; just type.
Each cJSON has, potentially, a child, siblings, value, a name.
The root object has: Object Type and a Child
The Child has name "name", with value "Jack ("Bee") Nimble", and a sibling:
Sibling has type Object, name "format", and a child.
That child has type String, name "type", value "rect", and a sibling:
Sibling has type Number, name "width", value 1920, and a sibling:
Sibling has type Number, name "height", value 1080, and a sibling:
Sibling hs type False, name "interlace", and a sibling:
Sibling has type Number, name "frame rate", value 24
Here's the structure:
typedef struct cJSON {
struct cJSON *next,*prev;
struct cJSON *child;
int type;
char *valuestring;
int valueint;
double valuedouble;
char *string;
} cJSON;
By default all values are 0 unless set by virtue of being meaningful.
next/prev is a doubly linked list of siblings. next takes you to your sibling,
prev takes you back from your sibling to you.
Only objects and arrays have a "child", and it's the head of the doubly linked list.
A "child" entry will have prev==0, but next potentially points on. The last sibling has next=0.
The type expresses Null/True/False/Number/String/Array/Object, all of which are #defined in
cJSON.h
A Number has valueint and valuedouble. If you're expecting an int, read valueint, if not read
valuedouble.
Any entry which is in the linked list which is the child of an object will have a "string"
which is the "name" of the entry. When I said "name" in the above example, that's "string".
"string" is the JSON name for the 'variable name' if you will.
Now you can trivially walk the lists, recursively, and parse as you please.
You can invoke cJSON_Parse to get cJSON to parse for you, and then you can take
the root object, and traverse the structure (which is, formally, an N-tree),
and tokenise as you please. If you wanted to build a callback style parser, this is how
you'd do it (just an example, since these things are very specific):
void parse_and_callback(cJSON *item,const char *prefix)
{
while (item)
{
char *newprefix=malloc(strlen(prefix)+strlen(item->name)+2);
sprintf(newprefix,"%s/%s",prefix,item->name);
int dorecurse=callback(newprefix, item->type, item);
if (item->child && dorecurse) parse_and_callback(item->child,newprefix);
item=item->next;
free(newprefix);
}
}
The prefix process will build you a separated list, to simplify your callback handling.
The 'dorecurse' flag would let the callback decide to handle sub-arrays on it's own, or
let you invoke it per-item. For the item above, your callback might look like this:
int callback(const char *name,int type,cJSON *item)
{
if (!strcmp(name,"name")) { /* populate name */ }
else if (!strcmp(name,"format/type") { /* handle "rect" */ }
else if (!strcmp(name,"format/width") { /* 800 */ }
else if (!strcmp(name,"format/height") { /* 600 */ }
else if (!strcmp(name,"format/interlace") { /* false */ }
else if (!strcmp(name,"format/frame rate") { /* 24 */ }
return 1;
}
Alternatively, you might like to parse iteratively.
You'd use:
void parse_object(cJSON *item)
{
int i; for (i=0;i<cJSON_GetArraySize(item);i++)
{
cJSON *subitem=cJSON_GetArrayItem(item,i);
// handle subitem.
}
}
Or, for PROPER manual mode:
void parse_object(cJSON *item)
{
cJSON *subitem=item->child;
while (subitem)
{
// handle subitem
if (subitem->child) parse_object(subitem->child);
subitem=subitem->next;
}
}
Of course, this should look familiar, since this is just a stripped-down version
of the callback-parser.
This should cover most uses you'll find for parsing. The rest should be possible
to infer.. and if in doubt, read the source! There's not a lot of it! ;)
In terms of constructing JSON data, the example code above is the right way to do it.
You can, of course, hand your sub-objects to other functions to populate.
Also, if you find a use for it, you can manually build the objects.
For instance, suppose you wanted to build an array of objects?
cJSON *objects[24];
cJSON *Create_array_of_anything(cJSON **items,int num)
{
int i;cJSON *prev, *root=cJSON_CreateArray();
for (i=0;i<24;i++)
{
if (!i) root->child=objects[i];
else prev->next=objects[i], objects[i]->prev=prev;
prev=objects[i];
}
return root;
}
and simply: Create_array_of_anything(objects,24);
cJSON doesn't make any assumptions about what order you create things in.
You can attach the objects, as above, and later add children to each
of those objects.
As soon as you call cJSON_Print, it renders the structure to text.
The test.c code shows how to handle a bunch of typical cases. If you uncomment
the code, it'll load, parse and print a bunch of test files, also from json.org,
which are more complex than I'd care to try and stash into a const char array[].
Enjoy cJSON!
- Dave Gamble, Aug 2009

514
plugins/omelasticsearch/cJSON/cjson.c Normal file
View File

@ -0,0 +1,514 @@
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
/* cJSON */
/* JSON parser in C. */
#include <string.h>
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <float.h>
#include <limits.h>
#include <ctype.h>
#include "cjson.h"
static const char *ep;
const char *cJSON_GetErrorPtr() {return ep;}
static int cJSON_strcasecmp(const char *s1,const char *s2)
{
if (!s1) return (s1==s2)?0:1;if (!s2) return 1;
for(; tolower(*s1) == tolower(*s2); ++s1, ++s2) if(*s1 == 0) return 0;
return tolower(*(const unsigned char *)s1) - tolower(*(const unsigned char *)s2);
}
static void *(*cJSON_malloc)(size_t sz) = malloc;
static void (*cJSON_free)(void *ptr) = free;
static char* cJSON_strdup(const char* str)
{
size_t len;
char* copy;
len = strlen(str) + 1;
if (!(copy = (char*)cJSON_malloc(len))) return 0;
memcpy(copy,str,len);
return copy;
}
void cJSON_InitHooks(cJSON_Hooks* hooks)
{
if (!hooks) { /* Reset hooks */
cJSON_malloc = malloc;
cJSON_free = free;
return;
}
cJSON_malloc = (hooks->malloc_fn)?hooks->malloc_fn:malloc;
cJSON_free = (hooks->free_fn)?hooks->free_fn:free;
}
/* Internal constructor. */
static cJSON *cJSON_New_Item()
{
cJSON* node = (cJSON*)cJSON_malloc(sizeof(cJSON));
if (node) memset(node,0,sizeof(cJSON));
return node;
}
/* Delete a cJSON structure. */
void cJSON_Delete(cJSON *c)
{
cJSON *next;
while (c)
{
next=c->next;
if (!(c->type&cJSON_IsReference) && c->child) cJSON_Delete(c->child);
if (!(c->type&cJSON_IsReference) && c->valuestring) cJSON_free(c->valuestring);
if (c->string) cJSON_free(c->string);
cJSON_free(c);
c=next;
}
}
/* Parse the input text to generate a number, and populate the result into item. */
static const char *parse_number(cJSON *item,const char *num)
{
double n=0,sign=1,scale=0;int subscale=0,signsubscale=1;
/* Could use sscanf for this? */
if (*num=='-') sign=-1,num++; /* Has sign? */
if (*num=='0') num++; /* is zero */
if (*num>='1' && *num<='9') do n=(n*10.0)+(*num++ -'0'); while (*num>='0' && *num<='9'); /* Number? */
if (*num=='.' && num[1]>='0' && num[1]<='9') {num++; do n=(n*10.0)+(*num++ -'0'),scale--; while (*num>='0' && *num<='9');} /* Fractional part? */
if (*num=='e' || *num=='E') /* Exponent? */
{ num++;if (*num=='+') num++; else if (*num=='-') signsubscale=-1,num++; /* With sign? */
while (*num>='0' && *num<='9') subscale=(subscale*10)+(*num++ - '0'); /* Number? */
}
n=sign*n*pow(10.0,(scale+subscale*signsubscale)); /* number = +/- number.fraction * 10^+/- exponent */
item->valuedouble=n;
item->valueint=(int)n;
item->type=cJSON_Number;
return num;
}
/* Render the number nicely from the given item into a string. */
char *cJSON_print_number(cJSON *item)
{
char *str;
double d=item->valuedouble;
if (fabs(((double)item->valueint)-d)<=DBL_EPSILON && d<=INT_MAX && d>=INT_MIN)
{
str=(char*)cJSON_malloc(21); /* 2^64+1 can be represented in 21 chars. */
if (str) sprintf(str,"%d",item->valueint);
}
else
{
str=(char*)cJSON_malloc(64); /* This is a nice tradeoff. */
if (str)
{
if (fabs(floor(d)-d)<=DBL_EPSILON) sprintf(str,"%.0f",d);
else if (fabs(d)<1.0e-6 || fabs(d)>1.0e9) sprintf(str,"%e",d);
else sprintf(str,"%f",d);
}
}
return str;
}
/* Parse the input text into an unescaped cstring, and populate item. */
static const unsigned char firstByteMark[7] = { 0x00, 0x00, 0xC0, 0xE0, 0xF0, 0xF8, 0xFC };
static const char *parse_string(cJSON *item,const char *str)
{
const char *ptr=str+1;char *ptr2;char *out;int len=0;unsigned uc,uc2;
if (*str!='\"') {ep=str;return 0;} /* not a string! */
while (*ptr!='\"' && *ptr && ++len) if (*ptr++ == '\\') ptr++; /* Skip escaped quotes. */
out=(char*)cJSON_malloc(len+1); /* This is how long we need for the string, roughly. */
if (!out) return 0;
ptr=str+1;ptr2=out;
while (*ptr!='\"' && *ptr)
{
if (*ptr!='\\') *ptr2++=*ptr++;
else
{
ptr++;
switch (*ptr)
{
case 'b': *ptr2++='\b'; break;
case 'f': *ptr2++='\f'; break;
case 'n': *ptr2++='\n'; break;
case 'r': *ptr2++='\r'; break;
case 't': *ptr2++='\t'; break;
case 'u': /* transcode utf16 to utf8. */
sscanf(ptr+1,"%4x",&uc);ptr+=4; /* get the unicode char. */
if ((uc>=0xDC00 && uc<=0xDFFF) || uc==0) break; // check for invalid.
if (uc>=0xD800 && uc<=0xDBFF) // UTF16 surrogate pairs.
{
if (ptr[1]!='\\' || ptr[2]!='u') break; // missing second-half of surrogate.
sscanf(ptr+3,"%4x",&uc2);ptr+=6;
if (uc2<0xDC00 || uc2>0xDFFF) break; // invalid second-half of surrogate.
uc=0x10000 | ((uc&0x3FF)<<10) | (uc2&0x3FF);
}
len=4;if (uc<0x80) len=1;else if (uc<0x800) len=2;else if (uc<0x10000) len=3; ptr2+=len;
switch (len) {
case 4: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6;
case 3: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6;
case 2: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6;
case 1: *--ptr2 =(uc | firstByteMark[len]);
}
ptr2+=len;
break;
default: *ptr2++=*ptr; break;
}
ptr++;
}
}
*ptr2=0;
if (*ptr=='\"') ptr++;
item->valuestring=out;
item->type=cJSON_String;
return ptr;
}
/* Render the cstring provided to an escaped version that can be printed. */
static char *print_string_ptr(const char *str)
{
const char *ptr;char *ptr2,*out;int len=0;unsigned char token;
if (!str) return cJSON_strdup("");
ptr=str;while ((token=*ptr) && ++len) {if (strchr("\"\\\b\f\n\r\t",token)) len++; else if (token<32) len+=5;ptr++;}
out=(char*)cJSON_malloc(len+3);
if (!out) return 0;
ptr2=out;ptr=str;
*ptr2++='\"';
while (*ptr)
{
if ((unsigned char)*ptr>31 && *ptr!='\"' && *ptr!='\\') *ptr2++=*ptr++;
else
{
*ptr2++='\\';
switch (token=*ptr++)
{
case '\\': *ptr2++='\\'; break;
case '\"': *ptr2++='\"'; break;
case '\b': *ptr2++='b'; break;
case '\f': *ptr2++='f'; break;
case '\n': *ptr2++='n'; break;
case '\r': *ptr2++='r'; break;
case '\t': *ptr2++='t'; break;
default: sprintf(ptr2,"u%04x",token);ptr2+=5; break; /* escape and print */
}
}
}
*ptr2++='\"';*ptr2++=0;
return out;
}
/* Invote print_string_ptr (which is useful) on an item. */
static char *print_string(cJSON *item) {return print_string_ptr(item->valuestring);}
/* Predeclare these prototypes. */
static const char *parse_value(cJSON *item,const char *value);
static char *print_value(cJSON *item,int depth,int fmt);
static const char *parse_array(cJSON *item,const char *value);
static char *print_array(cJSON *item,int depth,int fmt);
static const char *parse_object(cJSON *item,const char *value);
static char *print_object(cJSON *item,int depth,int fmt);
/* Utility to jump whitespace and cr/lf */
static const char *skip(const char *in) {while (in && *in && (unsigned char)*in<=32) in++; return in;}
/* Parse an object - create a new root, and populate. */
cJSON *cJSON_Parse(const char *value)
{
cJSON *c=cJSON_New_Item();
ep=0;
if (!c) return 0; /* memory fail */
if (!parse_value(c,skip(value))) {cJSON_Delete(c);return 0;}
return c;
}
/* Render a cJSON item/entity/structure to text. */
char *cJSON_Print(cJSON *item) {return print_value(item,0,1);}
char *cJSON_PrintUnformatted(cJSON *item) {return print_value(item,0,0);}
/* Parser core - when encountering text, process appropriately. */
static const char *parse_value(cJSON *item,const char *value)
{
if (!value) return 0; /* Fail on null. */
if (!strncmp(value,"null",4)) { item->type=cJSON_NULL; return value+4; }
if (!strncmp(value,"false",5)) { item->type=cJSON_False; return value+5; }
if (!strncmp(value,"true",4)) { item->type=cJSON_True; item->valueint=1; return value+4; }
if (*value=='\"') { return parse_string(item,value); }
if (*value=='-' || (*value>='0' && *value<='9')) { return parse_number(item,value); }
if (*value=='[') { return parse_array(item,value); }
if (*value=='{') { return parse_object(item,value); }
ep=value;return 0; /* failure. */
}
/* Render a value to text. */
static char *print_value(cJSON *item,int depth,int fmt)
{
char *out=0;
if (!item) return 0;
switch ((item->type)&255)
{
case cJSON_NULL: out=cJSON_strdup("null"); break;
case cJSON_False: out=cJSON_strdup("false");break;
case cJSON_True: out=cJSON_strdup("true"); break;
case cJSON_Number: out=cJSON_print_number(item);break;
case cJSON_String: out=print_string(item);break;
case cJSON_Array: out=print_array(item,depth,fmt);break;
case cJSON_Object: out=print_object(item,depth,fmt);break;
}
return out;
}
/* Build an array from input text. */
static const char *parse_array(cJSON *item,const char *value)
{
cJSON *child;
if (*value!='[') {ep=value;return 0;} /* not an array! */
item->type=cJSON_Array;
value=skip(value+1);
if (*value==']') return value+1; /* empty array. */
item->child=child=cJSON_New_Item();
if (!item->child) return 0; /* memory fail */
value=skip(parse_value(child,skip(value))); /* skip any spacing, get the value. */
if (!value) return 0;
while (*value==',')
{
cJSON *new_item;
if (!(new_item=cJSON_New_Item())) return 0; /* memory fail */
child->next=new_item;new_item->prev=child;child=new_item;
value=skip(parse_value(child,skip(value+1)));
if (!value) return 0; /* memory fail */
}
if (*value==']') return value+1; /* end of array */
ep=value;return 0; /* malformed. */
}
/* Render an array to text */
static char *print_array(cJSON *item,int depth,int fmt)
{
char **entries;
char *out=0,*ptr,*ret;int len=5;
cJSON *child=item->child;
int numentries=0,i=0,fail=0;
/* How many entries in the array? */
while (child) numentries++,child=child->next;
/* Allocate an array to hold the values for each */
entries=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!entries) return 0;
memset(entries,0,numentries*sizeof(char*));
/* Retrieve all the results: */
child=item->child;
while (child && !fail)
{
ret=print_value(child,depth+1,fmt);
entries[i++]=ret;
if (ret) len+=strlen(ret)+2+(fmt?1:0); else fail=1;
child=child->next;
}
/* If we didn't fail, try to malloc the output string */
if (!fail) out=(char*)cJSON_malloc(len);
/* If that fails, we fail. */
if (!out) fail=1;
/* Handle failure. */
if (fail)
{
for (i=0;i<numentries;i++) if (entries[i]) cJSON_free(entries[i]);
cJSON_free(entries);
return 0;
}
/* Compose the output array. */
*out='[';
ptr=out+1;*ptr=0;
for (i=0;i<numentries;i++)
{
strcpy(ptr,entries[i]);ptr+=strlen(entries[i]);
if (i!=numentries-1) {*ptr++=',';if(fmt)*ptr++=' ';*ptr=0;}
cJSON_free(entries[i]);
}
cJSON_free(entries);
*ptr++=']';*ptr++=0;
return out;
}
/* Build an object from the text. */
static const char *parse_object(cJSON *item,const char *value)
{
cJSON *child;
if (*value!='{') {ep=value;return 0;} /* not an object! */
item->type=cJSON_Object;
value=skip(value+1);
if (*value=='}') return value+1; /* empty array. */
item->child=child=cJSON_New_Item();
if (!item->child) return 0;
value=skip(parse_string(child,skip(value)));
if (!value) return 0;
child->string=child->valuestring;child->valuestring=0;
if (*value!=':') {ep=value;return 0;} /* fail! */
value=skip(parse_value(child,skip(value+1))); /* skip any spacing, get the value. */
if (!value) return 0;
while (*value==',')
{
cJSON *new_item;
if (!(new_item=cJSON_New_Item())) return 0; /* memory fail */
child->next=new_item;new_item->prev=child;child=new_item;
value=skip(parse_string(child,skip(value+1)));
if (!value) return 0;
child->string=child->valuestring;child->valuestring=0;
if (*value!=':') {ep=value;return 0;} /* fail! */
value=skip(parse_value(child,skip(value+1))); /* skip any spacing, get the value. */
if (!value) return 0;
}
if (*value=='}') return value+1; /* end of array */
ep=value;return 0; /* malformed. */
}
/* Render an object to text. */
static char *print_object(cJSON *item,int depth,int fmt)
{
char **entries=0,**names=0;
char *out=0,*ptr,*ret,*str;int len=7,i=0,j;
cJSON *child=item->child;
int numentries=0,fail=0;
/* Count the number of entries. */
while (child) numentries++,child=child->next;
/* Allocate space for the names and the objects */
entries=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!entries) return 0;
names=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!names) {cJSON_free(entries);return 0;}
memset(entries,0,sizeof(char*)*numentries);
memset(names,0,sizeof(char*)*numentries);
/* Collect all the results into our arrays: */
child=item->child;depth++;if (fmt) len+=depth;
while (child)
{
names[i]=str=print_string_ptr(child->string);
entries[i++]=ret=print_value(child,depth,fmt);
if (str && ret) len+=strlen(ret)+strlen(str)+2+(fmt?2+depth:0); else fail=1;
child=child->next;
}
/* Try to allocate the output string */
if (!fail) out=(char*)cJSON_malloc(len);
if (!out) fail=1;
/* Handle failure */
if (fail)
{
for (i=0;i<numentries;i++) {if (names[i]) cJSON_free(names[i]);if (entries[i]) cJSON_free(entries[i]);}
cJSON_free(names);cJSON_free(entries);
return 0;
}
/* Compose the output: */
*out='{';ptr=out+1;if (fmt)*ptr++='\n';*ptr=0;
for (i=0;i<numentries;i++)
{
if (fmt) for (j=0;j<depth;j++) *ptr++='\t';
strcpy(ptr,names[i]);ptr+=strlen(names[i]);
*ptr++=':';if (fmt) *ptr++='\t';
strcpy(ptr,entries[i]);ptr+=strlen(entries[i]);
if (i!=numentries-1) *ptr++=',';
if (fmt) *ptr++='\n';*ptr=0;
cJSON_free(names[i]);cJSON_free(entries[i]);
}
cJSON_free(names);cJSON_free(entries);
if (fmt) for (i=0;i<depth-1;i++) *ptr++='\t';
*ptr++='}';*ptr++=0;
return out;
}
/* Get Array size/item / object item. */
int cJSON_GetArraySize(cJSON *array) {cJSON *c=array->child;int i=0;while(c)i++,c=c->next;return i;}
cJSON *cJSON_GetArrayItem(cJSON *array,int item) {cJSON *c=array->child; while (c && item>0) item--,c=c->next; return c;}
cJSON *cJSON_GetObjectItem(cJSON *object,const char *string) {cJSON *c=object->child; while (c && cJSON_strcasecmp(c->string,string)) c=c->next; return c;}
/* Utility for array list handling. */
static void suffix_object(cJSON *prev,cJSON *item) {prev->next=item;item->prev=prev;}
/* Utility for handling references. */
static cJSON *create_reference(cJSON *item) {cJSON *ref=cJSON_New_Item();if (!ref) return 0;memcpy(ref,item,sizeof(cJSON));ref->string=0;ref->type|=cJSON_IsReference;ref->next=ref->prev=0;return ref;}
/* Add item to array/object. */
void cJSON_AddItemToArray(cJSON *array, cJSON *item) {cJSON *c=array->child;if (!item) return; if (!c) {array->child=item;} else {while (c && c->next) c=c->next; suffix_object(c,item);}}
void cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *item) {if (!item) return; if (item->string) cJSON_free(item->string);item->string=cJSON_strdup(string);cJSON_AddItemToArray(object,item);}
void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item) {cJSON_AddItemToArray(array,create_reference(item));}
void cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJSON *item) {cJSON_AddItemToObject(object,string,create_reference(item));}
cJSON *cJSON_DetachItemFromArray(cJSON *array,int which) {cJSON *c=array->child;while (c && which>0) c=c->next,which--;if (!c) return 0;
if (c->prev) c->prev->next=c->next;if (c->next) c->next->prev=c->prev;if (c==array->child) array->child=c->next;c->prev=c->next=0;return c;}
void cJSON_DeleteItemFromArray(cJSON *array,int which) {cJSON_Delete(cJSON_DetachItemFromArray(array,which));}
cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string) {int i=0;cJSON *c=object->child;while (c && cJSON_strcasecmp(c->string,string)) i++,c=c->next;if (c) return cJSON_DetachItemFromArray(object,i);return 0;}
void cJSON_DeleteItemFromObject(cJSON *object,const char *string) {cJSON_Delete(cJSON_DetachItemFromObject(object,string));}
/* Replace array/object items with new ones. */
void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem) {cJSON *c=array->child;while (c && which>0) c=c->next,which--;if (!c) return;
newitem->next=c->next;newitem->prev=c->prev;if (newitem->next) newitem->next->prev=newitem;
if (c==array->child) array->child=newitem; else newitem->prev->next=newitem;c->next=c->prev=0;cJSON_Delete(c);}
void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON *newitem){int i=0;cJSON *c=object->child;while(c && cJSON_strcasecmp(c->string,string))i++,c=c->next;if(c){newitem->string=cJSON_strdup(string);cJSON_ReplaceItemInArray(object,i,newitem);}}
/* Create basic types: */
cJSON *cJSON_CreateNull() {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_NULL;return item;}
cJSON *cJSON_CreateTrue() {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_True;return item;}
cJSON *cJSON_CreateFalse() {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_False;return item;}
cJSON *cJSON_CreateBool(int b) {cJSON *item=cJSON_New_Item();if(item)item->type=b?cJSON_True:cJSON_False;return item;}
cJSON *cJSON_CreateNumber(double num) {cJSON *item=cJSON_New_Item();if(item){item->type=cJSON_Number;item->valuedouble=num;item->valueint=(int)num;}return item;}
cJSON *cJSON_CreateString(const char *string) {cJSON *item=cJSON_New_Item();if(item){item->type=cJSON_String;item->valuestring=cJSON_strdup(string);}return item;}
cJSON *cJSON_CreateArray() {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_Array;return item;}
cJSON *cJSON_CreateObject() {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_Object;return item;}
/* Create Arrays: */
cJSON *cJSON_CreateIntArray(int *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateFloatArray(float *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateDoubleArray(double *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateStringArray(const char **strings,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateString(strings[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}

130
plugins/omelasticsearch/cJSON/cjson.h Normal file
View File

@ -0,0 +1,130 @@
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
#ifndef cJSON__h
#define cJSON__h
#ifdef __cplusplus
extern "C"
{
#endif
/* cJSON Types: */
#define cJSON_False 0
#define cJSON_True 1
#define cJSON_NULL 2
#define cJSON_Number 3
#define cJSON_String 4
#define cJSON_Array 5
#define cJSON_Object 6
#define cJSON_IsReference 256
/* The cJSON structure: */
typedef struct cJSON {
struct cJSON *next,*prev; /* next/prev allow you to walk array/object chains. Alternatively, use GetArraySize/GetArrayItem/GetObjectItem */
struct cJSON *child; /* An array or object item will have a child pointer pointing to a chain of the items in the array/object. */
int type; /* The type of the item, as above. */
char *valuestring; /* The item's string, if type==cJSON_String */
int valueint; /* The item's number, if type==cJSON_Number */
double valuedouble; /* The item's number, if type==cJSON_Number */
char *string; /* The item's name string, if this item is the child of, or is in the list of subitems of an object. */
} cJSON;
typedef struct cJSON_Hooks {
void *(*malloc_fn)(size_t sz);
void (*free_fn)(void *ptr);
} cJSON_Hooks;
/* Supply malloc, realloc and free functions to cJSON */
extern void cJSON_InitHooks(cJSON_Hooks* hooks);
/* Supply a block of JSON, and this returns a cJSON object you can interrogate. Call cJSON_Delete when finished. */
extern cJSON *cJSON_Parse(const char *value);
/* Render a cJSON entity to text for transfer/storage. Free the char* when finished. */
extern char *cJSON_Print(cJSON *item);
/* Render a cJSON entity to text for transfer/storage without any formatting. Free the char* when finished. */
extern char *cJSON_PrintUnformatted(cJSON *item);
/* Delete a cJSON entity and all subentities. */
extern void cJSON_Delete(cJSON *c);
/* Returns the number of items in an array (or object). */
extern int cJSON_GetArraySize(cJSON *array);
/* Retrieve item number "item" from array "array". Returns NULL if unsuccessful. */
extern cJSON *cJSON_GetArrayItem(cJSON *array,int item);
/* Get item "string" from object. Case insensitive. */
extern cJSON *cJSON_GetObjectItem(cJSON *object,const char *string);
/* For analysing failed parses. This returns a pointer to the parse error. You'll probably need to look a few chars back to make sense of it. Defined when cJSON_Parse() returns 0. 0 when cJSON_Parse() succeeds. */
extern const char *cJSON_GetErrorPtr();
/* These calls create a cJSON item of the appropriate type. */
extern cJSON *cJSON_CreateNull();
extern cJSON *cJSON_CreateTrue();
extern cJSON *cJSON_CreateFalse();
extern cJSON *cJSON_CreateBool(int b);
extern cJSON *cJSON_CreateNumber(double num);
extern cJSON *cJSON_CreateString(const char *string);
extern cJSON *cJSON_CreateArray();
extern cJSON *cJSON_CreateObject();
/* These utilities create an Array of count items. */
extern cJSON *cJSON_CreateIntArray(int *numbers,int count);
extern cJSON *cJSON_CreateFloatArray(float *numbers,int count);
extern cJSON *cJSON_CreateDoubleArray(double *numbers,int count);
extern cJSON *cJSON_CreateStringArray(const char **strings,int count);
/* Append item to the specified array/object. */
extern void cJSON_AddItemToArray(cJSON *array, cJSON *item);
extern void cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *item);
/* Append reference to item to the specified array/object. Use this when you want to add an existing cJSON to a new cJSON, but don't want to corrupt your existing cJSON. */
extern void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item);
extern void cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJSON *item);
/* Remove/Detatch items from Arrays/Objects. */
extern cJSON *cJSON_DetachItemFromArray(cJSON *array,int which);
extern void cJSON_DeleteItemFromArray(cJSON *array,int which);
extern cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string);
extern void cJSON_DeleteItemFromObject(cJSON *object,const char *string);
/* Update array items. */
extern void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem);
extern void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON *newitem);
/* rger: added helpers */
char *cJSON_print_number(cJSON *item);
#define cJSON_AddNullToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateNull())
#define cJSON_AddTrueToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateTrue())
#define cJSON_AddFalseToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateFalse())
#define cJSON_AddNumberToObject(object,name,n) cJSON_AddItemToObject(object, name, cJSON_CreateNumber(n))
#define cJSON_AddStringToObject(object,name,s) cJSON_AddItemToObject(object, name, cJSON_CreateString(s))
#ifdef __cplusplus
}
#endif
#endif

156
plugins/omelasticsearch/cJSON/test.c Normal file
View File

@ -0,0 +1,156 @@
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
#include <stdio.h>
#include <stdlib.h>
#include "cJSON.h"
/* Parse text to JSON, then render back to text, and print! */
void doit(char *text)
{
char *out;cJSON *json;
json=cJSON_Parse(text);
if (!json) {printf("Error before: [%s]\n",cJSON_GetErrorPtr());}
else
{
out=cJSON_Print(json);
cJSON_Delete(json);
printf("%s\n",out);
free(out);
}
}
/* Read a file, parse, render back, etc. */
void dofile(char *filename)
{
FILE *f=fopen(filename,"rb");fseek(f,0,SEEK_END);long len=ftell(f);fseek(f,0,SEEK_SET);
char *data=malloc(len+1);fread(data,1,len,f);fclose(f);
doit(data);
free(data);
}
/* Used by some code below as an example datatype. */
struct record {const char *precision;double lat,lon;const char *address,*city,*state,*zip,*country; };
/* Create a bunch of objects as demonstration. */
void create_objects()
{
cJSON *root,*fmt,*img,*thm,*fld;char *out;int i; /* declare a few. */
/* Here we construct some JSON standards, from the JSON site. */
/* Our "Video" datatype: */
root=cJSON_CreateObject();
cJSON_AddItemToObject(root, "name", cJSON_CreateString("Jack (\"Bee\") Nimble"));
cJSON_AddItemToObject(root, "format", fmt=cJSON_CreateObject());
cJSON_AddStringToObject(fmt,"type", "rect");
cJSON_AddNumberToObject(fmt,"width", 1920);
cJSON_AddNumberToObject(fmt,"height", 1080);
cJSON_AddFalseToObject (fmt,"interlace");
cJSON_AddNumberToObject(fmt,"frame rate", 24);
out=cJSON_Print(root); cJSON_Delete(root); printf("%s\n",out); free(out); /* Print to text, Delete the cJSON, print it, release the string.
/* Our "days of the week" array: */
const char *strings[7]={"Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"};
root=cJSON_CreateStringArray(strings,7);
out=cJSON_Print(root); cJSON_Delete(root); printf("%s\n",out); free(out);
/* Our matrix: */
int numbers[3][3]={{0,-1,0},{1,0,0},{0,0,1}};
root=cJSON_CreateArray();
for (i=0;i<3;i++) cJSON_AddItemToArray(root,cJSON_CreateIntArray(numbers[i],3));
/* cJSON_ReplaceItemInArray(root,1,cJSON_CreateString("Replacement")); */
out=cJSON_Print(root); cJSON_Delete(root); printf("%s\n",out); free(out);
/* Our "gallery" item: */
int ids[4]={116,943,234,38793};
root=cJSON_CreateObject();
cJSON_AddItemToObject(root, "Image", img=cJSON_CreateObject());
cJSON_AddNumberToObject(img,"Width",800);
cJSON_AddNumberToObject(img,"Height",600);
cJSON_AddStringToObject(img,"Title","View from 15th Floor");
cJSON_AddItemToObject(img, "Thumbnail", thm=cJSON_CreateObject());
cJSON_AddStringToObject(thm, "Url", "http:/*www.example.com/image/481989943");
cJSON_AddNumberToObject(thm,"Height",125);
cJSON_AddStringToObject(thm,"Width","100");
cJSON_AddItemToObject(img,"IDs", cJSON_CreateIntArray(ids,4));
out=cJSON_Print(root); cJSON_Delete(root); printf("%s\n",out); free(out);
/* Our array of "records": */
struct record fields[2]={
{"zip",37.7668,-1.223959e+2,"","SAN FRANCISCO","CA","94107","US"},
{"zip",37.371991,-1.22026e+2,"","SUNNYVALE","CA","94085","US"}};
root=cJSON_CreateArray();
for (i=0;i<2;i++)
{
cJSON_AddItemToArray(root,fld=cJSON_CreateObject());
cJSON_AddStringToObject(fld, "precision", fields[i].precision);
cJSON_AddNumberToObject(fld, "Latitude", fields[i].lat);
cJSON_AddNumberToObject(fld, "Longitude", fields[i].lon);
cJSON_AddStringToObject(fld, "Address", fields[i].address);
cJSON_AddStringToObject(fld, "City", fields[i].city);
cJSON_AddStringToObject(fld, "State", fields[i].state);
cJSON_AddStringToObject(fld, "Zip", fields[i].zip);
cJSON_AddStringToObject(fld, "Country", fields[i].country);
}
/* cJSON_ReplaceItemInObject(cJSON_GetArrayItem(root,1),"City",cJSON_CreateIntArray(ids,4)); */
out=cJSON_Print(root); cJSON_Delete(root); printf("%s\n",out); free(out);
}
int main (int argc, const char * argv[]) {
/* a bunch of json: */
char text1[]="{\n\"name\": \"Jack (\\\"Bee\\\") Nimble\", \n\"format\": {\"type\": \"rect\", \n\"width\": 1920, \n\"height\": 1080, \n\"interlace\": false,\"frame rate\": 24\n}\n}";
char text2[]="[\"Sunday\", \"Monday\", \"Tuesday\", \"Wednesday\", \"Thursday\", \"Friday\", \"Saturday\"]";
char text3[]="[\n [0, -1, 0],\n [1, 0, 0],\n [0, 0, 1]\n ]\n";
char text4[]="{\n \"Image\": {\n \"Width\": 800,\n \"Height\": 600,\n \"Title\": \"View from 15th Floor\",\n \"Thumbnail\": {\n \"Url\": \"http:/*www.example.com/image/481989943\",\n \"Height\": 125,\n \"Width\": \"100\"\n },\n \"IDs\": [116, 943, 234, 38793]\n }\n }";
char text5[]="[\n {\n \"precision\": \"zip\",\n \"Latitude\": 37.7668,\n \"Longitude\": -122.3959,\n \"Address\": \"\",\n \"City\": \"SAN FRANCISCO\",\n \"State\": \"CA\",\n \"Zip\": \"94107\",\n \"Country\": \"US\"\n },\n {\n \"precision\": \"zip\",\n \"Latitude\": 37.371991,\n \"Longitude\": -122.026020,\n \"Address\": \"\",\n \"City\": \"SUNNYVALE\",\n \"State\": \"CA\",\n \"Zip\": \"94085\",\n \"Country\": \"US\"\n }\n ]";
/* Process each json textblock by parsing, then rebuilding: */
doit(text1);
doit(text2);
doit(text3);
doit(text4);
doit(text5);
/* Parse standard testfiles:
/* dofile("../../tests/test1"); */
/* dofile("../../tests/test2"); */
/* dofile("../../tests/test3"); */
/* dofile("../../tests/test4"); */
/* dofile("../../tests/test5"); */
/* Now some samplecode for building objects concisely: */
create_objects();
return 0;
}

22
plugins/omelasticsearch/cJSON/tests/test1 Normal file
View File

@ -0,0 +1,22 @@
{
"glossary": {
"title": "example glossary",
"GlossDiv": {
"title": "S",
"GlossList": {
"GlossEntry": {
"ID": "SGML",
"SortAs": "SGML",
"GlossTerm": "Standard Generalized Markup Language",
"Acronym": "SGML",
"Abbrev": "ISO 8879:1986",
"GlossDef": {
"para": "A meta-markup language, used to create markup languages such as DocBook.",
"GlossSeeAlso": ["GML", "XML"]
},
"GlossSee": "markup"
}
}
}
}
}

11
plugins/omelasticsearch/cJSON/tests/test2 Normal file
View File

@ -0,0 +1,11 @@
{"menu": {
"id": "file",
"value": "File",
"popup": {
"menuitem": [
{"value": "New", "onclick": "CreateNewDoc()"},
{"value": "Open", "onclick": "OpenDoc()"},
{"value": "Close", "onclick": "CloseDoc()"}
]
}
}}

26
plugins/omelasticsearch/cJSON/tests/test3 Normal file
View File

@ -0,0 +1,26 @@
{"widget": {
"debug": "on",
"window": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
},
"image": {
"src": "Images/Sun.png",
"name": "sun1",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"
},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"name": "text1",
"hOffset": 250,
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}}

88
plugins/omelasticsearch/cJSON/tests/test4 Normal file
View File

@ -0,0 +1,88 @@
{"web-app": {
"servlet": [
{
"servlet-name": "cofaxCDS",
"servlet-class": "org.cofax.cds.CDSServlet",
"init-param": {
"configGlossary:installationAt": "Philadelphia, PA",
"configGlossary:adminEmail": "ksm@pobox.com",
"configGlossary:poweredBy": "Cofax",
"configGlossary:poweredByIcon": "/images/cofax.gif",
"configGlossary:staticPath": "/content/static",
"templateProcessorClass": "org.cofax.WysiwygTemplate",
"templateLoaderClass": "org.cofax.FilesTemplateLoader",
"templatePath": "templates",
"templateOverridePath": "",
"defaultListTemplate": "listTemplate.htm",
"defaultFileTemplate": "articleTemplate.htm",
"useJSP": false,
"jspListTemplate": "listTemplate.jsp",
"jspFileTemplate": "articleTemplate.jsp",
"cachePackageTagsTrack": 200,
"cachePackageTagsStore": 200,
"cachePackageTagsRefresh": 60,
"cacheTemplatesTrack": 100,
"cacheTemplatesStore": 50,
"cacheTemplatesRefresh": 15,
"cachePagesTrack": 200,
"cachePagesStore": 100,
"cachePagesRefresh": 10,
"cachePagesDirtyRead": 10,
"searchEngineListTemplate": "forSearchEnginesList.htm",
"searchEngineFileTemplate": "forSearchEngines.htm",
"searchEngineRobotsDb": "WEB-INF/robots.db",
"useDataStore": true,
"dataStoreClass": "org.cofax.SqlDataStore",
"redirectionClass": "org.cofax.SqlRedirection",
"dataStoreName": "cofax",
"dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
"dataStoreUrl": "jdbc:microsoft:sqlserver://LOCALHOST:1433;DatabaseName=goon",
"dataStoreUser": "sa",
"dataStorePassword": "dataStoreTestQuery",
"dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
"dataStoreLogFile": "/usr/local/tomcat/logs/datastore.log",
"dataStoreInitConns": 10,
"dataStoreMaxConns": 100,
"dataStoreConnUsageLimit": 100,
"dataStoreLogLevel": "debug",
"maxUrlLength": 500}},
{
"servlet-name": "cofaxEmail",
"servlet-class": "org.cofax.cds.EmailServlet",
"init-param": {
"mailHost": "mail1",
"mailHostOverride": "mail2"}},
{
"servlet-name": "cofaxAdmin",
"servlet-class": "org.cofax.cds.AdminServlet"},
{
"servlet-name": "fileServlet",
"servlet-class": "org.cofax.cds.FileServlet"},
{
"servlet-name": "cofaxTools",
"servlet-class": "org.cofax.cms.CofaxToolsServlet",
"init-param": {
"templatePath": "toolstemplates/",
"log": 1,
"logLocation": "/usr/local/tomcat/logs/CofaxTools.log",
"logMaxSize": "",
"dataLog": 1,
"dataLogLocation": "/usr/local/tomcat/logs/dataLog.log",
"dataLogMaxSize": "",
"removePageCache": "/content/admin/remove?cache=pages&id=",
"removeTemplateCache": "/content/admin/remove?cache=templates&id=",
"fileTransferFolder": "/usr/local/tomcat/webapps/content/fileTransferFolder",
"lookInContext": 1,
"adminGroupID": 4,
"betaServer": true}}],
"servlet-mapping": {
"cofaxCDS": "/",
"cofaxEmail": "/cofaxutil/aemail/*",
"cofaxAdmin": "/admin/*",
"fileServlet": "/static/*",
"cofaxTools": "/tools/*"},
"taglib": {
"taglib-uri": "cofax.tld",
"taglib-location": "/WEB-INF/tlds/cofax.tld"}}}

27
plugins/omelasticsearch/cJSON/tests/test5 Normal file
View File

@ -0,0 +1,27 @@
{"menu": {
"header": "SVG Viewer",
"items": [
{"id": "Open"},
{"id": "OpenNew", "label": "Open New"},
null,
{"id": "ZoomIn", "label": "Zoom In"},
{"id": "ZoomOut", "label": "Zoom Out"},
{"id": "OriginalView", "label": "Original View"},
null,
{"id": "Quality"},
{"id": "Pause"},
{"id": "Mute"},
null,
{"id": "Find", "label": "Find..."},
{"id": "FindAgain", "label": "Find Again"},
{"id": "Copy"},
{"id": "CopyAgain", "label": "Copy Again"},
{"id": "CopySVG", "label": "Copy SVG"},
{"id": "ViewSVG", "label": "View SVG"},
{"id": "ViewSource", "label": "View Source"},
{"id": "SaveAs", "label": "Save As"},
null,
{"id": "Help"},
{"id": "About", "label": "About Adobe CVG Viewer..."}
]
}}

424
plugins/omelasticsearch/omelasticsearch.c
View File

@ -11,11 +11,11 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
* -or-
* see COPYING.ASL20 in the source distribution
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@ -34,6 +34,10 @@
#include <signal.h>
#include <errno.h>
#include <time.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "cJSON/cjson.h"
#include "conf.h"
#include "syslogd-types.h"
#include "srUtils.h"
@ -54,18 +58,19 @@ DEFobjCurrIf(errmsg)
DEFobjCurrIf(statsobj)
statsobj_t *indexStats;
STATSCOUNTER_DEF(indexConFail, mutIndexConFail)
STATSCOUNTER_DEF(indexSubmit, mutIndexSubmit)
STATSCOUNTER_DEF(indexFailed, mutIndexFailed)
STATSCOUNTER_DEF(indexSuccess, mutIndexSuccess)
STATSCOUNTER_DEF(indexHTTPFail, mutIndexHTTPFail)
STATSCOUNTER_DEF(indexESFail, mutIndexESFail)
/* REST API for elasticsearch hits this URL:
* http://<hostName>:<restPort>/<searchIndex>/<searchType>
*/
typedef struct curl_slist HEADER;
typedef struct _instanceData {
uchar *server;
int port;
int replyLen;
int fdErrFile; /* error file fd or -1 if not open */
uchar *server;
uchar *uid;
uchar *pwd;
uchar *searchIndex;
@ -73,9 +78,14 @@ typedef struct _instanceData {
uchar *parent;
uchar *tplName;
uchar *timeout;
uchar *bulkId;
uchar *restURL; /* last used URL for error reporting */
uchar *errorFile;
char *reply;
sbool dynSrchIdx;
sbool dynSrchType;
sbool dynParent;
sbool dynBulkId;
sbool bulkmode;
sbool asyncRepl;
struct {
@ -104,7 +114,10 @@ static struct cnfparamdescr actpdescr[] = {
{ "bulkmode", eCmdHdlrBinary, 0 },
{ "asyncrepl", eCmdHdlrBinary, 0 },
{ "timeout", eCmdHdlrGetWord, 0 },
{ "template", eCmdHdlrGetWord, 1 }
{ "errorfile", eCmdHdlrGetWord, 0 },
{ "template", eCmdHdlrGetWord, 1 },
{ "dynbulkid", eCmdHdlrBinary, 0 },
{ "bulkid", eCmdHdlrGetWord, 0 },
};
static struct cnfparamblk actpblk =
{ CNFPARAMBLK_VERSION,
@ -114,6 +127,8 @@ static struct cnfparamblk actpblk =
BEGINcreateInstance
CODESTARTcreateInstance
pData->restURL = NULL;
pData->fdErrFile = -1;
ENDcreateInstance
BEGINisCompatibleWithFeature
@ -132,6 +147,8 @@ CODESTARTfreeInstance
curl_easy_cleanup(pData->curlHandle);
pData->curlHandle = NULL;
}
if(pData->fdErrFile != -1)
close(pData->fdErrFile);
free(pData->server);
free(pData->uid);
free(pData->pwd);
@ -139,6 +156,10 @@ CODESTARTfreeInstance
free(pData->searchType);
free(pData->parent);
free(pData->tplName);
free(pData->timeout);
free(pData->restURL);
free(pData->errorFile);
free(pData->bulkId);
ENDfreeInstance
BEGINdbgPrintInstInfo
@ -158,6 +179,10 @@ CODESTARTdbgPrintInstInfo
dbgprintf("\tdynamic parent=%d\n", pData->dynParent);
dbgprintf("\tasync replication=%d\n", pData->asyncRepl);
dbgprintf("\tbulkmode=%d\n", pData->bulkmode);
dbgprintf("\terrorfile='%s'\n", pData->errorFile == NULL ?
(uchar*)"(not configured)" : pData->errorFile);
dbgprintf("\tdynbulkid=%d\n", pData->dynBulkId);
dbgprintf("\tbulkid='%s'\n", pData->bulkId);
ENDdbgPrintInstInfo
@ -198,16 +223,26 @@ checkConn(instanceData *pData)
DBGPRINTF("omelasticsearch: checkConn() curl_easy_init() failed\n");
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
/* Bodypart of request not needed, so set curl opt to nobody and httpget, otherwise lib-curl could sigsegv */
curl_easy_setopt(curl, CURLOPT_HTTPGET, TRUE);
curl_easy_setopt(curl, CURLOPT_NOBODY, TRUE);
/* Only enable for debugging
curl_easy_setopt(curl, CURLOPT_VERBOSE, TRUE); */
cstr = es_str2cstr(url, NULL);
curl_easy_setopt(curl, CURLOPT_URL, cstr);
free(cstr);
pData->reply = NULL;
pData->replyLen = 0;
curl_easy_setopt(curl, CURLOPT_WRITEDATA, pData);
res = curl_easy_perform(curl);
if(res != CURLE_OK) {
DBGPRINTF("omelasticsearch: checkConn() curl_easy_perform() "
"failed: %s\n", curl_easy_strerror(res));
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
free(pData->reply);
DBGPRINTF("omelasticsearch: checkConn() completed with success\n");
finalize_it:
@ -227,7 +262,8 @@ ENDtryResume
/* get the current index and type for this message */
static inline void
getIndexTypeAndParent(instanceData *pData, uchar **tpls,
uchar **srchIndex, uchar **srchType, uchar **parent)
uchar **srchIndex, uchar **srchType, uchar **parent,
uchar **bulkId)
{
if(pData->dynSrchIdx) {
*srchIndex = tpls[1];
@ -235,15 +271,27 @@ getIndexTypeAndParent(instanceData *pData, uchar **tpls,
*srchType = tpls[2];
if(pData->dynParent) {
*parent = tpls[3];
if(pData->dynBulkId) {
*bulkId = tpls[4];
}
} else {
*parent = pData->parent;
if(pData->dynBulkId) {
*bulkId = tpls[3];
}
}
} else {
*srchType = pData->searchType;
if(pData->dynParent) {
*parent = tpls[2];
if(pData->dynBulkId) {
*bulkId = tpls[3];
}
} else {
*parent = pData->parent;
if(pData->dynBulkId) {
*bulkId = tpls[2];
}
}
}
} else {
@ -252,15 +300,27 @@ getIndexTypeAndParent(instanceData *pData, uchar **tpls,
*srchType = tpls[1];
if(pData->dynParent) {
*parent = tpls[2];
if(pData->dynBulkId) {
*bulkId = tpls[3];
}
} else {
*parent = pData->parent;
if(pData->dynBulkId) {
*bulkId = tpls[2];
}
}
} else {
*srchType = pData->searchType;
if(pData->dynParent) {
*parent = tpls[1];
if(pData->dynBulkId) {
*bulkId = tpls[2];
}
} else {
*parent = pData->parent;
if(pData->dynBulkId) {
*bulkId = tpls[1];
}
}
}
}
@ -271,10 +331,10 @@ static rsRetVal
setCurlURL(instanceData *pData, uchar **tpls)
{
char authBuf[1024];
char *restURL;
uchar *searchIndex;
uchar *searchType;
uchar *parent;
uchar *bulkId;
es_str_t *url;
int rLocal;
int r;
@ -286,7 +346,7 @@ setCurlURL(instanceData *pData, uchar **tpls)
r = es_addBuf(&url, "_bulk", sizeof("_bulk")-1);
parent = NULL;
} else {
getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent);
getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent, &bulkId);
r = es_addBuf(&url, (char*)searchIndex, ustrlen(searchIndex));
if(r == 0) r = es_addChar(&url, '/');
if(r == 0) r = es_addBuf(&url, (char*)searchType, ustrlen(searchType));
@ -305,11 +365,12 @@ setCurlURL(instanceData *pData, uchar **tpls)
if(r == 0) r = es_addBuf(&url, "parent=", sizeof("parent=")-1);
if(r == 0) r = es_addBuf(&url, (char*)parent, ustrlen(parent));
}
restURL = es_str2cstr(url, NULL);
curl_easy_setopt(pData->curlHandle, CURLOPT_URL, restURL);
free(pData->restURL);
pData->restURL = (uchar*)es_str2cstr(url, NULL);
curl_easy_setopt(pData->curlHandle, CURLOPT_URL, pData->restURL);
es_deleteStr(url);
DBGPRINTF("omelasticsearch: using REST URL: '%s'\n", restURL);
free(restURL);
DBGPRINTF("omelasticsearch: using REST URL: '%s'\n", pData->restURL);
if(pData->uid != NULL) {
rLocal = snprintf(authBuf, sizeof(authBuf), "%s:%s", pData->uid,
@ -320,7 +381,7 @@ setCurlURL(instanceData *pData, uchar **tpls)
rLocal);
ABORT_FINALIZE(RS_RET_ERR);
}
curl_easy_setopt(pData->curlHandle, CURLOPT_USERPWD, authBuf);
curl_easy_setopt(pData->curlHandle, CURLOPT_USERPWD, authBuf);
curl_easy_setopt(pData->curlHandle, CURLOPT_PROXYAUTH, CURLAUTH_ANY);
}
finalize_it:
@ -340,16 +401,15 @@ buildBatch(instanceData *pData, uchar *message, uchar **tpls)
uchar *searchIndex;
uchar *searchType;
uchar *parent;
uchar *bulkId = NULL;
DEFiRet;
# define META_STRT "{\"index\":{\"_index\": \""
# define META_TYPE "\",\"_type\":\""
# define META_PARENT "\",\"_parent\":\""
# define META_ID "\", \"_id\":\""
# define META_END "\"}}\n"
getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent);
dbgprintf("AAA: searchIndex: '%s'\n", searchIndex);
dbgprintf("AAA: searchType: '%s'\n", searchType);
dbgprintf("AAA: parent: '%s'\n", parent);
getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent, &bulkId);
r = es_addBuf(&pData->batch.data, META_STRT, sizeof(META_STRT)-1);
if(r == 0) r = es_addBuf(&pData->batch.data, (char*)searchIndex,
ustrlen(searchIndex));
@ -360,6 +420,10 @@ dbgprintf("AAA: parent: '%s'\n", parent);
if(r == 0) r = es_addBuf(&pData->batch.data, META_PARENT, sizeof(META_PARENT)-1);
if(r == 0) r = es_addBuf(&pData->batch.data, (char*)parent, ustrlen(parent));
}
if(bulkId != NULL) {
if(r == 0) r = es_addBuf(&pData->batch.data, META_ID, sizeof(META_ID)-1);
if(r == 0) r = es_addBuf(&pData->batch.data, (char*)bulkId, ustrlen(bulkId));
}
if(r == 0) r = es_addBuf(&pData->batch.data, META_END, sizeof(META_END)-1);
if(r == 0) r = es_addBuf(&pData->batch.data, (char*)message, length);
if(r == 0) r = es_addBuf(&pData->batch.data, "\n", sizeof("\n")-1);
@ -373,37 +437,192 @@ finalize_it:
RETiRet;
}
static rsRetVal
curlPost(instanceData *instance, uchar *message, int msglen, uchar **tpls)
/* write data error request/replies to separate error file
* Note: we open the file but never close it before exit. If it
* needs to be closed, HUP must be sent.
*/
static inline rsRetVal
writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg)
{
CURLcode code;
CURL *curl = instance->curlHandle;
char *rendered = NULL;
cJSON *errRoot;
cJSON *req;
cJSON *replyRoot = *pReplyRoot;
size_t toWrite;
ssize_t wrRet;
char errStr[1024];
DEFiRet;
if(instance->dynSrchIdx || instance->dynSrchType || instance->dynParent)
CHKiRet(setCurlURL(instance, tpls));
if(pData->errorFile == NULL) {
DBGPRINTF("omelasticsearch: no local error logger defined - "
"ignoring ES error information\n");
FINALIZE;
}
curl_easy_setopt(curl, CURLOPT_WRITEDATA, (char *)message);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, (char *)message);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, msglen);
dbgprintf("omelasticsearch: do curl_easy_perform()\n");
if(pData->fdErrFile == -1) {
pData->fdErrFile = open((char*)pData->errorFile,
O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE|O_CLOEXEC,
S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP);
if(pData->fdErrFile == -1) {
rs_strerror_r(errno, errStr, sizeof(errStr));
DBGPRINTF("omelasticsearch: error opening error file: %s\n", errStr);
ABORT_FINALIZE(RS_RET_ERR);
}
}
if((req=cJSON_CreateObject()) == NULL) ABORT_FINALIZE(RS_RET_ERR);
cJSON_AddItemToObject(req, "url", cJSON_CreateString((char*)pData->restURL));
cJSON_AddItemToObject(req, "postdata", cJSON_CreateString((char*)reqmsg));
if((errRoot=cJSON_CreateObject()) == NULL) ABORT_FINALIZE(RS_RET_ERR);
cJSON_AddItemToObject(errRoot, "request", req);
cJSON_AddItemToObject(errRoot, "reply", replyRoot);
rendered = cJSON_Print(errRoot);
/* we do not do real error-handling on the err file, as this finally complicates
* things way to much.
*/
DBGPRINTF("omelasticsearch: error record: '%s'\n", rendered);
toWrite = strlen(rendered);
wrRet = write(pData->fdErrFile, rendered, toWrite);
if(wrRet != (ssize_t) toWrite) {
DBGPRINTF("omelasticsearch: error %d writing error file, write returns %lld\n",
errno, (long long) wrRet);
}
cJSON_Delete(errRoot);
*pReplyRoot = NULL; /* tell caller not to delete once again! */
finalize_it:
free(rendered);
RETiRet;
}
static inline rsRetVal
checkResultBulkmode(instanceData *pData, cJSON *root)
{
int i;
int numitems;
cJSON *items;
cJSON *item;
cJSON *create;
cJSON *ok;
DEFiRet;
items = cJSON_GetObjectItem(root, "items");
if(items == NULL || items->type != cJSON_Array) {
DBGPRINTF("omelasticsearch: error in elasticsearch reply: "
"bulkmode insert does not return array, reply is: %s\n",
pData->reply);
ABORT_FINALIZE(RS_RET_DATAFAIL);
}
numitems = cJSON_GetArraySize(items);
DBGPRINTF("omelasticsearch: %d items in reply\n", numitems);
for(i = 0 ; i < numitems ; ++i) {
item = cJSON_GetArrayItem(items, i);
if(item == NULL) {
DBGPRINTF("omelasticsearch: error in elasticsearch reply: "
"cannot obtain reply array item %d\n", i);
ABORT_FINALIZE(RS_RET_DATAFAIL);
}
create = cJSON_GetObjectItem(item, "create");
if(create == NULL || create->type != cJSON_Object) {
DBGPRINTF("omelasticsearch: error in elasticsearch reply: "
"cannot obtain 'create' item for #%d\n", i);
ABORT_FINALIZE(RS_RET_DATAFAIL);
}
ok = cJSON_GetObjectItem(create, "ok");
if(ok == NULL || ok->type != cJSON_True) {
DBGPRINTF("omelasticsearch: error in elasticsearch reply: "
"item %d, prop ok (%p) not ok\n", i, ok);
ABORT_FINALIZE(RS_RET_DATAFAIL);
}
}
finalize_it:
if(iRet != RS_RET_OK) {
STATSCOUNTER_INC(indexESFail, mutIndexESFail);
}
RETiRet;
}
static inline rsRetVal
checkResult(instanceData *pData, uchar *reqmsg)
{
cJSON *root;
cJSON *ok;
DEFiRet;
root = cJSON_Parse(pData->reply);
if(root == NULL) {
DBGPRINTF("omelasticsearch: could not parse JSON result \n");
ABORT_FINALIZE(RS_RET_ERR);
}
if(pData->bulkmode) {
iRet = checkResultBulkmode(pData, root);
} else {
ok = cJSON_GetObjectItem(root, "ok");
if(ok == NULL || ok->type != cJSON_True) {
iRet = RS_RET_DATAFAIL;
}
}
/* Note: we ignore errors writing the error file, as we cannot handle
* these in any case.
*/
if(iRet == RS_RET_DATAFAIL) {
writeDataError(pData, &root, reqmsg);
iRet = RS_RET_OK; /* we have handled the problem! */
}
finalize_it:
if(root != NULL)
cJSON_Delete(root);
RETiRet;
}
static rsRetVal
curlPost(instanceData *pData, uchar *message, int msglen, uchar **tpls)
{
CURLcode code;
CURL *curl = pData->curlHandle;
DEFiRet;
pData->reply = NULL;
pData->replyLen = 0;
if(pData->dynSrchIdx || pData->dynSrchType || pData->dynParent)
CHKiRet(setCurlURL(pData, tpls));
curl_easy_setopt(curl, CURLOPT_WRITEDATA, pData);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, (char *)message);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, msglen);
code = curl_easy_perform(curl);
DBGPRINTF("omelasticsearch: curl_easy_perform() returned %lld\n", (long long) code);
switch (code) {
case CURLE_COULDNT_RESOLVE_HOST:
case CURLE_COULDNT_RESOLVE_PROXY:
case CURLE_COULDNT_CONNECT:
case CURLE_WRITE_ERROR:
STATSCOUNTER_INC(indexConFail, mutIndexConFail);
STATSCOUNTER_INC(indexHTTPFail, mutIndexHTTPFail);
DBGPRINTF("omelasticsearch: we are suspending ourselfs due "
"to failure %lld of curl_easy_perform()\n",
(long long) code);
return RS_RET_SUSPENDED;
ABORT_FINALIZE(RS_RET_SUSPENDED);
default:
STATSCOUNTER_INC(indexSubmit, mutIndexSubmit);
return RS_RET_OK;
break;
}
DBGPRINTF("omelasticsearch: pData replyLen = '%d'\n", pData->replyLen);
if (pData->replyLen > 0) {
pData->reply[pData->replyLen] = '\0'; /* Append 0 Byte if replyLen is above 0 - byte has been reserved in malloc */
}
DBGPRINTF("omelasticsearch: pData reply: '%s'\n", pData->reply);
CHKiRet(checkResult(pData, message));
finalize_it:
free(pData->reply);
RETiRet;
}
@ -421,10 +640,10 @@ ENDbeginTransaction
BEGINdoAction
CODESTARTdoAction
STATSCOUNTER_INC(indexSubmit, mutIndexSubmit);
if(pData->bulkmode) {
CHKiRet(buildBatch(pData, ppString[0], ppString));
} else {
dbgprintf("omelasticsearch: doAction calling curlPost\n");
CHKiRet(curlPost(pData, ppString[0], strlen((char*)ppString[0]),
ppString));
}
@ -434,12 +653,17 @@ ENDdoAction
BEGINendTransaction
char *cstr;
char *cstr = NULL;
CODESTARTendTransaction
dbgprintf("omelasticsearch: endTransaction init\n");
cstr = es_str2cstr(pData->batch.data, NULL);
dbgprintf("omelasticsearch: endTransaction, batch: '%s'\n", cstr);
CHKiRet(curlPost(pData, (uchar*) cstr, strlen(cstr), NULL));
/* End Transaction only if batch data is not empty */
if (pData->batch.data != NULL ) {
cstr = es_str2cstr(pData->batch.data, NULL);
dbgprintf("omelasticsearch: endTransaction, batch: '%s'\n", cstr);
CHKiRet(curlPost(pData, (uchar*) cstr, strlen(cstr), NULL));
}
else
dbgprintf("omelasticsearch: endTransaction, pData->batch.data is NULL, nothing to send. \n");
finalize_it:
free(cstr);
dbgprintf("omelasticsearch: endTransaction done with %d\n", iRet);
@ -449,35 +673,20 @@ ENDendTransaction
size_t
curlResult(void *ptr, size_t size, size_t nmemb, void *userdata)
{
unsigned int i;
char *p = (char *)ptr;
char *jsonData = (char *)userdata;
static char ok[] = "{\"ok\":true,";
instanceData *pData = (instanceData*) userdata;
char *buf;
size_t newlen;
ASSERT(size == 1);
DBGPRINTF("omelasticsearch request: %s\n", jsonData);
DBGPRINTF("omelasticsearch result: ");
for (i = 0; i < nmemb; i++)
DBGPRINTF("%c", p[i]);
DBGPRINTF("\n");
if (size == 1 &&
nmemb > sizeof(ok)-1 &&
strncmp(p, ok, sizeof(ok)-1) == 0) {
STATSCOUNTER_INC(indexSuccess, mutIndexSuccess);
dbgprintf("omelasticsearch ok\n");
} else {
dbgprintf("omelasticsearch fail\n");
STATSCOUNTER_INC(indexFailed, mutIndexFailed);
if (Debug) {
DBGPRINTF("omelasticsearch (fail) request: %s\n", jsonData);
DBGPRINTF("omelasticsearch (fail) result: ");
for (i = 0; i < nmemb; i++)
DBGPRINTF("%c", p[i]);
DBGPRINTF("\n");
}
newlen = pData->replyLen + size*nmemb;
if((buf = realloc(pData->reply, newlen + 1)) == NULL) {
DBGPRINTF("omelasticsearch: realloc failed in curlResult\n");
return 0; /* abort due to failure */
}
return size * nmemb;
memcpy(buf+pData->replyLen, p, size*nmemb);
pData->replyLen = newlen;
pData->reply = buf;
return size*nmemb;
}
@ -493,10 +702,10 @@ curlSetup(instanceData *pData)
}
header = curl_slist_append(NULL, "Content-Type: text/json; charset=utf-8");
curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, curlResult);
curl_easy_setopt(handle, CURLOPT_POST, 1);
curl_easy_setopt(handle, CURLOPT_POST, 1);
pData->curlHandle = handle;
pData->postHeader = header;
@ -533,6 +742,9 @@ setInstParamDefaults(instanceData *pData)
pData->asyncRepl = 0;
pData->bulkmode = 0;
pData->tplName = NULL;
pData->errorFile = NULL;
pData->dynBulkId= 0;
pData->bulkId = NULL;
}
BEGINnewActInst
@ -552,6 +764,8 @@ CODESTARTnewActInst
continue;
if(!strcmp(actpblk.descr[i].name, "server")) {
pData->server = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "errorfile")) {
pData->errorFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "serverport")) {
pData->port = (int) pvals[i].val.d.n, NULL;
} else if(!strcmp(actpblk.descr[i].name, "uid")) {
@ -578,12 +792,16 @@ CODESTARTnewActInst
pData->asyncRepl = pvals[i].val.d.n;
} else if(!strcmp(actpblk.descr[i].name, "template")) {
pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "dynbulkid")) {
pData->dynBulkId = pvals[i].val.d.n;
} else if(!strcmp(actpblk.descr[i].name, "bulkid")) {
pData->bulkId = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else {
dbgprintf("omelasticsearch: program error, non-handled "
"param '%s'\n", actpblk.descr[i].name);
}
}
if(pData->pwd != NULL && pData->uid == NULL) {
errmsg.LogError(0, RS_RET_UID_MISSING,
"omelasticsearch: password is provided, but no uid "
@ -608,6 +826,12 @@ CODESTARTnewActInst
"name for parent template given - action definition invalid");
ABORT_FINALIZE(RS_RET_CONFIG_ERROR);
}
if(pData->dynBulkId && pData->bulkId == NULL) {
errmsg.LogError(0, RS_RET_CONFIG_ERROR,
"omelasticsearch: requested dynamic bulkid, but no "
"name for bulkid template given - action definition invalid");
ABORT_FINALIZE(RS_RET_CONFIG_ERROR);
}
if(pData->bulkmode) {
pData->batch.currTpl1 = NULL;
@ -623,6 +847,7 @@ CODESTARTnewActInst
if(pData->dynSrchIdx) ++iNumTpls;
if(pData->dynSrchType) ++iNumTpls;
if(pData->dynParent) ++iNumTpls;
if(pData->dynBulkId) ++iNumTpls;
DBGPRINTF("omelasticsearch: requesting %d templates\n", iNumTpls);
CODE_STD_STRING_REQUESTnewActInst(iNumTpls)
@ -644,11 +869,29 @@ CODESTARTnewActInst
if(pData->dynParent) {
CHKiRet(OMSRsetEntry(*ppOMSR, 3, ustrdup(pData->parent),
OMSR_NO_RQD_TPL_OPTS));
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 4, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
} else {
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 3, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
}
} else {
if(pData->dynParent) {
CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->parent),
OMSR_NO_RQD_TPL_OPTS));
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 3, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
} else {
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
}
}
} else {
@ -658,12 +901,30 @@ CODESTARTnewActInst
if(pData->dynParent) {
CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->parent),
OMSR_NO_RQD_TPL_OPTS));
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 3, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
} else {
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
}
} else {
if(pData->dynParent) {
CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->parent),
OMSR_NO_RQD_TPL_OPTS));
}
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 2, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
} else {
if(pData->dynBulkId) {
CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->bulkId),
OMSR_NO_RQD_TPL_OPTS));
}
}
}
}
@ -693,6 +954,14 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
BEGINdoHUP
CODESTARTdoHUP
if(pData->fdErrFile != -1) {
close(pData->fdErrFile);
pData->fdErrFile = -1;
}
ENDdoHUP
BEGINmodExit
CODESTARTmodExit
@ -707,6 +976,7 @@ CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_doHUP
CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */
ENDqueryEtryPt
@ -725,15 +995,13 @@ CODEmodInit_QueryRegCFSLineHdlr
/* support statistics gathering */
CHKiRet(statsobj.Construct(&indexStats));
CHKiRet(statsobj.SetName(indexStats, (uchar *)"elasticsearch"));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"connfail",
ctrType_IntCtr, &indexConFail));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"submits",
CHKiRet(statsobj.SetName(indexStats, (uchar *)"omelasticsearch"));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"submitted",
ctrType_IntCtr, &indexSubmit));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed",
ctrType_IntCtr, &indexFailed));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"success",
ctrType_IntCtr, &indexSuccess));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed.http",
ctrType_IntCtr, &indexHTTPFail));
CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed.es",
ctrType_IntCtr, &indexESFail));
CHKiRet(statsobj.ConstructFinalize(indexStats));
ENDmodInit

25
plugins/omhiredis/README
View File

@ -1,29 +1,22 @@
Redis Outplug Plugin using hiredis library
tested in Centos 6.2
tested in Centos 6.2 and Archlinux
BUILDING THIS PLUGIN
Requires the hiredis C client library: https://github.com/antirez/hiredis/
Requires the hiredis C client library: https://github.com/redis/hiredis/
in your /etc/rsyslog.conf, together with other modules:
TODO
* Error handling for redis calls
* Integrating with impstats
* Clean up code
* Make it work with rsyslog batch mode
* Fix bugs
Brian Knox <bknox@talksum.com>
Brian Knox <briank@talksum.com>
---------------------------------------------------------------------------------------------
$ModLoad omhiredis.so # provides redis output
module(load="omhiredis")
$template TestRedis, "hincrby progcount %programname% 1"
template(name="simple_count" type="string" string="HINCRBY progcount %programname% 1")
if $msg then {
action(type="omhiredis", template="TestRedis")
}
action(name="simple_count_redis" type="omhiredis" queue.type="FixedArray" queue.size="10000" queue.dequeuebatchsize="100" template="simple_count")
---------------------------------------------------------------------------------------------
Note: dequeuebatchsize now sets the pipeline size for hiredis, allowing pipelining commands.
Note: this plugin will NOT handle full rsyslog messages properly yet. spaces in a property will
cause the redis command to be constructed improperly. a fix for this is in the works!

100
plugins/omhiredis/omhiredis.c
View File

@ -20,7 +20,6 @@
*/
#include "config.h"
#include <stdio.h>
#include <string.h>
@ -49,11 +48,16 @@ MODULE_CNFNAME("omhiredis")
DEF_OMOD_STATIC_DATA
DEFobjCurrIf(errmsg)
/* our instance data.
* this will be accessable
* via pData */
typedef struct _instanceData {
redisContext *conn;
uchar *server;
int port;
uchar *tplName;
redisContext *conn; /* redis connection */
uchar *server; /* redis server address */
int port; /* redis port */
uchar *tplName; /* template name */
redisReply **replies; /* array to hold replies from redis */
int count; /* count of command sent for current batch */
} instanceData;
@ -78,6 +82,7 @@ CODESTARTisCompatibleWithFeature
iRet = RS_RET_OK;
ENDisCompatibleWithFeature
/* called when closing */
static void closeHiredis(instanceData *pData)
{
if(pData->conn != NULL) {
@ -86,7 +91,8 @@ static void closeHiredis(instanceData *pData)
}
}
/* Free our instance data.
* TODO: free **replies */
BEGINfreeInstance
CODESTARTfreeInstance
closeHiredis(pData);
@ -100,7 +106,7 @@ CODESTARTdbgPrintInstInfo
/* nothing special here */
ENDdbgPrintInstInfo
/* establish our connection to redis */
static rsRetVal initHiredis(instanceData *pData, int bSilent)
{
char *server;
@ -115,55 +121,104 @@ static rsRetVal initHiredis(instanceData *pData, int bSilent)
if(!bSilent)
errmsg.LogError(0, RS_RET_SUSPENDED,
"can not initialize redis handle");
ABORT_FINALIZE(RS_RET_SUSPENDED);
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
finalize_it:
RETiRet;
}
rsRetVal writeHiredis(uchar *message, instanceData *pData)
{
redisReply *reply;
DEFiRet;
/* if we do not have a redis connection, call
* initHiredis and try to establish one */
if(pData->conn == NULL)
CHKiRet(initHiredis(pData, 0));
reply = redisCommand(pData->conn, (char*)message);
if (reply->type == REDIS_REPLY_ERROR) {
errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", reply->str);
dbgprintf("omhiredis: %s\n", reply->str);
freeReplyObject(reply);
/* try to append the command to the pipeline.
* REDIS_ERR reply indicates something bad
* happened, in which case abort. otherwise
* increase our current pipeline count
* by 1 and continue. */
int rc;
rc = redisAppendCommand(pData->conn, (char*)message);
if (rc == REDIS_ERR) {
errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", pData->conn->errstr);
dbgprintf("omhiredis: %s\n", pData->conn->errstr);
ABORT_FINALIZE(RS_RET_ERR);
} else {
freeReplyObject(reply);
}
pData->count++;
}
finalize_it:
RETiRet;
}
/* called when resuming from suspended state.
* try to restablish our connection to redis */
BEGINtryResume
CODESTARTtryResume
if(pData->conn == NULL)
iRet = initHiredis(pData, 0);
ENDtryResume
/* begin a transaction. for now does nothing.
* if I decide to use MULTI ... EXEC in the
* fture, this block should send the
* MULTI command to redis. */
BEGINbeginTransaction
CODESTARTbeginTransaction
dbgprintf("omhiredis: beginTransaction called\n");
ENDbeginTransaction
/* call writeHiredis for this log line,
* which appends it as a command to the
* current pipeline */
BEGINdoAction
CODESTARTdoAction
iRet = writeHiredis(ppString[0], pData);
CHKiRet(writeHiredis(ppString[0], pData));
iRet = RS_RET_DEFER_COMMIT;
finalize_it:
ENDdoAction
/* called when we have reached the end of a
* batch (queue.dequeuebatchsize). this
* iterates over the replies, putting them
* into the pData->replies buffer. we currently
* don't really bother to check for errors
* which should be fixed */
BEGINendTransaction
CODESTARTendTransaction
dbgprintf("omhiredis: endTransaction called\n");
int i;
pData->replies = malloc ( sizeof ( redisReply* ) * pData->count );
for ( i = 0; i < pData->count; i++ ) {
redisGetReply ( pData->conn, (void *)&pData->replies[i] );
/* TODO: add error checking here! */
free ( pData->replies[i] );
}
free ( pData->replies );
pData->count = 0;
ENDendTransaction
/* set defaults. note server is set to NULL
* and is set to a default in initHiredis if
* it is still null when it's called - I should
* probable just set the default here instead */
static inline void
setInstParamDefaults(instanceData *pData)
{
pData->server = NULL;
pData->port = 6379;
pData->tplName = NULL;
pData->count = 0;
}
/* here is where the work to set up a new instance
* is done. this reads the config options from
* the rsyslog conf and takes appropriate setup
* actions. */
BEGINnewActInst
struct cnfparamvals *pvals;
int i;
@ -222,18 +277,25 @@ BEGINmodExit
CODESTARTmodExit
ENDmodExit
/* register our plugin entry points
* with the rsyslog core engine */
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_TXIF_OMOD_QUERIES /* supports transaction interface */
ENDqueryEtryPt
/* note we do not support rsyslog v5 syntax */
BEGINmodInit()
CODESTARTmodInit
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* only supports rsyslog 6 configs */
CODEmodInit_QueryRegCFSLineHdlr
CHKiRet(objUse(errmsg, CORE_COMPONENT));
INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING);
if (!bCoreSupportsBatching) {
errmsg.LogError(0, NO_ERRCODE, "omhiredis: rsyslog core does not support batching - abort");
ABORT_FINALIZE(RS_RET_ERR);
}
DBGPRINTF("omhiredis: module compiled with rsyslog version %s.\n", VERSION);
ENDmodInit

8
plugins/omjournal/Makefile.am Normal file
View File

@ -0,0 +1,8 @@
pkglib_LTLIBRARIES = omjournal.la
omjournal_la_SOURCES = omjournal.c
omjournal_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBSYSTEMD_JOURNAL_CFLAGS)
omjournal_la_LDFLAGS = -module -avoid-version
omjournal_la_LIBADD = $(LIBSYSTEMD_JOURNAL_LIBS)
EXTRA_DIST =

187
plugins/omjournal/omjournal.c Normal file
View File

@ -0,0 +1,187 @@
/* omjournal.c
* send messages to the Linux Journal. This is meant to be used
* in cases where journal serves as the whole system log database.
* Note that we may get into a loop if journald re-injects messages
* into the syslog stream and we read that via imuxsock. Thus there
* is an option in imuxsock to ignore messages from ourselves
* (actually from our pid). So there are some module-interdependencies.
*
* Copyright 2013 Adiscon GmbH.
*
* This file is part of rsyslog.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* -or-
* see COPYING.ASL20 in the source distribution
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "config.h"
#include "rsyslog.h"
#include <stdio.h>
#include <stdarg.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <signal.h>
#include <errno.h>
#include <unistd.h>
#include "conf.h"
#include "syslogd-types.h"
#include "srUtils.h"
#include "template.h"
#include "module-template.h"
#include "errmsg.h"
#include <systemd/sd-journal.h>
MODULE_TYPE_OUTPUT
MODULE_TYPE_NOKEEP
MODULE_CNFNAME("omjournal")
DEFobjCurrIf(errmsg);
DEF_OMOD_STATIC_DATA
/* config variables */
typedef struct _instanceData {
} instanceData;
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
loadModConf = pModConf;
pModConf->pConf = pConf;
ENDbeginCnfLoad
BEGINendCnfLoad
CODESTARTendCnfLoad
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
runModConf = pModConf;
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
ENDfreeCnf
BEGINcreateInstance
CODESTARTcreateInstance
ENDcreateInstance
BEGINisCompatibleWithFeature
CODESTARTisCompatibleWithFeature
ENDisCompatibleWithFeature
BEGINfreeInstance
CODESTARTfreeInstance
ENDfreeInstance
BEGINnewActInst
CODESTARTnewActInst
/* Note: we currently do not have any parameters, so we do not need
* the lst ptr. However, we will most probably need params in the
* future.
*/
(void) lst; /* prevent compiler warning */
DBGPRINTF("newActInst (mmjournal)\n");
CODE_STD_STRING_REQUESTnewActInst(1)
CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG));
CHKiRet(createInstance(&pData));
/*setInstParamDefaults(pData);*/
CODE_STD_FINALIZERnewActInst
/* cnfparamvalsDestruct(pvals, &actpblk);*/
ENDnewActInst
BEGINdbgPrintInstInfo
CODESTARTdbgPrintInstInfo
ENDdbgPrintInstInfo
BEGINtryResume
CODESTARTtryResume
ENDtryResume
BEGINdoAction
msg_t *pMsg;
uchar *tag;
int lenTag;
int sev;
int r;
CODESTARTdoAction
pMsg = (msg_t*) ppString[0];
MsgGetSeverity(pMsg, &sev);
getTAG(pMsg, &tag, &lenTag);
/* we can use more properties here, but let's see if there
* is some real user interest. We can always add later...
*/
r = sd_journal_send("MESSAGE=%s", getMSG(pMsg),
"PRIORITY=%d", sev,
"SYSLOG_FACILITY=%d", pMsg->iFacility,
"SYSLOG_IDENTIFIER=%s", tag,
NULL);
/* FIXME: think about what to do with errors ;) */
(void) r; /* prevent compiler warning */
ENDdoAction
BEGINparseSelectorAct
CODESTARTparseSelectorAct
CODE_STD_STRING_REQUESTparseSelectorAct(1)
if(strncmp((char*) p, ":omjournal:", sizeof(":omjournal:") - 1)) {
errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED,
"omjournal supports only v6+ config format, use: "
"action(type=\"omjournal\" ...)");
}
ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED);
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
BEGINmodExit
CODESTARTmodExit
objRelease(errmsg, CORE_COMPONENT);
ENDmodExit
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
ENDqueryEtryPt
BEGINmodInit()
CODESTARTmodInit
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
DBGPRINTF("omjournal: module compiled with rsyslog version %s.\n", VERSION);
CHKiRet(objUse(errmsg, CORE_COMPONENT));
ENDmodInit

212
plugins/omlibdbi/omlibdbi.c
View File

@ -10,7 +10,7 @@
*
* File begun on 2008-02-14 by RGerhards (extracted from syslogd.c)
*
* Copyright 2008-2012 Adiscon GmbH.
* Copyright 2008-2013 Adiscon GmbH.
*
* This file is part of rsyslog.
*
@ -70,6 +70,7 @@ typedef struct _instanceData {
uchar *dbName; /* database to use */
unsigned uLastDBErrno; /* last errno returned by libdbi or 0 if all is well */
uchar *tplName; /* format template to use */
int txSupport; /* transaction support */
} instanceData;
typedef struct configSettings_s {
@ -81,15 +82,36 @@ typedef struct configSettings_s {
uchar *dbName; /* database to use */
} configSettings_t;
static configSettings_t cs;
uchar *pszFileDfltTplName; /* name of the default template to use */
struct modConfData_s {
rsconf_t *pConf; /* our overall config object */
uchar *dbiDrvrDir; /* where do the dbi drivers reside? */
uchar *tplName; /* default template */
};
static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */
static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */
static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */
/* tables for interfacing with the v6 config system */
/* module-global parameters */
static struct cnfparamdescr modpdescr[] = {
{ "template", eCmdHdlrGetWord, 0 },
{ "driverdirectory", eCmdHdlrGetWord, 0 }
};
static struct cnfparamblk modpblk =
{ CNFPARAMBLK_VERSION,
sizeof(modpdescr)/sizeof(struct cnfparamdescr),
modpdescr
};
/* action (instance) parameters */
static struct cnfparamdescr actpdescr[] = {
{ "server", eCmdHdlrGetWord, 1 },
{ "db", eCmdHdlrGetWord, 1 },
{ "uid", eCmdHdlrGetWord, 1 },
{ "pwd", eCmdHdlrGetWord, 1 },
{ "driverdirectory", eCmdHdlrGetWord, 0 },
{ "driver", eCmdHdlrGetWord, 1 },
{ "template", eCmdHdlrGetWord, 0 }
};
@ -99,6 +121,20 @@ static struct cnfparamblk actpblk =
actpdescr
};
/* this function gets the default template. It coordinates action between
* old-style and new-style configuration parts.
*/
static inline uchar*
getDfltTpl(void)
{
if(loadModConf != NULL && loadModConf->tplName != NULL)
return loadModConf->tplName;
else if(pszFileDfltTplName == NULL)
return (uchar*)" StdDBFmt";
else
return pszFileDfltTplName;
}
BEGINinitConfVars /* (re)set config variables to default values */
CODESTARTinitConfVars
@ -144,7 +180,6 @@ static void closeConn(instanceData *pData)
BEGINfreeInstance
CODESTARTfreeInstance
closeConn(pData);
free(pData->dbiDrvrDir);
free(pData->drvrName);
free(pData->host);
free(pData->usrName);
@ -227,7 +262,7 @@ static rsRetVal initConn(instanceData *pData, int bSilent)
# endif
if(pData->conn == NULL) {
errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize libdbi connection");
iRet = RS_RET_SUSPENDED;
ABORT_FINALIZE(RS_RET_SUSPENDED);
} else { /* we could get the handle, now on with work... */
/* Connect to database */
dbi_conn_set_option(pData->conn, "host", (char*) pData->host);
@ -238,8 +273,9 @@ static rsRetVal initConn(instanceData *pData, int bSilent)
if(dbi_conn_connect(pData->conn) < 0) {
reportDBError(pData, bSilent);
closeConn(pData); /* ignore any error we may get */
iRet = RS_RET_SUSPENDED;
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
pData->txSupport = dbi_conn_cap_get(pData->conn, "transaction_support");
}
finalize_it:
@ -295,12 +331,127 @@ CODESTARTtryResume
}
ENDtryResume
/* transaction support 2013-03 */
BEGINbeginTransaction
CODESTARTbeginTransaction
if(pData->conn == NULL) {
CHKiRet(initConn(pData, 0));
}
# if HAVE_DBI_TXSUPP
if (pData->txSupport == 1) {
if (dbi_conn_transaction_begin(pData->conn) != 0) {
const char *emsg;
dbi_conn_error(pData->conn, &emsg);
dbgprintf("libdbi server error: begin transaction "
"not successful: %s\n", emsg);
closeConn(pData);
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
}
# endif
finalize_it:
ENDbeginTransaction
/* end transaction */
BEGINdoAction
CODESTARTdoAction
dbgprintf("\n");
iRet = writeDB(ppString[0], pData);
CHKiRet(writeDB(ppString[0], pData));
# if HAVE_DBI_TXSUPP
if (pData->txSupport == 1) {
iRet = RS_RET_DEFER_COMMIT;
}
# endif
finalize_it:
ENDdoAction
/* transaction support 2013-03 */
BEGINendTransaction
CODESTARTendTransaction
# if HAVE_DBI_TXSUPP
if (dbi_conn_transaction_commit(pData->conn) != 0) {
const char *emsg;
dbi_conn_error(pData->conn, &emsg);
dbgprintf("libdbi server error: transaction not committed: %s\n",
emsg);
closeConn(pData);
iRet = RS_RET_SUSPENDED;
}
# endif
ENDendTransaction
/* end transaction */
BEGINbeginCnfLoad
CODESTARTbeginCnfLoad
loadModConf = pModConf;
pModConf->pConf = pConf;
pModConf->tplName = NULL;
bLegacyCnfModGlobalsPermitted = 1;
ENDbeginCnfLoad
BEGINsetModCnf
struct cnfparamvals *pvals = NULL;
int i;
CODESTARTsetModCnf
pvals = nvlstGetParams(lst, &modpblk, NULL);
if(pvals == NULL) {
errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "omlibdbi: error processing "
"module config parameters [module(...)]");
ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS);
}
if(Debug) {
dbgprintf("module (global) param blk for omlibdbi:\n");
cnfparamsPrint(&modpblk, pvals);
}
for(i = 0 ; i < modpblk.nParams ; ++i) {
if(!pvals[i].bUsed)
continue;
if(!strcmp(modpblk.descr[i].name, "template")) {
loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
if(pszFileDfltTplName != NULL) {
errmsg.LogError(0, RS_RET_DUP_PARAM, "omlibdbi: warning: default template "
"was already set via legacy directive - may lead to inconsistent "
"results.");
}
} else if(!strcmp(modpblk.descr[i].name, "driverdirectory")) {
loadModConf->dbiDrvrDir = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else {
dbgprintf("omlibdbi: program error, non-handled "
"param '%s' in beginCnfLoad\n", modpblk.descr[i].name);
}
}
bLegacyCnfModGlobalsPermitted = 0;
finalize_it:
if(pvals != NULL)
cnfparamvalsDestruct(pvals, &modpblk);
ENDsetModCnf
BEGINendCnfLoad
CODESTARTendCnfLoad
loadModConf = NULL; /* done loading */
/* free legacy config vars */
free(pszFileDfltTplName);
pszFileDfltTplName = NULL;
ENDendCnfLoad
BEGINcheckCnf
CODESTARTcheckCnf
ENDcheckCnf
BEGINactivateCnf
CODESTARTactivateCnf
runModConf = pModConf;
ENDactivateCnf
BEGINfreeCnf
CODESTARTfreeCnf
free(pModConf->tplName);
free(pModConf->dbiDrvrDir);
ENDfreeCnf
static inline void
setInstParamDefaults(instanceData *pData)
@ -311,6 +462,7 @@ setInstParamDefaults(instanceData *pData)
BEGINnewActInst
struct cnfparamvals *pvals;
uchar *tplToUse;
int i;
CODESTARTnewActInst
if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) {
@ -319,7 +471,6 @@ CODESTARTnewActInst
CHKiRet(createInstance(&pData));
setInstParamDefaults(pData);
CODE_STD_STRING_REQUESTnewActInst(1)
for(i = 0 ; i < actpblk.nParams ; ++i) {
if(!pvals[i].bUsed)
@ -332,28 +483,19 @@ CODESTARTnewActInst
pData->usrName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "pwd")) {
pData->pwd = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "driverdirectory")) {
pData->dbiDrvrDir = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "driver")) {
pData->drvrName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else if(!strcmp(actpblk.descr[i].name, "template")) {
pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL);
} else {
dbgprintf("ommysql: program error, non-handled "
dbgprintf("omlibdbi: program error, non-handled "
"param '%s'\n", actpblk.descr[i].name);
}
}
if(pData->tplName == NULL) {
CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) strdup(" StdDBFmt"),
OMSR_RQD_TPL_OPT_SQL));
} else {
CHKiRet(OMSRsetEntry(*ppOMSR, 0,
(uchar*) strdup((char*) pData->tplName),
OMSR_RQD_TPL_OPT_SQL));
}
tplToUse = (pData->tplName == NULL) ? (uchar*)strdup((char*)getDfltTpl()) : pData->tplName;
CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_RQD_TPL_OPT_SQL));
CODE_STD_FINALIZERnewActInst
dbgprintf("XXXX: added param, iRet %d\n", iRet);
cnfparamvalsDestruct(pvals, &actpblk);
ENDnewActInst
@ -369,7 +511,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
/* ok, if we reach this point, we have something for us */
CHKiRet(createInstance(&pData));
/* no create the instance based on what we currently have */
if(cs.drvrName == NULL) {
errmsg.LogError(0, RS_RET_NO_DRIVERNAME, "omlibdbi: no db driver name given - action can not be created");
@ -380,19 +521,17 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
/* NULL values are supported because drivers have different needs.
* They will err out on connect. -- rgerhards, 2008-02-15
*/
if(cs.host != NULL)
if((pData->host = (uchar*) strdup((char*)cs.host)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
if(cs.host != NULL)
CHKmalloc(pData->host = (uchar*) strdup((char*)cs.host));
if(cs.usrName != NULL)
if((pData->usrName = (uchar*) strdup((char*)cs.usrName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
if(cs.dbName != NULL)
if((pData->dbName = (uchar*) strdup((char*)cs.dbName)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
if(cs.pwd != NULL)
if((pData->pwd = (uchar*) strdup((char*)cs.pwd)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
CHKmalloc(pData->usrName = (uchar*) strdup((char*)cs.usrName));
if(cs.dbName != NULL)
CHKmalloc(pData->dbName = (uchar*) strdup((char*)cs.dbName));
if(cs.pwd != NULL)
CHKmalloc(pData->pwd = (uchar*) strdup((char*)cs.pwd));
if(cs.dbiDrvrDir != NULL)
if((pData->dbiDrvrDir = (uchar*) strdup((char*)cs.dbiDrvrDir)) == NULL) ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY);
CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_RQD_TPL_OPT_SQL, (uchar*) " StdDBFmt"));
CHKmalloc(loadModConf->dbiDrvrDir = (uchar*) strdup((char*)cs.dbiDrvrDir));
CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_RQD_TPL_OPT_SQL, getDfltTpl()));
CODE_STD_FINALIZERparseSelectorAct
ENDparseSelectorAct
@ -413,7 +552,10 @@ ENDmodExit
BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_QUERIES
CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */
ENDqueryEtryPt
@ -443,8 +585,12 @@ CODESTARTmodInit
INITLegCnfVars
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
# ifndef HAVE_DBI_TXSUPP
DBGPRINTF("omlibdbi: no transaction support in libdbi\n");
# warning libdbi too old - transactions are not enabled (use 0.9 or later)
# endif
CHKiRet(objUse(errmsg, CORE_COMPONENT));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &cs.dbiDrvrDir, STD_LOADABLE_MODULE_ID));
CHKiRet(regCfSysLineHdlr2((uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &cs.dbiDrvrDir, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriver", 0, eCmdHdlrGetWord, NULL, &cs.drvrName, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbihost", 0, eCmdHdlrGetWord, NULL, &cs.host, STD_LOADABLE_MODULE_ID));
CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbiusername", 0, eCmdHdlrGetWord, NULL, &cs.usrName, STD_LOADABLE_MODULE_ID));

4
plugins/ommail/ommail.c
View File

@ -176,7 +176,7 @@ WriteRcpts(instanceData *pData, uchar *pszOp, size_t lenOp, int iStatusToCheck)
for(pRcpt = pData->md.smtp.lstRcpt ; pRcpt != NULL ; pRcpt = pRcpt->pNext) {
dbgprintf("Sending '%s: <%s>'\n", pszOp, pRcpt->pszTo);
CHKiRet(Send(pData->md.smtp.sock, (char*)pszOp, lenOp));
CHKiRet(Send(pData->md.smtp.sock, ": <", sizeof(": <") - 1));
CHKiRet(Send(pData->md.smtp.sock, ":<", sizeof(":<") - 1));
CHKiRet(Send(pData->md.smtp.sock, (char*)pRcpt->pszTo, strlen((char*)pRcpt->pszTo)));
CHKiRet(Send(pData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1));
if(iStatusToCheck >= 0)
@ -522,7 +522,7 @@ sendSMTP(instanceData *pData, uchar *body, uchar *subject)
CHKiRet(Send(pData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1));
CHKiRet(readResponse(pData, &iState, 250));
CHKiRet(Send(pData->md.smtp.sock, "MAIL FROM: <", sizeof("MAIL FROM: <") - 1));
CHKiRet(Send(pData->md.smtp.sock, "MAIL FROM:<", sizeof("MAIL FROM:<") - 1));
CHKiRet(Send(pData->md.smtp.sock, (char*)pData->md.smtp.pszFrom, strlen((char*)pData->md.smtp.pszFrom)));
CHKiRet(Send(pData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1));
CHKiRet(readResponse(pData, &iState, 250));

14
plugins/ommongodb/ommongodb.c
View File

@ -33,9 +33,9 @@
#include <stdint.h>
#include <time.h>
#include <mongo.h>
#include <json/json.h>
#include <json.h>
/* For struct json_object_iter, should not be necessary in future versions */
#include <json/json_object_private.h>
#include <json_object_private.h>
#include "rsyslog.h"
#include "conf.h"
@ -236,11 +236,11 @@ getDefaultBSON(msg_t *pMsg)
gint64 ts_gen, ts_rcv; /* timestamps: generated, received */
int secfrac;
procid = MsgGetProp(pMsg, NULL, PROP_PROGRAMNAME, NULL, &procid_len, &procid_free);
tag = MsgGetProp(pMsg, NULL, PROP_SYSLOGTAG, NULL, &tag_len, &tag_free);
pid = MsgGetProp(pMsg, NULL, PROP_PROCID, NULL, &pid_len, &pid_free);
sys = MsgGetProp(pMsg, NULL, PROP_HOSTNAME, NULL, &sys_len, &sys_free);
msg = MsgGetProp(pMsg, NULL, PROP_MSG, NULL, &msg_len, &msg_free);
procid = MsgGetProp(pMsg, NULL, PROP_PROGRAMNAME, NULL, &procid_len, &procid_free, NULL);
tag = MsgGetProp(pMsg, NULL, PROP_SYSLOGTAG, NULL, &tag_len, &tag_free, NULL);
pid = MsgGetProp(pMsg, NULL, PROP_PROCID, NULL, &pid_len, &pid_free, NULL);
sys = MsgGetProp(pMsg, NULL, PROP_HOSTNAME, NULL, &sys_len, &sys_free, NULL);
msg = MsgGetProp(pMsg, NULL, PROP_MSG, NULL, &msg_len, &msg_free, NULL);
// TODO: move to datetime? Refactor in any case! rgerhards, 2012-03-30
ts_gen = (gint64) datetime.syslogTime2time_t(&pMsg->tTIMESTAMP) * 1000; /* ms! */

32
plugins/ommysql/ommysql.c
View File

@ -189,7 +189,6 @@ static rsRetVal initMySQL(instanceData *pData, int bSilent)
ASSERT(pData != NULL);
ASSERT(pData->f_hmysql == NULL);
pData->f_hmysql = mysql_init(NULL);
if(pData->f_hmysql == NULL) {
errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize MySQL handle");
@ -219,10 +218,12 @@ static rsRetVal initMySQL(instanceData *pData, int bSilent)
pData->f_dbpwd, pData->f_dbname, pData->f_dbsrvPort, NULL, 0) == NULL) {
reportDBError(pData, bSilent);
closeMySQL(pData); /* ignore any error we may get */
iRet = RS_RET_SUSPENDED;
ABORT_FINALIZE(RS_RET_SUSPENDED);
}
mysql_autocommit(pData->f_hmysql, 0);
}
finalize_it:
RETiRet;
}
@ -241,6 +242,7 @@ rsRetVal writeMySQL(uchar *psz, instanceData *pData)
/* see if we are ready to proceed */
if(pData->f_hmysql == NULL) {
CHKiRet(initMySQL(pData, 0));
}
/* try insert */
@ -272,12 +274,28 @@ CODESTARTtryResume
}
ENDtryResume
BEGINbeginTransaction
CODESTARTbeginTransaction
CHKiRet(writeMySQL((uchar*)"START TRANSACTION", pData));
finalize_it:
ENDbeginTransaction
BEGINdoAction
CODESTARTdoAction
dbgprintf("\n");
iRet = writeMySQL(ppString[0], pData);
CHKiRet(writeMySQL(ppString[0], pData));
iRet = RS_RET_DEFER_COMMIT;
finalize_it:
ENDdoAction
BEGINendTransaction
CODESTARTendTransaction
if (mysql_commit(pData->f_hmysql) != 0) {
dbgprintf("mysql server error: transaction not committed\n");
iRet = RS_RET_SUSPENDED;
}
ENDendTransaction
static inline void
setInstParamDefaults(instanceData *pData)
@ -305,7 +323,7 @@ CODESTARTnewActInst
CHKiRet(createInstance(&pData));
setInstParamDefaults(pData);
CODE_STD_STRING_REQUESTnewActInst(1)
CODE_STD_STRING_REQUESTparseSelectorAct(1)
for(i = 0 ; i < actpblk.nParams ; ++i) {
if(!pvals[i].bUsed)
continue;
@ -437,6 +455,7 @@ BEGINqueryEtryPt
CODESTARTqueryEtryPt
CODEqueryEtryPt_STD_OMOD_QUERIES
CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */
ENDqueryEtryPt
@ -459,6 +478,11 @@ INITLegCnfVars
*ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */
CODEmodInit_QueryRegCFSLineHdlr
CHKiRet(objUse(errmsg, CORE_COMPONENT));
INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING);
if(!bCoreSupportsBatching) {
errmsg.LogError(0, NO_ERRCODE, "ommysql: rsyslog core too old");
ABORT_FINALIZE(RS_RET_ERR);
}
/* we need to init the MySQL library. If that fails, we cannot run */
if(

Some files were not shown because too many files have changed in this diff Show More