166 Commits

Author SHA1 Message Date
Rainer Gerhards
a9dd12b967
queue: permit ability to double size at shutdown
This prevents message loss due to "queue full" when re-enqueueing data
under quite exotic settings.

see also https://github.com/rsyslog/rsyslog/issues/3941#issuecomment-549765813
closes https://github.com/rsyslog/rsyslog/issues/4020
2020-03-03 16:14:31 +01:00
Rainer Gerhards
cbcaf2c7e5
core: add global parameter "security.abortOnIDResolutionFail"
This parameter controls whether or not rsyslog aborts when a name ID
lookup fails (for user and group names). This is necessary as a security
measure, as otherwise the wrong permissions can be assigned or privileges
are not dropped.

CHANGE OF BEHAVIOR
The default for this parameter is "on". In previous versions, the default
was "off" (by virtue) of this parameter not existing. As such, existing
configurations may now error out.

We have decided to accept this change of behavior because of the potential
security implications.

closes https://github.com/rsyslog/rsyslog/issues/4164
2020-02-11 12:54:19 +01:00
Dagobert Michelsen
585d7b1a99 Use cast to long for pid_t to avoid type error 2020-01-06 14:34:04 +01:00
Rainer Gerhards
f9706e6d7c
core: do not unnecessarily set hostname on each HUP 2019-08-20 11:10:12 +02:00
Rainer Gerhards
55d4c7dbf2
imrelp: fix cosmetic race in recently-added code
It is extremely unlikly that the race could have caused harm, except
during test runs with thread sanitizer (TSAN) enabled.
2019-08-05 09:31:18 +02:00
Rainer Gerhards
0b2477a9f5
imrelp bugfix: hang after HUP
termination condition was not properly checked; this lead to
premature termination after patch 1c8712415b9 was applied.

It is open to debate if patch 1c8712415b9 changed the module
interface. Actually it looks like this was previously not
well thought out.

closes https://github.com/rsyslog/rsyslog/issues/3760
2019-07-31 10:35:58 +02:00
Rainer Gerhards
359288d788
global config: new parameters for ruleset queue defaults
specifically:

* default.ruleset.queue.timeoutshutdown
* default.ruleset.queue.timeoutactioncompletion
* default.ruleset.queue.timeoutenqueue
* default.ruleset.queue.timeoutworkerthreadshutdown

closes https://github.com/rsyslog/rsyslog/issues/3656
2019-05-09 15:18:55 +02:00
Rainer Gerhards
d9d12071f3
change default of internalmsg.severity global parameter, add tests
also fix a problem in handling this parameter when rsyslog processed
internal messages itself (it did not work). As the parameter was
introduced today, we do not flag this follow-up commit as "bugfix".
The issue was noticed when we added the additional tests.

see also https://github.com/rsyslog/rsyslog/issues/3650
see also https://github.com/rsyslog/rsyslog/issues/3639
2019-05-09 12:13:45 +02:00
Rainer Gerhards
160f2d7a52
new global config parameter "internalmsg.severity"
permits to specify a severity filter for internal message. Only
messages with this severity level or more severe are logged.

Orignally this was done in rsyslog.conf as usual: you can filter
rsyslog messages on severity, just like any other. But with systemd,
we now emit primarily to the journal, and this is outside of rsyslog's
rule engine and so regular filters do not apply (at least in regard
to the journal). Logging to journal is good, because finally
folks begin to see the messages (traditional distro configs discard
them, for whatever is the reason).

This commit implements a global setting for a severity-based filter
for internal messages, before submitted to journal. So it's not 100%
of what rsyslog can do, but at least some way to customize.

see also https://github.com/rsyslog/rsyslog/issues/3639
2019-05-07 11:27:08 +02:00
Rainer Gerhards
37187427bf
dnscache: fix looking issue detected by Coverity Scan
This leads to a refactoring of the looking code; issue was caused
by new TTL cache expiration code which placed not semantics on the
cache. These were not properly handled under all circumstances.
2019-04-16 15:41:32 +02:00
Rainer Gerhards
6f3f6c47ae
dnscache: add global parameter dnscache.default.ttl
This permits to control default TTL for cache entries. If set
to 0, the DNS cache is effectively disabled.

closes https://github.com/rsyslog/rsyslog/issues/49
closes https://github.com/rsyslog/rsyslog/issues/1487
2019-04-12 17:26:29 +02:00
9c60ae361e testbench: reorganized testcases for server/client certless comm
- Corrected ANON Cipher handling in ossl / gtls driver.
- removed error when no CA is configured for ANON Mode in gtls dirver.
- Set GNUTLS Debug level to 2, so we see more informations about gnutls errors
  in rsyslog debug mode.
- fixed tcpdump parameters not using TLS in manytcp-too-few-tls-vg.sh
- fixed minor memory leak in shutdown destructor of ossl tls driver.
2019-02-21 17:43:35 +01:00
Joan Sala
9315787c6d report child process exit status according to config parameter
Add new global setting 'reportChildProcessExits' with possible values
'none|errors|all' (default 'errors'), and new global function
'glblReportChildProcessExit' to report the exit status of a child
process according to the setting.

Invoke the report function whenever rsyslog reaps a child, namely in:
- rsyslogd.c (SIGCHLD signal handler)
- omprog
- mmexternal
- srutils.c (execProg function, invoked from stream.c and omshell)

Remove redundant "reaped by main loop" info log in omprog.

Promote debug message in mmexternal indicating that the child has
terminated prematurely to a warning log, like in omprog.

Closes #3281
2018-12-21 20:29:01 +01:00
Rainer Gerhards
e2b87769f1
global object: add ability to set action queue timeout defaults
while this is useful for users as well, we have done it so
that we can handle slow CI systems during CI runs. It is also
required for massively parallel testing, which makes each
individual test rather slow.

With the new settings, the testbench framework can now set
longer timeouts by defaults. Also updated framework accordingly.
2018-12-20 13:56:30 +01:00
Rainer Gerhards
d09416c95f
core: add operating state file capability (new feature)
adds global(operatingStateFile="fn") and related handling (see doc
for details).

closes https://github.com/rsyslog/rsyslog/issues/1721
2018-10-29 15:59:47 +01:00
PascalWithopf
19133327cc correct codestyle in rsyslog 2018-07-31 09:44:27 +02:00
Rainer Gerhards
e200b2192f add new global config parameter "inputs.timeout.shutdown" 2018-07-27 09:09:38 +02:00
Rainer Gerhards
2656afd4f5
build system: do not disable testbench tests via --disable-liblogging-stdlog 2018-07-02 11:13:05 +02:00
Rainer Gerhards
8f8ed81e5e
add global directive "abortOnUncleanConfig"
This provides a new-style alternative to $AbortOnUncleanConfig.
Note that a testbench test was changed to the new syntax. Adding
an additional test did not look useful, as the testbench still
sufficiently tests old and new method.

closes https://github.com/rsyslog/rsyslog/issues/2744
2018-06-27 11:17:48 +02:00
Rainer Gerhards
90308823df core: consistent handling of oversize input messages
In the community we frequently discuss handling of oversize messages.
David Lang rightfully suggested to create a central capability inside
rsyslog core to handle them.

We need to make a distinction between input and output messages. Also,
input messages frequently need to have some size restrictions done at
a lower layer (e.g. protocol layer) for security reasons. Nevertheless,
we should have a central capability

* for cases where it need not be handled at a lower level
* as a safeguard when a module invalidly emits it (imfile is an example,
  see https://github.com/rsyslog/rsyslog/pull/2632 for a try to fix it
  on the module level - we will replace that with the new capability
  described here).

The central capability works on message submission, and so cannot be
circumvented. It has these capabilities:

* overisze message handling modes:
  - truncate message
  - split message
    this is of questionable use, but also often requested. In that mode,
    the oversize message content is split into multiple messages. Usually,
    this ends up with message segments where all but the first is lost
    anyhow as the regular filter rules do not match the other fragments.
    As it is requested, we still implemented it.
  - accept message as is, even if oversize
    This may be required for some cases. Most importantly, it makes
    quite some sense when writing messages to file, where oversize
    does not matter (accept from a DoS PoV).
* report message to a special "oversize message log file" (not via the
  regular engine, as that would obviously cause another oversize message)

This commit, as the title says, handles oversize INPUT messages.

see also https://github.com/rsyslog/rsyslog/issues/2190
closes https://github.com/rsyslog/rsyslog/issues/2681
closes https://github.com/rsyslog/rsyslog/issues/498

Note: this commit adds global parameters:
 * "oversizemsg.errorfile",
   is used to specify the location of the oversize message log file.
 * "oversizemsg.report",
   is used to control if an error shall be reported when an oversize
   message is seen. The default it "on".
 * add global parameter "oversizemsg.input.mode"
   is used to specify the mode with which oversized messages will
   be handled.
2018-05-14 11:24:20 +02:00
Rainer Gerhards
bd6f016db0 add global option internal.developeronly.options
This permits the ability to set some developer-specific behaviour
changes, e.g. for testbench testing or things like this. NEVER to
be used by users.
2018-04-28 12:46:32 +02:00
Rainer Gerhards
6d850a1aa9 fix behaviour not totally defined by C standard
see also: https://stackoverflow.com/questions/32916575/how-to-use-zd-specifier-with-printf

detected by lgtm static analyzer
2018-03-17 10:26:27 +01:00
Rainer Gerhards
c9fa875c44 fix global parameter name shutdown.enable.ctl-c
the config system actually does not permit "-" inside parameter
names. This has now been changed to "shutdown.enable.ctlc".

Note: this was never released and only present in 8.33 pre-release
versions.

closes https://github.com/rsyslog/rsyslog/issues/2482
2018-02-15 18:44:32 +01:00
Jan Gerhards
97c223a198 glbl: use new errmsg interface
see also https://github.com/rsyslog/rsyslog/issues/1684
2018-02-02 10:00:05 +01:00
Rainer Gerhards
b9cac88204 add global config setting shutdown.enable.ctl-c to enable ctl-c
It can be useful to ctl-c out of rsyslog, even if it is started via
-n, especially inside containers.  The new config setting permits
to enable this capability.

closes https://github.com/rsyslog/rsyslog/issues/533
2018-01-08 18:10:47 +01:00
Rainer Gerhards
e05b3ff0fd core/glbl: remove long-unused option $optimizeforuniprocessor
This was still available, but had no effect (for ~10 years or so). The
plumbing is now removed. If someone tries to use the option, an
error message is generated.

closes https://github.com/rsyslog/rsyslog/issues/2280
2017-12-21 14:50:35 +01:00
Rainer Gerhards
ba66b20d20 core bugfix: race on LocalHostIP property during startup
The way the default was set could lead to a race if e.g. two internal
messages were emitted at startup when the property was not yet set. This
has been seen to happen in practice. It had no bad effect except a very
small stationary memory leak, but made using thread analyzers unreliable
(as it was rightfully detected as a problem).

closes https://github.com/rsyslog/rsyslog/issues/2012
2017-12-19 18:53:18 +01:00
Rainer Gerhards
1e9579b544 core bugfix: potential segfault on startup
timezone info table was "sorted" even though it may be NULL

Detected by clang ubsan.
2017-12-14 17:25:17 +01:00
Rainer Gerhards
6e473f3ba0 "fix" Coverity valse positive, CID 185368 2017-11-22 08:34:15 +01:00
Rainer Gerhards
b4f0e83c45 debug system: fix cosmetic file handle leak
one file handle is lost per run if a debug file is specified via config
as well es environment variable.

Detected by Coverity scan, CID 185420
2017-11-21 08:32:56 +01:00
PascalWithopf
e485c5c5bc codestyle: shorten lines to max 130 chars 2017-11-15 09:04:42 +01:00
Rainer Gerhards
9074b5ab97 core: potential NULL pointer dereference in config parsing
This happens if there is a problem with the timezone parameters.
Affects only startup, once started, no problem exists.

Detected by Coverty scan; CID 185414
2017-10-26 13:07:13 +02:00
Rainer Gerhards
ad040017e8 Merge pull request #1807 from rgerhards/case-sensitive
handle (JSON) variables in case-insensitive way
2017-10-16 11:09:56 +02:00
Rainer Gerhards
6fe8153be9 handle (JSON) variables in case-insensitive way
The variable system inside rsyslog is JSON based (for easy consumption
of JSON input, the prime source of structured data). In JSON, keys
("variable names") are case-sensitive. This causes constant problems
inside rsyslog configurations. A major nit is that case-insensitivity
option inside templates (even if turned on) does not work with JSON
keys because they of inner workings*1.

It is much more natural to treat keys in a case-INsensitive way (e.g.
"$!Key" and "$!key" are the same). We do not expect any real problems
out of this, as key names only differing in case is highly unlikely.
However, as it is possible we provide a way to enable case-sensitivity
via the global(variables.casesensitve="on") global config object.

Note that the default is to do case-insensitive matches. The reason
is that this is the right thing to do in almost all cases, and we do
not envision any problems at all in existing deployments.

*1 Note: case-insensitivity in rsyslog is achieved by converting all
names to lower case. So that the higher speed of strcmp() can be used.
The template option does actually that, convert the template keys to
lower case. Unfortunately, this does not work with JSON, as JSON keys
are NOT converted to lower case.

closes https://github.com/rsyslog/rsyslog/issues/1805
2017-10-13 09:39:56 +02:00
Rainer Gerhards
3d7223dd25 core: add ability to limit number of error messages going to stderr
This permits to put a hard limit on the number of messages that can
go to stderr. If for nothing else, this capability is helpful for the
testbench. It permits to reduce spamming the test log while still
providing the ability to see initial error messages. Might also be
useful for some practical deployments.

closes https://github.com/rsyslog/rsyslog/issues/1841
2017-10-12 16:14:13 +02:00
Rainer Gerhards
eac37a891b bugfix: rsyslog aborts on startup if certificate files cannot be accessed
When the global defaults for certificates are set and the certificate
files cannot be accessed, rsyslog aborts. This is caused by a failure
in certificate-file-validity check code. It can only happen right at startup.

Also, the invalid certificate file is still set, causing follow-on problems.

closes https://github.com/rsyslog/rsyslog/issues/1786
2017-09-21 12:07:25 +02:00
Jan Gerhards
000d55910e core bugfix: cosmetic memory leak
closes https://github.com/rsyslog/rsyslog/issues/1736
2017-08-23 10:06:14 +02:00
Jan Gerhards
edfc4396a3 core: add parameters debug.file and debug.whitelist
allows to generate debug log output only of specific files
2017-08-08 12:12:39 +02:00
Jan Gerhards
ac9874cf49 core: make rsyslog internal message rate-limiter configurable
closes https://github.com/rsyslog/rsyslog/issues/1670
2017-07-16 13:34:09 +02:00
PascalWithopf
5d0bcd69a2 gtls: improve error if certificate file can't be opened
closes https://github.com/rsyslog/rsyslog/issues/1121
2017-06-27 11:04:15 +02:00
PascalWithopf
8ceaddde35 imfile: add posibility to truncate msg
closes https://github.com/rsyslog/rsyslog/issues/1552
2017-05-19 12:04:02 +02:00
Jiri Vymazal
5d36b82088 performing MainqObj destruction only when not NULL already 2017-04-03 14:09:49 +02:00
Rainer Gerhards
304937104c Merge pull request #1443 from rgerhards/i-1442
bugfix: rsyslog identifies itself as "liblogging-stdlog" in internal …
2017-03-07 08:44:41 +01:00
Rainer Gerhards
8f5c32c3f4 add new global parameter "umask"
This is equivalent to "$umask" and permits to convert that construct
to new-style config format.

closes https://github.com/rsyslog/rsyslog/issues/1382
2017-03-06 11:35:11 +01:00
Rainer Gerhards
9e7d57a925 bugfix: rsyslog identifies itself as "liblogging-stdlog" in internal messages
This occured when liblogging-stdlog was used, and was used by default (without
explicit configuration). This is a regression of the new default, which does
not correctly call stdlog_open() in the default case.

closes https://github.com/rsyslog/rsyslog/issues/1442
2017-03-04 10:46:24 +01:00
Radovan Sroka
d649d77cd5 Fixed segfault in glblDoneLoadCnf()
This is caused by uninitialized net static structure in glbl objUse(net)
is not called after start of glbl module.

It looks like it is not possible to use objUse(net) when glbl module
starts. So we call it inside of glblDoneLoadCnf().
2017-02-23 15:11:11 +01:00
Rainer Gerhards
ae20a4d1e9 Revert "Sanitizer longmsg fix2" 2017-02-20 10:21:15 +01:00
Rainer Gerhards
cce49b0f79 core: prevent too-long messages from entering the system
MsgSetRawMsg() did not check max message size. So if an input module
submitted a too-long message, it was silently accepted. This now adds
a check and truncates the message if too long. If not turned off, this
will also be logged as a rsyslog error message.

TODO: currently only partly implemented!
2017-02-16 19:21:00 +01:00
Rainer Gerhards
43db4f8233 Merge pull request #1362 from PascalWithopf/linelength
codestyle: line length adjusted
2017-01-17 10:26:23 +01:00
Pascal Withopf
b63a6b9010 codestyle: line length adjusted 2017-01-17 08:54:37 +01:00