Adds the ability to remove unnecessary spaces from genrated json
to conserve space, e.g. on disk.
Maintainer edit: this replaces orginal PR which was too hard to
rebase. This was caused by too-slow reaction of the rsyslog team.
see also: https://github.com/rsyslog/rsyslog/pull/2925
(this is the original PR)
closes: https://github.com/rsyslog/rsyslog/issues/2913
Multiple intermediate CAs were not honored because strtok() modified
the original config value at the first comma. This led to only the
first file being considered. This change preserves the original value
and validates files using a duplicate buffer.
Impact: Users can reliably specify multiple CA intermediates via
NetstreamDriverCAExtraFiles; TLS chains that require intermediates
now validate as expected. Tests added.
Before: parsing mutated the stored string, effectively truncating at
the first comma and ignoring subsequent CA files.
After: we strdup() for validation, leave the original string intact,
and set it only if all files are accessible. Error handling follows
existing RS_RET patterns; temporary buffer is freed on all paths.
Additionally, add a test (OpenSSL backend) that generates a root,
two intermediates, and leaf certs on both client and server. The test
verifies chains with openssl, aligns CN/PermittedPeer, and exercises
a full send/receive path to guard against regressions. Makefile is
updated to include the new test in TESTS and EXTRA_DIST.
Fixes: https://github.com/rsyslog/rsyslog/issues/5207
This commit applies the new canonical formatting style using `clang-format` with custom settings (notably 4-space indentation), as part of our shift toward automated formatting normalization.
⚠️ No functional changes are included — only whitespace and layout modifications as produced by `clang-format`.
This change is part of the formatting modernization strategy discussed in:
https://github.com/rsyslog/rsyslog/issues/5747
Key context:
- Formatting is now treated as a disposable view, normalized via tooling.
- The `.clang-format` file defines the canonical style.
- A fixup script (`devtools/format-code.sh`) handles remaining edge cases.
- Formatting commits are added to `.git-blame-ignore-revs` to reduce noise.
- Developers remain free to format code however they prefer locally.
This commit performs a broad modernization of widely used rsyslog
macros to align with modern C practices and support automated
formatting tools like clang-format. The changes focus on improving
syntactic regularity, readability, and tooling compatibility — without
altering behavior.
Macros refactored in this commit now follow a consistent,
statement-like form with explicit trailing semicolons. Where
applicable, macro blocks that define module interfaces (`queryEtryPt`)
have been updated to use simple `if` statements instead of `else if`
chains. While this slightly increases evaluation time, the affected
functions are only called once per module during load time to register
supported interfaces — making the performance cost irrelevant in
practice.
These improvements serve multiple purposes:
- Enable reliable clang-format usage without mangling macro logic
- Simplify reasoning about macro-expanded code for human readers
- Reduce style drift and merge conflicts
- Facilitate development for contributors using assistive tools
- Support future formatting pipelines using:
1. `clang-format`
2. a post-fixup normalization script
Refactored macros:
- MODULE_TYPE_NOKEEP
- MODULE_TYPE_KEEP
- MODULE_TYPE_INPUT
- MODULE_TYPE_OUTPUT
- MODULE_TYPE_FUNCTION
- MODULE_TYPE_PARSER
- MODULE_TYPE_LIB
- DEF_IMOD_STATIC_DATA
- DEF_OMOD_STATIC_DATA
- DEF_PMOD_STATIC_DATA
- DEF_FMOD_STATIC_DATA
- DEFobjStaticHelpers
- SIMP_PROP(...)
And all `queryEtryPt()` dispatch macros:
- CODEqueryEtryPt_STD_MOD_QUERIES
- CODEqueryEtryPt_STD_OMOD_QUERIES
- CODEqueryEtryPt_STD_OMODTX_QUERIES
- CODEqueryEtryPt_STD_OMOD8_QUERIES
- CODEqueryEtryPt_TXIF_OMOD_QUERIES
- CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
- CODEqueryEtryPt_STD_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_QUERIES
- CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
- CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES
- CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
- CODEqueryEtryPt_STD_PMOD_QUERIES
- CODEqueryEtryPt_STD_PMOD2_QUERIES
- CODEqueryEtryPt_STD_FMOD_QUERIES
- CODEqueryEtryPt_STD_SMOD_QUERIES
- CODEqueryEtryPt_doHUPWrkr
- CODEqueryEtryPt_doHUP
This general modernization reduces macro misuse, improves DX, and
lays the foundation for a robust, automated style normalization
system.
See also: https://github.com/rsyslog/rsyslog/issues/5747
This patch implements a simple round-robin load balancer
for omfwd. It provides equal distribution of load to a pool
of target servers.
The code currently has no different modes and no special tuning
for the load balancer. However, it works very well in the most
common use cases. Furthermore, it provides a solid base on which
more elaborate functionality can be build if there is need to.
The new functionality is fully backwards compatible with previous
configuration settings.
New action() config params:
* pool.resumeinterval
New/"changed" rstats counters
Each target receives its own set of pstats counters. Most
importantly this is the case for byte counts. That counter retains
the same naming, but there may now be multiple of these counters,
one for each target ip, port tuple.
New pstats message count to target
Among others, this can be used for checking that the load balancer
works as intended. The so-far byte count emitted does not provide
a clear indication of how many messages the targets had actually
processed.
For obvious reasons, this message count makes most sense in
advanced load balancing scenarios, but also provides additional
insight into round-robin. Non-matches indicate that targets
went offline, and we can now evaluate the impact this had
on processing.
- re-design rebind functionality
This now works at the transaction level. It causes a rebind of all
pool members. Previous code did not work 100% correct since for a
couple of years now (after output batching integration).
As cleanup, rebindInterval support has been removed from tcpClt,
because omfwd is the only user. This permits a cleaner code path.
We also noticed a bug with rebindInterval: it caused some mild
message duplication for quite some time. This went unnoticed.
To address that efficiently, rebindInterval in the future will
be considered once per batch. That means up to (maxBatchSize - 1)
messages may be transmitted more than the rebindinterval is.
That's the cleanest mode of operation and should not make any
difference for real deployments.
Some additional work done in this commit:
netstream: harden component against upper-layer logic errors
network subsystem: better handle API errors and provide more info
omfwd: add new parameter "iobuffer.maxsize"
add new global parameter debug.abortoninternalerror and use it
This parameter permits to make test runs fail when an internal error
is detected and gracefully handled by rsyslog. While it is great to
have it gracefully handled in practice, we should not accept this
during testing. The new parameter permits to abort in this case and
emits the related error message beforehand. It is turned on by
default in our regular tests.
add dedicated error code for "hard" program errors
omfwd: some cleanup + error message fix + new debug level messages
imptcp: improve error messages
add omfwd option to NOT do extended connection check
also output wrkr id in some omfwd messages (primarily debugging aid)
better debug info via LogMsg() interface
improve messages regarding imptcp and omfwd suspension / thread IDs
refactor and enchance minitcpsrvr for mimicing died servers
new global (debugging) option, correction of an informational msg
add global option allmessagestostderr
add new tests
- output all loaded ciphers and engines.
- Add new global option "defaultopensslengine" to customize the
default openssl engine. If not defined, openssl will handle the
default engine automatically.
- Add simple openssl performance test with defaultopensslengine
set to rdrand (Intel).
- removed unneeded testcase files in runtime folder.
- corrected whitelist settings for debug.files in TLS testcases
Even if hostname FQDN is configured, rsyslog internal messages generated
after rsyslog startup and before the first HUP will not necessarily have
FQDN but instead only the shortname of the local host. This commit
fixes the situation.
Special thanks to github user eciii for doing a great bug analysis
and helping us considerably to fix the issue.
closes https://github.com/rsyslog/rsyslog/issues/5218
Defines whether rsyslog should drop capabilities at startup or not.
By default, it is set to "on". Until this point, if the project was
compiled with --enable-libcap-ng option, capabilities were
automatically dropped. This is configurable now.
Defines how rsyslog should behave in case something went wrong
when capabilities were to be dropped. Default value is "on",
in which case rsyslog exits on a libcapng related error.
Closes#5096
rsyslog.conf may affect the host's local name. These changes were
so far only activated after the first HUP. This patch now ensures
that the configured local host name is applied correctly throughout
all processing, including early startup.
This patch causes a slight change of behaviour. However, the behaviour
was inconsitent before. Now it is consistent and according to the config.
Please note: this patch also exposes a global entry point via "regular"
dynamic loading as this makes things much easier to do. This is in-line
with ongoing simplification effort.
Finally, we also remove a CI test that we do no longer need because
the problem covered is now addressed differently and the original issue
can no longer occur.
closes https://github.com/rsyslog/rsyslog/issues/4975
similiar to "abortONUncleanConfig", this parameter aborts rsyslog
when a queue has problems during startup. Some users perfer rsyslog
to terminate in this case. By default, nothing changes.
closes https://github.com/rsyslog/rsyslog/issues/4902
This change allows to include extra CA files
so that no "unable to get issuer certificates" issue
is obtained when using chained cert files.
Proposed new parameter name is "NetstreamDriverCAExtraFiles"
fixes#4851
Signed-off-by: Sergio Arroutbi <sarroutb@redhat.com>
This permits to turn off rsyslog's single-message compression extension
when it interferes with non-syslog message processing (the parser
subsystem expects syslog messages, not generic text)
closes https://github.com/rsyslog/rsyslog/issues/4598
This parameter controls whether or not rsyslog aborts when a name ID
lookup fails (for user and group names). This is necessary as a security
measure, as otherwise the wrong permissions can be assigned or privileges
are not dropped.
CHANGE OF BEHAVIOR
The default for this parameter is "on". In previous versions, the default
was "off" (by virtue) of this parameter not existing. As such, existing
configurations may now error out.
We have decided to accept this change of behavior because of the potential
security implications.
closes https://github.com/rsyslog/rsyslog/issues/4164
termination condition was not properly checked; this lead to
premature termination after patch 1c8712415b9 was applied.
It is open to debate if patch 1c8712415b9 changed the module
interface. Actually it looks like this was previously not
well thought out.
closes https://github.com/rsyslog/rsyslog/issues/3760
also fix a problem in handling this parameter when rsyslog processed
internal messages itself (it did not work). As the parameter was
introduced today, we do not flag this follow-up commit as "bugfix".
The issue was noticed when we added the additional tests.
see also https://github.com/rsyslog/rsyslog/issues/3650
see also https://github.com/rsyslog/rsyslog/issues/3639
permits to specify a severity filter for internal message. Only
messages with this severity level or more severe are logged.
Orignally this was done in rsyslog.conf as usual: you can filter
rsyslog messages on severity, just like any other. But with systemd,
we now emit primarily to the journal, and this is outside of rsyslog's
rule engine and so regular filters do not apply (at least in regard
to the journal). Logging to journal is good, because finally
folks begin to see the messages (traditional distro configs discard
them, for whatever is the reason).
This commit implements a global setting for a severity-based filter
for internal messages, before submitted to journal. So it's not 100%
of what rsyslog can do, but at least some way to customize.
see also https://github.com/rsyslog/rsyslog/issues/3639
This leads to a refactoring of the looking code; issue was caused
by new TTL cache expiration code which placed not semantics on the
cache. These were not properly handled under all circumstances.
- Corrected ANON Cipher handling in ossl / gtls driver.
- removed error when no CA is configured for ANON Mode in gtls dirver.
- Set GNUTLS Debug level to 2, so we see more informations about gnutls errors
in rsyslog debug mode.
- fixed tcpdump parameters not using TLS in manytcp-too-few-tls-vg.sh
- fixed minor memory leak in shutdown destructor of ossl tls driver.
Add new global setting 'reportChildProcessExits' with possible values
'none|errors|all' (default 'errors'), and new global function
'glblReportChildProcessExit' to report the exit status of a child
process according to the setting.
Invoke the report function whenever rsyslog reaps a child, namely in:
- rsyslogd.c (SIGCHLD signal handler)
- omprog
- mmexternal
- srutils.c (execProg function, invoked from stream.c and omshell)
Remove redundant "reaped by main loop" info log in omprog.
Promote debug message in mmexternal indicating that the child has
terminated prematurely to a warning log, like in omprog.
Closes#3281
while this is useful for users as well, we have done it so
that we can handle slow CI systems during CI runs. It is also
required for massively parallel testing, which makes each
individual test rather slow.
With the new settings, the testbench framework can now set
longer timeouts by defaults. Also updated framework accordingly.
This provides a new-style alternative to $AbortOnUncleanConfig.
Note that a testbench test was changed to the new syntax. Adding
an additional test did not look useful, as the testbench still
sufficiently tests old and new method.
closes https://github.com/rsyslog/rsyslog/issues/2744
In the community we frequently discuss handling of oversize messages.
David Lang rightfully suggested to create a central capability inside
rsyslog core to handle them.
We need to make a distinction between input and output messages. Also,
input messages frequently need to have some size restrictions done at
a lower layer (e.g. protocol layer) for security reasons. Nevertheless,
we should have a central capability
* for cases where it need not be handled at a lower level
* as a safeguard when a module invalidly emits it (imfile is an example,
see https://github.com/rsyslog/rsyslog/pull/2632 for a try to fix it
on the module level - we will replace that with the new capability
described here).
The central capability works on message submission, and so cannot be
circumvented. It has these capabilities:
* overisze message handling modes:
- truncate message
- split message
this is of questionable use, but also often requested. In that mode,
the oversize message content is split into multiple messages. Usually,
this ends up with message segments where all but the first is lost
anyhow as the regular filter rules do not match the other fragments.
As it is requested, we still implemented it.
- accept message as is, even if oversize
This may be required for some cases. Most importantly, it makes
quite some sense when writing messages to file, where oversize
does not matter (accept from a DoS PoV).
* report message to a special "oversize message log file" (not via the
regular engine, as that would obviously cause another oversize message)
This commit, as the title says, handles oversize INPUT messages.
see also https://github.com/rsyslog/rsyslog/issues/2190
closes https://github.com/rsyslog/rsyslog/issues/2681
closes https://github.com/rsyslog/rsyslog/issues/498
Note: this commit adds global parameters:
* "oversizemsg.errorfile",
is used to specify the location of the oversize message log file.
* "oversizemsg.report",
is used to control if an error shall be reported when an oversize
message is seen. The default it "on".
* add global parameter "oversizemsg.input.mode"
is used to specify the mode with which oversized messages will
be handled.
the config system actually does not permit "-" inside parameter
names. This has now been changed to "shutdown.enable.ctlc".
Note: this was never released and only present in 8.33 pre-release
versions.
closes https://github.com/rsyslog/rsyslog/issues/2482
