This improves code readability and maintainability by correcting
typos in comments. While non-functional, clear comments reduce
confusion for contributors and support long-term project quality.
Before: comments contained minor typos and inconsistencies.
After: comments use corrected spelling and clearer wording.
Impact: none (no functional or behavioral changes).
The patch updates comment text only, without modifying logic,
interfaces, or runtime behavior. No changes to queue semantics,
transactions, or module interactions are introduced.
This aligns with ongoing maintenance efforts to keep the codebase
clean and easier to understand for contributors and reviewers.
Fixes: https://github.com/rsyslog/rsyslog/issues/6023
AI-Agent: Copilot 2026-03
Non-technical: centralize and reuse rate-limit definitions so admins
can apply consistent policies across listeners. This is part of an
ongoing series to improve rate limiting and its manageability.
Before: inputs set per-listener interval/burst ad hoc.
After: inputs can reference a named ratelimit() policy shared across
listeners; per-listener values remain as fallback.
Impact: New ratelimit() object and RateLimit.Name param for imtcp/imptcp.
If a policy file is configured but libyaml is unavailable, config fails.
Technical details:
- Add top-level ratelimit() Rainerscript object. Parsed in rsconf and
stored in a central registry (hashtable + rwlock) on rsconf.
- New runtime API: ratelimitAddConfig(), ratelimitNewFromConfig(), plus
cfgs init/destruct on rsconf lifecycle.
- imtcp/imptcp accept RateLimit.Name; when set, tcpsrv/imptcp build the
ratelimiter from the named policy; otherwise legacy interval/burst is
used. Thread-safety retained via ratelimitSetThreadSafe().
- tcpsrv gains ownership helpers for listener params and frees them on
errors; imtcp explicitly transfers ownership and nulls the pointer.
- Optional libyaml: detected at configure; runtime parser loads simple
key/value policy files (interval, burst, severity).
- Docs: new ratelimit object page; imtcp/imptcp parameter references and
module docs updated; design-decisions note added for libyaml.
- Tests: add ratelimit_name.sh (guarded for imtcp+imptcp) to validate
named policy application and observable throttling.
Refs: https://github.com/rsyslog/rsyslog/issues/6201
With the help of AI-Agents: Antigravity
imudp: add ratelimit.name support
This commit adds the `ratelimit.name` parameter to imudp, allowing listeners
to utilize the global rate limit registry (shared state).
Features:
- New `ratelimit.name` string parameter.
- Integration with `ratelimitNewFromConfig`.
- Strict mutual exclusivity: specifying `ratelimit.name` prohibits the use
of legacy per-listener parameters (`ratelimit.burst`, `ratelimit.interval`).
If a conflict occurs, an error is logged and the named rate limit takes precedence.
- Updated documentation.
- New regression test `tests/imudp_ratelimit_name.sh`.
With the help of AI Agent: Google Antigravity
Non-technical: improves operator ergonomics and closes a feature gap
with imptcp. Enables regex-based start-of-frame detection and optional
multi-line message handling on TCP inputs.
Impact: New config params; defaults keep existing behavior unchanged.
Before: imtcp framed messages via octet-counting or LF delimiter only.
After: imtcp can treat lines not starting a new frame as continuations
(MultiLine) and can split frames on a regex start pattern.
Technical:
- Adds imtcp params: MultiLine (bool) and framing.delimiter.regex (string).
Regex compilation happens in tcpsrv on listener creation; errors if
regex is set without FEATURE_REGEXP.
- tcps_sess adds a regex-aware path that tracks current-line offset,
runs the compiled regex on line starts, and uses a second buffer to
handle split packets cleanly. On >2x max-line without a match, we
submit and reset to avoid unbounded growth.
- Introduces input state eInMsgCheckMultiLine and LF lookahead to decide
continuation vs new frame; when at buffer end, defers the decision to
the next packet.
- Updates processDataRcvd signature to accept a movable cursor and
buffer length for lookahead; DataRcvd passes these and advances the
pointer accordingly.
- Wires regexp object usage in tcpsrv/tcps_sess init/exit; frees compiled
patterns on listener teardown and error paths. Tests cover both new
code paths (regex framing and multi-line).
Closes https://github.com/rsyslog/rsyslog/issues/5637
This implements certificate revocation checking using OCSP (RFC 6960)
for the OpenSSL network stream driver. The feature is disabled by
default and can be enabled via the new StreamDriver.TlsRevocationCheck
configuration parameter.
This is a rebased and refactored version of the original implementation
by Daniel Gollub from June 2020, updated to work with the current main
branch and enhanced with proper plumbing, security hardening, tests,
and documentation.
OCSP Implementation:
- Implements OCSP (RFC 6960) for certificate revocation checking
- Supports OCSP over HTTP transport protocol (HTTPS not implemented)
- Supports Nonce extension for replay protection
- Uses "strict" revocation policy (any OCSP error fails verification)
- Does not support TLS OCSP stapling
- CRL-only certificates are not supported
Configuration Parameter:
- New parameter: StreamDriver.TlsRevocationCheck (binary, default: off)
- Can be set at module or input level
- Disabled by default for backward compatibility and to avoid
unexpected blocking I/O in existing configurations
- Only applies to OpenSSL driver (not available for GnuTLS/mbedTLS)
Usage:
module(load="imtcp" StreamDriver.Name="ossl"
StreamDriver.Mode="1"
StreamDriver.AuthMode="x509/name"
StreamDriver.TlsRevocationCheck="on")
Full Plumbing Through Network Stack:
- imtcp: Added iStrmTlsRevocationCheck parameter parsing and config
- tcpsrv: Added DrvrTlsRevocationCheck field and SetDrvrTlsRevocationCheck()
- netstrms: Added Set/Get functions for revocation check configuration
- netstrm: Added SetDrvrTlsRevocationCheck() pass-through
- nsd interface: Bumped version 18 -> 19, added SetTlsRevocationCheck()
- nsd_ossl: Implemented SetTlsRevocationCheck(), stores flag in SSL ex_data
- nsd_gtls: Added stub returning RS_RET_VALUE_NOT_SUPPORTED
- nsd_mbedtls: Added stub returning RS_RET_VALUE_NOT_SUPPORTED
- nsd_ptcp: Added stub returning RS_RET_VALUE_NOT_SUPPORTED
Security Hardening:
- Fixed OCSP_basic_verify() to not use OCSP_TRUSTOTHER flag (prevents
forged OCSP responses from rogue responder certificates)
- Added Content-Length validation (1MB limit) to prevent memory
exhaustion attacks from malicious OCSP responders
- Changed SSL ex_data index from 2 to 3 to avoid collision with imdtls
- Added proper struct field initialization and copying in AcceptConnReq
- Added socket read/write timeouts (SO_RCVTIMEO/SO_SNDTIMEO) to prevent
indefinite blocking during OCSP response I/O (BIO_gets, BIO_write,
d2i_OCSP_RESPONSE_bio operations now bound by OCSP_TIMEOUT)
Compatibility:
- Added OpenSSL 1.0.2 compatibility (CentOS 7 support)
- Disabled OCSP for WolfSSL builds (API not available)
- Fixed variable shadowing warnings
Known Limitations (documented in code and user documentation):
- OCSP checks perform blocking network I/O (DNS + socket operations)
during TLS handshake, which can cause latency of up to 5 seconds
per OCSP responder
- Potential DoS vector: malicious certificates with multiple slow/
unresponsive OCSP responder URLs can block worker threads
- No async OCSP support or response caching (future enhancement)
Tests:
- imtcp-tls-ossl-revocationcheck-off.sh: Verifies parameter can be
set to "off" and normal TLS operation works
- imtcp-tls-gtls-revocationcheck-error.sh: Verifies error message
when attempting to enable OCSP with unsupported GnuTLS driver
Documentation:
- Created comprehensive parameter reference page
- Added EXPERIMENTAL FEATURE warning about blocking I/O and DoS risks
- Integrated into imtcp module documentation
- Documented usage examples and important considerations
Changes from original implementation by Daniel Gollub:
- Moved OCSP functions from nsd_ossl.c to net_ossl.c (new location
for SSL helper functions in current codebase)
- Updated to use SSL_CTX directly instead of separate trusted_issuers
- Added full parameter plumbing through all network stack layers
- Added StreamDriver.TlsRevocationCheck configuration parameter
- Added security hardening (OCSP_TRUSTOTHER fix, Content-Length
validation, ex_data index collision fix, socket timeout fix)
- Added OpenSSL 1.0.2 and WolfSSL compatibility
- Added support for all NSD drivers (stub implementations)
- Added comprehensive tests and documentation
- Fixed variable shadowing and compiler warnings
- Adapted to current code structure and formatting standards
Original-Author: Daniel Gollub <dgollub@att.com>
Co-authored-by: Daniel Gollub <dgollub@att.com>
With the help of AI-Agents: GitHub Copilot CLI
Worker threads need to be assigned the "rsyslog default stack size".
This is especially an issue with musl lib, which has a very small stack
size. As such, rsyslog can segfault without specifying our expected
stack size.
closes https://github.com/rsyslog/rsyslog/issues/6357
The worker pool is normally stopped inside Run(), but when the input
thread terminates unexpectedly (for example via pthread_cancel during a
shutdown race) we can reach the destructor while workers are still
running. Those workers retain pointers into pLstnPorts and associated
ratelimit instances; freeing that memory while they are active results
in use-after-free crashes (observed with Valgrind when restarting imtcp
while omfwd is connected over TLS).
Ensure the pool is torn down here as a last-resort safety net so every
worker has exited before we release listener state.
* imtcp: include peer ports in runtime log messages
Ensure tcpsrv and session diagnostics report the sender's source port alongside the existing host/IP fields so administrators can pinpoint connections precisely. This updates imtcp connection notices, close/error paths, and framing warnings to use a consistent "host (ip:port)" style already present in TLS mismatch logs.
With the help of AI-Agent: Codex
This builds on "PR#6121 net: Add NetworkNamespace APIS"
to add Network Namespace support to imtcp module. This
extends imtcp to support a wider range of Unix/Linux
environments (or any environment supporting network
namespaces).
The imtcp module is enhanced to accept a NetworkNamespace
parameter, both as a default at the module level, and
on a per-instance basis.
The tcpsrv module is enhanced to allow the NetworkNamespace
to be applied to a listener's configuration parameters.
Finally, the netstrm module is enhanced to switch namespaces
before invoking the downstream (driver specific) LstnInit
function.
A new test imtcp-netns (and associated imtcp-netns-vg) is
added to test this functionality. This must be run as root
(technically it must be run by a user with CAP_SYS_ADMIN
capabilities, as network namespace creating/change is
required).
A slight change to diag.sh is made to allow passing $RS_REDIR
to valgrind (as $RS_REDIR is used in the imtcp-netns.sh
test for some negative cases).
Signed-off-by: Billie Alsup <balsup@cisco.com>
Some deployments need to disambiguate multiple senders sharing an IP,
for example autossh or similar tunnel setups. Exposing the source port
improves observability and lets pipelines key on a stable tuple.
Impact: new property/JSON field; tcps_sess IF v4; out-of-tree modules
must rebuild.
Before: messages exposed fromhost and fromhost-ip only.
After: messages also expose fromhost-port and jsonmesg includes it.
Introduce PROP_FROMHOST_PORT and wire it through msg.{h,c}. For TCP,
capture the remote port on accept, store it in tcps_sess, and attach it
to the msg on submit. For other inputs, resolveDNS derives the port from
the sockaddr when available; local inputs return an empty string. Add a
getter, duplication and destructor handling, and name<->ID mapping. Add
the field to jsonmesg output. Update docs, lexer keywords, and the
external plugin interface doc (property is modifiable). Bump
tcps_sessCURR_IF_VERSION to 4 and add SetHostPort() to the interface.
Include a focused test (fromhost-port.sh) that verifies the property.
Non-technical rationale: allow identification by (fromhost-ip,
fromhost-port) where IP alone is shared across systems (e.g., autossh).
With help from AI-Agents: ChatGPT
Fix compilation issues on macOS/Darwin systems and enhance cross-platform
compatibility for BSD variants:
- Add Darwin-specific pthread_setname_np call in tcpsrv.c with enhanced
platform-specific conditional compilation
- macOS (__APPLE__): Single parameter, returns int
- FreeBSD/NetBSD: Two parameters, returns void
- Linux/glibc (default): Two parameters, returns int
This prevents compilation failures across all BSD systems where the
function signature and return type differ, while maintaining existing
compatibility with macOS and Linux systems.
- Add _PATH_UTMP fallback definition in omusrmsg.c for systems
without paths.h or missing _PATH_UTMP definition
- Remove trailing empty line in cfsysline.c for consistency
The change adds proper platform-specific conditional compilation
with clear documentation for each variant and maintains error
checking where the return value is available.
Impact: Fixes build regressions on Darwin and BSD systems while
preserving backward compatibility with existing platforms.
Refs: https://github.com/rsyslog/rsyslog/pull/6069
Refs: https://github.com/rsyslog/rsyslog/pull/5635
Refs: https://github.com/Homebrew/homebrew-core/issues/221869
Refs: https://github.com/Homebrew/homebrew-core/issues/226378
Improve maintainability and robustness of the TCP server by clarifying
locking/ownership, tightening invariants, and simplifying queueing.
Also fix a long-standing pragma macro typo across the tree.
Impact: Internal behavior only. EPOLL re-arm now occurs while holding
pSess->mut; starvation cap counts only successful reads.
Before/After:
Before: EPOLL re-arm happened after leaving the critical section; read
starvation cap counted loop iterations; closeSess() sometimes unlocked;
select_* helpers used on non-epoll path; enqueueWork() returned status.
After: EPOLLONESHOT is re-armed before unlocking; starvation cap counts
only RS_RET_OK reads; closeSess() never unlocks; poll_* helpers replace
select_*; enqueueWork() is void (best-effort).
Technical details:
- Replace notifyReArm() with rearmIoEvent() (EPOLL_CTL_MOD with
EPOLLONESHOT|EPOLLET; asserts efd/sock; logs on failure).
- doReceive(): explicit state machine; would-block path re-arms before
unlock; close path unlocks then calls closeSess(); starvation handoff
enqueues without re-arming.
- Initialize ioDirection for listener and session descriptors; add
assert(sock >= 0) and widespread ATTR_NONNULL annotations.
- startWrkrPool(): single finalize rollback (cancel/join partial
threads; destroy cond/mutex); stopWrkrPool(): destroy cond/mutex.
- enqueueWork(): FIFO append under lock and cond signal; returns void.
- Cleanup hardening on construct failure: free ppLstn, ppLstnPort,
ppioDescrPtr; free fromHostIP on SessAccept() error.
- Non-epoll: rename select_Add/Poll/IsReady -> poll_*; RunPoll() uses
poll_* and sets sane ioDirection defaults.
- Typo fix: standardize PRAGMA_IGNORE_Wswitch_enum in header and all
users (action.c, rainerscript.c, template.c, tcpsrv.c).
While at the time of merge I was confident it would fix a data race,
the root cause has now surfaced to be a simple state advancement
(single-threaded) bug. See commit c5fd73499 for details. As such
I revert this patch. it caused no harm, but complicates code, adds
a bit of computation and is no longer needed.
I cannot 100% outrule it might have addressed some edge cases.
I know an environment where we can verify this within the next
month or so. If, unexpectedly, this shows regressions, we can
re-enable the patch. But I am 99.99% sure it is not needed.
see also: https://github.com/rsyslog/rsyslog/pull/5806
see also: https://github.com/rsyslog/rsyslog/pull/5993
inQueue guarded against double-queuing of descriptors.
The server uses EPOLLONESHOT and only one dispatcher, so
a descriptor is enqueued at most once before being
processed. The extra flag and atomic helpers are dead
code. Removing them simplifies queue management.
AI-Agent: ChatGPT
This fixes a real-world segfault seen in production. The loop could
continue after closing a session and touch freed state on EPOLL builds.
The change makes the termination explicit to prevent crashes.
Impact: prevents segfaults in rare error paths; loop ends earlier after
a close, which may skip remaining iteration work for that session.
Before/After: before, the read/error branch could continue after
closeSess() and use freed pioDescr/pSess; after, we set do_run=0 and
exit the loop, avoiding any further dereference.
Technical: closeSess() on EPOLL removes the fd from epoll and frees the
io descriptor; callers must not access pioDescr or pSess after return.
We now set do_run=0 right after CHKiRet(closeSess(...)) in the read/
error path, matching the documented contract that both pointers become
invalid. We also improve closeSess() documentation to state pre/post
conditions, unlocking behavior, and epoll semantics, and fix a comment
typo ("epoll_Ctl"). No API/ABI change; action queues and stats are
unchanged. HUP and OMODTX semantics are unaffected.
This commit fixes a race condition that could occur when two threads tried to close the same TCP session simultaneously. This could lead to an "epoll_ctl failed: Bad file descriptor" error message.
The fix introduces an atomic flag `being_closed` to the `tcps_sess_t` struct. This flag is used to ensure that the session-closing logic is executed only once per session.
This commit also corrects the logic of the atomic check to prevent the race condition correctly.
This commit fixes a race condition that could occur when two threads tried to close the same TCP session simultaneously. This could lead to an "epoll_ctl failed: Bad file descriptor" error message.
The fix introduces an atomic flag `being_closed` to the `tcps_sess_t` struct. This flag is used to ensure that the session-closing logic is executed only once per session.
This commit applies the new canonical formatting style using `clang-format` with custom settings (notably 4-space indentation), as part of our shift toward automated formatting normalization.
⚠️ No functional changes are included — only whitespace and layout modifications as produced by `clang-format`.
This change is part of the formatting modernization strategy discussed in:
https://github.com/rsyslog/rsyslog/issues/5747
Key context:
- Formatting is now treated as a disposable view, normalized via tooling.
- The `.clang-format` file defines the canonical style.
- A fixup script (`devtools/format-code.sh`) handles remaining edge cases.
- Formatting commits are added to `.git-blame-ignore-revs` to reduce noise.
- Developers remain free to format code however they prefer locally.
This commit performs a broad modernization of widely used rsyslog
macros to align with modern C practices and support automated
formatting tools like clang-format. The changes focus on improving
syntactic regularity, readability, and tooling compatibility — without
altering behavior.
Macros refactored in this commit now follow a consistent,
statement-like form with explicit trailing semicolons. Where
applicable, macro blocks that define module interfaces (`queryEtryPt`)
have been updated to use simple `if` statements instead of `else if`
chains. While this slightly increases evaluation time, the affected
functions are only called once per module during load time to register
supported interfaces — making the performance cost irrelevant in
practice.
These improvements serve multiple purposes:
- Enable reliable clang-format usage without mangling macro logic
- Simplify reasoning about macro-expanded code for human readers
- Reduce style drift and merge conflicts
- Facilitate development for contributors using assistive tools
- Support future formatting pipelines using:
1. `clang-format`
2. a post-fixup normalization script
Refactored macros:
- MODULE_TYPE_NOKEEP
- MODULE_TYPE_KEEP
- MODULE_TYPE_INPUT
- MODULE_TYPE_OUTPUT
- MODULE_TYPE_FUNCTION
- MODULE_TYPE_PARSER
- MODULE_TYPE_LIB
- DEF_IMOD_STATIC_DATA
- DEF_OMOD_STATIC_DATA
- DEF_PMOD_STATIC_DATA
- DEF_FMOD_STATIC_DATA
- DEFobjStaticHelpers
- SIMP_PROP(...)
And all `queryEtryPt()` dispatch macros:
- CODEqueryEtryPt_STD_MOD_QUERIES
- CODEqueryEtryPt_STD_OMOD_QUERIES
- CODEqueryEtryPt_STD_OMODTX_QUERIES
- CODEqueryEtryPt_STD_OMOD8_QUERIES
- CODEqueryEtryPt_TXIF_OMOD_QUERIES
- CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
- CODEqueryEtryPt_STD_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_QUERIES
- CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
- CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES
- CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
- CODEqueryEtryPt_STD_PMOD_QUERIES
- CODEqueryEtryPt_STD_PMOD2_QUERIES
- CODEqueryEtryPt_STD_FMOD_QUERIES
- CODEqueryEtryPt_STD_SMOD_QUERIES
- CODEqueryEtryPt_doHUPWrkr
- CODEqueryEtryPt_doHUP
This general modernization reduces macro misuse, improves DX, and
lays the foundation for a robust, automated style normalization
system.
See also: https://github.com/rsyslog/rsyslog/issues/5747
This patch adds an inQueue flag with its own mutex to each
tcpsrv_io_descr_t structure. The flag prevents multiple worker threads
from processing the same descriptor at the same time.
The change was motivated by segmentation faults reported in production
systems after commit ad1fd21, which introduced a worker thread pool to
imtcp. We could not reproduce the faults ourselves, but code analysis
suggests several race conditions may exist.
In particular:
- epoll_wait may return the same descriptor multiple times. This is not
expected, as we use EPOLLONESHOT. However, if a thread does not clear
or re-arm the event quickly enough, or in edge cases involving race
conditions and rapid I/O activity, duplicate delivery may still occur.
- If a descriptor is enqueued more than once, multiple threads may
process and free it in parallel, causing use-after-free errors.
- closeSess releases the session mutex before destroying the session and
descriptor. A second thread might still be waiting to acquire the
mutex and access the now-freed memory.
- shutdown is unordered: stopWrkrPool waits for threads to join, but the
work queue may still contain descriptors that will be processed after
their memory has been freed.
- pending epoll events for a socket may still be processed after
epoll_ctl(..., DEL) was called, leading to access to invalid memory.
The patch:
- Adds an inQueue flag to each descriptor and a mutex to protect it.
- Prevents enqueueWork from queuing a descriptor already in queue.
- Clears the flag when dequeueing the descriptor.
- Initializes and destroys the new mutex at listener startup/cleanup.
While unverified, we believe this patch is a safe and helpful change.
It may fix the reported crashes and in general improves correctness.
The analysis and draft of this patch were created with help from a
Codex-based AI agent. Final review and edits were done by a human.
Performance will be evaluated during PR review.
This was an overlook during the refactoring of tcpsrv.c earlier in 2025.
We kept the line for some tests, but actually it needs to be removed
as notifyReArm() now has the proper code to handle these cases.
The left-over TODO could cause confusion during code review.
This fix prevents polluting the message log and solves a long-standing
testbench issue where it can occur during high test concurrency due to
sluggish timing.
among others, remove some warning suppressions by "fixing" the
respective constructs with work-arounds (root cause is compilers
do not handle enums in switch well).
Depending on circumstances, tcpsrv worker threads did effectively
busy-wait on io events to handle. Not always, but often. This was
caused be improperly re-arming the inotify subsystem.
This effected overall system performance, but not general rsyslog
stability. The bug was introduced on March 1st 2025 into the
daily stable build.
closes: https://github.com/rsyslog/rsyslog/issues/5623
As a result, it did not show up in pstats. Alos, we now use the input name
in worker thread to easily identify where they belong to. As thread names
have very limited length, the thread naming now is
"w<worker-number>/<input-name>".
This affects primarily imtcp. The counter was not maintained at all,
it now is incremented whenever a processing loop of the worker thread
is completed. Note that, for busy senders, can mean a large number
of iterations. The other counters tell the the details of operation.
That also means that very busy threads can be detected by looking at
the ratio of runs vs. actual operations.
This patch also fixes an invalid mutex being locked on systems that
do not support atomic instructions (a copy&paste errror).
This commit significantly enhances imtcp by introducing a fully
functional worker thread pool, enabling true multi-threaded
processing for better scalability under high loads. This is
particularly beneficial when using TLS connections.
Notable changes:
- Implemented a complete worker pool for imtcp.
- Introduced the `workerthreads` config parameter for tuning
concurrency.
- Improved epoll efficiency by enabling edge-triggered mode.
- Added starvation handling via `starvationProtection.maxReads`.
- Refactored session accept logic and optimized network object
handling.
- Removed an obsolete network driver layer for event notification.
- Fixed multiple issues related to message timing, EPOLLERR
handling, and tests.
- Improved performance in poll() mode by reducing redundant
allocations.
- Introduced new CI tests for imtcp without epoll.
- Allowed disabling imtcp tests via a new configure switch.
- Added new impstats counters for worker thread pool statistics.
Details:
- The worker pool replaces an outdated experimental
implementation.
- If `workerthreads=1`, no worker pool is created to minimize
context switches.
- Moves worker pool variables inside `tcpsrv` instance to
prevent conflicts.
- Extracts session `accept()` logic into a dedicated function
for clarity.
- Fixes message ordering inconsistencies in multi-threaded
scenarios.
- Properly handles `EPOLLERR` notifications to improve error
resilience.
- Optimizes poll() mode by avoiding unnecessary reallocation
of file descriptors.
- Replaces the old network driver layer for event notification
with a streamlined solution.
- Now uses **conditional compilation** to select the best
method (epoll or poll) at build time.
- This significantly reduces code complexity, improves
maintainability, and boosts performance.
- The previous "thread pool" was a rough experiment that did
not perform significantly better than single-threaded mode.
- The **new implementation** allows multiple worker threads
on platforms with `epoll`.
- On non-epoll systems, an optimized **poll() based
single-threaded approach** is used, which is expected to
perform better than the old "thread pool."
- Adds `pthread_setname_np` only when available to improve
portability.
- Fixes test cases that assumed strict message timing, which
was unreliable.
- Reduces test parallelism for TSAN CI runs to prevent
resource exhaustion.
- Moves a test case to `imdiag` to ensure stable execution.
- Provides a new CI environment to verify `imtcp` behavior
without epoll.
- Introduces `--enable-imtcp-tests` configure switch for test
flexibility.
- Improves debug logging and adds better error handling for
worker pool startup.
New configuration parameters:
- `workerthreads`: Defines the number of worker threads for
imtcp. If set to 1, no worker pool is created.
- `starvationProtection.maxReads`: Defines the maximum number
of consecutive reads a worker can perform before being
interrupted to allow other sessions to be processed.
New impstats counters (emitted only when `workerthreads > 1`):
- `runs`: Number of times the worker thread has been invoked.
- `read`: Number of read calls performed by the worker.
For TLS, this includes read/write calls.
- `accept`: Number of `accept()` calls handled by the worker.
- `starvation_protect`: Number of times a socket was sent
back to the queue due to reaching the maximum number of
consecutive requests, ensuring fair scheduling of sessions.
These changes significantly enhance rsyslog’s TCP handling
performance and stability, particularly in high-volume
environments.
Closes#5529, #5532, #5578, #5580.
A new "socketBacklog" parameter has been added to the imtcp module, allowing
users to override the default TCP SYN backlog size. Previously, the backlog
was set to roughly 10% of the configured max sessions, which remains the
default if the parameter is not specified. This enhancement enables better
configuration for high-performance servers. The parameter name aligns with
the "socketBacklog" parameter in imptcp for consistency.
The "socketBacklog" parameter should be set based on the anticipated connection
rate and the server's ability to handle incoming connections. For high-performance
environments with heavy traffic, a larger value may be needed to avoid dropped
connections during bursts. If unsure, leave the parameter unset to use the default
(10% of max sessions), which is suitable for typical workloads.
- output all loaded ciphers and engines.
- Add new global option "defaultopensslengine" to customize the
default openssl engine. If not defined, openssl will handle the
default engine automatically.
- Add simple openssl performance test with defaultopensslengine
set to rdrand (Intel).
- removed unneeded testcase files in runtime folder.
- corrected whitelist settings for debug.files in TLS testcases
Add this both as module an input parameter. Complements already-existing
config param notifyonconnectionclose and mirrors the similar feature from
imptcp.
The module parameter acts as default, similarly to notifyonconnectionclose.
Note that in contrast to imptcp, we emit IP addresses and not host
names. This sticks with the traditional semantics of imtcp.
Note that we also fixed a mislading error message in the case when a
disallowed sender tried to connect.
Thanks to John Chivian for suggesting the addition.
There was a rare possibility that the E_AGAIN/E_INTERRUPT handling
could cause an infinite loop (100% CPU Usage), for example when a TLS
handshake is interrupted at a certain stage.
- After gnutls_record_recv is called, and E_AGAIN/E_INTERRUPT error
occurs, we need to do additional read/write direction handling
with gnutls_record_get_direction.
- After the second call of gnutls_record_recv (Expand buffer)
we needed to also check the eror codes for E_AGAIN/E_INTERRUPT
to do propper errorhandling.
- Add extra debug output based on ossl driver.
- Potential fix for 100% CPU Loop Receiveloop after gtlsRecordRecv
in doRetry call.
see also: https://github.com/rsyslog/rsyslog/issues/4818
When the pthread library fails, errno is referenced even though errno is not set.
Fix to refer to the return code of the pthread library instead of errno.
tcpsrv.c:992:1: error: label at end of compound statement
finalize_it:
^~~~~~~~~~~
Quoting from pthread.h:
pthread_cleanup_push and pthread_cleanup_pop are macros and must always
be used in matching pairs at the same nesting level of braces.
Amends commit bcdd220142ec9eb106550195ba331fd114adb0bd.
Test should have been added to commit 3d23c7ac8aea but was forgotten.
This test also showed an issue on Solaris, where NULL values cannot
be used in printf functions. This has also been addressed.
see also: https://github.com/rsyslog/rsyslog/pull/4627
Modules (like imtcp and imdiag) which use tcpsrv could abort or
otherwise malfunction if no listener for a specific input could
be started.
Found during implementing a new feature, no report from practice.
But could very well happen.
