Why:
The de-flake campaign exposed lookup-table reload tests that could resume
after HUP while the reload worker was still installing the replacement
table. The wait helper saw no pending reload and injected messages against
stale or stub lookup state.
Impact:
Lookup reload waiters now observe the full reload lifecycle.
Before/After:
Before, only queued reload requests were pending; after, an active reload
also remains pending until the table swap completes.
Technical Overview:
Track the interval after the reloader consumes do_reload but before
lookupDoReload() returns. lookupPendingReloadCount() now treats that
interval as pending, so imdiag's AwaitLookupTableReload command cannot
return while a reload is still applying. Initialize and clear the new state
alongside the existing reloader flags to keep startup, activation, and
shutdown state consistent.
Validation:
- ./autogen.sh --enable-debug --enable-testbench --enable-imdiag --enable-omstdout
- make -j$(nproc) check TESTS=""
- ./tests/array_lookup_table.sh
- ./tests/lookup_table_bad_configs.sh
- git diff --check
- Ubuntu 26.04 dev container focused lookup reload run, 9/9 pass:
array_lookup_table.sh array_lookup_table-vg.sh
array_lookup_table_misuse-vg.sh lookup_table_bad_configs.sh
lookup_table_bad_configs-vg.sh lookup_table_rscript_reload.sh
lookup_table_rscript_reload-vg.sh
lookup_table_rscript_reload_without_stub.sh
lookup_table_rscript_reload_without_stub-vg.sh
With the help of AI-Agents: OpenAI Codex
Why: old crash/hang reports showed lookup-table shutdown could
touch pthread state that had not been fully activated.
Impact: daemonized lookup-table shutdown now has focused
regression coverage.
Before/After: reloader state was assumed; destruction now
follows explicit initialized and started flags.
Technical Overview:
Track initialized pthread primitives and the reloader thread
start state in lookup_ref_t. Only join the reloader and
destroy synchronization objects when activation created them.
Move lookup reloader thread naming into the running thread via
pthread_self(), avoiding use of an uninitialized pthread_t.
Add lookupKeyLocked() so RainerScript lookup() keeps table and
key-type consistency without recursively acquiring the rwlock.
Add a daemon-mode shutdown regression with lookup_table() and
lookup().
closes https://github.com/rsyslog/rsyslog/issues/1071
closes https://github.com/rsyslog/rsyslog/issues/5301
With the help of AI-Agents: Codex, Copernicus
Why: duplicate lookup_table names create unreachable table entries and can
exhaust reloader resources before reporting the real configuration problem.
Impact: configurations with duplicate lookup_table names now fail validation
with a clear diagnostic. Unique lookup tables and reload recovery for
initially invalid table files remain unchanged.
Before and after: duplicates were accepted and only the first table was usable.
Now duplicate names are rejected before allocating another table.
Technical Overview:
- Parse lookup_table name, file, and reloadOnHUP before creating the table
object.
- Check the current config set for an existing lookup table name.
- Report duplicate names as RS_RET_CONFIG_ERROR with a stable message.
- Keep existing registration behavior for initially invalid table files so
later HUP reload can recover them.
- Add RainerScript and YAML regression tests for duplicate lookup table names.
closes https://github.com/rsyslog/rsyslog/issues/5316
With the help of AI-Agents: Codex
Why:
Older platforms need consistent formatted string allocation, and the
remaining copy helpers kept triggering review noise around classic C
string APIs.
A major motivation is to avoid very common AI review false positives:
those tools often do not understand the actual scope and safety checks,
and then mechanically flag strcpy-style APIs despite the surrounding
bounds and initialization logic being correct.
Impact: string allocation and bounded copy paths are now explicit and
portable across the tree.
Before/After: ad hoc unsafe string helpers remained; now allocation and
bounded copies follow one portable pattern.
Technical Overview:
Add a complete asprintf and vasprintf compatibility layer with shared
prototypes so older libc variants build without local wrappers.
Replace repo-wide strcpy, strcat, strncat, sprintf, and direct strncpy
uses with explicit memcpy-based bounded copies or exact-width byte
copies as appropriate for each destination.
Add rsCStrAppendParts() for incremental string assembly so callers can
build pre-sized buffers without repeated snprintf return handling.
Update the unicode helper copy routine so existing ustrncpy() call sites
no longer route to libc strncpy semantics.
This also removes a broad class of review distractions from automated AI
reviewers that key off banned function names without understanding the
actual copy contract at the call site.
Extend the stringbuf unit coverage for the new append helper and the
formatted-allocation compatibility path.
With the help of AI-Agents: Codex
Why
Guard obvious zero-size allocation edges and remove a couple of
stale or misleading notes in the imtcp/tcpsrv path so future
reviews see the intended behavior directly in code.
Impact
Empty key and lookup-table files now fail before zero-size
allocations; OpenSSL accepted sessions use per-session callback
context.
Before/After
Before, empty files could flow into malloc(0)-style paths and the
OpenSSL accept path stored listener-owned callback context.
After, those runtime paths fail early and accepted OpenSSL sessions
carry their own callback context.
Technical Overview
Reject empty key files in the libgcry and OpenSSL runtime helpers
before allocating buffers from sb.st_size.
Reject empty lookup-table files before allocating and parsing the
JSON payload.
Update accepted OpenSSL sessions to store pNew-owned callback state
in SSL ex-data instead of the listener object's pointers.
Remove a stale TODO in the imtcp octet-count parser because the
zero-count case already falls into the framing-error path.
Add a short comment that maxReads==0 is intentional and documented
behavior that disables starvation protection.
With the help of AI-Agents: Codex
Previously, the `lookupTableReloader` thread held the `reloader_mut` mutex
for the entire duration of the reload operation (`lookupDoReload`). Since
reloading a lookup table involves file I/O and parsing, it can take a
significant amount of time. If a SIGHUP (or other reload trigger) occurred
while a reload was in progress, the main thread (executing `lookupReload`)
would fail to acquire the mutex via `pthread_mutex_trylock`, resulting in
the error "Device or resource busy" and the reload request being dropped.
This commit modifies `lookupTableReloader` to release the mutex before
calling `lookupDoReload` and re-acquire it afterwards. This allows
`lookupReload` to successfully acquire the mutex, queue a new reload request
(by setting `do_reload = 1`), and update the stub value if necessary.
To ensure thread safety, `lookupDoReload` now accepts the `stub_val` as an
argument (instead of accessing the shared `stub_value_for_reload_failure`
member directly) and takes ownership of it (freeing it upon completion).
`lookupTableReloader` extracts the stub value to a local variable while
holding the lock before passing it to `lookupDoReload`.
The `ATTR_NONNULL` attribute on `lookupDoReload` was updated to `ATTR_NONNULL(1)`
to allow the `stub_val` argument to be NULL, fixing a build failure with
`-Werror`.
Fixes#5628
Co-authored-by: rgerhards <1482123+rgerhards@users.noreply.github.com>
This commit applies the new canonical formatting style using `clang-format` with custom settings (notably 4-space indentation), as part of our shift toward automated formatting normalization.
⚠️ No functional changes are included — only whitespace and layout modifications as produced by `clang-format`.
This change is part of the formatting modernization strategy discussed in:
https://github.com/rsyslog/rsyslog/issues/5747
Key context:
- Formatting is now treated as a disposable view, normalized via tooling.
- The `.clang-format` file defines the canonical style.
- A fixup script (`devtools/format-code.sh`) handles remaining edge cases.
- Formatting commits are added to `.git-blame-ignore-revs` to reduce noise.
- Developers remain free to format code however they prefer locally.
This commit performs a broad modernization of widely used rsyslog
macros to align with modern C practices and support automated
formatting tools like clang-format. The changes focus on improving
syntactic regularity, readability, and tooling compatibility — without
altering behavior.
Macros refactored in this commit now follow a consistent,
statement-like form with explicit trailing semicolons. Where
applicable, macro blocks that define module interfaces (`queryEtryPt`)
have been updated to use simple `if` statements instead of `else if`
chains. While this slightly increases evaluation time, the affected
functions are only called once per module during load time to register
supported interfaces — making the performance cost irrelevant in
practice.
These improvements serve multiple purposes:
- Enable reliable clang-format usage without mangling macro logic
- Simplify reasoning about macro-expanded code for human readers
- Reduce style drift and merge conflicts
- Facilitate development for contributors using assistive tools
- Support future formatting pipelines using:
1. `clang-format`
2. a post-fixup normalization script
Refactored macros:
- MODULE_TYPE_NOKEEP
- MODULE_TYPE_KEEP
- MODULE_TYPE_INPUT
- MODULE_TYPE_OUTPUT
- MODULE_TYPE_FUNCTION
- MODULE_TYPE_PARSER
- MODULE_TYPE_LIB
- DEF_IMOD_STATIC_DATA
- DEF_OMOD_STATIC_DATA
- DEF_PMOD_STATIC_DATA
- DEF_FMOD_STATIC_DATA
- DEFobjStaticHelpers
- SIMP_PROP(...)
And all `queryEtryPt()` dispatch macros:
- CODEqueryEtryPt_STD_MOD_QUERIES
- CODEqueryEtryPt_STD_OMOD_QUERIES
- CODEqueryEtryPt_STD_OMODTX_QUERIES
- CODEqueryEtryPt_STD_OMOD8_QUERIES
- CODEqueryEtryPt_TXIF_OMOD_QUERIES
- CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
- CODEqueryEtryPt_STD_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_QUERIES
- CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
- CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES
- CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
- CODEqueryEtryPt_STD_PMOD_QUERIES
- CODEqueryEtryPt_STD_PMOD2_QUERIES
- CODEqueryEtryPt_STD_FMOD_QUERIES
- CODEqueryEtryPt_STD_SMOD_QUERIES
- CODEqueryEtryPt_doHUPWrkr
- CODEqueryEtryPt_doHUP
This general modernization reduces macro misuse, improves DX, and
lays the foundation for a robust, automated style normalization
system.
See also: https://github.com/rsyslog/rsyslog/issues/5747
Commit ffaf6dc4620da added proper variable sync, but dropped the check
if active count had actually changed. As such, the output was always
generated, which could pollute the log heavily.
also fixes some codestyle issues introduced by earlier commits.
With the help of AI-Agent: Codex 2025-06
While we do not like rexeg-matches for performance reasons, they are
well known and appreciated by users. With the new table type, we
add a lookup capability for partial matches, but at the price
of much higher ressource use. It still is useful, e.g. to classify
events as "noise" events in a simple manner.
If something goes really wrong, a lookup table's name would not
be set. That could lead to a NULL pointer access. HOWEVER, this
would require serious bugs in config parameter parsing, as the
lookup table name is a required parameter and the parser will
error out if not set.
So the bug is mostly cosmetic - but it does not hurt to handle
this case, of course.
Based on discussion, the error level is being adjusted to reflect
that the action (reloading the lookup table) is normal/expected
and not an actual error condition.
refs rsyslog/rsyslog#2713
Trying to understand CID 185407, which looks like a false positive. On
the way to this, add attributation which could increase correctness of
compiler code unerstanding and trigger different set of error messages.
lookup table could loop if error in lookupDoStub() occurs
also fixes coverty scan CID 185315 (IDENTICAL_BRANCHES due to
CHKiRet() immediately followed by finalize_it)
