Work towards full completion of the module’s interface as it exits PoC.
The new name senderid (formerly: template) matches intent (identify a
sender, not an output format) and avoids confusion. Also make state
persistence safer via atomic writes.
Merged a bit pre-completion in order to get a static analyzer fix
into the main code base. Some additional cleanup PR will follow.
State is much cleaner now than in PoC.
Impact: Configs must use senderid=; statefile is now required. Tests
and docs updated.
Before/After: action(... template="name") -> action(... senderid="name").
Technical details:
- Replace the config parameter and instance field; drop legacy template
parsing in parseSelectorAct. The module now reads only senderid=.
- Enforce required statefile (descriptor + runtime) and cache a temp
path "<statefile>.tmp" to ensure rename() is atomic. Provide a
one-time fallback build if the cache is missing (e.g., reload flow).
- Add built-in template "StdOmSenderTrack-senderid" yielding
"%fromhost-ip%" and use it as the default senderid template.
- Improve memory ownership (free cached temp path/default template) and
keep existing locking/queue semantics unchanged.
- Update tests to use senderid= and prune PoC doc note accordingly.
see also: https://github.com/rsyslog/rsyslog/issues/5599
createInstance() left stateFile and bMain uninitialized. When a
statefile was configured, activateCnf() attempted to free the garbage
pointer, leading to free(): invalid pointer at startup. Initialize
these fields to sane defaults.
Fixes https://github.com/rsyslog/rsyslog/issues/5730
AI-Agent: ChatGPT
Trailing semicolons in atomic helper macros caused double
semicolons when the macros expanded, producing build warnings.
Macros now omit semicolons and call sites add them explicitly.
STATSCOUNTER_DEF updated to terminate the generated mutex line.
AI-Agent: ChatGPT
When the main thread cancel imjournal thread, the thread exits without
setting sd_journal to NULL because sd_journal_close() contain cancel point.
This leads to a double free scenario where:
1. The thread cancel occurs during sd_journal_close()
2. The main thread then calls imjournal's afterrun function
3. sd_journal_close() is called again on the already-freed sd_journal
This item was create in 2007 and has proven in ~20yrs of practice
to be no issue. Even more so, this is now in legacy code that will
no longer be updated at all. So the TODO no longer applies and could
be removed.
Removed the global pthread_rwlock_t rwlock_hmysql and related locking
around worker-local MYSQL *hmysql handles. Each worker owns its handle
exclusively, so no cross-thread access occurs and the global lock adds
unnecessary overhead.
Commit a3b2983 (2021) introduced this lock without clearly stating the
underlying issue. A thorough architectural review and targeted analysis
found no flaw in rsyslog’s code or in the mariadb/mysql client libraries.
Web research also showed no known relevant defects. The most plausible
explanation is that the original change addressed a CI-induced race,
which would be consistent with other CI flake fixes we have made in
recent years.
Given the absence of a reproducible problem and the fact that the
locking code itself introduced theoretical races (never observed in
practice), we are reverting a3b2983’s behavior. We will monitor for
any regressions.
closes https://github.com/rsyslog/rsyslog/issues/5890
We need to validate the locking method - in theory, multiple access to
the mysql handle should not be possible, because instance data is
worker-local. However, the following 2021 commit suggests there were some
issue. I tend to say the commit was wrong, but this needs to be checked
in more depth, while we need some better doc on the current locking method
right now. So we document existing and valid behaviour and will follow up later.
related commit:
a3b2983342
Introduce a new lifecycle callback—`checkParserInst`—to perform
configuration sanity checks on parser instances immediately after they’re
created. This establishes a standardized validation point (similar to
`checkCnf` in other module types) without altering existing parser logic.
By wiring `checkParserInst` into:
- the module template (macros for definition and registration),
- the module loader (`doModInit`) with graceful fallback,
- the runtime configuration flow (`rsconf.c`) just after
`newParserInst`,
and by providing empty stubs in all current parser modules (contrib and
plugins), we now have a clear, uniform spot to add parser-specific
validation rules in subsequent patches. This improves future
maintainability and robustness of parser configuration handling.
Initialize cs.fCreateMode to 0644 instead of -1 to prevent state files
from being created with all permission bits set when legacy configuration
is used. The setModCnf function is only called with modern configuration,
leaving cs.fCreateMode at -1 for legacy configs, which results in wrong
open() calls.
The omelasticsearch module previously failed to correctly handle HTTP
error status codes returned by the server. The logic checked for
libcurl transport errors but explicitly ignored
'CURLE_HTTP_RETURNED_ERROR', causing server-side errors to be
treated as successful submissions.
This led to silent failures in critical scenarios, especially
authentication (401 Unauthorized) and authorization (403
Forbidden) errors. The plugin would not suspend or report an
error, and data could be lost without any clear error indication.
This commit resolves the issue by:
- Retrieving the HTTP status code after each request using
'curl_easy_getinfo'.
- Adding an explicit check for 401 and 403 status codes.
- Logging a specific "authentication failed" error and suspending
the plugin if these errors are detected.
This ensures that users receive immediate and clear feedback on
authentication problems, preventing silent data loss and aiding in
faster troubleshooting.
With AI support: Codex, Gemini
This commit applies the new canonical formatting style using `clang-format` with custom settings (notably 4-space indentation), as part of our shift toward automated formatting normalization.
⚠️ No functional changes are included — only whitespace and layout modifications as produced by `clang-format`.
This change is part of the formatting modernization strategy discussed in:
https://github.com/rsyslog/rsyslog/issues/5747
Key context:
- Formatting is now treated as a disposable view, normalized via tooling.
- The `.clang-format` file defines the canonical style.
- A fixup script (`devtools/format-code.sh`) handles remaining edge cases.
- Formatting commits are added to `.git-blame-ignore-revs` to reduce noise.
- Developers remain free to format code however they prefer locally.
This commit adds the initial implementation of the `mmaitag` module,
a pluggable AI-based message classification processor.
The module supports two providers:
- `gemini`: uses the Gemini API via libcurl and JSON-C
- `gemini_mock`: a deterministic in-memory mock used for testing
Each log message is classified individually and tagged with a label
stored in a configurable message variable (default: `$.aitag`).
Included:
- Provider abstraction layer (`ai_provider.h`)
- Gemini provider with API key support (inline or via file)
- Prompt customization and input property selection
- Two regression tests: basic and invalid-key scenarios
- Sphinx documentation under `doc/configuration/modules`
Note: mmaitag is in its infancy and primarily a PoC. Future work
will improve batching, performance, and add more providers.
Refs:
- https://github.com/rsyslog/rsyslog/issues/5747 (AI-first strategy)
With the help of AI Agents: Codex, Gemini
This commit extends the impstats input module to support a new
"prometheus" format for exporting statistics. The output is
generated via statsobj.c, which already provides Prometheus-
compatible formatting.
To enable this, a new "format" config option value ("prometheus")
was added and recognized by impstats.c. When selected, impstats
invokes the appropriate statsobj function to generate output in
Prometheus exposition format.
The default behavior remains unchanged. Existing formats such as
"json" and "legacy" are not modified.
A new test has been added under tests/ to validate correct
Prometheus output generation. The test verifies that the format
header and at least one representative metric line conform to the
Prometheus text format specification.
This enhancement enables rsyslog to integrate more easily with
Prometheus-based monitoring stacks and paves the way for native
metrics scraping without external translation layers.
When enabled, state files are deleted when the watched file is moved or
rotated away. This prevents accumulation of stale state files from
rotated logs. By default the state file is kept. Does not change
default behavior for existing inputs. The config option is available as
per-input and per-module parameter.
This commit performs a broad modernization of widely used rsyslog
macros to align with modern C practices and support automated
formatting tools like clang-format. The changes focus on improving
syntactic regularity, readability, and tooling compatibility — without
altering behavior.
Macros refactored in this commit now follow a consistent,
statement-like form with explicit trailing semicolons. Where
applicable, macro blocks that define module interfaces (`queryEtryPt`)
have been updated to use simple `if` statements instead of `else if`
chains. While this slightly increases evaluation time, the affected
functions are only called once per module during load time to register
supported interfaces — making the performance cost irrelevant in
practice.
These improvements serve multiple purposes:
- Enable reliable clang-format usage without mangling macro logic
- Simplify reasoning about macro-expanded code for human readers
- Reduce style drift and merge conflicts
- Facilitate development for contributors using assistive tools
- Support future formatting pipelines using:
1. `clang-format`
2. a post-fixup normalization script
Refactored macros:
- MODULE_TYPE_NOKEEP
- MODULE_TYPE_KEEP
- MODULE_TYPE_INPUT
- MODULE_TYPE_OUTPUT
- MODULE_TYPE_FUNCTION
- MODULE_TYPE_PARSER
- MODULE_TYPE_LIB
- DEF_IMOD_STATIC_DATA
- DEF_OMOD_STATIC_DATA
- DEF_PMOD_STATIC_DATA
- DEF_FMOD_STATIC_DATA
- DEFobjStaticHelpers
- SIMP_PROP(...)
And all `queryEtryPt()` dispatch macros:
- CODEqueryEtryPt_STD_MOD_QUERIES
- CODEqueryEtryPt_STD_OMOD_QUERIES
- CODEqueryEtryPt_STD_OMODTX_QUERIES
- CODEqueryEtryPt_STD_OMOD8_QUERIES
- CODEqueryEtryPt_TXIF_OMOD_QUERIES
- CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES
- CODEqueryEtryPt_STD_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_QUERIES
- CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES
- CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES
- CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES
- CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES
- CODEqueryEtryPt_STD_PMOD_QUERIES
- CODEqueryEtryPt_STD_PMOD2_QUERIES
- CODEqueryEtryPt_STD_FMOD_QUERIES
- CODEqueryEtryPt_STD_SMOD_QUERIES
- CODEqueryEtryPt_doHUPWrkr
- CODEqueryEtryPt_doHUP
This general modernization reduces macro misuse, improves DX, and
lays the foundation for a robust, automated style normalization
system.
See also: https://github.com/rsyslog/rsyslog/issues/5747
* Add note for indentation cleanup
* Converted tabs to spaces in various Python scripts to match PEP 8 indentation.
With the help of AI-Agent: Codex, Google Gemini Code review
among others, remove some warning suppressions by "fixing" the
respective constructs with work-arounds (root cause is compilers
do not handle enums in switch well).
Easier to maintain than the off-project C source. The checks
are pretty trivial.
This also fixes some nits that the old checker did not detect.
closes https://github.com/rsyslog/rsyslog/issues/5631
Commit ffaf6dc4620da added proper variable sync, but dropped the check
if active count had actually changed. As such, the output was always
generated, which could pollute the log heavily.
also fixes some codestyle issues introduced by earlier commits.
With the help of AI-Agent: Codex 2025-06
The omsendertrack module is designed to track and report statistics for
message senders across all inputs in Rsyslog. It periodically outputs a
JSON file containing information about each sender.
Note: This commit provides minimalistic basic functionality as a PoC.
We will check it's usefulness in practice and expect follow-up PRs
to enhance functionality and include feedback from early testing.
However, this module is solid, just feature limited.
see also: https://github.com/rsyslog/rsyslog/issues/5599
