This replaces the former clang static analyzer check via buildbot, which needed quite some baby-sitting due to environment constraints. For the next time, run it via github action.
Note that we use the same core script to run the analyzer like we did previously. This means we can also switch back with ease.
Modernize our dev/CI matrix to match current distro lines and reduce
toolchain drift. This keeps project-provided images aligned with what
contributors run locally and moves us toward “project-supported” bases.
Impact: CI matrix refresh only; no runtime behavior change intended.
Before: CI used Fedora 35/36 and Debian 10/11 dev-base images.
After: CI uses Fedora 41/42 and Debian 11/13 dev-base images.
Update run_checks.yml to use new container tags and pass consistent
configure extras (incl. enabling dtls in relevant jobs). Add Dockerfiles
and helper scripts for debian:13 and fedora:{41,42}. Bump the cached
ZooKeeper tarball to 3.9.3 across legacy images. Debian 13 adds a small
libgcrypt-config wrapper that defers to pkg-config to keep builds working.
Fedora images refresh toolchains (clang18 et al.) and re-sync module flags.
No API/ABI changes; this is build/CI plumbing only.
closes: https://github.com/rsyslog/rsyslog/issues/5989
This adds repository-level instructions to guide GitHub Copilot in
producing higher quality and safer pull requests. It is part of the
"AI First" approach, aiming to improve contributor experience and
reduce review friction.
Impact: no change to runtime or user-facing behavior.
The new files under `.github/` provide scoped rules for Copilot across
documentation, plugins, and contrib modules. They emphasize alignment
with AGENTS.md, developer checklists, and module maps. Guardrails cover
quoting, doc structure, config conventions, and mandatory source
citations. The intent is to ensure AI-assisted edits follow existing
rsyslog patterns and remain minimal and test-backed.
These changes are additive only and have no effect on rsyslog code,
APIs, or operational semantics. They prepare the repo for more
consistent AI-assisted contributions.
This modernizes contributor experience by simplifying PR templates and
removing the lengthy GDPR disclaimer that often discouraged or confused
new contributors. The update encourages more participation and shows how
responsible AI can be used to improve open source workflows.
Impact: none on runtime behavior; contributor workflow improved.
Before: PR template included long GDPR block; commit message rules were
scattered and partly implicit.
After: PR template is concise, GDPR text removed, and commit assistant
usage is documented across README, CONTRIBUTING, and AGENTS.md.
Technical changes include:
- PR template: drop GDPR notice, add commit-assistant references.
- CONTRIBUTING.md: add explicit commit rules and workflow guidance.
- AGENTS.md: require canonical base prompt and commit-first workflow.
- README.md: point to assistant and updated guidance.
- base_prompt.txt: enforce "Findings:" colon format.
- Minor formatting corrections in comments.
Introduce a GitHub Actions workflow that automatically runs
devtools/format-code.sh on every pull request touching source files.
If formatting changes are detected, the check fails and provides clear
instructions for contributors—run the formatter locally and enable
“Allow edits from maintainers” so fixes can be applied.
This prevents code‑style drift and eliminates style‑only update noise.
This patch improves the GitHub Actions PR validation workflow:
- Added comprehensive inline documentation for clarity.
- Corrected branch reference from 'master' to 'main'
in `tests/CI/PR_validation.sh`.
These changes enhance CI/CD clarity and maintainability.
With support from AI Agents: Gemini
- Consolidate change detection into a single “Check for doc changes” step
- Remove separate determine_changes_type and doc_only_success_check jobs
- Ensure the CI job always returns an exit status (satisfies branch protection)
- Update checkout action version and file‐pattern list for code changes
- Bump copyright year to 2025
Create doc_build.yml for GitHub Actions. The workflow builds
Sphinx docs when *.rst files under doc/ change. It still runs for
other pull requests but exits early so branch protection succeeds
without running make.
AI-Agent: Codex
AI tools and newcomers often execute test scripts
without running autoreconf, configure or make. This
causes missing binaries and confusing failures.
The diag.sh stub now self-bootstraps when needed:
- Run autoreconf -fi if configure is missing.
- Run configure with testbench flags if Makefile is
missing.
- Build tests/tcpflood and tools/rsyslogd via
make check TESTS="".
This makes tests more robust against improper
environment setup.
Enhances the 'compile check' GitHub Actions workflow to intelligently
skip resource-intensive compilation when Pull Requests (PRs) only
involve documentation or other non-source code modifications.
This optimization improves CI efficiency and reduces build times for
documentation-only changes, while still enforcing full compilation for
actual code modifications.
Key changes include:
- **Removed `paths-ignore`**: The workflow now always triggers, allowing for granular job control.
- **Introduced `determine_changes_type` job**: This job uses `tj-actions/changed-files` to accurately detect if a PR consists solely of ignored files (e.g., `ChangeLog`, `*.md`, `*.txt`, `doc/**`).
- **Conditional `doc_only_success_check` job**: A lightweight job that runs and passes quickly when only documentation changes are present, satisfying branch protection requirements.
- **Conditional `run` job**: The main compilation job now only executes if actual source code changes are detected, preventing unnecessary builds.
This ensures that all required status checks for branch protection are met,
whether the PR touches code or just documentation.
This commit significantly enhances imtcp by introducing a fully
functional worker thread pool, enabling true multi-threaded
processing for better scalability under high loads. This is
particularly beneficial when using TLS connections.
Notable changes:
- Implemented a complete worker pool for imtcp.
- Introduced the `workerthreads` config parameter for tuning
concurrency.
- Improved epoll efficiency by enabling edge-triggered mode.
- Added starvation handling via `starvationProtection.maxReads`.
- Refactored session accept logic and optimized network object
handling.
- Removed an obsolete network driver layer for event notification.
- Fixed multiple issues related to message timing, EPOLLERR
handling, and tests.
- Improved performance in poll() mode by reducing redundant
allocations.
- Introduced new CI tests for imtcp without epoll.
- Allowed disabling imtcp tests via a new configure switch.
- Added new impstats counters for worker thread pool statistics.
Details:
- The worker pool replaces an outdated experimental
implementation.
- If `workerthreads=1`, no worker pool is created to minimize
context switches.
- Moves worker pool variables inside `tcpsrv` instance to
prevent conflicts.
- Extracts session `accept()` logic into a dedicated function
for clarity.
- Fixes message ordering inconsistencies in multi-threaded
scenarios.
- Properly handles `EPOLLERR` notifications to improve error
resilience.
- Optimizes poll() mode by avoiding unnecessary reallocation
of file descriptors.
- Replaces the old network driver layer for event notification
with a streamlined solution.
- Now uses **conditional compilation** to select the best
method (epoll or poll) at build time.
- This significantly reduces code complexity, improves
maintainability, and boosts performance.
- The previous "thread pool" was a rough experiment that did
not perform significantly better than single-threaded mode.
- The **new implementation** allows multiple worker threads
on platforms with `epoll`.
- On non-epoll systems, an optimized **poll() based
single-threaded approach** is used, which is expected to
perform better than the old "thread pool."
- Adds `pthread_setname_np` only when available to improve
portability.
- Fixes test cases that assumed strict message timing, which
was unreliable.
- Reduces test parallelism for TSAN CI runs to prevent
resource exhaustion.
- Moves a test case to `imdiag` to ensure stable execution.
- Provides a new CI environment to verify `imtcp` behavior
without epoll.
- Introduces `--enable-imtcp-tests` configure switch for test
flexibility.
- Improves debug logging and adds better error handling for
worker pool startup.
New configuration parameters:
- `workerthreads`: Defines the number of worker threads for
imtcp. If set to 1, no worker pool is created.
- `starvationProtection.maxReads`: Defines the maximum number
of consecutive reads a worker can perform before being
interrupted to allow other sessions to be processed.
New impstats counters (emitted only when `workerthreads > 1`):
- `runs`: Number of times the worker thread has been invoked.
- `read`: Number of read calls performed by the worker.
For TLS, this includes read/write calls.
- `accept`: Number of `accept()` calls handled by the worker.
- `starvation_protect`: Number of times a socket was sent
back to the queue due to reaching the maximum number of
consecutive requests, ensuring fair scheduling of sessions.
These changes significantly enhance rsyslog’s TCP handling
performance and stability, particularly in high-volume
environments.
Closes#5529, #5532, #5578, #5580.
We are seeing an increasing number of flaky tests with hard-to-explain
failure conditions. Upon analysis it looks like this primarily stems
back to overloaded CI VMs. This can be the case because we have
performance enhanced rsyslog of the past month, which means the VMs
have higher load (due to better parallel processing) at the same time.
This is especially the case with CPU intense checks like TSAN.
In order to address this, we reduce the number of parallel tests.
The platforms that do not have support for atomic operations become
more and more exoctic. Even more so, only Solaris builders now test
that inside the rsyslog CI. To "harden" the CI system against a
potential (temporary) unavailability of Solaris, we now also do
a build where we intentionally disable atomics. This will trigger
build issues on all platforms. This method also permits us to detect
problems quicker, as the solaris builders are slow.
Note: we temporarily disable omamqp1 testing while we search for
the missing dependency.
We also add a special setting that permits us to control if
known-(very)-flaky test shall be run.
closes https://github.com/rsyslog/rsyslog/issues/5546
Note: The upcoming gnu23 C standard is overdoing it with type-safety. Inside
rsyslog, we historically have method tables for generic calls, which
keeps the code small and easy to understand. This would not decently be
possible with the new type-safety requirements.
So this commit works around these warning in a way that pretends to
provide more type safety. We have done this in the least intrusive
way to reduce the risk for regressions in code that works well in
since decades. Also note that the code already does parameter
validation.
There would have been more elaborate ways to make gnu23 compile happy,
e.g. by using a union of structs to provide the data element. Some folks
consider this type safe. In reality, it is not a bit better than
traditional C without types at all. Because the caller still needs to
ensure it picks the right struct from the union. As this approach
would also have larger regeression potential, we have not used it.
Right now, we have suppressed some of the new warnings, as working
around them would have required an even larger time budget and
potentially larger regression potential. In the long term we may
want to look into enabling them, as they would potentially be
beneficial for new code not involving method tables.
Some nits, however, were detected and have been fixed.
This patch also "fixes" some false positive test failures, mostly
be disabling some test functionality after confirmation these are
flakes.
see also https://github.com/rsyslog/rsyslog/issues/5507
also bump default development container to Ubuntu 24.04
This commit also contains some nitfixes that were detected
by the newer CI components. If it is just nitfixes, we do not
create separate commits for them.
- Extracted basic OpenSSL helper functions into own module net_ossl.h/net_ossl.c
Both are compiled into lmnsd_ossl.
- Cleanup of OpenSSL code, fixed minor compiler and linking issues.
- Added DTLS Sender option DTLS into tcpflood for testbench.
- Add initial implementation of imdtls input module. Added to configure and makefile
- Add initial implementation of omdtls output module. Added to configure and makefile
- Add multiple basic tests for imdtls receiving data by using tcpflood.
- Add multiple send-receive test for imdtls and omdtls based on existing tls tests.
- Add timeout and sessionbreak tests for imdtls stress testing.
closes: https://github.com/rsyslog/rsyslog/issues/5211
FileCreateMode allows to set the default file mode bits
when creating new files. As of now, it has only impact on the state file.
Add test suite as well.
Minor indentation fix in run_journal.yml
This is just the "host" for containers, so no big deal. Looks like
Ubuntu 18.04 is no longer supported by github, at least all actions
hang in "queued" status without proceeding.
The zstd library provides better and faster compression than zlib.
This patch integrates zstd as a dynamically-loadable functionality.
As such, no further dependencies need to be added to the rsyslog
base package.
Due to the increased performance, usage of zstd is highly recommended
for high-volume use cases.
This patch also refactor zlib compression in order to unify handling
in both compression cases.
Matrix feature was initially barely usable because github did not permit
to restart just failed jobs. This has changed. So we now migrate back to
the use of matrix feature where it is useful.
We also reduce the number of plain compile tests to newest and oldest
compiler version only. This saves CI ressources. The important other
ones (distro default!) are used during other CI tests anyways.
We also fix some minor coding errors detected by new compiler builds.
None of them seems to be related to any real issues.
