Rename the Snare Windows Security parser module from "mmsnarewinsec"
to "mmsnareparse" for clearer naming and consistency with other parser
modules. Update code identifiers, build system, docs, tests, CI flags,
and paths accordingly.
What changed
- MODULE_CNFNAME set to "mmsnareparse"; default macro renamed
- Log/error/debug tags updated to "mmsnareparse"
- plugins/mmsnarewinsec/* moved to plugins/mmsnareparse/*
- configure.ac:
- add --enable-mmsnareparse
- AM_CONDITIONAL(ENABLE_MMSNAREPARSE)
- AC_CONFIG_FILES now includes plugins/mmsnareparse/Makefile
- Makefile.am subdir switch to plugins/mmsnareparse
- Tests renamed and updated (scripts and testsuites directory)
- Docs page renamed and examples updated
- CI workflow uses --enable-mmsnareparse
- Rebase to main; resolved configure.ac conflict
Impact
- Backwards-incompatible module name and configure flag changes.
Migration
- Config: module(load="mmsnareparse"), action(type="mmsnareparse")
- Build: use --enable-mmsnareparse
Co-authored-by: alorbach <alorbach@adiscon.com>
Real-world tarballs must be self-contained for doc builds. This change
verifies release completeness by building the docs from the tarball,
not the checkout.
Impact: Release tarballs now include additional doc assets; CI fails
early if a doc-required file is missing.
Before: CI built Sphinx docs directly from the repo tree. Missing files
could be masked by in-tree paths. After: CI creates a 'make dist'
tarball, unpacks it, and builds docs from that tree to catch omissions.
Technically, the doc workflow installs build deps, runs autoreconf and
a minimal './configure', issues 'make dist', unpacks into
'doc-builder/', and runs 'make html' under 'doc-builder/doc'. The
artifact path is updated accordingly.
For distribution, 'configure.ac' now composes DOC_FILES from
'doc/source' (*.rst, *.conf, *.jpg, *.png) and explicitly adds
'tutorials/cert-script.tar.gz', 'doc/Makefile', and
'doc/ai/module_map.yaml'. The duplicate listing of
'doc/ai/module_map.yaml' is removed from 'Makefile.am' EXTRA_DIST to
avoid drift; DOC_FILES remains the single source of truth for doc
payloads.
* ci/tests: add Elasticsearch 8 workflow
- allow the testbench to honour RSYSLOG_TESTBENCH_EXTERNAL_ES_URL
so externally managed Elasticsearch services can be reused without local
start/stop logic
- provision Elasticsearch 8 via GitHub Actions and run the omelasticsearch
suite inside the development container
With the help of AI-Agent: ChatGPT
Implement complete NXLog Snare-formatted Windows Security event parser
with multi-format support (RFC5424/RFC3164), 100+ field patterns, and
advanced features including GUID/IP/timestamp type detection, runtime
configuration, enhanced validation modes, and comprehensive test suite.
Features:
- Parse major Windows security event types (4624, 4625, 4634, etc.)
- Extract structured data into configurable JSON containers (!win default)
- Handle modern Windows telemetry (LAPS, TLS, WDAC, WUFB, Kerberos)
- Type-aware parsing with validation and fallback handling
- Runtime configuration support for custom field patterns
- Thread-safe design with no shared mutable state
- 9 comprehensive test scripts covering all functionality
Impact: Enables structured analysis of Windows Security events for
SIEM integration, threat detection, and compliance reporting while
preserving original payloads for forensic investigation.
Files: contrib/mmsnarewinsec/, tests/mmsnarewinsec-*.sh,
doc/source/configuration/modules/mmsnarewinsec.rst
Add comprehensive macOS CI support with two new GitHub Actions
workflows:
- run_macos.yml: PR-triggered CI with matrix strategy covering macOS
13–15, x64/arm64 architectures, and sanitizer combinations (none,
ASAN, TSAN)
- run_macos_weekly.yml: Scheduled weekly testing with full matrix
coverage and automated failure reporting via GitHub issues
- Set sin_len in tests/diagtalker.c on macOS to fix connect() EINVAL,
unblocking TLS certvalid tests.
Root cause and fix details (macOS testbench)
- Why tests failed
On macOS 14 the imdiag control listener often ended up IPv6-only.
The plain TCP listener creates an IPv6 socket and sets IPV6_V6ONLY;
the companion IPv4 bind can fail on macOS when sharing an ephemeral
port, leaving only the IPv6 listener active. Our injector
tests/diagtalker.c was IPv4-only (AF_INET to 127.0.0.1), so it could
not reach the imdiag port, causing repeated connect retries and
timeouts. CI logs showed “cannot connect to 127.0.0.1:<port> …
Connection refused” alongside benign OpenSSL anon-mode warnings.
- What we changed
1) Made the injector dual-stack by switching diagtalker to
getaddrinfo(AF_UNSPEC) and trying both IPv6 and IPv4 (with fallback
to 127.0.0.1 and ::1). This removes the hard dependency on IPv4
reachability when the listener is IPv6-only on macOS.
2) Added an opt-in testbench knob to enforce IPv4 where appropriate:
generate_conf() now honors RSTB_FORCE_IPV4=1 (or
RSTB_NET_IPPROTO=ipv4-only) to inject
global(net.ipprotocol="ipv4-only"). We enable this only in the IPv4
test variant so the IPv6 wrapper remains pure IPv6.
- Impact
The injector/listener address-family mismatch is eliminated, resolving
the macOS connect() failures and unblocking the TLS “certvalid” and
anonymous tests on macOS runners.
Refs: https://github.com/rsyslog/rsyslog/issues/5635
Refs: https://github.com/Homebrew/homebrew-core/pull/221869
Refs: https://github.com/Homebrew/homebrew-core/pull/226378
Add an openEuler 24.03 LTS development container and wire it into CI to
validate builds on that platform.
Why: expand RPM-based coverage and catch distro-specific build issues
early.
Impact: CI-only. No runtime or API changes.
Before: no openEuler container or CI job; build breakage went unnoticed.
After: dedicated container and matrix entry compile and run unit tests
on openEuler.
Notes: module and test coverage may differ on openEuler; track gaps in
follow-up issues.
Non-technical: test flakiness makes it hard to validate unrelated changes.
This aligns omelasticsearch tests with ES 7.14 defaults to get the
testbench back to a deterministic state and pave the way for further
modernization.
Impact: test behavior changes; one test skipped; CI coverage slightly reduced.
Before: tests mixed ES 6-era types and ad-hoc tarball picks; deprecation
checks intermittently failed and retries were brittle. After: tests use the
7.14.1 tarball via diag.sh default, typeless mappings, and `_doc` type in
omelasticsearch actions; known-flaky bulk-retry test is skipped for now.
Technical details:
- Add `searchType="_doc"` to all omelasticsearch actions and update index
provisioning to typeless mappings compatible with ES 7.14.
- Remove script-level `ES_DOWNLOAD` overrides to follow diag.sh's 7.14.1
default, keeping test scripts and helper defaults in sync.
- Drop deprecation-log assertion in `es-searchType-empty.sh` to prevent
spurious failures specific to ES 6-era types.
- Temporarily skip `es-bulk-retry.sh` (exit 77) pending a rewrite of retry
semantics under ES 7.x.
- CI: export `VERBOSE=1`; disable Kafka and Elasticsearch tests in the
affected matrix job to keep CI green while ES/Kafka suites are refactored.
VERBOSE ensures test logs are emitted to stderr and as such are
visible in CI test runs.
- Minor whitespace/indent cleanups; no runtime code or plugin behavior
changes.
Motivation: code coverage reports were incomplete. This lays a better
base for consistent reporting via GitHub Actions, with room for follow-ups.
It also removes a test flake source in Kafka jobs.
Impact: CI/tests only; no runtime behavior or ABI changes expected.
Before: Coverage uploads were inconsistent; Kafka tests could hang while
reading from /dev/urandom to generate topic names.
After: Coverage is collected with lcov and uploaded via a dedicated GH
Action; Kafka topics use fast $RANDOM-based hex, avoiding early-boot
entropy stalls.
Technical details:
- Add two workflows: "codecov base" and "codecov kafka" on Ubuntu 24.04.
Use lcov capture with unexecuted blocks and prune common noise; upload
with token for same-repo PRs and tokenless for forks.
- Update .codecov.yml: add path fixes for container (/rsyslog) and
runner layouts; explicitly set comment: false and patch: false.
- Bump actions/checkout to v4 in existing workflows; add an actionlint
job to catch YAML problems early.
- Switch codecov jobs in container matrix to 24.04 images.
- Improve run-ci.sh lcov invocation to be more tolerant of line/macro
mismatches.
- Testbench: replace /dev/urandom topic generation with 8-char hex from
$RANDOM; adjust diag.sh path/quoting for zookeeper helper.
The civetweb library is not packaged in a thread-safe way and as such
TSAN always fails due to a civetweb data race by calling gmtime.
Nothing to do so far against that.
We try sid to gain early access to new compiler versions. It needs
to be show if sid causes too many false positives (due to its own
instability) and if we manage to regenerate the compiler frequently
enough.
Publish PR previews for both `pull_request` and `pull_request_target`
events. Previously, the preview job only ran for `pull_request`, so runs
from `pull_request_target` were skipped.
Details
- Build Sphinx docs with rsyslog Makefile (make html, warnings-as-errors).
- Deploy preview to GitHub Pages under "pr-<num>/" using configure,
upload, and deploy actions.
- Post/update PR comment with live preview and workflow/artifacts links.
- Cleanup preview folder on PR close.
- Grant required permissions (pages, id-token, issues, pull-requests);
keep yamllint clean.
- Remove old "Deploy GitHub Pages" job.
- Trigger on both PR events; include event name in concurrency group to
avoid cross-run cancellation.
- Checkout PR head SHA for PR events (fallback to github.sha).
- Gate heavy steps on `doc/**/*.rst` changes for both PR event types.
- Restrict publish to same-repo PRs; fork PRs remain excluded by policy.
Impact
- PR doc previews publish for same-repo PRs regardless of which PR event
triggered the workflow.
- Non-doc PRs still skip heavy steps; no change in behavior there.
Co-authored-by: alorbach <alorbach@adiscon.com>
Replace deprecated bash uploader with Codecov's official uploader.
The script now downloads the uploader binary, verifies its SHA256
checksum, and runs it to submit coverage results.
The change keeps run-ci.sh CI-agnostic.
With the help of AI-Agent: ChatGPT
Add concurrency groups with cancel-in-progress to all pull request
workflows so new pushes abort outdated jobs.
Most importantly this saves ressources and also makes follow-up
PR runs much quicker.
with the help of ChatGPT
Add a GitHub Actions job using codespell to check Sphinx docs in doc/source.
This helps catch spelling errors early and improves documentation quality.
With help from AI-Agent: ChatGPT
- fixed typos, corrected small text issues, and adjusted minor wording in messageparser.rst and multi_ruleset.rst.
- removing second 'be' & changing the verb form from activate to activates.
Only updates to documentation files (.rst, .md, .txt) for minor fixes. No source code or runtime behavior is affected.
* CI: fix welcome message activation mode to make it really work
We need to run the repo-provided script to have sufficient acces rights.
Otherwise, it will not work with PRs from forked repos (which is the
norm).
Plus some robustness amendments.
* Update .github/workflows/contributor-welcome.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
intent is two-fold:
- act welcoming and provide some guidance for PR contributors
- do a bit of checking so maintainer get's an idea if this might
be a spammy account (the world is a bad place...). Also
prevent rsyslog from being used for social engineering other
repos.
This method will be improved in the future and is work in progress.
Note: rsyslog welcomes small PRs, even single typo fixes, when
they are useful to the project. We shall, however, reject frequent
single typo fixes from same account and ask them to combine the
commits. Reason is this might be a social engineering tactic to
make rsyslog itself or other repos build unrooted trust in the
account.
Multiple intermediate CAs were not honored because strtok() modified
the original config value at the first comma. This led to only the
first file being considered. This change preserves the original value
and validates files using a duplicate buffer.
Impact: Users can reliably specify multiple CA intermediates via
NetstreamDriverCAExtraFiles; TLS chains that require intermediates
now validate as expected. Tests added.
Before: parsing mutated the stored string, effectively truncating at
the first comma and ignoring subsequent CA files.
After: we strdup() for validation, leave the original string intact,
and set it only if all files are accessible. Error handling follows
existing RS_RET patterns; temporary buffer is freed on all paths.
Additionally, add a test (OpenSSL backend) that generates a root,
two intermediates, and leaf certs on both client and server. The test
verifies chains with openssl, aligns CN/PermittedPeer, and exercises
a full send/receive path to guard against regressions. Makefile is
updated to include the new test in TESTS and EXTRA_DIST.
Fixes: https://github.com/rsyslog/rsyslog/issues/5207
