This parses the LEEF message (if it is) and creates a JSON subtree.
The current implementation is PoC and will be provided to gather
early review.
Changes, including breaking changes, may happen in future versions of
this module.
We invalidly specified the source distribution, which does not include
the necessary class files. This lead to zookeeper start failure and thus
no kafka tests being executed.
* tests: improve kafka startup readiness handling
* add generic TCP wait helper and kafka readiness check that replaces fixed sleeps in start_kafka
* verify zookeeper client port availability after startup
* replace ad-hoc sleeps in kafka-focused tests with wait_for_kafka_startup
* rely on kafka-topics probe instead of port checks
* share kafka layout helper to avoid duplicated logic
* align kafkapid emptiness checks for consistent style
With the help of AI-Agent: ChatGPT
Non-technical: test flakiness makes it hard to validate unrelated changes.
This aligns omelasticsearch tests with ES 7.14 defaults to get the
testbench back to a deterministic state and pave the way for further
modernization.
Impact: test behavior changes; one test skipped; CI coverage slightly reduced.
Before: tests mixed ES 6-era types and ad-hoc tarball picks; deprecation
checks intermittently failed and retries were brittle. After: tests use the
7.14.1 tarball via diag.sh default, typeless mappings, and `_doc` type in
omelasticsearch actions; known-flaky bulk-retry test is skipped for now.
Technical details:
- Add `searchType="_doc"` to all omelasticsearch actions and update index
provisioning to typeless mappings compatible with ES 7.14.
- Remove script-level `ES_DOWNLOAD` overrides to follow diag.sh's 7.14.1
default, keeping test scripts and helper defaults in sync.
- Drop deprecation-log assertion in `es-searchType-empty.sh` to prevent
spurious failures specific to ES 6-era types.
- Temporarily skip `es-bulk-retry.sh` (exit 77) pending a rewrite of retry
semantics under ES 7.x.
- CI: export `VERBOSE=1`; disable Kafka and Elasticsearch tests in the
affected matrix job to keep CI green while ES/Kafka suites are refactored.
VERBOSE ensures test logs are emitted to stderr and as such are
visible in CI test runs.
- Minor whitespace/indent cleanups; no runtime code or plugin behavior
changes.
Split the module parameter docs into standalone reference pages and
replace the inline tables with list tables that include the summaries.
Add the hidden toctree for the new parameter files.
Provide different file name for zookeeper tar - it looks like apache
changed the name (not sure, but it worked previously).
Also now cached the file on rsyslog.com, so that we have reliable
access even on name change or apache download rate-limiting.
Admins often report "gibberish" when a TLS-enabled sender connects to a
plain imtcp port. Making the mismatch explicit reduces operator confusion
and support churn, and points directly to remediation.
Impact: logs one explicit error per mismatched connection; no change to
parsing or transport on plain listeners.
Before/After: before, ClientHello bytes were ingested as binary with no
hint; after, imtcp detects a TLS ClientHello on ptcp and logs a clear
message with a troubleshooting URL.
Technically, we add a small per-session probe in tcps_sess_t and sample
the first 5 bytes of new sessions. If the record header matches a TLS
handshake (type 0x16, version 0x03.00–0x04, length 40–16384) and the
listener is plain TCP (streamDriver.mode=0), we emit a single error and
disable further probing for that session. The probe is called from
DataRcvd() and returns RS_RET_SERVER_NO_TLS when triggered; the session
is otherwise left untouched.
Runtime: introduce RS_RET_SERVER_NO_TLS (-2465) to tag the condition.
Docs: add imtcp troubleshooting section and a dedicated FAQ page.
Tests/tools: add test imtcp-tls-gibberish.sh and extend tcpflood with
-H to send only a ClientHello (OpenSSL and GnuTLS paths tolerate early
termination and non-blocking I/O for this mode).
This commit introduces cross-platform helper scripts to streamline the
rsyslog documentation build process and updates the build instructions
for better user experience.
Changes:
- Add build-doc-linux.sh: Linux helper script with virtual environment
management, pip bootstrapping, and flexible build options
- Add build-doc-windows.ps1: Windows PowerShell script with equivalent
functionality for Windows environments
- Update doc/BUILDS_README.md: Add quick links and simplified build
instructions for maintainers
- Update doc/README.md: Add Linux distribution package installation
instructions and quickstart guide with helper script usage
Key features:
- Automatic virtual environment creation and management
- Fallback support for systems without python3-venv (uses virtualenv)
- Cross-platform compatibility (Linux/Windows)
- Support for multiple output formats (HTML, EPUB)
- Strict mode option for treating warnings as errors
- Automatic pip bootstrapping for environments without ensurepip
- Clean build option to remove previous build artifacts
The scripts handle common pain points in documentation builds:
- Missing python3-venv package on some distributions
- Missing pip in virtual environments
- Complex manual setup procedures
- Platform-specific differences
This improves the contributor experience by providing one-command
documentation builds while maintaining backward compatibility with
manual setup procedures.
Co-authored-by: alorbach <alorbach@adiscon.com>
AI-Agent: Cursor
Add kafkaHeader parameter to define key/value pairs
that are attached as headers to every produced message.
Require librdkafka v0.11 for header support. Update
configure checks, docs and add a regression test.
closes: https://github.com/rsyslog/rsyslog/issues/5185
With help of AI-Agent: OpenAI ChatGPT
The AC_CHECK_PROG macro requires a value-if-found parameter
and an optional value-if-not-found parameter
Fix by adding the value-if-found parameter
Also handle the situation where there's no libgcrypt.pc
file for libgcrypt versions before 1.8.4
Improve discoverability of rsyslog-specific AI helpers for users and
contributors. This supports onboarding and troubleshooting and aligns
with the project's AI-first strategy.
Impact: docs-only; adds a new page under Getting Started navigation.
there was "workaround" introduced in the container which prevented
to detect some issues in libgrypt-related code. This was probably done
to get newer version in dev env without the need to adapt code to
broken libgrypt API level in new version.
That prevented issues in https://github.com/rsyslog/rsyslog/pull/5406
to be deteced.
We are redesigning the new user onboarding doc. This commit
provides very first experience documentation. Some is duplicate
content to the previous version, which we keep for now.
A cleanup will follow later. We want to gain some feedback first.
Thus the early merge.
Handle post-handshake KeyUpdate by driving a minimal non-blocking read when the
TLS library requests READ during Send(). This prevents stalls when servers send
TLS 1.3 KeyUpdate and aligns behavior with RFC 8446 §4.6.3.
- nsd_ossl.c: SSL_ERROR_WANT_READ => small SSL_read(), then retry write
- nsd_gtls.c: E_AGAIN/E_INTERRUPTED with READ direction => small gnutls_record_recv(), then retry write
Backward-compatible and only active when the TLS stack signals a need to read.
closes: https://github.com/rsyslog/rsyslog/issues/5627
most importantly, header file now includes comments that enable
tooltip-like behaviour in IDEs. Also includes antipaterns, which
is useful for developers and hopefully also for AI to detect
them e.g. in code reviews (and get it right in AI-generated code).
This moves NetworkNamespace functionality into
the net module. This allows the same code to
be reused across multiple tools and plugins.
The first usage is with omfwd, which is changed
to use the common net implementation. Note
the net implementation is based on the original
omfwd implementation. Subsequent PRs will be
opened for integrating this into omuxsock and
imtcp.
The original test case tcp_forwarding_ns_tpl.sh
was broken due to use of single quotes rather
than double quotes, thus preventing the proper
port number argument to be passed to the
listener. Note this test must be run as
root.
Development has occurred across Fedora41 and
Fedora42, which uses glibc 2.40 and glibc2.41
respectively. The valgrind suppressions are
updated to handle new glibc issues
accordingly.
Four new functions are callable through the
net module. These allow one to save a handle
to the current namespace, switch to a new
namespace by name, and restore the namespace
from the saved handle. A fourth higher
level wrapper is used to open a socket in
a named network namespace, and handles the
invocation of the lower level functions.
Ideally this would be the only public
function, however it simplifies integration
into imtcp in the future (which doesn't
directly open sockets). This may change
in the future as network namespaces are
integrated into more modules and plugins.
Signed-off-by: Billie Alsup <balsup@cisco.com>
