Add ignoreTrailingPattern.regex parameter to support POSIX extended
regular expressions for dynamic trailing data removal. This enables
matching patterns with variable prefixes (e.g., numeric prefixes) that
cannot be handled by the static ignoreTrailingPattern parameter.
The new parameter is mutually exclusive with ignoreTrailingPattern.
When a regex pattern is provided, it is compiled during action instance
creation using regcomp with REG_EXTENDED. Compilation errors are reported
via regerror and cause configuration load to abort. The compiled regex
is freed in freeInstance using regfree.
The detect_and_truncate_trailing_extradata function now checks
ignoreTrailingPattern_isRegex to determine whether to use strstr
(static) or regexec (regex) for pattern matching. For regex matches,
the entire last token (including the matched prefix) is truncated and
stored in $!extradata_section, consistent with static pattern behavior.
Documentation updated to clarify truncation behavior and provide regex
examples. Test suite extended with mmsnareparse-trailing-extradata-regex.sh
covering various numeric prefix scenarios. All test cases updated to use
"custom_section" terminology for consistency.
Impact:
- Backward compatible: existing ignoreTrailingPattern continues to work
- New functionality: regex support for dynamic pattern matching
- Configuration validation: mutual exclusivity enforced at load time
Tests:
- mmsnareparse-trailing-extradata.sh (regression test)
- mmsnareparse-trailing-extradata-regex.sh (new regex test)
Co-authored-by: alorbach <alorbach@adiscon.com>
... for trailing extra-data removal.
Add configurable mechanism to detect and remove trailing extra-data sections
from messages before parsing. This addresses cases where third-party enrichers
append non-standard data (e.g., "enrichment_section: fromhost-ip=...") that
can interfere with Snare event parsing.
The ignoreTrailingPattern parameter can be set at both module and action
levels, with action-level values overriding module defaults. When configured,
the parser searches for the pattern in trailing positions (after the last
tab-separated token). If found, the message is truncated at the start of the
last token, removing the entire trailing section including any preceding
content in that token (e.g., dynamic numeric prefixes).
The truncated extra-data section is optionally exposed as a !extradata_section
message property, allowing downstream processing to access the removed content
if needed (e.g., for extracting sender IP addresses).
Implementation details:
- Pattern matching is literal string-based (not regex)
- Truncation only occurs when pattern appears in valid trailing positions
- Conservative detection for non-tab messages (last 20% or 200 chars)
- Proper memory management for pattern strings and extra-data sections
- No changes to existing behavior when parameter is not set
Added test case mmsnareparse-trailing-extradata.sh with anonymized sample
data validating Event ID 13 parsing with trailing enrichment section.
Updated documentation in doc/source/configuration/modules/mmsnareparse.rst
with parameter description and usage notes.
docs: enhance AGENTS.md with WSL build/test instructions
Added complete dependency installation, module-specific configure examples,
test execution patterns, and debugging workflow based
on actual development sessions.
Co-authored-by: alorbach <alorbach@adiscon.com>
Add support for Microsoft Sysinternals Sysmon events to the mmsnareparse
plugin using an external JSON definition file (sysmon_definitions.json).
This enables generic parsing of Sysmon events without hardcoding
event-specific logic, making it extensible for other event channels.
Key changes:
- Enhanced locate_snare_payload() to detect Sysmon events when MSWinEventLog
is in syslog tag (RFC3164 parsing scenario)
- Updated populate_event_metadata() to extract Channel from raw message
when version/channel fields are removed by syslog parser
- Fixed key-value parsing to handle single-space-separated pairs in
Sysmon descriptions (e.g., "User: CORP\NETWORK SERVICE")
- Improved pattern selection to prefer EventData section patterns when
sectionName is NULL, fixing User field storage location
- Added sysmon_definitions.json with event type mappings and field patterns
- Added test case mmsnareparse-sysmon.sh validating Event IDs 1, 3, and 5
The implementation is generic and extensible - other event channels can
be added by creating additional JSON definition files following the same
structure.
Co-authored-by: alorbach <alorbach@adiscon.com>
Test imrelp-tls-cfgcmd.sh seems to fail but still report success, which
can create a false impression.
Suspected root cause is in librelp. It may also be that the test
actually succeeds, where it just looks like it failed because of abort
of tcpflood testing tool (due to librelp bug).
The whole point of the test is that an error is generated, and this may
very well happen. And only tcpflood aborts because of the librelp bug.
Core file detection than jumps in, and invalidly treats the tcpflood
core file as a test failure.
While this is investigated, the test will be skipped. Highly like that
this needs to be forwarded either to librelp or is a native tcpflood
bug.
Thanks to Chris Hofstaedtler for reporting the bug and Michael Biebl for
forwarding it upstream.
see also: https://github.com/rsyslog/rsyslog/issues/6267
Rename the Snare Windows Security parser module from "mmsnarewinsec"
to "mmsnareparse" for clearer naming and consistency with other parser
modules. Update code identifiers, build system, docs, tests, CI flags,
and paths accordingly.
What changed
- MODULE_CNFNAME set to "mmsnareparse"; default macro renamed
- Log/error/debug tags updated to "mmsnareparse"
- plugins/mmsnarewinsec/* moved to plugins/mmsnareparse/*
- configure.ac:
- add --enable-mmsnareparse
- AM_CONDITIONAL(ENABLE_MMSNAREPARSE)
- AC_CONFIG_FILES now includes plugins/mmsnareparse/Makefile
- Makefile.am subdir switch to plugins/mmsnareparse
- Tests renamed and updated (scripts and testsuites directory)
- Docs page renamed and examples updated
- CI workflow uses --enable-mmsnareparse
- Rebase to main; resolved configure.ac conflict
Impact
- Backwards-incompatible module name and configure flag changes.
Migration
- Config: module(load="mmsnareparse"), action(type="mmsnareparse")
- Build: use --enable-mmsnareparse
Co-authored-by: alorbach <alorbach@adiscon.com>
This tool requires rsyslog to build with TLS support. If not present,
the test will always fail. This is solved by executing it only if gnutls
is enabled. As this is a fequently tested environment, this does not
reduce test coverage. It is easier to do then checking for both gnutls
and openssl.
Many thanks to Michael Biebl for bringing this to our attention.
closes https://github.com/rsyslog/rsyslog/issues/6224
Replace the retry-count loop with a deadline-based poll so the helper\nfails after a sensible timeout instead of hanging when impstats stops\nemitting metrics. Use a longer default window for the Valgrind variant\nso it tolerates the slower runtime.\n\nAI-Agent: ChatGPT
Plain JSON embedded in text is common in production logs. This change
lets users parse such logs without cookies, improving ease of use and
lowering onboarding friction while keeping legacy behavior intact.
Before/After: cookie-only JSON -> find-json parses first top-level {}.
Impact: Default behavior unchanged. New mode and counters are opt-in.
Technical details:
- Add action parameter `mode` with `cookie` (default) and `find-json`.
The new mode scans for the first `{` and uses json_tokener to validate
a complete top-level object; quotes/escapes are respected.
- Add `max_scan_bytes` (default 65536) to bound scanning work and
`allow_trailing` (default on) to accept or reject non-whitespace data
after the parsed object. On reject/fail we return RS_RET_NO_CEE_MSG and
fall back to {"msg":"..."} while preserving parsesuccess semantics.
- Expose per-worker scan counters via statsobj/impstats and rsyslogctl:
scan.attempted, scan.found, scan.failed, scan.truncated. Counters are
active only in find-json mode and are resettable.
- Use length-aware cookie parsing (getMSG/getMSGLen) and keep legacy
RS_RET codes. Cookie mode behavior remains unchanged.
- Update docs: module overview, parameter references, statistics section
(impstats usage), and examples incl. mixed-mode routing. Add developer
engine overview page.
- Add tests for basic scanning, trailing control, scan limit, invalid
JSON, invalid mode, and parser validation edge cases.
With the help of AI Agent: Copilot
- wrap omfwd-lb-1target-retry-full_buf.sh around the shared skeleton so the scenario is retried once before reporting failure, easing TCP buffer timing races
- document follow-on stabilisation ideas for minitcpsrvr coordination and omfwd diagnostics
AI-Agent: Codex
* ci/tests: add Elasticsearch 8 workflow
- allow the testbench to honour RSYSLOG_TESTBENCH_EXTERNAL_ES_URL
so externally managed Elasticsearch services can be reused without local
start/stop logic
- provision Elasticsearch 8 via GitHub Actions and run the omelasticsearch
suite inside the development container
With the help of AI-Agent: ChatGPT
Real-world need: unflatten dotted JSON into nested objects, and optionally
flatten back for downstream tools. This introduces a general transformer
with a dedicated output tree. Interface is intentionally unstable.
Impact: New module behind --enable-mmjsontransform; no default behavior
changes. New tests and docs added. Parameters and behavior may change.
Add mmjsontransform, a message modification module that rewrites dotted
JSON keys. By default it "unflattens" an input object to nested containers
and stores the result in a configured output property. A mode parameter
also supports "flatten" to collapse nested trees into dotted keys. The
action refuses to overwrite an existing destination, validates that input
is a JSON object, and reports conflicts with precise key paths. Per-action
config is immutable; workers hold pointers only, so no extra locking. Docs
(Sphinx + parameter refs) and doxygen coverage included, plus a regression
test exercising nested arrays/objects. Build system and CI scripts gain
--enable-mmjsontransform and a basic test hook. An experimental companion
mmjsonrewrite module is wired similarly for dotted-key expansion.
Before/After: Previously no built-in JSON un/flatten; now an action can
unflatten (default) or flatten JSON into a separate message property.
With the help of AI Agents: ChatGPT codex, gemini
Parsing QRadar-formatted JSON is common in real-world pipelines and
migration projects. This change adds CI coverage to ensure rsyslog
can ingest and re-emit such events without semantic drift.
Impact: CI-only; no runtime behavior change.
Add testsuites/qradar_json with two representative QRadar events. A
new Python validator compares input vs. emitted JSON after removing
the CEE cookie, ignoring whitespace and key order for semantic
equality. A shell fallback uses jq for normalized compares and a
basic field presence check when jq is unavailable. Increase the
tests/diagtalker input buffer from 2 KiB to 10 KiB so large QRadar
payloads are not truncated during test runs.
BEFORE: No semantic check for QRadar JSON; large messages could
truncate in diagtalker.
AFTER: CI asserts input/output JSON equivalence; diagtalker handles
~10 KiB lines without truncation.
Implement complete NXLog Snare-formatted Windows Security event parser
with multi-format support (RFC5424/RFC3164), 100+ field patterns, and
advanced features including GUID/IP/timestamp type detection, runtime
configuration, enhanced validation modes, and comprehensive test suite.
Features:
- Parse major Windows security event types (4624, 4625, 4634, etc.)
- Extract structured data into configurable JSON containers (!win default)
- Handle modern Windows telemetry (LAPS, TLS, WDAC, WUFB, Kerberos)
- Type-aware parsing with validation and fallback handling
- Runtime configuration support for custom field patterns
- Thread-safe design with no shared mutable state
- 9 comprehensive test scripts covering all functionality
Impact: Enables structured analysis of Windows Security events for
SIEM integration, threat detection, and compliance reporting while
preserving original payloads for forensic investigation.
Files: contrib/mmsnarewinsec/, tests/mmsnarewinsec-*.sh,
doc/source/configuration/modules/mmsnarewinsec.rst
Subtree templates copied data into the worker buffer but left lenStr at
zero. Output modules that respect lenStr (omfwd, omfile, others) therefore
emitted empty payloads even though the buffer held valid JSON.
Set lenStr to the subtree length immediately after the memcpy. This aligns
the subtree branch with the existing regular/jsonftree/strgen paths and
restores correct forwarding behaviour for all modules.
Add regression coverage:
* retain omfwd-subtree-tpl.sh to prove network forwarding now delivers the
subtree payload
* add omfile-subtree-jsonf.sh to exercise subtree data consumed via
exec_template() and rendered through an option.jsonf list template
Before: subtree templates built the JSON text but omfwd saw lenStr=0 and
sent empty frames or files.
After: lenStr matches the copied bytes, so modules transmit the expected
JSON content.
Closes: https://github.com/rsyslog/rsyslog/issues/6206
Add comprehensive macOS CI support with two new GitHub Actions
workflows:
- run_macos.yml: PR-triggered CI with matrix strategy covering macOS
13–15, x64/arm64 architectures, and sanitizer combinations (none,
ASAN, TSAN)
- run_macos_weekly.yml: Scheduled weekly testing with full matrix
coverage and automated failure reporting via GitHub issues
- Set sin_len in tests/diagtalker.c on macOS to fix connect() EINVAL,
unblocking TLS certvalid tests.
Root cause and fix details (macOS testbench)
- Why tests failed
On macOS 14 the imdiag control listener often ended up IPv6-only.
The plain TCP listener creates an IPv6 socket and sets IPV6_V6ONLY;
the companion IPv4 bind can fail on macOS when sharing an ephemeral
port, leaving only the IPv6 listener active. Our injector
tests/diagtalker.c was IPv4-only (AF_INET to 127.0.0.1), so it could
not reach the imdiag port, causing repeated connect retries and
timeouts. CI logs showed “cannot connect to 127.0.0.1:<port> …
Connection refused” alongside benign OpenSSL anon-mode warnings.
- What we changed
1) Made the injector dual-stack by switching diagtalker to
getaddrinfo(AF_UNSPEC) and trying both IPv6 and IPv4 (with fallback
to 127.0.0.1 and ::1). This removes the hard dependency on IPv4
reachability when the listener is IPv6-only on macOS.
2) Added an opt-in testbench knob to enforce IPv4 where appropriate:
generate_conf() now honors RSTB_FORCE_IPV4=1 (or
RSTB_NET_IPPROTO=ipv4-only) to inject
global(net.ipprotocol="ipv4-only"). We enable this only in the IPv4
test variant so the IPv6 wrapper remains pure IPv6.
- Impact
The injector/listener address-family mismatch is eliminated, resolving
the macOS connect() failures and unblocking the TLS “certvalid” and
anonymous tests on macOS runners.
Refs: https://github.com/rsyslog/rsyslog/issues/5635
Refs: https://github.com/Homebrew/homebrew-core/pull/221869
Refs: https://github.com/Homebrew/homebrew-core/pull/226378
* AI: align AGENTS.md and README; expand harness tips
This refines testbench docs for both AI agents and human operators.
Goal: clearer authoring guidance and a smoother quickstart so agents
produce higher-quality tests and contributors run them reliably.
Before/After: scattered guidance across files -> cross-linked docs with
explicit helper usage, Valgrind wrapper patterns, and quickstart.
Non-technical rationale:
This improves maintainability and AI readiness by giving agents concrete
patterns and reducing duplicate boilerplate. It also aligns operator docs
with the harness so CI and local runs behave consistently.
With the help of AI Agents: ChatGPT, gemini
Use a documentation-only IP to avoid confusion and follow best practice.
This prevents irritation when users or scanners see a private address in
samples and wonder if rsyslog is trying to contact it.
Impact: Visible change in sample payloads, tests, and docs (host field).
Technical: Replace the hardcoded "172.20.245.8" in imdiag's generated
messages with a constant default hostname "192.0.2.8" from RFC 5737
(TEST-NET-1). Update tests (e.g., omrabbitmq, perctile, mmexternal,
clickhouse) and template examples to match. No API/ABI change; no effect
on action queues, OMODTX, retry/duplicate semantics, or HUP behavior.
This is purely about sample message contents emitted by the diag tool.
Before/After: samples used 172.20.245.8 -> now 192.0.2.8.
See also: https://gitee.com/src-openeuler/rsyslog/issues/I7V4I8
Fixes: https://github.com/rsyslog/rsyslog/issues/6190
Newer Elasticsearch versions reject typed APIs. This adapts the module
so shipping works out of the box with modern clusters and keeps the
codebase aligned with typeless ES conventions.
Impact: bulk metadata no longer includes _type unless explicitly set.
Old setups that relied on a default "events" type (ES < 8) may need
explicit configuration.
Technically, the default searchType is now NULL
(OMES_SEARCHTYPE_DEFAULT). setPostURL continues to route requests to
the typeless /_doc endpoint, but bulk metadata is generated without a
_type field when searchType is unset. The legacy default "events" for
ES < 8 is removed. Tests are updated to stop passing searchType, and
the searchType-empty test is dropped to reflect the new default. No
OMODTX or action-queue semantics change.
Closes: https://github.com/rsyslog/rsyslog/issues/5060
We want easy nested JSON to match common schemas (e.g., Elastic ECS)
without external processors. This introduces an opt-in mode so existing
jsonf users keep exact behavior while enabling structured output when
requested.
Impact: No change unless option.jsonftree is enabled. With jsonftree,
dotted outnames render as nested objects; empty containers are skipped.
On name collisions (object vs value), we fall back to flat rendering.
Before: jsonf always emitted flat name/value pairs, even for dotted
outnames. After: jsonf remains flat by default; enabling jsonftree makes
"host.hostname" and "host.ip" render as {"host":{"hostname":...,"ip":...}}.
Technically, we add option.jsonftree to templates. When set, we lazily
build a per-template JSON tree (tplJsonNode) from dotted segments and
render it in one pass, reusing existing jsonf formatting for leaves.
The tree state is tracked on the template and freed on template delete.
Config parsing enforces mutual exclusivity among sql, stdsql, json,
jsonf, and jsonftree. Constants record bJSONf to reuse serialized
fragments. Tests cover nested output and pure-json cases using
option.jsonftree.
Make default handling more natural for modern logs, especially JSON
lines. This aligns with common expectations of line-separated records
and reduces surprises when viewing files with standard tools.
Impact: Default changes; messages without trailing LF now get one added.
Backward-compatibility: We accept this change because the common and
portable convention is LF-terminated records. Most users expect this,
and many tools assume it. The risk is limited to consumers that require
byte-identical output without a final LF. Those users can restore the
old behavior by setting addLF="off" globally or per action.
There was also some clean-up of variable names and the removal of
accidentally included obsolete-legacy style config directive.
Previously, omfile wrote rendered messages as-is unless addLF was set.
This could yield "incomplete lines" when templates did not terminate
with LF. We now default addLF to on and only append when the message
does not already end with LF, preserving existing full-line writes.
The change is implemented by making the instance default true; module-
and action-level settings continue to override. Behavior remains
transparent with compression and crypto providers; no ABI/API impact.
Before: messages lacking LF were written without a final newline.
After: such messages are written with a single LF appended.
* omfile: add addLF option to append newline
Add a boolean addLF option for omfile that appends a trailing LF when
the rendered message does not already end with one. The setting is
available as both a module and action parameter (default off) and it is
propagated to legacy configuration and signature providers.
Documentation and a regression test cover the new behaviour.
With the help of AI-Agent: ChatGPT
This parses the LEEF message (if it is) and creates a JSON subtree.
The current implementation is PoC and will be provided to gather
early review.
Changes, including breaking changes, may happen in future versions of
this module.
We invalidly specified the source distribution, which does not include
the necessary class files. This lead to zookeeper start failure and thus
no kafka tests being executed.
* tests: improve kafka startup readiness handling
* add generic TCP wait helper and kafka readiness check that replaces fixed sleeps in start_kafka
* verify zookeeper client port availability after startup
* replace ad-hoc sleeps in kafka-focused tests with wait_for_kafka_startup
* rely on kafka-topics probe instead of port checks
* share kafka layout helper to avoid duplicated logic
* align kafkapid emptiness checks for consistent style
With the help of AI-Agent: ChatGPT
Non-technical: test flakiness makes it hard to validate unrelated changes.
This aligns omelasticsearch tests with ES 7.14 defaults to get the
testbench back to a deterministic state and pave the way for further
modernization.
Impact: test behavior changes; one test skipped; CI coverage slightly reduced.
Before: tests mixed ES 6-era types and ad-hoc tarball picks; deprecation
checks intermittently failed and retries were brittle. After: tests use the
7.14.1 tarball via diag.sh default, typeless mappings, and `_doc` type in
omelasticsearch actions; known-flaky bulk-retry test is skipped for now.
Technical details:
- Add `searchType="_doc"` to all omelasticsearch actions and update index
provisioning to typeless mappings compatible with ES 7.14.
- Remove script-level `ES_DOWNLOAD` overrides to follow diag.sh's 7.14.1
default, keeping test scripts and helper defaults in sync.
- Drop deprecation-log assertion in `es-searchType-empty.sh` to prevent
spurious failures specific to ES 6-era types.
- Temporarily skip `es-bulk-retry.sh` (exit 77) pending a rewrite of retry
semantics under ES 7.x.
- CI: export `VERBOSE=1`; disable Kafka and Elasticsearch tests in the
affected matrix job to keep CI green while ES/Kafka suites are refactored.
VERBOSE ensures test logs are emitted to stderr and as such are
visible in CI test runs.
- Minor whitespace/indent cleanups; no runtime code or plugin behavior
changes.
Provide different file name for zookeeper tar - it looks like apache
changed the name (not sure, but it worked previously).
Also now cached the file on rsyslog.com, so that we have reliable
access even on name change or apache download rate-limiting.
Admins often report "gibberish" when a TLS-enabled sender connects to a
plain imtcp port. Making the mismatch explicit reduces operator confusion
and support churn, and points directly to remediation.
Impact: logs one explicit error per mismatched connection; no change to
parsing or transport on plain listeners.
Before/After: before, ClientHello bytes were ingested as binary with no
hint; after, imtcp detects a TLS ClientHello on ptcp and logs a clear
message with a troubleshooting URL.
Technically, we add a small per-session probe in tcps_sess_t and sample
the first 5 bytes of new sessions. If the record header matches a TLS
handshake (type 0x16, version 0x03.00–0x04, length 40–16384) and the
listener is plain TCP (streamDriver.mode=0), we emit a single error and
disable further probing for that session. The probe is called from
DataRcvd() and returns RS_RET_SERVER_NO_TLS when triggered; the session
is otherwise left untouched.
Runtime: introduce RS_RET_SERVER_NO_TLS (-2465) to tag the condition.
Docs: add imtcp troubleshooting section and a dedicated FAQ page.
Tests/tools: add test imtcp-tls-gibberish.sh and extend tcpflood with
-H to send only a ClientHello (OpenSSL and GnuTLS paths tolerate early
termination and non-blocking I/O for this mode).
Add comprehensive macOS CI support with two new GitHub Actions workflows:
- run_macos.yml: PR-triggered CI with matrix strategy covering macOS 13-15,
x64/arm64 architectures, and sanitizer combinations (none, ASAN, TSAN)
- run_macos_weekly.yml: Scheduled weekly testing with full matrix coverage
and automated failure reporting via GitHub issues
Enhance test diagnostics in tests/diag.sh:
- Add separate startup timeout handling for valgrind vs normal runs
- Implement immediate crash detection during rsyslog startup
- Add comprehensive core dump analysis with platform-specific debuggers
- Improve macOS-specific error reporting and system information gathering
- Add disk space monitoring to prevent core dump creation failures
- tests/kafka: give unique names in sndrcv_kafka_multi_topics
Rename the second omkafka action to a distinct name (kafka-fwd-2).
Using the same action name twice can lead to only one effective
publisher with newer builds, which caused
tests/sndrcv_kafka_multi_topics.sh to publish to a single topic and
fail the final count check.
Fix macOS-specific test issues:
- Skip failing TCP tests on Darwin due to TCP chunking behavior
- Add maxframesize parameter to imptcp tests for better reliability
The CI workflows include proper sanitizer configuration, core dump handling,
disk space management, and comprehensive error reporting to improve
debugging capabilities on macOS platforms.
Impact: Enables continuous testing on macOS, improves test reliability,
and provides better diagnostics for platform-specific issues.
Fixes: https://github.com/rsyslog/rsyslog/issues/5629
Refs: https://github.com/rsyslog/rsyslog/pull/5635
Refs: https://github.com/Homebrew/homebrew-core/issues/221869
Refs: https://github.com/Homebrew/homebrew-core/issues/226378
Add kafkaHeader parameter to define key/value pairs
that are attached as headers to every produced message.
Require librdkafka v0.11 for header support. Update
configure checks, docs and add a regression test.
closes: https://github.com/rsyslog/rsyslog/issues/5185
With help of AI-Agent: OpenAI ChatGPT
This builds on "PR#6121 net: Add NetworkNamespace APIS"
to add Network Namespace support to imtcp module. This
extends imtcp to support a wider range of Unix/Linux
environments (or any environment supporting network
namespaces).
The imtcp module is enhanced to accept a NetworkNamespace
parameter, both as a default at the module level, and
on a per-instance basis.
The tcpsrv module is enhanced to allow the NetworkNamespace
to be applied to a listener's configuration parameters.
Finally, the netstrm module is enhanced to switch namespaces
before invoking the downstream (driver specific) LstnInit
function.
A new test imtcp-netns (and associated imtcp-netns-vg) is
added to test this functionality. This must be run as root
(technically it must be run by a user with CAP_SYS_ADMIN
capabilities, as network namespace creating/change is
required).
A slight change to diag.sh is made to allow passing $RS_REDIR
to valgrind (as $RS_REDIR is used in the imtcp-netns.sh
test for some negative cases).
Signed-off-by: Billie Alsup <balsup@cisco.com>
This moves NetworkNamespace functionality into
the net module. This allows the same code to
be reused across multiple tools and plugins.
The first usage is with omfwd, which is changed
to use the common net implementation. Note
the net implementation is based on the original
omfwd implementation. Subsequent PRs will be
opened for integrating this into omuxsock and
imtcp.
The original test case tcp_forwarding_ns_tpl.sh
was broken due to use of single quotes rather
than double quotes, thus preventing the proper
port number argument to be passed to the
listener. Note this test must be run as
root.
Development has occurred across Fedora41 and
Fedora42, which uses glibc 2.40 and glibc2.41
respectively. The valgrind suppressions are
updated to handle new glibc issues
accordingly.
Four new functions are callable through the
net module. These allow one to save a handle
to the current namespace, switch to a new
namespace by name, and restore the namespace
from the saved handle. A fourth higher
level wrapper is used to open a socket in
a named network namespace, and handles the
invocation of the lower level functions.
Ideally this would be the only public
function, however it simplifies integration
into imtcp in the future (which doesn't
directly open sockets). This may change
in the future as network namespaces are
integrated into more modules and plugins.
Signed-off-by: Billie Alsup <balsup@cisco.com>
Some deployments need to disambiguate multiple senders sharing an IP,
for example autossh or similar tunnel setups. Exposing the source port
improves observability and lets pipelines key on a stable tuple.
Impact: new property/JSON field; tcps_sess IF v4; out-of-tree modules
must rebuild.
Before: messages exposed fromhost and fromhost-ip only.
After: messages also expose fromhost-port and jsonmesg includes it.
Introduce PROP_FROMHOST_PORT and wire it through msg.{h,c}. For TCP,
capture the remote port on accept, store it in tcps_sess, and attach it
to the msg on submit. For other inputs, resolveDNS derives the port from
the sockaddr when available; local inputs return an empty string. Add a
getter, duplication and destructor handling, and name<->ID mapping. Add
the field to jsonmesg output. Update docs, lexer keywords, and the
external plugin interface doc (property is modifiable). Bump
tcps_sessCURR_IF_VERSION to 4 and add SetHostPort() to the interface.
Include a focused test (fromhost-port.sh) that verifies the property.
Non-technical rationale: allow identification by (fromhost-ip,
fromhost-port) where IP alone is shared across systems (e.g., autossh).
With help from AI-Agents: ChatGPT
