rsyslog/tests/AGENTS.md

AGENTS.md – Testbench subtree

This guide covers everything under tests/, including shell test cases, helpers, suppressions, and supporting binaries. Pair it with tests/README for human- and CI-facing run instructions; this document focuses on authoring guidance and the knobs that matter most to AI agents.

Purpose & scope

• The directory implements the Automake testbench that exercises rsyslog.
• Each .sh script is a standalone scenario that can be executed directly or through make check.
• Treat tests/ as the single recursive test-owning subtree. Keep new unit test sources under tests/unit/ or similar subfolders, but register and run them from tests/Makefile.am instead of creating another recursive tests/.../Makefile.am harness. Recursive Automake propagates make check TESTS=... into every subdirectory, so splitting test ownership across multiple subdirs creates fragile name collisions and selection failures.
• Use this guide together with the top-level AGENTS.md and the component guide that matches the module you are testing.

Writing & updating tests

• Base new shell tests on existing ones; include . $srcdir/diag.sh to gain the helper functions (timeouts, Valgrind integration, rsyslogd launch helpers).
• Every new or changed test must document the exact behavior, regression, or invariant it covers. Add or refresh a short comment near the top of the test when the current intent is missing, stale, or vague. For timing, retry, sampling, concurrency, or negative-path tests, also explain the oracle: what proves success or failure, and why any wait or threshold exists. When changing a test, verify that the head comment still matches the actual setup, stimulus, oracle, and pass/fail conditions after the edit; update it in the same commit if it does not.
• For diagnostics emitted by rsyslog itself, prefer asserting the configured rsyslog output destination, usually testbench omfile output such as RSYSLOG_OUT_LOG, after synchronized shutdown. Do not use rsyslogd stdout/stderr as the oracle unless the behavior being tested is specifically process-level stdout/stderr emission, startup before configuration is usable, or another documented case where the message cannot pass through rsyslog's normal output path. Explain such exceptions in the test header.
• Prefer harness helpers such as content_check, content_count_check, custom_content_check, check_not_present, cmp_exact, command_deny, and require_plugin over ad-hoc shell to keep diagnostics uniform. In particular, use content_check "needle" "$file" instead of hand-written grep ... || { cat "$file"; error_exit 1; } blocks when asserting log or output content.
• Config format coverage: When a module parameter or config object is tested via RainerScript, add a companion YAML test (or extend an existing yaml-<area>-*.sh) that exercises the same parameter. Both frontends share the same backend, but bugs can exist in the YAML parser alone. Name YAML tests yaml-<area>-<what>.sh and use a .yaml extension for the config fixture in tests/testsuites/ so the YAML loader is triggered automatically. Register the test in tests/Makefile.am under the same module conditionals as its RainerScript counterpart. See the rsyslog_config skill for full conventions.
• Name Valgrind-enabled wrappers with the -vg.sh suffix and toggle Valgrind by exporting USE_VALGRIND before including the non-vg script using the . command. Use tests/timereported-utc-vg.sh as the reference layout: it includes the base scenario using the . command instead of copying it and demonstrates paring back emitted messages when the underlying test is slow—especially important under Valgrind. Older wrappers still duplicate logic; prefer the modern pattern when touching them.
• Put auxiliary binaries next to their scripts (e.g. *.c programs compiled via the Automake harness) and list them in tests/Makefile.am. Unit-test sources may live in tests/unit/, but the owning harness remains tests/Makefile.am.
• Keep long-lived configuration snippets in tests/testsuites/ and reuse them instead of copying large config blocks into multiple scripts.
• Document new environment flags or helper functions inside diag.sh so other tests can discover them. Mention the addition in tests/README if operators should know about the new knob.

Running tests locally

• Build rsyslog first using the efficient incremental command:

  make -j$(nproc) check TESTS=""
  

  This ensures the testbench can load freshly built binaries and modules. If the Makefile is missing, see "Step 2" in the top-level AGENTS.md.
• Execute individual scenarios directly for quick feedback (./tests/imfile-basic.sh).
• Use make check TESTS='script.sh' when you need Automake logging, parallelisation control, or to exercise the Valgrind wrappers.
• For unit binaries registered in tests/Makefile.am, use make check TESTS='binary_name' from the repository root so Automake builds the required runtime libraries before entering tests/.

Multi-Module Test Guards

When adding a test that requires multiple modules (e.g., imtcp AND imptcp), you MUST wrap the test definition in tests/Makefile.am with significantly separate conditionals for ALL required modules. Do not assume one implies the others. Example:

if ENABLE_IMTCP
if ENABLE_IMPTCP
TESTS += multi_module_test.sh
endif
endif

Failing to do this breaks the build on systems where only one of the modules is enabled.

• Remove stale .log/.trs files before re-running a flaky test to avoid Automake caching previous outcomes.
• For configuration validation changes, run ./tests/validation-run.sh to confirm both failure and success paths.

Debugging & environment control

• tests/diag.sh documents environment variables such as SUDO, USE_VALGRIND, RSYSLOG_DEBUG, and timeout overrides; prefer these knobs over ad-hoc sleep loops. tests/README mirrors the operator-facing knobs so tooling and human docs stay aligned.
• Use USE_GDB=1 make <test>.log to pause execution and attach a debugger as described in tests/README.
• Keep suppression files (e.g. *.supp) current when adding new Valgrind noise; failing to do so will cause CI false positives.

Enabling Debug Output

To enable rsyslog debug logging for a test, temporarily uncomment these lines in tests/diag.sh (around lines 88-89):

export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction nostdout"
export RSYSLOG_DEBUGLOG="log"

This creates a log file in the tests directory with detailed execution traces.

Important: Remember to re-comment these lines after debugging to avoid cluttering test output.

Preventing Test Cleanup for Inspection

To examine test output files after a test runs, temporarily comment out exit_test at the end of the test script:

# exit_test  # Temporarily disabled to inspect logs

This preserves:

• rstb_*.out.log - The actual test output
• rstb_*.conf - The generated rsyslog configuration
• log - Debug log (if enabled)
• rstb_*.input* - Test input files

Example Debugging Workflow

1. Enable debug output in diag.sh:

   # Uncomment lines 88-89
   export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction nostdout"
   export RSYSLOG_DEBUGLOG="log"
   

2. Disable cleanup in your test script:

   # Comment the exit_test line
   #exit_test
   

3. Run the test:

   cd tests
   ./mmsnareparse-trailing-extradata.sh
   

4. Examine output:

   # Check actual output vs expected
   cat rstb_*.out.log
   
   # Search debug log for specific patterns
   grep "extradata_section" log
   grep "Truncated trailing" log
   

5. Restore test environment:

   • Re-comment debug exports in diag.sh
   • Uncomment exit_test in your test script
   • Clean up test artifacts: rm -f rstb_* log

Understanding Test Output

When a test fails with content_check, the error shows:

FAIL: content_check failed to find "expected content"
FILE "rstb_*.out.log" content:
     1  actual line 1
     2  actual line 2

This helps identify:

• What the test expected vs what was produced
• Whether the module parsed the message correctly
• If fields are populated as expected

YAML-only test mode

generate_conf --yaml-only creates a pure-YAML preamble (no RainerScript preamble) that is used as the rsyslogd startup configuration directly. Use it when a test must validate YAML-loader behaviour or when no RainerScript is desired.

How it works

• generate_conf --yaml-only [instance] writes ${TESTCONF_NM}[instance].yaml containing version: 2, global:, and testbench_modules: (imdiag setup). testbench_modules: is a YAML key understood by rsyslogd as an alias for modules: and is reserved for testbench infrastructure — it avoids any conflict with the test's own modules: section.
• Tests add their own modules: section (and inputs:, rulesets:, etc.) via add_yaml_conf.
• add_yaml_conf 'fragment' [instance] appends arbitrary YAML to the same file.
• add_yaml_imdiag_input [instance] is a historical compatibility helper. imdiag startup detection is configured by generate_conf --yaml-only via module-scoped testbench parameters, so new tests should not call it.
• startup_common detects RSYSLOG_YAML_ONLY=1 and passes the .yaml file to rsyslogd instead of the usual .conf file.

Limitations

The following testbench features are not available in yaml-only mode:

Feature	Reason	Workaround
Legacy $ directives	Legacy syntax is not parsed by the YAML loader	Use v2 RainerScript (module(), input()) or YAML keys instead

Note

: Startup detection uses the imdiag port file in both RainerScript and yaml-only modes. The .started marker file mechanism has been removed; the imdiag port file is the sole startup signal in all modes.

Example test structure

. ${srcdir:=.}/diag.sh init
require_plugin imtcp
export NUMMESSAGES=100
export QUEUE_EMPTY_CHECK_FUNC=wait_file_lines
generate_conf --yaml-only
# Test-specific modules in their own modules: section (testbench_modules: is in preamble)
add_yaml_conf 'modules:'
add_yaml_conf '  - load: "../plugins/imtcp/.libs/imtcp"'
add_yaml_conf ''
add_yaml_conf 'inputs:'
add_yaml_conf "  - type: imtcp"
add_yaml_conf "    port: \"0\""
add_yaml_conf "    listenPortFileName: \"${RSYSLOG_DYNNAME}.tcpflood_port\""
add_yaml_conf "    ruleset: main"
add_yaml_conf 'rulesets:'
add_yaml_conf '  - name: main'
add_yaml_conf '    script: |'
add_yaml_conf "      action(type=\"omfile\" file=\"${RSYSLOG_OUT_LOG}\")"
startup
# ... test body ...
exit_test

Naming and registration

• Name yaml-only tests yaml-<area>-yamlonly.sh (or append -yamlonly to an existing test name) so they are easy to find.
• Register them in tests/Makefile.am under TESTS_LIBYAML.

Coordination

• When adding tests for a plugin or runtime subsystem, mention them in the component’s AGENTS.md so future authors know smoke coverage exists.
• Update KNOWN_ISSUES or module metadata if a test encodes a known bug or a skipped scenario.
• If a change requires additional docker services or fixtures, document setup steps in tests/CI/README (or create it) and link from the relevant module guide.