21035 Commits

Author SHA1 Message Date
Rainer Gerhards
91eb4a154c
Merge pull request #7164 from rsyslog/codex/fix-missing-limits.h-in-yamlconf.c
runtime: include <limits.h> for INT_MAX in yamlconf
2026-06-02 20:38:03 +02:00
Rainer Gerhards
10500d52ad
Merge pull request #7160 from rsyslog/codex/propose-fix-for-find-json-vulnerability
mmjsonparse: fix find-json ownership and scan bounds
2026-06-02 20:37:35 +02:00
Rainer Gerhards
83f456afc3
Merge pull request #7163 from rgerhards/codex/i2524-abort-unclean-config
rainerscript: accept optimizer NOP statements
2026-06-02 20:33:53 +02:00
Rainer Gerhards
609561b086
Merge pull request #7166 from rsyslog/codex/propose-fix-for-disk-queue-recovery-issue
runtime/queue: reset sizeOnDisk after safe recovery
2026-06-02 20:32:57 +02:00
Rainer Gerhards
3111d5d2d9
Merge pull request #7167 from rgerhards/codex/ci-make-check-verbose
ci: enable verbose make check output
2026-06-02 20:28:16 +02:00
Rainer Gerhards
9ba48c1ebc ci: enable verbose make check output
Why: CI failures need enough test harness output for quick diagnosis.

Impact: CI make check jobs now print verbose Automake test details.

Before/After: Failed test logs were easier to miss; CI now requests verbose test output.

Technical Overview: Add VERBOSE=1 to direct make check invocations in the CI workflows. Apply the same setting in devtools/run-ci.sh so shared container-based CI paths inherit verbose Automake test logging. This keeps the change limited to diagnostic output and does not alter the selected tests.

With the help of AI-Agents: Codex
2026-06-02 19:10:49 +02:00
Rainer Gerhards
af3e8bf725
Merge pull request #7076 from rsyslog/codex/fix-malformed-json-output-in-jsonftree
template: fix jsonftree fallback to valid flat JSON
2026-06-02 18:22:46 +02:00
Rainer Gerhards
02bb4bc19d runtime/queue: reset sizeOnDisk after safe recovery
Why:

Safe disk queue recovery reconstructs queue files from scratch.

Keeping stale on-disk byte accounting can make the new queue appear

full and block later enqueues when maxDiskSpace is configured.

Impact:

Recovered disk queues no longer retain stale disk usage limits.

Before/After:

Before: recovered queue could stay blocked by old sizeOnDisk value.

After: recovery clears sizeOnDisk with other queue size counters.

Technical Overview:

- extend qqueueResetRecoveredQueueSize() to reset disk byte accounting

  for disk and disk-assisted queues

- keep existing queue-size and dequeue-log resets unchanged

- apply the change only in the shared recovery reset helper to keep

  behavior centralized

With the help of AI-Agents: GPT-5.3-Codex
2026-06-02 18:21:39 +02:00
Rainer Gerhards
5aa6a5cde7 runtime: include limits.h for INT_MAX in yamlconf 2026-06-02 18:20:13 +02:00
Rainer Gerhards
883fdb4575 rainerscript: accept optimizer NOP statements
Explicit continue statements become S_NOP nodes while the optimizer walks the statement tree. That state is expected: removeNOPs() drops those nodes after optimization, so logging an optimizer error during traversal was misleading and could make strict config validation appear dirty.

Treat S_NOP as a normal intermediate and cover the regression with an AbortOnUncleanConfig -N1 test for the continue-in-if pattern from #2524 and #2568.

Fixes #2524.

Fixes #2568.
2026-06-02 18:03:54 +02:00
Rainer Gerhards
5d22640d0c Merge remote-tracking branch 'upstream/main' into babysit-7160
# Conflicts:
#	plugins/mmjsonparse/mmjsonparse.c
2026-06-02 18:02:02 +02:00
Rainer Gerhards
3d9af5299b
Merge pull request #6753 from Cropi/fix/imfile-inotify-fd-release-on-delete
imfile: release deleted-file FDs after FILE_DELETE_DELAY in inotify mode
2026-06-02 17:55:37 +02:00
Rainer Gerhards
9127557bc1
Merge pull request #7158 from rsyslog/codex/propose-fix-for-udp-memory-exhaustion-vulnerability
sidecar: cap UDP burst buffer memory in UDP mode
2026-06-02 17:25:22 +02:00
Rainer Gerhards
2f706538b7
Merge pull request #7159 from rsyslog/codex/propose-fix-for-wolfssl-send-retry-issue
tls: propagate wolfSSL send-side read retry
2026-06-02 17:22:18 +02:00
Rainer Gerhards
68c80ed5cf tls: bound wolfSSL send-side read retries 2026-06-02 16:12:55 +02:00
Rainer Gerhards
d0a910f5a8 tls: keep wolfSSL send retry local 2026-06-02 15:36:40 +02:00
Rainer Gerhards
02c85f12e2 sidecar: cap UDP burst buffer by total bytes 2026-06-02 15:26:44 +02:00
Rainer Gerhards
50bdea0958
Merge pull request #7162 from rgerhards/codex/i5693-omhttp-retry
omhttp: avoid retry-ruleset self-stall
2026-06-02 15:18:23 +02:00
Rainer Gerhards
5d24127501 omhttp: avoid retry-ruleset self-stall
Why:

A short HTTP outage can leave omhttp retry.ruleset work stuck.

The worker may wait at the full-delay mark on its own retry queue.

Impact:

Retry rulesets keep bounded backpressure and gain recovery coverage.

Before/After:

Before, retry messages used FULL_DELAY. After, they use LIGHT_DELAY.

The worker can keep draining retry work after the outage.

Technical Overview:

- Change retry.ruleset message flow control to LIGHT_DELAY.

- Keep existing fire-and-forget retry enqueue ownership unchanged.

- Add an omhttp helper mode for time-bounded HTTP failures.

- Add an omhttp retry.ruleset recovery regression test.

- Register the test in the existing OMHTTP test family.

Refs: https://github.com/rsyslog/rsyslog/issues/5693

With the help of AI-Agents: OpenAI Codex
2026-06-02 14:54:44 +02:00
Rainer Gerhards
9050500eb6 mmjsonparse: clear JSON pointer after ownership transfer 2026-06-02 14:41:18 +02:00
Rainer Gerhards
97f2b96525
Merge pull request #6991 from rgerhards/codex/issue-3297-1891-mmpstrucdata
mmpstrucdata: support custom SD containers
2026-06-02 14:25:36 +02:00
Rainer Gerhards
66423127ee mmjsonparse: fix find-json ownership and scan bounds 2026-06-02 14:17:47 +02:00
Rainer Gerhards
bcda60a369 mmpstrucdata: support custom SD containers
Why:
Operators need parsed RFC5424 structured data below a predictable
JSON object name, and old large structured-data messages could exceed
module-local assumptions.

Impact:
Adds two action parameters, expands structured-data length storage,
and enables mmpstrucdata in the Ubuntu 26.04 TSAN lane.

Before/After:
Before, mmpstrucdata always wrote `rfc5424-sd` and could fail on very
large SD. After, the container is configurable and SD size is bounded.

Technical Overview:
- Add `container` with default `rfc5424-sd`.
- Add `maxStructuredDataSize`, defaulting to global max message size.
- Store message structured-data length as `rs_size_t` instead of 16 bit.
- Allocate parameter-value parse buffers from the actual SD length.
- Preserve RFC3164/no-SD skip behavior and represent RFC5424 NILVALUE
  as JSON null under the configured container.
- Document RainerScript and YAML usage and add parameter reference pages.
- Add focused regression tests for custom containers, NILVALUE, large SD,
  oversized SD, and RFC3164 skip behavior.
- Remove the old mmpstrucdata TSAN suppression and codify container
  clean/rebuild guidance for local validation.

Special credits:
Bogoslovskii Fedor provided good analysis that facilitated parts of the
implementation.

Closes https://github.com/rsyslog/rsyslog/issues/3297
Closes https://github.com/rsyslog/rsyslog/issues/1891

With the help of AI-Agents: Codex

Co-authored-by: Bogoslovskii Fedor <Fedorasta@gmail.com>
2026-06-02 14:00:15 +02:00
Rainer Gerhards
9489aafdeb
Merge pull request #7051 from rsyslog/cursor/critical-correctness-bugs-c15a
stats: avoid notifier teardown deadlocks
2026-06-02 13:17:33 +02:00
Rainer Gerhards
5cd7d6e9da
Merge pull request #7145 from rgerhards/codex/msgprop-coverage
tests: cover template property transformations
2026-06-02 12:57:51 +02:00
Rainer Gerhards
0250238e6e tls: propagate wolfSSL send-side read retry
Why:

wolfSSL send retries must yield when a nonblocking read cannot

make progress. Swallowing that retry can keep a worker in the

SSL_write loop without waiting for socket readiness.

Impact:

wolfSSL TLS sends now return RS_RET_RETRY instead of spinning.

Before/After:

Before, send-side WANT_READ could busy-loop on read retry; after,

the retry is propagated to the existing event-loop path.

Technical Overview:

The wolfSSL-specific SSL_ERROR_WANT_READ path still drives a local

osslRecordRecv() so wolfSSL can process read-side TLS traffic while

sending. When that read reports RS_RET_RETRY, Send() now aborts with

RS_RET_RETRY instead of clearing retry state and immediately calling

SSL_write() again. Successful local reads continue to clear retry

state before retrying SSL_write(), matching the generic helper's

state handling without changing successful progress behavior.

With the help of AI-Agents: GPT-5.3-Codex
2026-06-02 10:38:09 +02:00
Rainer Gerhards
2b4a2457d9
Merge pull request #7154 from rgerhards/codex/local-validation-cleanup
devtools: fold local review experiment into planner
2026-06-02 10:24:41 +02:00
Rainer Gerhards
ed91aebc42 devtools: fold local review experiment into planner
Why:
The old AI local-review workflow was experimental, duplicated newer
validation guidance, and still carried stale review behavior on main.

Impact:
Local validation keeps useful checks in the maintained planner and
removes the obsolete experiment while retaining shared prompt assets.

Before/After:
Before, local review policy lived partly in an unused AI script and a
deleted workflow entry point. After, the planner and skills own the
reusable checks and prompt guidance.

Technical Overview:
Remove ai/local-review-workflow.sh and .agent/workflows/audit.md.
Teach devtools/local-validation-plan.sh to derive its default base from
RSYSLOG_LOCAL_VALIDATION_BASE, rsyslog.localValidationBase, or the
worktree HEAD reflog baseline.
Use the same base for local Cubic review so committed branch changes are
reviewed against the worktree creation point.
Add advisory raw allocation and test antipattern scans, and run fast mock
distcheck for distribution-risk test/build changes.
Fold the old audit prompt guidance into the local container testing skill
as late manual prompt audits without launching another AI CLI.
Document ai/ as the central shared prompt-asset library and remove stale
references to deleted workflow paths.

With the help of AI-Agents: OpenAI Codex
2026-06-02 10:15:24 +02:00
Rainer Gerhards
a49e032a84
Merge pull request #7157 from rgerhards/codex/daily-focused-cache-handling
ci: keep daily Kafka configure fresh
2026-06-02 09:52:10 +02:00
Rainer Gerhards
90131e971e ci: keep daily Kafka configure fresh
Why: Daily focused runs are intended to exercise the current tree and current runner environment without stale cross-run configure state.

Impact: The daily Kafka lane no longer reuses config.cache, setup failures do not trigger secondary coverage failures, and the tracking issue identifies failed lanes.

Before/After: Before, a restored Kafka config.cache could reject new coverage flags and obscure the root cause with coverage cleanup failures. After, Kafka configures fresh and failure reports name the failed lane and artifact.

Technical Overview: Removed the actions/cache step for the daily Kafka configure cache.

Technical Overview: Removed --cache-file=config.cache from the Kafka configure command.

Technical Overview: Added an id to the prepare step and run coverage/Codecov only after configure succeeds.

Technical Overview: Extended the report job to read workflow jobs and include failed daily lanes plus expected ci-failure artifact names in the tracking issue.

Technical Overview: Added actions:read permission for the report job so it can inspect run jobs.

Issue: https://github.com/rsyslog/rsyslog/issues/7156

With the help of AI-Agents: Codex
2026-06-02 09:47:10 +02:00
Rainer Gerhards
372e6e8a8c
packaging: enable HTTPS by default for ETL Vespa output (#7087)
* packaging: enable HTTPS by default for ETL Vespa output

* packaging: expose Vespa HTTPS toggles

* packaging: use ETL Vespa env defaults directly

* packaging: default ETL TLS envs in entrypoint
2026-06-02 09:00:27 +02:00
Tamir Suliman
2de8fa287b
omkafka: fix HUP deadlock when doAction holds mut_doAction (#7129)
Why: HUP can permanently hang omkafka while messages are being
produced. Deployments that reload via logrotate or manual SIGHUP
need the action to recover without a forced kill.

Before/After: before, doHUP could enter Kafka handle teardown outside
the doAction serialization domain. After, HUP first enters
mut_doAction and only then touches file mutexes or the Kafka handle.

Technical overview:
- doAction and pollCallbackThread hold mut_doAction while polling
  librdkafka.
- Callbacks from that poll path may write stats or error files.
- doHUP now takes mut_doAction before mutErrFile and mutStatsFile.
- Kafka handle recreate/poll continues inside that same outer critical
  section.
- This keeps runtime ordering consistent:
  mut_doAction -> narrower file mutexes / rkLock.
- tryResume already follows the same outer mut_doAction rule.

Impact: HUP handling waits for any in-progress Kafka action before
closing files or recreating/polling the handle.

Fixes: https://github.com/rsyslog/rsyslog/issues/4230

With the help of AI-Agents: Claude Sonnet 4.6 2026-05, Codex
2026-06-02 08:20:28 +02:00
Rainer Gerhards
4be3191ded
ChangeLog: update 8.2606 entries (#7153)
Why:
Keep the scheduled 8.2606 release notes current after recent
user-visible and maintainer-visible changes accumulated.

Impact:
Release-note readers see the important recent changes.

Before/After:
Before, the block stopped at older entries. After, it covers
recent selected fixes and features without listing generic churn.

Technical Overview:
- Add selective entries for new modules and user-visible fixes.
- Keep broad hardening and testbench work under the existing umbrella.
- Preserve the established 8.2606 ChangeLog style and issue links.
- Leave build, CI, and low-signal documentation churn out of the block.

With the help of AI-Agents: Codex
2026-06-02 08:12:23 +02:00
Rainer Gerhards
804d9f80ae
ChangeLog: update 8.2606 entries
Why:
Keep the scheduled 8.2606 release notes current after recent
user-visible and maintainer-visible changes accumulated.

Impact:
Release-note readers see the important recent changes.

Before/After:
Before, the block stopped at older entries. After, it covers
recent selected fixes and features without listing generic churn.

Technical Overview:
- Add selective entries for new modules and user-visible fixes.
- Keep broad hardening and testbench work under the existing umbrella.
- Preserve the established 8.2606 ChangeLog style and issue links.
- Leave build, CI, and low-signal documentation churn out of the block.

With the help of AI-Agents: Codex
2026-06-01 18:57:28 +02:00
Rainer Gerhards
fb0572f847
translate: cover script serialization (#7152)
* translate: cover script serialization
2026-06-01 18:45:57 +02:00
Rainer Gerhards
6444ff67e9
Merge pull request #7151 from rgerhards/codex/dev-env-lcov
dev_env: include lcov in Ubuntu coverage images
2026-06-01 17:23:12 +02:00
Rainer Gerhards
deedd5a146 dev_env: include lcov in Ubuntu coverage images 2026-06-01 16:06:18 +02:00
Rainer Gerhards
4b5ca397fa
Merge pull request #7149 from rgerhards/codex/daily-codecov
ci: upload Codecov from daily focused tests
2026-06-01 15:31:17 +02:00
Rainer Gerhards
3d3a06bc5c ci: fail daily coverage filtering errors 2026-06-01 14:57:57 +02:00
Rainer Gerhards
c081e92510 ci: upload Codecov from daily focused tests 2026-06-01 14:51:27 +02:00
Rainer Gerhards
33bf706e76 tests: cover additional message property shapes 2026-06-01 14:49:45 +02:00
Rainer Gerhards
ac6c8f02a3
Merge pull request #7103 from rgerhards/template-regex-bounds-fix
template: reject invalid regex match selectors
2026-06-01 14:37:02 +02:00
Rainer Gerhards
b1fe3c3eaa tests: extend template property coverage matrix 2026-06-01 14:10:55 +02:00
Rainer Gerhards
d2602020dd tests: gate faketime coverage by libfaketime 2026-05-31 21:03:21 +02:00
Rainer Gerhards
ae6980c523 tests: broaden template property coverage 2026-05-31 20:51:20 +02:00
Rainer Gerhards
fcec80bbd0
Merge pull request #7141 from rgerhards/codex/tsan-deadlock-detection
ci: enable TSan deadlock detection
2026-05-31 19:13:35 +02:00
Rainer Gerhards
f081fd5aaa tests: cover template property transformations 2026-05-31 19:13:23 +02:00
Rainer Gerhards
f914dfc862
Merge pull request #7140 from rgerhards/codex/i5132-action-resume-interval
action: avoid committing suspended retry batches
2026-05-31 17:38:08 +02:00
Rainer Gerhards
bd20d87fa2
Merge pull request #7142 from rgerhards/codex/i3485-3487-omclickhouse-errors
omclickhouse: report HTTP response errors
2026-05-31 17:35:56 +02:00
Rainer Gerhards
8299ecfc1e
Merge pull request #7144 from rgerhards/codex/narrow-service-relevance
tests: narrow service relevance for test files
2026-05-31 17:32:47 +02:00