334 Commits

Author SHA1 Message Date
Rainer Gerhards
ad625a64c8
Merge pull request #6651 from rgerhards/codex-container-build-ci
ci: add container image build workflow
2026-03-20 14:09:15 +01:00
Rainer Gerhards
5650dbc71c
ci: add container image build workflow
Why: the rsyslog container family needs dedicated CI before release
publishing is automated, but the workflow also needs to stay usable
with branch protection on unrelated PRs.

Impact: pull requests always get a container CI result, while the
actual image build only runs when the container packaging subtree or
this workflow file changes.

Before/After: there was no dedicated container image CI workflow;
now the image family has an explicit build-only workflow with clear
skip behavior for unrelated changes.

Technical Overview:
Add a dedicated GitHub Actions workflow for the rsyslog container
family.

Run the workflow on every pull request so it can be used as a branch
protection check.

Scope push builds to changes under packaging/docker/rsyslog and this
workflow file.

Use changed-files in the PR path so unrelated pull requests succeed
with an explicit skip step instead of rebuilding container images.

Build the full layered image family through the existing Docker
Makefile using a CI-specific image tag derived from the commit SHA.

Document the trigger and skip intent with comments in the workflow.

With the help of AI-Agents: Codex
2026-03-20 13:51:48 +01:00
Rainer Gerhards
6e0ce2d2d8
ci: skip doc preview comment on fork PRs
Why
Fork-origin pull requests run with a read-only token even when the job
requests comment-write permissions. That made the documentation preview
comment step fail with a 403 after the docs themselves built
successfully.

Impact
Fork PRs no longer fail the docs workflow just because the preview
comment cannot be posted.

Before/After
Before, fork PRs could show a red docs-related check due to a failed
comment write. After, the comment job only runs when the PR branch is
in the same repository.

Technical Overview
Guard the `publish_pr_preview` job in `doc_build.yml` so it only runs
for pull requests whose head repository matches `github.repository`.

This preserves preview comments for same-repo PRs while avoiding the
GitHub token permission trap for fork-origin contributions.

Validation
Reviewed the failing workflow log for PR 6649 and confirmed the error
was a 403 from `issues.createComment`, not a docs build failure.

With the help of AI-Agents: Codex
2026-03-20 13:40:26 +01:00
Rainer Gerhards
86c9e04a2f ci: stop PR docs from overwriting Pages
Why: PR preview runs publish a Pages artifact that only contains
`pr-<num>/`, which replaces the production docs site when deployed.

Impact: PR doc workflows keep producing downloadable HTML artifacts
and status comments, but no longer deploy to GitHub Pages.

Before/After: before a later PR preview run could turn `/` and `/doc/`
into 404s; after only the main docs deploy workflow updates Pages.

Technical Overview:
Remove the top-level Pages and id-token permissions from the doc
build workflow.

Convert the PR preview job from a Pages deployment job into a PR
comment-only job.

Drop the artifact download, Pages artifact preparation, configure-pages,
upload-pages-artifact, and deploy-pages steps from PR runs.

Keep the PR comment, but point reviewers to the workflow run and the
uploaded HTML artifact instead of a Pages preview URL.

With the help of AI-Agents: Codex
2026-03-20 08:35:23 +01:00
Rainer Gerhards
b840717d8f
ci: fix yamllint changed-file handling
Why: the workflow reconstructs changed YAML paths inside a heredoc,
which can hand yamllint malformed filenames.

Impact: changed YAML files are linted reliably even when the action
returns multiline path output.

Before/After: before the job could pass a path with a stray trailing
backslash to yamllint; after it reads the changed-file list from the
environment and lints each file directly.

Technical Overview:
Keep using tj-actions/changed-files with newline-separated output.

Pass the changed-file list through an environment variable instead of
splicing it into the shell script body.

Read the list line by line in bash and invoke yamllint once per file.

This preserves the changed-files-only scope while avoiding shell
reconstruction bugs in the workflow step.

With the help of AI-Agents: Codex
2026-03-17 17:38:55 +01:00
Rainer Gerhards
6ae7f6b0f0 ci: extend focused repo policy checks
Why: the first deterministic policy workflow covers the most obvious
repo rules, but it still leaves new-module build wiring and parameter
doc coverage to human review.

Impact: the focused review now also checks top-level build wiring for
new modules and warns when new module parameters lack reference docs.

Before/After: before the workflow only checked tests, doc distribution,
and basic module onboarding; after it also covers module build manifests
and parameter reference doc sync.

Technical Overview:
Extend the review package builder with deterministic facts for new
module build wiring against the top-level Makefile.am and configure.ac.

Add a parameter-doc-sync rule that compares newly introduced
cnfparamdescr names against the expected files under
`doc/source/reference/parameters/`.

Teach the deterministic evaluator about the two new checks and update
the workflow trigger/doc text to match the expanded rule set.

With the help of AI-Agents: Codex
2026-03-16 15:51:45 +01:00
Rainer Gerhards
8fd70ab92b
Merge pull request #6617 from rgerhards/feature/post-quantum-tls
tls: add native post-quantum guidance and tests
2026-03-16 12:31:57 +01:00
Rainer Gerhards
b91a0eeb6b
Merge pull request #6626 from rgerhards/feature/yamllint-120
ci: relax yamllint line length to 120
2026-03-16 12:16:59 +01:00
Rainer Gerhards
56a9e73fc6
Merge pull request #6622 from alorbach/copilot/review-issue-6620
doc: Fix Mermaid diagrams and unify doc build output
2026-03-16 12:16:30 +01:00
Rainer Gerhards
23743a635a ci: quote yamllint file arguments safely
Why: the changed-file output was being expanded as a raw shell string,
which can break on paths with spaces or leading dashes.

Impact: yamllint now receives the changed YAML paths as proper quoted
arguments.

Before/After: before the workflow interpolated `all_changed_files`
directly into the shell command; after it reads newline-separated paths
into an array and passes them safely.

Technical Overview:
Configure `tj-actions/changed-files` to emit newline-separated paths.

Use `mapfile` in the lint step and invoke yamllint with the resulting
quoted array.

With the help of AI-Agents: Codex
2026-03-16 12:06:58 +01:00
Rainer Gerhards
81f3e1ce63 ci: wrap yamllint config line
Why: the inline yamllint override itself exceeded the new 120-character
limit and triggered the warning this branch is trying to address.

Impact: the workflow keeps the 120-character override without linting
itself.

Before/After: before the new override line was 123 characters; after it
is split across shell continuation lines.

Technical Overview:
Rewrite the yamllint invocation in the workflow as a multi-line shell
command while keeping the same inline config.

With the help of AI-Agents: Codex
2026-03-16 12:06:20 +01:00
Rainer Gerhards
15c35d7ce3 ci: relax yamllint line length to 120
Why: the default relaxed profile still warns on lines longer than 80
characters, which is too short for some workflow and YAML content.

Impact: yamllint will now allow lines up to 120 characters before
warning.

Before/After: before yamllint warned at 80 characters even in relaxed
mode; after the line-length rule is capped at 120.

Technical Overview:
Keep the workflow on the relaxed yamllint profile and override only
the line-length maximum in the inline configuration string.

With the help of AI-Agents: Codex
2026-03-16 12:02:13 +01:00
Rainer Gerhards
7a409ea6a8 ci: add focused repo policy review
Why: rsyslog CI already covers generic quality well, but it lacked
repository-specific policy checks in a few recurring areas.

Impact: pull requests now get a focused deterministic policy check
for test registration, doc distribution sync, and new-module
onboarding.

Before/After: before these repository rules depended on reviewer
memory; after CI checks them directly and reports all findings in one
run.

Technical Overview:
Add a dedicated workflow that only triggers for policy-relevant
changes and builds a focused review package from the pull-request
diff.

Evaluate the focused checks deterministically for tests, docs, and
new modules, then normalize the results into a workflow summary.

Fail the workflow only for deterministic policy violations, while
still printing advisory warnings so contributors can address all
follow-up in one iteration.

With the help of AI-Agents: Codex
2026-03-16 09:19:10 +01:00
b4d8463e56 doc: Fix Mermaid diagrams and unify build output to build/
Why:
  Mermaid diagrams fail on GitHub Pages PR previews and when opening
  built HTML via file://. ES module imports use wrong paths and CORS
  blocks file://. Two build paths (build/ vs build/html/) caused
  confusion and inconsistent fix application.

Impact:
  Doc build output path unified; Mermaid diagrams render in offline,
  online, and RPM builds.

Before/After:
  Before: Diagrams broken on pr-N previews and file://; two output
  paths. After: Single build/ path; diagrams work everywhere.

Technical Overview:
  - conf.py: Simplify sphinxcontrib-mermaid monkey-patch; force
    Dagre renderer; improve UMD/file:// comments.
  - fix-mermaid-offline.py: Remove ELK script first (regex order fix);
    add inline ESM import handling for HTTP; compute _static path per
    file depth.
  - Makefile, inside_docker, CI: Switch -M html to -b html; output
    to build/ instead of build/html/.
  - build_rag_db.py: Use build/.doctrees for -b html.
  - doc_build.yml: Add conf.py and fix-mermaid-offline.py to
    changed-files; update artifact paths to doc/build.

Fixes: https://github.com/rsyslog/rsyslog/issues/6620
2026-03-13 15:27:38 +01:00
1382d04062 ci: upgrade RPM CI from Rocky Linux 8 / EPEL-8 to Rocky Linux 9 / EPEL-9
Why:
  Align CI with current Rocky Linux and EPEL versions for ongoing
  support and security.

Impact:
  RPM build job and mock defaults now target EL9; artifact names change.

Before/After:
  Before: Rocky Linux 8 container, epel-8 repo, powertools, epel-8-x86_64.
  After:  Rocky Linux 9 container, epel-9 repo, crb, epel-9-x86_64.

Technical Overview:
  - .github/workflows/run_checks.yml: Use quay.io/rockylinux/rockylinux:9
    and EPEL-9 baseurl; enable crb instead of powertools; set artifact
    name to rpms-epel-9-x86_64.
  - devtools/run-rpm-build.sh: Default MOCK_CONFIG to epel-9-x86_64 so
    local/CI runs match the new environment.
2026-03-12 12:31:01 +01:00
Rainer Gerhards
d128ef88a5 tls: add native post-quantum guidance and tests
Why
Native post-quantum TLS support should be usable and testable on newer
distro baselines without adding provider-mode compatibility work for
older platforms.

Impact
Rsyslog now has native-PQ smoke tests, clearer TLS diagnostics, updated
CI baselines and helper images, and a new post-quantum tutorial for
supported distros.

Before/After
Before: Fedora CI still targeted Fedora 41, PQ-capable TLS settings had
no dedicated rsyslog tests or user-facing tutorial, and stricter clang
builds could fail on warning-group handling.
After: CI targets Fedora 43, native PQ usage is documented and smoke-
tested, helper images include the required tools, and the branch builds
and tests cleanly with the newer compiler/container combinations.

Technical Overview
The CI matrix now replaces the Fedora 41 lane with Fedora 43 and adds a
matching Fedora 43 development image.
The Debian 13 and Fedora 43 development containers now install the
GnuTLS CLI utilities needed for native PQ capability checks.
The OpenSSL TLS config path logs clearer messages when a command or
value is unavailable on the native OpenSSL build.
The GnuTLS TLS config path reports unsupported priority-string options
more explicitly.
Two new shell tests add native PQ smoke coverage for OpenSSL and GnuTLS
using the existing gnutlsPriorityString control surface.
Those tests self-skip unless the local native TLS libraries expose the
required hybrid group support.
The imtcp parameter docs and omfwd docs now explain the native-only PQ
support policy and include example configurations.
A new tutorial documents native PQ usage for OpenSSL and GnuTLS on
supported newer distro versions.
The shared runtime warning policy in rsyslog.h now tolerates clang
handling of unknown warning groups so older and newer clang lanes remain
warning-free under the existing finalize_it error-handling pattern.
Testbench follow-ups harden omfwd-lb-susp with isolated retry attempts,
skip rcvr_fail_restore on ARM where it is timing-flaky, and keep local
SC2181 suppressions where if-exec rewrites would reduce shell-script
usability.
The Fedora 43 Dockerfile now cleans the dnf cache after install and
locally suppresses the non-useful DL3041 package-version pinning warning.
Older distro versions remain intentionally unsupported for PQ in this
phase because we expect users to move to newer baselines first.
If there is demand later, older-version support can be considered in a
separate effort.

With the help of AI-Agents: Codex
2026-03-11 15:10:24 +01:00
Rainer Gerhards
5737692fc2
Merge pull request #6611 from alorbach/add/deploy-rpm-ci
packaging/rpm: add RPM build CI from rsyslog-pkg-rhel-centos
2026-03-11 10:09:01 +01:00
Rainer Gerhards
0fa27bb07e
github: assign maintainers to CODEOWNERS coverage
Why
Ensure GitHub code owner review applies consistently to repository
metadata, CI, build tooling, and the default source tree.

Impact
Security-sensitive paths now require review from the listed
maintainers when branch protection enforces code owner approval.

Before/After
Before: only workflow files named a single owner.
After: metadata, CI, build tooling, and all other paths map to both
maintainers.

Technical Overview
Expand `.github/CODEOWNERS` from workflow-specific entries to a
small explicit rule set.

Add root rules for `.gitmodules` and `.gitattributes`.

Replace the separate workflow globs with a single
`.github/workflows/*` rule.

Add ownership for `autogen.sh`, `configure.ac`, `Makefile.am`, and
`devtools/` to cover build and release tooling.

Add a final `*` rule so the core source tree and remaining repository
paths are owned by the maintainers.

With the help of AI-Agents: Codex
2026-03-10 19:01:26 +01:00
478d218c24 rpm: packaging overhaul, CI job, and review fixes
- Add package_build_rpm CI job (Rocky Linux 8, mock, EPEL 8)
- Add devtools/run-rpm-build.sh for CI RPM builds via mock
- Add packaging/rpm/ with build-rpms.sh, config.sh, mock configs
- Add packaging/rpm/rpmbuild/ (SPECS, SOURCES) for rsyslog and deps
- Main spec: rsyslog-v8-stable.spec (EL8/EL9, RHEL 7 dropped)
- qpid-proton tarballs as Source5/Source6 with SHA256 checksums
- libfastjson4 Source0: GitHub refs/tags/v%{version} URL
- build-rpms.sh: atomic downloads, GPG --batch --pinentry-mode loopback
- Fix duplicate changelog NVR 8.2008.0-2 -> 8.2008.0-3 (Sep 29 2020)
- .gitignore: root-anchored /build-result/
- doc/BUILDS_README.md: document why pip is needed for RPM Sphinx build

closes: https://github.com/rsyslog/rsyslog/issues/6584
2026-03-05 11:49:10 +01:00
Rainer Gerhards
18f6a5e2e6
doc: add deterministic HTML link validation in docs CI
Why:
Broken local links in generated docs can slip through Sphinx success and
only surface after deployment.

Impact:
Docs CI now fails on invalid generated local links and missing anchors.

Before/After:
Before, docs CI built HTML but did not reliably validate generated links.
After, docs CI runs deterministic HTML link checks on built output.

Technical Overview:
Add doc/tools/check-html-links.py to parse generated HTML and validate
local href/src/srcset targets deterministically.
Support base href resolution to match browser-local URL resolution.
Validate optional fragment anchors using id/name lookup.
Decode percent-encoded URL paths before filesystem target checks.
Guard against paths that escape the build root after normalization.
Update doc_build workflow to run the checker after Sphinx HTML build.
Use the HTML output root (doc-builder/doc/build/html) for correct
root-absolute link validation behavior.

With the help of AI-Agents: codex
2026-02-25 09:54:21 +01:00
Rainer Gerhards
9217eefc40
CI: increase permitted macOS runner timeout
Practice shows that the timeout is no longer sufficient and needs
to be bumped.
2026-02-23 16:09:10 +01:00
Rainer Gerhards
f68532193b
ci: restrict doc deployment to official repository
This prevents documentation deployment workflows from running on forks,
saving GitHub Actions minutes and reducing noise for contributors.
The workflow will now only execute when the repository is
'rsyslog/rsyslog'.
2026-02-19 18:50:00 +01:00
Rainer Gerhards
8f1c0a3e83
ci: enable core dumps in ASAN builds
AddressSanitizer by default sets disable_coredump=1, which prevents
core file generation even when abort() is called. This commit adds
disable_coredump=0 to ASAN_OPTIONS in both macOS and ARM64 workflows
to ensure core dumps are generated on crashes.

This allows diag.sh to find and analyze core files for better
debugging of test failures in ASAN-enabled CI jobs.

Affected workflows:
- macOS CI (ASAN builds)
- ARM64 CI (native ASAN builds)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-19 17:13:59 +01:00
d34fc12574 ci: add DOC_BASE_URL variable for docs Sitemap base
Sitemap uses DOC_BASE_URL/doc/sitemap.xml when the repository
variable is set; otherwise falls back to the github.io URL for forks.

Technical overview:
- Add DOC_BASE_URL env to the Prepare Pages step (from vars.DOC_BASE_URL).
- Introduce DEFAULT_BASE from github.repository for backward compatibility.
- BASE="${DOC_BASE_URL:-$DEFAULT_BASE}" selects override or fallback.
- Document the variable in the workflow header comment.
- No change to build artifact layout or deployment flow.
2026-02-19 12:52:42 +01:00
7d36865842 doc: fix release_string_detail NameError, narrow deploy to main only
Impact: Doc build succeeds with release_type=dev from env; deploy runs
only on push to main.

Before: Env override with release_type=dev hit NameError in html_title
block; deploy ran on main, master, and pr/publish-doc.
After: release_string_detail set when release_type=dev; deploy on main.

Technical Overview:
- doc/source/conf.py: Set release_string_detail = 'simple' when
  release_type == 'dev' in the env override block so html_title and
  epub blocks no longer reference an undefined variable.
- doc_deploy_main.yml: Replace branches with [main] (drop master and
  pr/publish-doc).
2026-02-18 19:52:09 +01:00
6de9790f11 doc: add GitHub Actions workflow to deploy docs with GA to Pages
Why: Enable automated deployment of rsyslog documentation to GitHub
Pages with Google Analytics, using the same build setup as production.

Impact: New workflow, build script, and Docker image updates; removes
cleanup_pr_preview job from doc_build.yml.

Before: Documentation deploy was manual; PR preview folders lingered
on gh-pages after PR close.
After: Push to main triggers build and deploy to /doc/ with GA; root
redirects to /doc/; robots.txt restricts crawling to /doc/ only;
PR preview cleanup removed in favor of Pages artifact deploy.

Technical Overview:
- Add .github/workflows/doc_deploy_main.yml: triggers on push to main,
  master, pr/publish-doc or workflow_dispatch; builds in Docker with
  rsyslog_dev_doc_base_ubuntu:22.04; deploys under /doc/ via
  upload-pages-artifact; runs container as -u "$(id -u):$(id -g)";
  adds root index.html redirect, robots.txt (Allow /doc/, Sitemap).
- Add doc/tools/inside_docker_doc_html.sh: uses pre-built venv from
  image; sets RSYSLOG_DOC_VERSION/RSYSLOG_DOC_RELEASE_TYPE; runs
  sphinx-build with Furo/sitemap; applies Mermaid fix.
- doc/source/conf.py: support RSYSLOG_DOC_VERSION and
  RSYSLOG_DOC_RELEASE_TYPE env vars; format rst_prolog for non-git
  builds; use release_type in release string (Gemini feedback).
- Add doc/tools/pages-root-index.html (redirect / to /doc/),
  doc/tools/pages-robots.txt (Allow /doc/, Disallow /, Sitemap).
- packaging/docker/dev_env/ubuntu/doc_base/22.04: add python3-venv;
  pre-build /opt/rsyslog-doc-venv from doc/requirements.txt; build.sh
  runs from repo root for COPY.
- doc/Makefile.am: add inside_docker_doc_html.sh, pages-root-index.html,
  pages-robots.txt to EXTRA_DIST.
- doc_build.yml: remove cleanup_pr_preview job.

Requires GOOGLE_ANALYTICS_ID repository secret.
2026-02-18 18:00:23 +01:00
Rainer Gerhards
07917ca1b8
configure: default-enable impstats push build support (#6542)
* build: default-enable impstats-push and align CI containers

Enable impstats-push by default and keep configure strict when dependencies are missing.

Update CI/container definitions for distro differences (CentOS/OpenEuler/Ubuntu and workflow overrides), add explicit --disable-impstats-push where impstats is disabled, and fix impstats protobuf generation for distcheck/VPATH builds.
2026-02-16 14:33:09 +01:00
cd85debac0 ci: extend GitHub Actions for ARM (armhf QEMU, arm64 native)
Why:
Enable CI validation on ARM architectures so platform-specific regressions
are caught before merge. arm64 uses native GitHub runners; armhf uses QEMU
because GitHub does not offer 32-bit ARM runners. Network namespace tests may
fail under QEMU where the mount syscall is not properly emulated.

Impact:
- New arm_CI job runs on armhf and arm64 when relevant files change.
- arm64 uses native ubuntu-24.04-arm runner; armhf uses QEMU on x64.
- Netns tests skip gracefully (exit 77) if ip netns add fails.

Before:
- No ARM CI; netns tests could fail with cryptic errors under QEMU.

After:
- arm_CI: armhf (QEMU, reduced test set) and arm64 (native, expanded tests, ASan).
- require_netns_capable() in diag.sh; netns tests call it and skip when unavailable.
- ratelimit double-free fixed in ratelimit.c.
- skip_ASAN() in diag.sh; empty-hostname, omfile-read-only* skip when ASan enabled
  (LD_PRELOAD/read-only behavior conflicts).

Technical Overview:
- Add arm_CI job to run_checks.yml: matrix over armhf/arm64. armhf:
  runs-on ubuntu-24.04, QEMU + Docker Buildx, reduced configure (disable-default-tests,
  many modules disabled). arm64: runs-on ubuntu-24.04-arm (native), expanded
  configure (default tests, gnutls, relp, imfile, etc.). Conditional QEMU
  setup only for armhf.
- Add devtools/ci/Dockerfile.arm: Ubuntu 24.04 with build tools, gnutls,
  libestr, libfastjson, zlib, iproute2, libgcrypt, librelp, uuid, libyaml
  (for arm64 expanded build).
- Add require_netns_capable() to diag.sh; use in imtcp-netns.sh,
  uxsock_multiple_netns.sh, tcp_forwarding_ns_tpl.sh.
- Add skip_ASAN() to diag.sh; use in empty-hostname.sh, omfile-read-only.sh,
  omfile-read-only-errmsg.sh.
- Add devtools/ci/Dockerfile.arm to arm_CI changed-files filter.
- Quote $GITHUB_OUTPUT and $GITHUB_STEP_SUMMARY in clang static analyzer steps.
- Fix double-free in ratelimit.c: shared->name is the hashtable key, freed by
  hashtable_destroy; remove redundant free(shared->name) in ratelimitFreeShared.
2026-02-12 22:31:56 +01:00
Rainer Gerhards
9e7cb9556b
Merge pull request #6540 from rgerhards/codex-i-victorialogs-ci
omhttp: add VictoriaLogs jsonline CI test
2026-02-09 09:24:44 +01:00
Rainer Gerhards
88bcf7ee48
Merge pull request #6504 from rgerhards/feature/impstats-victoriametrics-push
impstats: add VictoriaMetrics push support via Prometheus Remote Write
2026-02-08 17:12:03 +01:00
Rainer Gerhards
1d32f29035
omhttp: add VictoriaLogs jsonline CI test
Why
VictoriaLogs jsonline is a target deployment path for omhttp users and
we need a direct integration signal in PR CI.

Impact
Adds a real-container omhttp->VictoriaLogs validation path and a scoped
CI job for relevant PRs.

Before/After
Before: no CI test validated omhttp against VictoriaLogs jsonline.
After: PRs touching omhttp or this test run a minimal live integration
check.

Technical Overview
Add tests/omhttp-victorialogs-jsonline.sh to send batched newline JSONL
payloads with omhttp to /insert/jsonline and verify indexed results via
/select/logsql/query.

Use jsonf list templating and a per-run marker to isolate records during
query validation. Keep transport on plain HTTP for CI simplicity.

Register the test in tests/Makefile.am under TESTS_OMHTTP so it is part
of testbench distribution and invocable as a single .log target.

Add a new run_checks.yml job named victorialogs_CI that starts a
VictoriaLogs service container, runs only
omhttp-victorialogs-jsonline.log, and gates execution with changed-files
filters for the test, omhttp components, and the workflow itself.

With the help of AI-Agents: Codex (GPT-5)
2026-02-08 17:06:24 +01:00
Rainer Gerhards
f045a81f02
ci: consolidate PR workflows into run_checks
Move standalone PR workflows into .github/workflows/run_checks.yml so compile acts as the common gate for downstream CI jobs.

This ensures macOS, journal, codecov (base/kafka), elasticsearch, clang analyzer, and kafka distcheck do not run when the initial compile matrix fails.

Also align changed-files logic to avoid broad workflow globs and let non-compile jobs ignore doc/Makefile.am, while compile intentionally keeps doc/Makefile.am in scope to satisfy branch-protection expectations.
2026-02-07 13:16:25 +01:00
Rainer Gerhards
d9489d70c9
ci: unify compile and check workflow for PRs
Other github-action coordination mechanisms are too complex, so better
have one large action file.
2026-02-06 17:34:39 +01:00
Rainer Gerhards
653a97fa08
CI: fix workflow, did the trigger conditions incorrectly 2026-02-06 16:56:56 +01:00
Rainer Gerhards
c253fc36cc
impstats: add VictoriaMetrics push via Remote Write
Adds first-class integration with VictoriaMetrics to simplify ops
dashboards and move toward project-supported telemetry without
sidecar collectors.

Impact: New optional feature (off by default). No behavior change
unless configured via push.* parameters.

Before: impstats could only log locally or emit text formats.
After: impstats can push counters to Prometheus-compatible endpoints.

Technical: implement a native Prometheus Remote Write path in
impstats, encoding counters to protobuf and compressing with snappy
over HTTP via libcurl. Replace interim text parsing with a new
statsobj v14 API (GetAllCounters) that iterates raw uint64 counters,
keeps atomic reads for IntCtr and best-effort reads for Int. Add
metric builder with Prometheus-compliant sanitization and the naming
pattern <origin>_<name>_<counter>_total. Provide TLS knobs (CA, mTLS,
insecureSkipVerify), static/dynamic labels, timeout, and optional
batching by bytes/series. Build is gated behind
--enable-impstats-push with protobuf-c/snappy/curl checks. Ship docs,
basic/VM integration tests, and a GitHub Actions workflow using a
VictoriaMetrics service; TSAN jobs disable impstats-push.

Configuration: push.url, push.labels, push.timeout.ms,
push.label.{instance,job,origin,name}, push.tls.{cafile,certfile,
keyfile,insecureSkipVerify}, push.batch.{maxBytes,maxSeries}.

With the Help of AI Agents: ChatGPT codex 5.2
2026-02-06 16:02:08 +01:00
Rainer Gerhards
59aa87218a
github action security follow up (#6532)
* CI: enable run_checks for forks & secure workflows

Previously, the `run_checks` workflow was restricted to run only
for PRs originating from the same repository. This prevented
integration tests from running on PRs submitted from forks.

This change enables `run_checks` for forks by:
1.  Removing the repository check in the `if` condition.
2.  Adding `permissions: contents: read` to ensure the workflow
    runs with minimal privileges, mitigating security risks.
3.  Updating the `checkout` step to explicitly use the PR's head
    repository and ref.
4.  Fetching the `upstream` remote to ensure the git history is
    complete for `tj-actions/changed-files`.

Additionally, a `.github/CODEOWNERS` file is added to require
review from @rgerhards for any changes to `.github/workflows/`,
further securing the CI configuration against malicious PRs.

Co-authored-by: rgerhards <1482123+rgerhards@users.noreply.github.com>
2026-02-06 15:57:17 +01:00
Rainer Gerhards
5045b24202 ci: gate checks after compile
Why: reduce CI runtime and improve green IT/cost efficiency by stopping redundant work on failures.

Impact: compile matrix cancels on first failure; checks wait on compile.

Before/After: checks ran in parallel; now they run after a successful compile.

Technical Overview:

- Enable fail-fast in compile matrix to cancel remaining jobs.

- Trigger run_checks via workflow_run on compile completion.

- Guard run_checks to PRs with successful compile outcomes.

- Restrict run_checks to same-repo PRs to avoid elevated token risk.

- Recompute change filters in run_checks with changed-files.

- Skip container CI when no relevant changes are detected.

- No local tests run (workflow changes only).

With the help of AI-Agents: Codex
2026-02-05 16:58:14 +01:00
Rainer Gerhards
ece2e239ae
CI: do no longer validate commit style
Thanks to AI, the commit style is very often not correct in interim
states. As such this check has become a very noisy signal and is no
longer worth it.

Instead, maintainers need to be a bit more careful during merge. But,
honestly, this almost-red check required that as well.
2026-02-04 15:33:32 +01:00
Rainer Gerhards
0ae7417cf5
CI: remove clutter from MacOS TSAN/ASAN error reports 2026-01-28 17:20:08 +01:00
Rainer Gerhards
89993d1f86
CI: do not compile on pure doc change
The doc Makefile.am did trigger the general rule for running full
builds, which is not necessary as it only contains doc-related
stuff.
2026-01-17 12:41:40 +01:00
Rainer Gerhards
f6ace2702f
build: move doc EXTRA_DIST into doc/Makefile.am
This refactor cleans up the root Makefile.am and aligns documentation
packaging with standard Automake structure. It improves maintainability
and keeps doc build logic localized, which helps container/CI setups.

BEFORE: Root Makefile.am listed all doc files via EXTRA_DIST.
AFTER:  doc/Makefile.am owns the doc EXTRA_DIST and Sphinx targets.

Technical details:
- Added "doc" to SUBDIRS so Automake processes the directory.
- Added "doc/Makefile" to AC_CONFIG_FILES in configure.ac.
- Removed the hand-written doc/Makefile in favor of Automake-managed
  doc/Makefile.am. The new file provides convenience targets
  (html-local, html-with-sitemap, singlehtml, json, alljson, rag-db)
  that wrap sphinx-build, and a clean-local rule.
- Updated doc/.gitignore to ignore generated Makefile/Makefile.in.
- The list of documentation assets was moved to doc/Makefile.am
  (paths made relative to doc/). No runtime or ABI changes expected;
  dist tarball contents intended to be unchanged.

Impact: Build/packaging only. CI or scripts calling old doc targets may
need to switch to "make -C doc html-local".

With the help of AI Agents: Google Jules, cubic dev ai, ChatGPT codex
2026-01-17 12:15:49 +01:00
Rainer Gerhards
911d030e10 CI: add support for wolfssl testing 2026-01-15 15:08:39 +01:00
Rainer Gerhards
f91adbefc4
CI: run SAN, UBSAN under Ubuntu 24.04 for more modern tooling
- core: suppress pointer mismatch for UBSAN
  The supression is just an interim solution until the pointer issue
  has been fully analyzed and aligned (or considered OK w/ reasoning).
- suppress an issue in libcivetweb
2026-01-01 13:10:30 +01:00
Rainer Gerhards
37b16cf8ad
doc: add RAG Knowledge Base extraction and build integration
This commit adds a context-aware RAG dataset generator and integrates
it into the build system to support advanced AI-driven documentation
workflows. It ensures a consistent knowledge base for RAG pipelines.

Impact: Adds doc/build_rag_db.py; output in build/rag/ (cleaned).

Before:
- No automated way to generate RAG-ready JSON from Sphinx doctrees.
- Nested code blocks in lists and notes were skipped during extraction.
- Fragile string parsing and broad exception handling in extractor.
- Data loss occurred due to hard truncation at 2000 characters.
- CI only ran HTML build, not RAG generation.

After:
- Recursive walker captures all nested code and prose thematic blocks.
- Thematic merging creates context-rich chunks (~2000 chars).
- Non-lossy chunking logic flushes buffer at natural boundaries.
- Metadata (module, scope, item) and syntax templates injected.
- Robust regex-based parameter parsing and specific error handling.
- Module-level constants for node types improve readability.
- Makefile target 'json-formatter' (and 'rag-db') automated the build.
- Project docs (README.md, AGENTS.md) updated with instructions.
- GitHub Actions workflow builds and uploads RAG artifact.

AI-Agent: Antigravity
2025-12-23 11:18:45 +01:00
Rainer Gerhards
3f52f6b829
CI: build doc on pull_request, not pull_request_target
With _target, we always get outdated doc into CI, which does not
really work for testing. So we switch back to just pull_request.
In this case, we cannot deploy the doc, but that's kind of OK.

We can probably later clean up the workflow once we know this change
here actually fixes the situation.
2025-12-21 16:18:27 +01:00
3e4e0fc4d7 Fix doc build CI security, spelling, and RST build errors
Security & CI improvements:
- Remove pull_request_target trigger (security vulnerability)
- Simplify checkout to use default behavior (matches other workflows)
- Reduce fetch-depth from 8 to 2 for change detection
- Simplify conditional logic throughout workflow
- Enable fork PR support: allow fork PRs to build docs and produce artifacts
- Make Pages deployment conditional: only deploy for same-repo PRs
- Update PR comments to handle fork vs same-repo PRs appropriately

Documentation fixes:
- Fix 3 instances of "it's" → "its" in reliable_logging.rst
- Fix 22 RST build errors: replace :json: roles with inline code in
  6 mmdarwin parameter files

The workflow now follows the same pattern as 19 other workflows in the
repository, improves security posture, enables fork contributors to build
documentation, and fixes all documentation build errors.
2025-12-18 09:45:06 +01:00
17890316fb
Merge pull request #6362 from alorbach/pr-issue-6361
Refactor: Rename omotlp module to omotel
2025-12-17 12:32:14 +01:00
Cursor Agent
fbdf736092 Remove macos-13 from CI and weekly jobs
Co-authored-by: alorbach <alorbach@adiscon.com>
2025-12-17 10:06:00 +00:00
c9f56709a7 Refactor: rename omotlp module to omotel
Rename the OpenTelemetry output module from "omotlp" to "omotel" across
the entire codebase. This includes directory, file, function, type, and
constant names, as well as build system configuration, documentation,
and test files.

Changes:
- Directory: plugins/omotlp/ → plugins/omotel/
- Source files: omotlp.c → omotel.c, omotlp_http.c → omotel_http.c,
  omotlp_http.h → omotel_http.h
- Code: all function names, types, constants (OMOTLP_* → OMOTEL_*)
- Build: configure.ac (--enable-omotlp → --enable-omotel,
  OMOTLP_HTTP_* → OMOTEL_HTTP_*), Makefile.am files
- Docs: omotlp.rst → omotel.rst, all examples updated
- Tests: omotlp-*.sh → omotel-*.sh, content updated
- Tasks: omotlp_*.md → omotel_*.md
- Config: module_map.yaml, AGENTS.md

Impact:
- Module name in rsyslog.conf: "omotlp" → "omotel"
- Build flag: --enable-omotlp → --enable-omotel
- No functional changes, pure refactoring

closes: https://github.com/rsyslog/rsyslog/issues/6361
2025-12-17 10:50:06 +01:00
d42711de10 omotlp: introduce OTLP/HTTP log exporter
OpenTelemetry adoption: provide a first-party path to ship rsyslog
logs directly to OTLP collectors, enabling cleaner OTel pipelines and
container-friendly deployments without sidecars.

Impact: new output module; user-visible config surface; off by default
unless explicitly enabled at build and in config.

Before: no native OpenTelemetry (OTLP) exporter in rsyslog.
After: new "omotlp" action streams logs via OTLP/HTTP JSON with
configurable batching (count/bytes/timeout), optional gzip, retry/
backoff, TLS/mTLS, custom headers, and proxy support.

Technically, the action holds immutable config while each worker owns
an HTTP client and a batch buffer guarded by a mutex. A small flush
thread handles timeout-based flushes; batches also flush immediately on
thresholds and when the action queue transaction completes. HTTP 2xx
acknowledges and clears the batch; 4xx drops it; 5xx maps to
RS_RET_SUSPENDED for retry by the action queue. Stats per instance track
batches submitted/success/dropped/retried, HTTP 4xx/5xx, records.sent,
and cumulative request latency. Parameters honor OTEL_* env var
fallbacks when not explicitly set. Build is gated behind
--enable-omotlp; docs and tests cover batching, compression, TLS/mTLS,
proxy, and trace-correlation flows.
2025-12-05 06:44:30 +01:00