doc: add security triage rubric for AI agents

Why: automated security reviews need a shared proof standard so
hardening opportunities are not over-stated as confirmed vulnerabilities.

Impact: contributor-facing guidance now distinguishes confirmed issues,
potential issues, hardening, and invalid findings before severity or CWE
language is used.

Before/After: agent guidance pointed at documentation structure, but did
not define a security-finding evidence bar; the new rubric documents the
required source, reachability, sink, missing guard, and impact checks.

Technical Overview:
Add doc/ai/security_triage_rubric.md with classification, proof, CWE,
severity, rsyslog-specific, test, wording, and inline-comment guidance.
Link the rubric from the root AGENTS.md and doc/ai AGENTS.md files.
List the new file in doc/ai/README.md and doc/Makefile.am so it is easy
to discover and packaged with the documentation support files.

With the help of AI-Agents: Codex
This commit is contained in:
Rainer Gerhards 2026-05-10 15:03:08 +02:00
parent d2bc55836a
commit 9b466b89d1
7 changed files with 157 additions and 2 deletions

View File

@ -36,6 +36,20 @@ Every documentation page must include:
> [!IMPORTANT]
> **Trigger Side-Effect**: If you add, move, or remove any `.rst` file, YOU MUST follow the [`rsyslog_doc_dist`](../rsyslog_doc_dist/SKILL.md) skill to update `doc/Makefile.am` and run the **extended distribution check**.
> [!IMPORTANT]
> **Sample Config Security Review**: If you add or materially update a sample
> rsyslog configuration, you MUST review that specific configuration for
> security posture before finishing the documentation change. Check whether the
> example exposes listeners, weakens authentication or TLS, writes to sensitive
> paths, follows unsafe file/link behavior, uses overly broad permissions,
> enables compatibility fallbacks, or recommends defaults that are unsuitable
> for new deployments. Use `doc/ai/security_triage_rubric.md` to classify any
> concern as a confirmed issue, potential issue, hardening, or not actionable.
> If this review exposes vulnerable or unsafe code behavior, do not treat it as
> documentation-only work: either fix the code in the same change when the scope
> is clear and bounded, or record the implementation follow-up with enough
> evidence for maintainers to reproduce and prioritize it.
### 2. Module Documentation
- **Parameters**: Use the `include` directive to pull parameter details from `doc/source/reference/parameters/`.
- **Anchors**: Use explicit anchors (e.g., `.. _parameter_name:`) for consistent linking.

View File

@ -31,6 +31,10 @@ Follow these three steps for a typical development task:
- **Architecture**: Microkernel core (`runtime/`) + Loadable Plugins (`plugins/`)
- **Metadata**: Every module directory contains `MODULE_METADATA.yaml`.
- **Knowledge Base**: `doc/ai/` contains canonical patterns for RAG ingestion.
- **Security Triage**: [`doc/ai/security_triage_rubric.md`](./doc/ai/security_triage_rubric.md)
defines how AI agents must distinguish confirmed issues from potential
issues, hardening, and invalid findings before using security severity or CWE
language.
## Container Images

View File

@ -42,8 +42,9 @@ The current stable release is listed on the
## Response
rsyslog is maintained by a small team. Security reports receive priority
handling, and we aim to provide an initial response within 90 days where
possible.
handling, and we aim to acknowledge reports within 7 days where possible.
Final fixes and coordinated disclosure timelines are handled separately, as
described below.
## Requirements for a Valid Report

View File

@ -61,6 +61,7 @@ EXTRA_DIST = \
ai/drift_monitoring.md \
ai/mermaid_rules.md \
ai/module_map.yaml \
ai/security_triage_rubric.md \
ai/structure_and_paths.md \
ai/templates/template-concept.rst \
ai/templates/template-tutorial.rst \

View File

@ -7,6 +7,7 @@ This file defines specific guidelines for AI assistants working on the documenta
- **Authoring Guidelines:** [`doc/ai/authoring_guidelines.md`](./authoring_guidelines.md)
- **Structure and Paths:** [`doc/ai/structure_and_paths.md`](./structure_and_paths.md)
- **Terminology:** [`doc/ai/terminology.md`](./terminology.md)
- **Security Triage Rubric:** [`doc/ai/security_triage_rubric.md`](./security_triage_rubric.md)
- **Doc Assistant Prompt:** [`ai/rsyslog_doc_assistant/base_prompt.txt`](../../ai/rsyslog_doc_assistant/base_prompt.txt)
## Critical Requirements for Documentation
@ -23,6 +24,13 @@ All documentation files (especially module documentation) must strictly adhere t
## Common Tasks
### Security and Hardening Reviews
- Read `security_triage_rubric.md` before classifying audit findings.
- Treat unproven issues as hardening unless attacker control, reachability, and
impact are demonstrated.
- Use CWE labels only when the CWE mapping is proven by a reproducer or direct
code proof.
### Updating Module Documentation
- Ensure the module page (`doc/source/configuration/modules/<module>.rst`) has the required `.. meta::` block.
- Verify that the "Purpose" or introductory section is wrapped in `.. summary-start` and `.. summary-end`.

View File

@ -12,6 +12,7 @@
| `authoring_guidelines.md` | Required blocks, tone, section order |
| `mermaid_rules.md` | Diagram syntax rules |
| `terminology.md` | Canonical rsyslog vocabulary |
| `security_triage_rubric.md` | Security finding proof, severity, and hardening classification rules |
| `chunking_and_embeddings.md` | RAG extraction schema and chunk structure |
| `crosslinking_and_nav.md` | Navigation and cross-reference patterns |
| `drift_monitoring.md` | Detecting doc/code drift |

View File

@ -0,0 +1,126 @@
# Security Triage Rubric for AI Agents
**Purpose:** Help AI assistants and automated reviews separate confirmed
security issues from potential issues, hardening opportunities, and invalid
findings.
**Audience:** Agents performing security review, hardening work, or PR
preparation in the rsyslog repository.
## Required Classification
Classify each finding before proposing or implementing a fix:
| Classification | Required evidence |
| --- | --- |
| Confirmed issue | Working reproducer, or direct code proof equivalent to a reproducer, demonstrating: attacker-controlled input, reachable code path, and impact. |
| Potential issue | Plausible source-to-sink path and plausible impact, but missing a reproducer or an important environmental proof. |
| Hardening | Robustness improvement where exploitability is config-only, deployment-dependent, testbench-only, best-effort stats related, or not demonstrated. |
| Not actionable | Attacker control, reachability, or impact does not hold after code review. |
Default to **hardening** when proof is incomplete.
## Proof Standard
Do not call a finding a vulnerability solely because a risky pattern appears in
the code. First prove the following:
1. **Source:** Identify whether the input is remote attacker-controlled, local
user-controlled, configuration-controlled, testbench-controlled, or internal.
2. **Reachability:** Show the build option, module configuration, and runtime
path needed to reach the code.
3. **Sink:** Name the exact write, read, allocation, parser, path operation,
format operation, command boundary, or synchronization point that creates the
risk.
4. **Missing guard:** Identify the absent or insufficient check, sanitizer,
escaping step, size guard, atomic operation, or lifecycle pairing.
5. **Impact:** Demonstrate what changes for the attacker or operator: memory
corruption, arbitrary file write, file read, message loss, forged output,
denial of service, routing confusion, or only noisy statistics.
A working reproducer is preferred. Direct code proof can substitute for a
reproducer only when the source, sink, missing guard, and impact are explicit
and mechanically clear.
## CWE Use
Use CWE labels only for confirmed issues.
For potential issues, keep CWE mappings in internal notes unless the proof is
complete. If a CWE is obvious from direct code reasoning, still name the exact
source, sink, guard, and impact. Avoid public PR titles or descriptions that
claim a CWE when the finding is only hardening.
## Severity Guidance
Rate severity by demonstrated impact and realistic preconditions:
| Severity | Guidance |
| --- | --- |
| Critical | Default-reachable unauthenticated remote code execution, equivalent memory corruption with reliable control, or arbitrary privileged file write with realistic attacker control. |
| High | Attacker-controlled memory corruption, privileged file replacement, or major integrity impact with plausible deployment assumptions. |
| Medium | Meaningful integrity, confidentiality, denial-of-service, routing, or log-forgery impact that depends on a non-default but realistic deployment. |
| Low | Config-only, local-only with strong preconditions, testbench-only, noisy stats, diagnostic-only, or defense-in-depth hardening. |
| None | Not reachable, already guarded, intended behavior, or no demonstrated impact. |
Configuration control is usually administrator-equivalent in rsyslog. Treat
config-only behavior as hardening or low severity unless there is a separate,
realistic path for a lower-privileged actor to influence that configuration.
## Rsyslog-Specific Checks
Apply these checks before filing or implementing a finding:
- **Stats are best effort.** Atomic operations are acceptable on Linux. Do not
add full mutex synchronization solely to make stats exact; noisy stats are
preferable to hot-path contention.
- **Testbench modules are not product exposure by default.** Modules such as
imdiag may intentionally expose diagnostic controls when explicitly enabled
for tests. Treat misuse as operational risk unless default product exposure is
demonstrated.
- **Config paths differ from runtime input paths.** A path chosen by the
administrator is not automatically attacker input. For path findings, state
who controls each path component.
- **Legacy behavior may be compatibility-controlled.** For breaking hardening
defaults, prefer the global `compatibility.defaults.secure` policy:
`strict`, `warn`, and `backward-compatible`.
- **Syntax compatibility is separate from runtime hardening.** Use
`compatibility.configformat.*` only for legacy syntax controls, not for
runtime security defaults.
## Test Expectations
Do not add placeholder tests.
Tests should exercise the changed path and fail without the fix when practical.
Register shell tests through `tests/Makefile.am`; do not create separate
recursive test harnesses below `tests/`. Prefer focused `diag.sh` tests over
large end-to-end suites unless the risk crosses module boundaries.
## Public Wording
Use wording that matches the proof:
- Confirmed issue: describe the demonstrated issue and impact precisely.
- Potential issue: say what remains unproven.
- Hardening: use terms such as "harden", "bound", "validate", "avoid",
"preserve", or "make robust".
Avoid overstating exploitability. Do not use "critical", "RCE", "arbitrary",
or CWE labels in PR titles unless the proof standard is met.
## Inline Comments
Prefer agent-facing documentation for broad policy. Use inline comments only
when they preserve a local invariant that future maintainers could otherwise
break, for example:
- A non-obvious size calculation before appending variable-length metadata.
- A deliberate compatibility fallback controlled by
`compatibility.defaults.secure`.
- A concurrency decision where stats are intentionally best effort.
- A lifecycle pairing such as an initialization helper that needs destruction
when atomic operations are disabled or unavailable.
Avoid inline comments that merely restate code or label a block as "security"
without explaining the invariant.