rsyslog/rsyslog
1
0
Fork 0
mirror of https://github.com/rsyslog/rsyslog.git synced 2025-12-15 19:50:40 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #5807 from rgerhards/doc-strategy

Browse Source 
doc: provide strategy document
This commit is contained in:
Rainer Gerhards 2025-07-17 13:15:34 +02:00 committed by GitHub
commit 5601e8a1a3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 415 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

299
doc/IMPLEMENTATION_PLAN.md Normal file
View File

@ -0,0 +1,299 @@
# Rsyslog Documentation: Implementation & Quality Improvement Plan
**Version: 1.0**
**Related Document:** `STRATEGY.md`
## 1. Introduction
This document is the practical playbook for executing the official Rsyslog Documentation Strategy. It breaks down the high-level strategy into concrete phases and actionable tasks. Its purpose is to guide the development team through the setup, migration, automation, and long-term improvement of the documentation.
---
## 2. Phase 1: Foundational Setup
This phase focuses on creating the new structure and environment.
### Task 2.1: Establish the New Directory Structure
The first step is to create the new, hierarchical directory structure within `doc/source/`. This provides a clean foundation for all subsequent work.
**Action:** Create the following directories and move the existing `.rst` files into their most logical new home. For files that are difficult to categorize, place them in a temporary `_needs_triage` directory.
```
doc/
├── examples/ \# For validated, standalone rsyslog.conf examples
└── source/
├── \_static/ \# For CSS, images, etc.
├── \_templates/ \# For custom HTML templates
├── \_needs\_triage/ \# Temporary home for uncategorized old files
├── 01\_getting\_started/
├── 02\_concepts/
├── 03\_configuration/
├── 04\_modules/
├── 05\_how-to\_guides/
├── 06\_reference/
├── 07\_tutorials/
├── 08\_deployment\_guides/
│ ├── container\_deployment/
│ └── aws\_deployment/
└── man/ \# For the RST source of man pages
````
### Task 2.2: Configure `conf.py` for the New System
To power our new features, we must update the Sphinx configuration file.
**Action:** Edit `doc/source/conf.py` and ensure the following extensions are added to the `extensions` list.
```python
# doc/source/conf.py
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx', # For linking to external docs
'sphinx.ext.githubpages',
'sphinx_mermaid', # For Mermaid diagrams
'sphinx_sitemap', # For generating sitemap.xml
# 'breathe', # Add this once Doxygen is set up
]
# Add this configuration for intersphinx
intersphinx_mapping = {
'agent': ('URL_TO_WINDOWS_AGENT_DOCS', None),
}
# Add this for sitemap generation
html_baseurl = '[https://www.rsyslog.com/doc/](https://www.rsyslog.com/doc/)'
# Add this for the "Edit on GitHub" link
html_context = {
"display_github": True,
"github_user": "rsyslog",
"github_repo": "rsyslog",
"github_version": "master",
"conf_py_path": "/doc/source/",
}
````
-----
## 3\. Phase 2: Content Migration and Enhancement
This phase involves moving existing content into the new system and refactoring it to meet our quality standards.
### Task 3.1: Migrating Man Pages to RST
**Goal:** Convert a legacy man page into a Sphinx-manageable RST file.
**Steps:**
1. **Create RST File:** For `rsyslogd.8`, create a new file at `doc/source/man/rsyslogd.rst`.
2. **Copy Content:** Copy the raw text content from the old man page into the new RST file.
3. **Add Metadata:** Add the required metadata fields at the top of the RST file. This is what the man page builder uses to create the file correctly.
```rst
..
:manpage_url: [https://www.rsyslog.com/doc/man/rsyslogd.8.html](https://www.rsyslog.com/doc/man/rsyslogd.8.html)
rsyslogd
########
:program:`rsyslogd` - reliable and extended syslogd
..
This is the rst source for the rsyslogd man page.
It is generated as part of the rsyslog build process.
.. program:: rsyslogd
.. sectionauthor:: Rainer Gerhards <rgerhards@adiscon.com>
..
(rest of man page content follows, formatted in RST)
```
4. **Update Makefile:** Ensure the `Makefile` contains a target to build the man pages.
```makefile
# In your main Makefile
man:
@$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
```
5. **Build and Validate:** Run `make man` and check the output in the `build/man` directory.
### Task 3.2: Building the Validated Example Library
**Goal:** Create a repository of tested, reliable configuration examples.
**Steps:**
1. **Identify Candidates:** Review the existing rsyslog testbench to identify self-contained, high-value configuration examples.
2. **Extract and Sanitize:** Copy these configurations into new, clean `.conf` files within the `doc/examples/` directory (e.g., `doc/examples/basic_forwarding.conf`). Remove any testbench-specific scaffolding.
3. **Create Validation Script:** Create a CI script that iterates through all files in `doc/examples/` and runs `rsyslogd -N1` against them to check for syntax validity.
4. **Refactor Existing Docs:** Search the documentation for inline `.. code-block::` sections that contain full configurations. Replace them with the `.. literalinclude::` directive pointing to the corresponding validated file.
```rst
.. literalinclude:: ../examples/basic_forwarding.conf
:language: rsyslog-conf
```
-----
## 4\. Phase 3: Automation and CI Integration
This phase involves the core engineering work to automate our quality gates.
### Task 4.1: Implement the "Drift Detector" Linter
**Goal:** Create a CI script that automatically detects technical inaccuracies in the documentation.
**Specification:**
* **Language:** Python (recommended, for ease of integration with Sphinx).
* **Input:**
1. An "authoritative spec" file (`spec.json`) generated by the code analysis agent, containing a list of all valid parameters and modules.
2. The set of all `.rst` files in the `doc/source/` directory.
* **Process:**
1. The script reads `spec.json` into memory.
2. It iterates through each `.rst` file.
3. It uses regular expressions to find all potential rsyslog parameter names (e.g., `\$[a-zA-Z0-9_]+`, `[a-zA-Z]+\s*\(.*?\)`).
4. For each term found, it checks if the term exists in the `spec.json`.
5. If an unknown term is found, the script prints an error message indicating the term and the file/line number where it was found, then exits with a non-zero status code, failing the CI build.
### Task 4.2: Implement the Code Analysis Agent
**Goal:** Use Doxygen and Breathe to generate reference documentation directly from C code comments.
**Steps:**
1. **Define Comment Standard:** Establish a mandatory, standardized Doxygen comment block format for all developers to use. (See **Appendix B**).
2. **Configure Doxygen:** Create a `Doxyfile` configuration. Key settings:
* `GENERATE_XML = YES`
* `GENERATE_HTML = NO`
* `GENERATE_LATEX = NO`
* Set `INPUT` to the relevant source code directories.
3. **Integrate with Breathe:**
* Add `breathe` to the `extensions` list in `conf.py`.
* Configure the path to the Doxygen XML output: `breathe_projects = { "rsyslog": "path/to/doxygen/xml/" }`
4. **Create RST Stubs:** In the `06_reference/` directory, create stub `.rst` files that use Breathe directives to pull in the documentation.
```rst
// In doc/source/06_reference/imfile.rst
Module: imfile
==============
.. doxygenmodule:: imfile
:members:
:undoc-members:
```
5. **Generate `spec.json`:** The Doxygen XML output is machine-readable. Create a simple script that parses this XML and generates the `spec.json` file needed by the Drift Detector linter.
### Task 4.3: Create the Full Documentation CI Pipeline
**Goal:** A complete GitHub Actions workflow that automates all checks.
**Action:** Create a file at `.github/workflows/docs.yml`.
```yaml
name: 'Documentation CI'
on:
pull_request:
paths:
- 'doc/**'
- '.github/workflows/docs.yml'
jobs:
build_and_test_docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install -r doc/requirements.txt
# Add other dependencies like doxygen, etc.
- name: Generate Authoritative Spec
run: |
# Command to run doxygen and the spec generator script
make doc-spec
- name: Validate Examples
run: |
# Command to run the example validation script
python doc/ci/validate_examples.py
- name: Run Drift Detector Linter
run: |
# Command to run the drift detector
python doc/ci/drift_detector.py
- name: Build Sphinx Docs
run: |
make -C doc html
```
-----
## 5\. Phase 4: Iterative Quality Improvement
This phase is a continuous, long-term process.
### Task 5.1: The RST Quality Rubric
Use this checklist when refactoring an old documentation file to meet the new standard.
* [ ] **Headings:** Are the heading levels consistent with the style guide (See **Appendix A**)?
* [ ] **Semantic Markup:** Have plain backticks (`` ` ``) been replaced with specific semantic roles where appropriate (e.g., `:program:`, `:file:`, `:option:`)?
* [ ] **Cross-References:** Are all mentions of other modules, concepts, or sections linked using the `:ref:` or `:doc:` role?
* [ ] **Examples:** Are all configuration examples either short, illustrative snippets or replaced with a `.. literalinclude::` directive pointing to a validated example?
* [ ] **Clarity:** Is the language clear, concise, and unambiguous?
### Task 5.2: Prioritization Strategy
It's impossible to refactor everything at once. Prioritize the work in this order:
1. **High-Traffic Pages:** Start with the Getting Started guide, core configuration files, and the most popular modules.
2. **Community Pain Points:** Address pages that are frequently mentioned in GitHub issues or community forums as being confusing or incorrect.
3. **Core Concepts:** Ensure the files in `02_concepts/` are crystal clear, as they form the foundation of user understanding.
-----
## 6\. Appendices
### Appendix A: Recommended RST Style Guide
* **Page Title (H1):** `################` (over and under)
* **Chapter (H2):** `****************` (over and under)
* **Section (H3):** `================` (underline only)
* **Subsection (H4):** `----------------` (underline only)
* **Line Length:** Wrap all prose at 88 characters to improve readability in code editors.
### Appendix B: Sample Standardized Code Comment (Doxygen)
All new functions and configuration parameters in the C code should use this format.
```c
/**
* @brief A brief, one-sentence description of the parameter.
*
* A more detailed, multi-sentence description can follow here.
* It should explain what the parameter does, its purpose, and any
* important side effects or interactions.
*
* @param[in] name The name of the parameter. (Example for a function)
*
* @default "default_value"
* @module-type output
* @supported-since 8.2404.0
* @see another_function()
*
* @return A description of the return value.
*/
```

10
doc/README.md
View File

@ -31,17 +31,17 @@ provide dev and official release builds of the documentation.
1. Log in with a GitHub account.
2. Fork the official https://github.com/rsyslog/rsyslog repo.
3. Create a new branch off of the latest `master` branch.
3. Create a new branch off of the latest `main` branch.
4. Make your changes in the **`doc/`** subfolder.
5. Commit to the new branch in your fork.
6. Submit a Pull Request (PR) for review
(https://github.com/rsyslog/rsyslog/pulls). **Note that PRs will be automatically AI reviewed.**
7. Stop making any changes to your new branch now that you've submitted a
Pull Request for review. Instead, create a new branch from your `master`
Pull Request for review. Instead, create a new branch from your `main`
branch while you wait for feedback from the doc team.
8. A team member will review and offer feedback on your work. After
feedback has been given and you've made all necessary changes, your
PR will be accepted and merged into the official `master` branch.
PR will be accepted and merged into the official `main` branch.
9. At this point, delete the branch you submitted the PR from and start
a new one for your next round of work.
@ -148,12 +148,12 @@ later steps are identical, so we've covered those steps in one place.
1. `python -m pip install -r requirements.txt`
2. Clone the official Git repo:
1. `git clone https://github.com/rsyslog/rsyslog.git`
3. Check out either the current stable or development (aka, "master") branch:
3. Check out either the current stable or development (aka, "main") branch:
1. `cd rsyslog/doc`
1. `git checkout BRANCH_NAME_HERE`
- Choose the `v8-stable` branch for coverage of features currently
available in the latest stable release.
- Choose the `master` branch for coverage of upcoming features and fixes.
- Choose the `main` branch for coverage of upcoming features and fixes.
4. **Optional:** If you've previously cloned the repo, run `git pull` to update it
with new changes before continuing.

111
doc/STRATEGY.md Normal file
View File

@ -0,0 +1,111 @@
# The Rsyslog Documentation Strategy
**Version: 1.0**
**Related Document:** `IMPLEMENTATION_PLAN.md`
**Status: Active**
This document outlines the official strategy for the rsyslog project's documentation. It is a living document, subject to revision as our tools and processes evolve. Its purpose is to ensure our documentation is **authoritative**, **sustainable**, and **intelligent**, serving both human system administrators and AI agents as a primary source of truth.
## Core Principles
1. **Docs-as-Code**: Documentation is treated with the same rigor as application code. It lives in the same repository, is version-controlled, and is integrated into our CI/CD pipelines.
2. **Single Source of Truth**: We will relentlessly eliminate information duplication. A piece of information should exist in one place and be referenced everywhere else.
3. **Automation First**: We will automate every process possible, from generation and validation to drift detection and contribution checks.
4. **Hybrid Model**: We recognize that great documentation requires both machine-driven accuracy for reference material and human-authored clarity for narrative content.
---
## Pillar 1: The Automated Foundation (Code-Driven Reference)
This pillar ensures that our reference documentation is always an accurate reflection of the source code, even if manual updates are missed during development.
* **Authoritative Source**: The C source code is the ultimate source of truth for all modules, configuration parameters, and directives.
* **Generation Toolchain**:
* **Doxygen**: Parses standardized comment blocks and code structure from `.c` files into an intermediate XML format.
* **Sphinx**: The core documentation build system.
* **Breathe**: A Sphinx extension that bridges Doxygen and Sphinx, rendering the XML as clean HTML and man pages.
* **Handling Undocumented Changes**:
1. A CI job runs nightly (or on commit) to analyze the source code and compare it to the current documentation.
2. If this "code analysis agent" detects a new or changed parameter that is not documented, it will **not** fail the build.
3. Instead, it will automatically create a **new issue** in the project's GitHub issue tracker, tagged with `documentation` and `autogenerated`. This creates a trackable "documentation debt" item.
---
## Pillar 2: The Narrative Layer (Human-Authored Guides)
This layer provides context, learning paths, and practical solutions. This content is manually authored but automatically validated.
* **Validated Example Library**:
* All complete configuration examples will be stored in a dedicated `doc/examples/` directory.
* A CI job will test these examples for syntactic validity to ensure they are guaranteed to work.
* Tutorials and guides will **not** contain inline, hardcoded configurations. They will use Sphinx's `.. literalinclude:: ../examples/my_validated_example.conf` directive to pull in these tested examples.
* **Diagrams as Code**:
* All diagrams and schematics will be created using **Mermaid.js**.
* The `sphinx-mermaid` extension will be used to render these text-based diagrams directly within the documentation, making them version-controllable.
* **Content Migration**: High-value, existing content (such as from `rainer.gerhards.net`) will be migrated, converted to RST, and integrated directly into the `doc/source/tutorials/` section, enriching it with cross-references and standard formatting.
---
## Pillar 3: Quality & Contribution Framework
This pillar defines the processes that ensure our documentation is high-quality, trustworthy, and easy to contribute to.
* **Documentation CI Pipeline**: Any pull request that modifies documentation will trigger a CI pipeline that:
1. **Builds the Docs**: Runs `make html` to ensure no Sphinx errors.
2. **Runs the "Drift Detector"**: A custom linter script scans all `.rst` files for rsyslog-specific technical terms (e.g., `$WorkDirectory`) and validates them against the authoritative list generated from the source code. An unknown term will fail the build, preventing typos and outdated information from being merged.
* **AI-Powered Review**: As a supplementary check, a GitHub Action can be configured to use an LLM to review the clarity, grammar, and style of documentation changes in a pull request, posting feedback as a comment.
* **User Feedback**: Every page will feature an "Edit on GitHub" link to provide a low-friction way for users to suggest improvements. This is configured in `conf.py` via the `html_context`.
---
## Pillar 4: Distribution & Format Integration
This pillar defines how documentation is delivered to end-users and how legacy formats are handled.
* **Man Page Generation**:
* Man pages are not written manually. They are **generated** from the same RST source files as the HTML documentation.
* Content from existing man pages will be migrated to RST files under `doc/source/man/`.
* The `Makefile` will include the `make man` target, which uses Sphinx's man-page builder. The generated pages will be included in the release tarball.
* **Official Versioning Policy**:
> This online documentation reflects the latest development version (`main` branch) of rsyslog. It contains the most current information but may include features not yet available in a stable release.
>
> For stable versions, the authoritative documentation is bundled within the official release tarballs. Linux distributions like Red Hat, Ubuntu, and others build their documentation packages directly from these tarballs. **If you have installed rsyslog via a distribution's package manager, please refer to the documentation provided by that package as your primary source of truth.**
---
## Pillar 5: Ecosystem Integration (External & Platform Docs)
This pillar defines how we integrate with related external projects and platform-specific documentation.
* **External Projects (e.g., Rsyslog Windows Agent)**:
* To link to related but separate documentation sets, we will use a combination of `git submodule` and the Sphinx `intersphinx` extension.
* This allows us to cross-reference content between projects seamlessly, making it feel like a single, unified documentation set to the user.
* **Cloud & Container Deployments**:
* A dedicated top-level section, `08_deployment_guides/`, will be created in the documentation.
* This section will contain sub-folders for major platforms (e.g., `container_deployment/`, `aws_deployment/`), providing comprehensive, validated guides for modern deployment scenarios.
---
## Pillar 6: Content Amplification & Authority Building
This pillar outlines our strategy to combat misinformation and establish the official documentation as the canonical source of truth.
* **Technical SEO**:
* We will use the `sphinx-sitemap` extension to generate a `sitemap.xml` to improve indexing.
* We will embed `Schema.org` structured data (e.g., `TechArticle`, `HowTo`) into our HTML templates to provide explicit context to search engines and AI crawlers.
* **Strategic Syndication**:
* Key tutorials will be periodically published on high-authority external sites (e.g., dev.to).
* **Crucially**, these syndicated articles **must** use a `rel="canonical"` link pointing back to the original article in the official rsyslog documentation to centralize SEO authority.
* **Answer Hubs**: When answering questions on sites like Stack Overflow, community members are encouraged to provide a brief summary and then link directly to the relevant page in the official documentation.
---
## Maintaining This Strategy
This document is the "single source of truth" for our documentation process.
* **Location**: It is stored at `doc/STRATEGY.md` within the main rsyslog repository.
* **Discoverability**: A `CONTRIBUTING.md` file at the repository root links directly to this strategy, and a pull request template (`.github/pull_request_template.md`) includes a checklist item requiring contributors to have read it.
* **Enforcement**: The strategy is primarily enforced through the automated CI checks. If the CI pipeline passes, the contribution is considered compliant.