testbench: add root-level diag.sh stub and simplify test invocation

- Introduce root-level diag.sh stub to auto-forward to tests/diag.sh, preventing
  human and AI errors when sourcing the harness and removing the need to `cd`
  into the tests directory.
- Update AGENTS.md to configure once and invoke tests via `./tests/<script>.sh`,
  providing unfiltered stdout/stderr without a CI wrapper and improving
  convenience for human developers.
This commit is contained in:
Rainer Gerhards 2025-06-21 14:39:34 +02:00
parent cb4aef11b0
commit 15796f1047
No known key found for this signature in database
GPG Key ID: 0CB6B2A8BE80B499
2 changed files with 59 additions and 14 deletions

View File

@ -99,30 +99,26 @@ Instead, AI agents should invoke individual test scripts directly. This yields u
---
### Running Individual Tests (AIAgent Best Practice)
### Running Individual Tests (AI-Agent Best Practice)
1. **Configure the project** (once per session):
```bash
./configure --enable-imdiag --enable-testbench
```
2. **Enter the tests directory**:
2. **Invoke your test**:
```bash
cd tests/
./tests/<test-script>.sh
```
3. **Run a specific test script**:
For example:
```bash
./rscript_re_extract.sh
```
_Include the `.sh` suffix for direct execution; omit it when using `make check TESTS=`._
4. **Return to the project root**:
```bash
cd ..
./tests/manytcp-too-few-tls-vg.sh > /tmp/test.log && tail -n20 /tmp/test.log
```
> **Why direct invocation?**
> - Prints failures and debug messages straight to stdout
> - No need to open or grep through `tests/test-suite.log`
> - Faster turnaround for focused test work
3. **Why this works**
- Each test script transparently finds and loads the test harness
- You get unfiltered stdout/stderr without any CI wrapper
- No manual `cd` or log-file parsing required
> **Note for Docker-based agents**
> Agents such as Codex run inside their own Docker containers and cannot invoke the official CI Docker image. They should rely on the local `./configure` + direct script execution workflow within their container, rather than trying to spin up `rsyslog/rsyslog_dev_base_*` images.

49
diag.sh Executable file
View File

@ -0,0 +1,49 @@
#!/usr/bin/env bash
## diag.sh stub for rsyslog test harness forwarding
##
## This stub lives at the repo root so that every tests
## “. ${srcdir:=.}/diag.sh” line loads the real harness in tests/.
##
## It performs:
## 1) Locate project root (where this stub lives)
## 2) Ensure srcdir points to tests/ under project root
## 3) Identify which test script sourced this stub
## 4) Change into that tests directory so relative paths work
## If `cd` fails (e.g. malformed caller path or missing dir),
## the stub aborts immediately with a clear error. This prevents
## sourcing the wrong harness or causing misleading test failures.
## 5) Print a one-line notice about forwarding
## 6) Source the real tests/diag.sh, passing any args (init, kill, etc.)
##
## Note on directory management:
## We could use `pushd`/`popd` to manage a directory stack, but since
## each test script runs in its own shell process, a simple `cd` is
## sufficient. No `popd` is needed—when the test process exits, its
## working directory is discarded.
##
## Usage in test scripts:
## . ${srcdir:=.}/diag.sh init
## → this stub handles locating and sourcing tests/diag.sh automatically.
# 1) Determine project root (where this stub lives)
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# 2) Point srcdir at the real harness directory
: "${srcdir:="${PROJECT_ROOT}/tests"}"
# 3) Find which test script called us (strip leading "./")
CALLER="${BASH_SOURCE[1]#./}" # e.g. "tests/foo.sh"
TEST_DIR="${PROJECT_ROOT}/$(dirname "${CALLER}")"
# 4) Notify user/agent of forwarding action
echo "diag.sh stub: forwarding to ${srcdir}/diag.sh in ${TEST_DIR}"
# 5) Change into the tests directory (abort on failure)
cd "${TEST_DIR}" || {
echo "FATAL: cannot cd to ${TEST_DIR}" >&2
exit 1
}
# 6) Source the real harness, passing along all arguments
. "${srcdir}/diag.sh" "$@"