rsyslog/rsyslog
1
0
Fork 0
mirror of https://github.com/rsyslog/rsyslog.git synced 2026-08-24 19:24:14 +02:00
Code Issues Packages Projects Releases Wiki Activity

ci: publish Debian daily builds to package archive

Browse Source 
Why:
Daily stable packages need durable multi-version retention without using
GitHub Pages as an ever-growing package store.

Impact:
Debian 13 amd64 package builds can publish to a future Spaces archive.

Before/After:
Before, staging replaced a Pages repository. After, Actions maintains an
append-only signed APT archive with at least five years of retained builds.

Technical Overview:
Build and verify Debian 13 packages in Actions, incrementally merge signed
APT indexes, and publish immutable packages and snapshots to an S3-compatible
archive. Scheduled publishing stays disabled until DigitalOcean is ready.

With the help of AI-Agents: Codex
This commit is contained in:
Rainer Gerhards 2026-07-24 10:13:23 +02:00
parent 7d8aee240f
commit a49973fcba
8 changed files with 1315 additions and 752 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

85
.github/DEBIAN_DAILY_STABLE.md vendored Normal file
View File

@ -0,0 +1,85 @@
# Debian daily stable package archive
The `debian daily stable` workflow builds rsyslog from `v8-stable` for Debian
13 (`trixie`) on `amd64`. Pull requests that change the archive automation run
the same build without publishing. Scheduled publishing remains disabled until
the DigitalOcean archive is provisioned.
The workflow owns package construction, APT metadata generation, signing,
upload ordering, and post-publication installation verification. DigitalOcean
Spaces is passive S3-compatible storage behind its CDN; no package-building
service runs there.
## Archive layout and retention
The initial repository URL ends in:
```text
/apt/daily-stable/debian/13
```
This leaves room for later channels, Debian and Ubuntu versions, RPM-based
distributions, and additional architectures. Debian packages and source
artifacts use immutable paths below `pool/`. Every run also records its
manifest, checksums, build information, and build log below:
```text
snapshots/YYYY-MM-DD/PACKAGE_VERSION/
```
The APT indexes retain every published package version. The Space must not have
a lifecycle rule that removes package-pool, by-hash, or snapshot objects before
five years. Retention starts with the first successful publication; the
workflow does not synthesize historical builds.
Only the small current `Packages.xz` and `Sources.xz` indexes are downloaded
before a publication. They are merged with the new build, so daily Actions
traffic does not grow with the full archive size. Immutable objects are
uploaded first, signed mutable metadata last, and `InRelease` last of all.
## DigitalOcean setup contract
Create one public-read Standard Spaces bucket with object versioning enabled.
Give the workflow a bucket-scoped read/write Spaces key; do not give it a
DigitalOcean account API token. Enable the Spaces CDN and route the final
package hostname to it.
Configure these GitHub repository variables:
- `DEBIAN_DAILY_STABLE_ENABLED`: set to `true` only after a manual publication
and installation test succeeds.
- `DEBIAN_DAILY_STABLE_SPACE_BUCKET`: Space name.
- `DEBIAN_DAILY_STABLE_SPACE_ENDPOINT`: regional S3 endpoint, for example
`https://fra1.digitaloceanspaces.com`.
- `DEBIAN_DAILY_STABLE_SPACE_REGION`: matching region, for example `fra1`.
- `DEBIAN_DAILY_STABLE_REPO_URL`: public URL including
`/apt/daily-stable/debian/13`.
- `DEBIAN_DAILY_STABLE_GPG_FINGERPRINT`: full fingerprint of the archive
signing key.
Create a protected `debian-daily-stable` GitHub Environment and add:
- `DEBIAN_DAILY_STABLE_SPACE_ACCESS_KEY`
- `DEBIAN_DAILY_STABLE_SPACE_SECRET_KEY`
- `DEBIAN_DAILY_STABLE_GPG_PRIVATE_KEY`
- `DEBIAN_DAILY_STABLE_GPG_PASSPHRASE`
The passphrase may be empty only if the archive key was intentionally created
without one. Keep stable-release publishing in a separate environment when it
is added later.
## Activation
Before enabling the schedule:
1. Run the workflow manually with publication disabled and review its package
artifact.
2. Provision the Space, CDN, DNS, signing key, variables, and secrets.
3. Run it manually with publication enabled.
4. Confirm that signed metadata verification and the clean Debian 13 package
installation both pass.
5. Set `DEBIAN_DAILY_STABLE_ENABLED` to `true`.
There is deliberately no five-year GitHub Actions artifact retention. The
short-lived Actions artifact transfers one build between jobs; DigitalOcean
Spaces is the durable archive.

16
.github/debian-ci-policy.yml vendored
View File

@ -15,6 +15,22 @@
{
"patch": "queue-quarantine-runtime-disk-corruption-safely.patch",
"reason": "Debian latest carries a queue disk-corruption quarantine patch that is already effectively upstream and now applies in reverse."
},
{
"patch": "config-keep-empty-include-globs-silent.patch",
"reason": "Debian latest carries the upstream empty IncludeConfig glob behavior from commit e0651c019 and the cherry-picked patch no longer applies."
},
{
"patch": "testbench-add-daemon-termination-oracle.patch",
"reason": "Debian latest carries the upstream daemon termination oracle from commit 37a624442 and the cherry-picked patch no longer applies."
},
{
"patch": "Revert-tests-keep-epoll-preload-helper-test-only.patch",
"reason": "The Debian revert conflicts with the current upstream test-only preload-helper layout; daily packages must not install the fault-injection helper."
},
{
"patch": "Use-check_LTLIBRARIES-to-generate-liboverride_-test-libra.patch",
"reason": "Current upstream already uses check_LTLIBRARIES and TEST_LIBOVERRIDE_LDFLAGS for the preload helpers, so the Debian patch applies in reverse."
}
],
"supplemental_build_deps": [],

1
.github/workflows/collect_flake_evidence.yml vendored
View File

@ -13,6 +13,7 @@ name: collect flake evidence
- imbeats tests
- imhttp Prometheus scrape
- impstats push to VictoriaMetrics
- debian daily stable
types: [completed]
workflow_dispatch:
inputs:

689
.github/workflows/debian_daily_stable.yml vendored Normal file
View File

@ -0,0 +1,689 @@
# Copyright 2026 Rainer Gerhards and Others
#
# https://github.com/rsyslog/rsyslog
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
name: debian daily stable
'on':
pull_request:
paths:
- '.github/debian-ci-policy.yml'
- '.github/scripts/debian_package_build.sh'
- '.github/workflows/debian_daily_stable.yml'
- 'devtools/release/debian-daily-stable.sh'
workflow_dispatch:
inputs:
source_ref:
description: rsyslog source branch, tag, or commit to package
required: true
default: v8-stable
type: string
publish_to_archive:
description: Publish to the configured DigitalOcean Spaces archive
required: false
default: false
type: boolean
schedule:
- cron: '23 3 * * *'
concurrency:
group: debian-daily-stable
cancel-in-progress: false
env:
DEBIAN_CI_POLICY_FILE: .github/debian-ci-policy.yml
DEBIAN_CI_HELPER: .github/scripts/debian_package_build.sh
DEBIAN_DAILY_STABLE_HELPER: devtools/release/debian-daily-stable.sh
DEBIAN_PACKAGING_REPO: https://salsa.debian.org/debian/rsyslog.git
DEBIAN_PACKAGING_BRANCH: debian/latest
DEBIAN_BUILD_ROOT: /tmp/rsyslog-debian-daily-stable-build
DEBIAN_SUITE: trixie
DEBIAN_COMPONENT: main
DEBIAN_ARCH: amd64
PACKAGE_CHANNEL: daily-stable
PACKAGE_DISTRO: debian
PACKAGE_DISTRO_VERSION: '13'
SPACE_PREFIX: apt/daily-stable/debian/13
jobs:
preflight:
name: preflight
runs-on: ubuntu-24.04
if: github.repository == 'rsyslog/rsyslog'
permissions:
contents: read
outputs:
should_run: ${{ steps.decision.outputs.should_run }}
should_publish: ${{ steps.decision.outputs.should_publish }}
source_ref: ${{ steps.decision.outputs.source_ref }}
steps:
- name: Decide whether this run is active
id: decision
env:
EVENT_NAME: ${{ github.event_name }}
MANUAL_PUBLISH: ${{ inputs.publish_to_archive }}
MANUAL_SOURCE_REF: ${{ inputs.source_ref }}
PR_SOURCE_SHA: ${{ github.event.pull_request.head.sha }}
SCHEDULE_ENABLED: ${{ vars.DEBIAN_DAILY_STABLE_ENABLED }}
run: |
set -euo pipefail
should_run=false
should_publish=false
source_ref=v8-stable
case "$EVENT_NAME" in
pull_request)
should_run=true
source_ref="$PR_SOURCE_SHA"
;;
workflow_dispatch)
should_run=true
source_ref="${MANUAL_SOURCE_REF:-v8-stable}"
if [ "${MANUAL_PUBLISH:-false}" = "true" ]; then
should_publish=true
fi
;;
schedule)
if [ "${SCHEDULE_ENABLED:-false}" = "true" ]; then
should_run=true
should_publish=true
fi
;;
esac
{
echo "should_run=$should_run"
echo "should_publish=$should_publish"
echo "source_ref=$source_ref"
} >> "$GITHUB_OUTPUT"
build:
name: build Debian 13 amd64 packages
needs: preflight
if: needs.preflight.outputs.should_run == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 120
container:
image: debian:trixie
options: --user root
permissions:
contents: read
outputs:
archive_date: ${{ steps.version.outputs.archive_date }}
version: ${{ steps.version.outputs.version }}
source_sha: ${{ steps.source.outputs.source_sha }}
steps:
- name: Checkout archive automation
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Checkout source to package
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
path: rsyslog-source
persist-credentials: false
ref: ${{ needs.preflight.outputs.source_ref }}
- name: Record packaged source revision
id: source
run: |
set -euo pipefail
source_sha="$(git -C rsyslog-source rev-parse HEAD)"
{
echo "source_sha=$source_sha"
echo "SOURCE_GIT_SHA=$source_sha"
echo "RSYSLOG_SOURCE_DIR=$GITHUB_WORKSPACE/rsyslog-source"
} | tee -a "$GITHUB_ENV" >> "$GITHUB_OUTPUT"
- name: Install prerequisites
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" install_prereqs
apt-get install -y --no-install-recommends \
apt-utils \
curl \
gnupg \
xz-utils
- name: Test incremental archive generation
run: |
devtools/ci-flake-phase.sh run debian-archive-self-test custom -- \
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" self-test
- name: Generate daily package version
id: version
run: |
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" version
- name: Load Debian CI policy
run: |
mkdir -p "$DEBIAN_BUILD_ROOT"
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" load_policy \
"$GITHUB_WORKSPACE/$DEBIAN_CI_POLICY_FILE" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy"
- name: Fetch Debian packaging baseline
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" fetch_debian_packaging \
"$DEBIAN_PACKAGING_REPO" \
"$DEBIAN_PACKAGING_BRANCH" \
"$DEBIAN_BUILD_ROOT/debian-packaging"
- name: Apply packaging baseline policy
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" apply_control_replacements \
"$DEBIAN_BUILD_ROOT/debian-packaging/control" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/control_replacements.tsv" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/control_replacements.reasons.txt"
- name: Install Debian build dependencies
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" install_build_deps \
"$DEBIAN_BUILD_ROOT/debian-packaging/control" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/supplemental_build_deps.txt"
- name: Generate rsyslog dist tarball
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" run_dist_build \
"$GITHUB_WORKSPACE/rsyslog-source"
- name: Locate rsyslog dist tarball
run: |
dist_tarball="$(
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" find_dist_tarball \
"$GITHUB_WORKSPACE/rsyslog-source"
)"
echo "DIST_TARBALL=$dist_tarball" >> "$GITHUB_ENV"
- name: Unpack tarball and inject Debian packaging
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" unpack_source_tree \
"$GITHUB_WORKSPACE/rsyslog-source" \
"$DIST_TARBALL" \
"$DEBIAN_BUILD_ROOT/debian-packaging" \
"$DEBIAN_BUILD_ROOT/debian-src"
- name: Stamp daily stable changelog
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" stamp-changelog \
"$DEBIAN_BUILD_ROOT/debian-src" \
"$VERSION"
- name: Apply Debian CI policy
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" apply_not_installed_policy \
"$DEBIAN_BUILD_ROOT/debian-src" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/not_installed_paths.txt" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/not_installed_paths.reasons.txt"
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" resolve_patch_policy \
"$DEBIAN_BUILD_ROOT/debian-src" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/allowed_patch_skips.txt" \
"$DEBIAN_BUILD_ROOT/debian-ci-policy/allowed_patch_skips.reasons.txt" \
strict
- name: Build source and binary packages
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
devtools/ci-flake-phase.sh run debian-package-build custom -- \
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" build-package \
"$GITHUB_WORKSPACE/rsyslog-source" \
"$DEBIAN_BUILD_ROOT/debian-src" \
"$DIST_TARBALL" \
"$GITHUB_WORKSPACE/debian-daily-stable-artifacts" \
"$VERSION" \
"$DEBIAN_BUILD_ROOT/debian-daily-stable-build.log"
- name: Upload failure evidence
if: failure()
uses: ./.github/actions/upload-flake-evidence
with:
job-name: Debian 13 daily stable package build
- name: Generate artifact manifest
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" manifest \
"$GITHUB_WORKSPACE/debian-daily-stable-artifacts" \
"$VERSION" \
"$DEBIAN_SUITE" \
"$DEBIAN_ARCH" \
"$PACKAGE_CHANNEL" \
"$PACKAGE_DISTRO" \
"$PACKAGE_DISTRO_VERSION"
- name: Upload Debian package artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: debian-daily-stable-${{ steps.version.outputs.version }}
path: debian-daily-stable-artifacts/
retention-days: 14
if-no-files-found: error
- name: Summarize build
env:
SOURCE_REF: ${{ needs.preflight.outputs.source_ref }}
VERSION: ${{ steps.version.outputs.version }}
run: |
{
echo "### Debian daily stable build"
echo
echo "- Version: \`$VERSION\`"
echo "- Source ref: \`$SOURCE_REF\`"
echo "- Source commit: \`$SOURCE_GIT_SHA\`"
echo "- Target: Debian 13 (\`$DEBIAN_SUITE\`), \`$DEBIAN_ARCH\`"
echo "- Archive prefix: \`$SPACE_PREFIX\`"
echo
echo "Artifacts:"
find debian-daily-stable-artifacts -maxdepth 1 -type f \
-printf '%f\n' | sort |
while IFS= read -r artifact; do
printf -- "- \`%s\`\n" "$artifact"
done
} >> "$GITHUB_STEP_SUMMARY"
publish:
name: publish DigitalOcean Spaces APT archive
needs:
- preflight
- build
if: needs.preflight.outputs.should_publish == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 30
environment: debian-daily-stable
permissions:
contents: read
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DEBIAN_DAILY_STABLE_SPACE_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEBIAN_DAILY_STABLE_SPACE_SECRET_KEY }}
AWS_DEFAULT_REGION: ${{ vars.DEBIAN_DAILY_STABLE_SPACE_REGION }}
SPACE_BUCKET: ${{ vars.DEBIAN_DAILY_STABLE_SPACE_BUCKET }}
SPACE_ENDPOINT: ${{ vars.DEBIAN_DAILY_STABLE_SPACE_ENDPOINT }}
DEBIAN_DAILY_STABLE_GPG_PRIVATE_KEY: ${{ secrets.DEBIAN_DAILY_STABLE_GPG_PRIVATE_KEY }}
DEBIAN_DAILY_STABLE_GPG_PASSPHRASE: ${{ secrets.DEBIAN_DAILY_STABLE_GPG_PASSPHRASE }}
steps:
- name: Checkout archive automation
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install repository and Spaces tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
apt-utils \
awscli \
gnupg \
xz-utils
- name: Validate archive configuration
run: |
set -euo pipefail
for variable in \
AWS_ACCESS_KEY_ID \
AWS_SECRET_ACCESS_KEY \
AWS_DEFAULT_REGION \
SPACE_BUCKET \
SPACE_ENDPOINT \
DEBIAN_DAILY_STABLE_GPG_PRIVATE_KEY; do
[ -n "${!variable:-}" ] || {
echo "$variable is empty" >&2
exit 1
}
done
- name: Download Debian package artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: debian-daily-stable-${{ needs.build.outputs.version }}
path: debian-daily-stable-artifacts
- name: Verify artifact checksums
run: |
cd debian-daily-stable-artifacts
sha256sum -c SHA256SUMS
- name: Import archive signing key
run: |
set -euo pipefail
install -m 700 -d "$HOME/.gnupg"
printf '%s\n' "$DEBIAN_DAILY_STABLE_GPG_PRIVATE_KEY" |
gpg --batch --import
gpg --batch --list-secret-keys
- name: Download current indexes
run: |
set -euo pipefail
mkdir -p "apt-repo/dists/$DEBIAN_SUITE/$DEBIAN_COMPONENT/binary-$DEBIAN_ARCH"
mkdir -p "apt-repo/dists/$DEBIAN_SUITE/$DEBIAN_COMPONENT/source"
download_if_present() {
local relative_path="$1"
local key="$SPACE_PREFIX/$relative_path"
local remote_key
remote_key="$(
aws s3api list-objects-v2 \
--endpoint-url "$SPACE_ENDPOINT" \
--bucket "$SPACE_BUCKET" \
--prefix "$key" \
--max-keys 1 \
--query 'Contents[0].Key' \
--output text
)"
if [ "$remote_key" = "$key" ]; then
aws s3 cp \
--endpoint-url "$SPACE_ENDPOINT" \
"s3://$SPACE_BUCKET/$key" \
"apt-repo/$relative_path"
fi
}
download_if_present \
"dists/$DEBIAN_SUITE/$DEBIAN_COMPONENT/binary-$DEBIAN_ARCH/Packages.xz"
download_if_present \
"dists/$DEBIAN_SUITE/$DEBIAN_COMPONENT/source/Sources.xz"
- name: Generate signed incremental APT repository
run: |
"$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" generate-repo \
"$GITHUB_WORKSPACE/debian-daily-stable-artifacts" \
"$GITHUB_WORKSPACE/apt-repo" \
"$DEBIAN_SUITE" \
"$DEBIAN_COMPONENT" \
"$DEBIAN_ARCH"
- name: Prepare immutable build snapshot
env:
ARCHIVE_DATE: ${{ needs.build.outputs.archive_date }}
VERSION: ${{ needs.build.outputs.version }}
run: |
set -euo pipefail
snapshot_dir="apt-repo/snapshots/$ARCHIVE_DATE/$VERSION"
mkdir -p "$snapshot_dir"
find debian-daily-stable-artifacts -maxdepth 1 -type f \
\( -name 'manifest.json' -o -name 'SHA256SUMS' \
-o -name '*.changes' -o -name '*.buildinfo' \
-o -name 'build.log' \) \
-exec cp -a {} "$snapshot_dir/" \;
- name: Publish immutable packages and snapshots
run: |
set -euo pipefail
upload_immutable() {
local path="$1"
local relative_path="${path#apt-repo/}"
local key="$SPACE_PREFIX/$relative_path"
local remote_key local_hash remote_hash
remote_key="$(
aws s3api list-objects-v2 \
--endpoint-url "$SPACE_ENDPOINT" \
--bucket "$SPACE_BUCKET" \
--prefix "$key" \
--max-keys 1 \
--query 'Contents[0].Key' \
--output text
)"
local_hash="$(sha256sum "$path" | awk '{print $1}')"
if [ "$remote_key" = "$key" ]; then
remote_hash="$(
aws s3 cp \
--quiet \
--endpoint-url "$SPACE_ENDPOINT" \
"s3://$SPACE_BUCKET/$key" - |
sha256sum | awk '{print $1}'
)"
[ "$remote_hash" = "$local_hash" ] || {
echo "immutable archive collision at $key" >&2
exit 1
}
return
fi
aws s3 cp \
--only-show-errors \
--endpoint-url "$SPACE_ENDPOINT" \
--cache-control 'public,max-age=31536000,immutable' \
--metadata "sha256=$local_hash" \
"$path" \
"s3://$SPACE_BUCKET/$key"
}
while IFS= read -r -d '' path; do
upload_immutable "$path"
done < <(
find apt-repo/pool apt-repo/snapshots \
"apt-repo/dists/$DEBIAN_SUITE" \
-type f \
\( -path '*/pool/*' -o -path '*/snapshots/*' \
-o -path '*/by-hash/*' \) \
-print0
)
- name: Publish archive key and mutable metadata
run: |
set -euo pipefail
upload_metadata() {
local path="$1"
local relative_path="${path#apt-repo/}"
aws s3 cp \
--only-show-errors \
--endpoint-url "$SPACE_ENDPOINT" \
--cache-control 'public,max-age=60,must-revalidate' \
"$path" \
"s3://$SPACE_BUCKET/$SPACE_PREFIX/$relative_path"
}
upload_metadata apt-repo/rsyslog-archive-keyring.asc
while IFS= read -r -d '' path; do
upload_metadata "$path"
done < <(
find "apt-repo/dists/$DEBIAN_SUITE" -type f \
! -path '*/by-hash/*' \
! -name Release \
! -name Release.gpg \
! -name InRelease \
-print0
)
upload_metadata "apt-repo/dists/$DEBIAN_SUITE/Release"
upload_metadata "apt-repo/dists/$DEBIAN_SUITE/Release.gpg"
upload_metadata "apt-repo/dists/$DEBIAN_SUITE/InRelease"
verify:
name: verify published Debian 13 repository
needs:
- preflight
- build
- publish
if: needs.preflight.outputs.should_publish == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 20
container:
image: debian:trixie
options: --user root
environment: debian-daily-stable
permissions:
contents: read
env:
REPO_URL: ${{ vars.DEBIAN_DAILY_STABLE_REPO_URL }}
EXPECTED_GPG_FINGERPRINT: ${{ vars.DEBIAN_DAILY_STABLE_GPG_FINGERPRINT }}
steps:
- name: Checkout archive automation
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install verification tools
run: |
apt-get update
apt-get install -y --no-install-recommends \
ca-certificates \
curl \
gnupg \
xz-utils
- name: Wait for CDN and verify signed metadata
env:
VERSION: ${{ needs.build.outputs.version }}
run: |
set -euo pipefail
[ -n "$REPO_URL" ] || {
echo "DEBIAN_DAILY_STABLE_REPO_URL is empty" >&2
exit 1
}
[ -n "$EXPECTED_GPG_FINGERPRINT" ] || {
echo "DEBIAN_DAILY_STABLE_GPG_FINGERPRINT is empty" >&2
exit 1
}
devtools/ci-flake-phase.sh begin debian-published-metadata custom
set +e
verification_rc=1
for attempt in $(seq 1 20); do
if "$GITHUB_WORKSPACE/$DEBIAN_DAILY_STABLE_HELPER" verify-repo \
"$REPO_URL" \
"$DEBIAN_SUITE" \
"$DEBIAN_COMPONENT" \
"$DEBIAN_ARCH" \
"$VERSION" \
"$EXPECTED_GPG_FINGERPRINT"; then
verification_rc=0
break
fi
echo "Repository not ready yet, retrying ($attempt/20)..."
if [ "$attempt" -lt 20 ]; then
sleep 30
fi
done
set -e
devtools/ci-flake-phase.sh end \
debian-published-metadata custom "$verification_rc"
if [ "$verification_rc" -ne 0 ]; then
echo "Repository verification did not succeed before timeout." >&2
exit "$verification_rc"
fi
- name: Install the published package
env:
VERSION: ${{ needs.build.outputs.version }}
run: |
set -euo pipefail
devtools/ci-flake-phase.sh begin debian-published-install custom
set +e
(
set -euo pipefail
curl -fsSL "$REPO_URL/rsyslog-archive-keyring.asc" |
gpg --dearmor -o /usr/share/keyrings/rsyslog-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/rsyslog-archive-keyring.gpg] $REPO_URL $DEBIAN_SUITE $DEBIAN_COMPONENT" \
> /etc/apt/sources.list.d/rsyslog-daily-stable.list
apt-get update
apt-get install -y "rsyslog=$VERSION"
rsyslogd -v
) 2>&1 | tee .ci/flake-evidence/logs/debian-published-install.log
install_status=("${PIPESTATUS[@]}")
set -e
devtools/ci-flake-phase.sh end \
debian-published-install custom "${install_status[0]}"
exit "${install_status[0]}"
- name: Upload publication failure evidence
if: failure()
uses: ./.github/actions/upload-flake-evidence
with:
job-name: Debian 13 daily stable publication verification
report_failure:
name: report failure
needs:
- preflight
- build
- publish
- verify
if: >-
always() &&
github.event_name != 'pull_request' &&
needs.preflight.outputs.should_run == 'true' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-24.04
permissions:
issues: write
contents: read
steps:
- name: Create or update failure issue
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
VERSION: ${{ needs.build.outputs.version }}
BUILD_RESULT: ${{ needs.build.result }}
PUBLISH_RESULT: ${{ needs.publish.result }}
VERIFY_RESULT: ${{ needs.verify.result }}
with:
script: |
const version = process.env.VERSION || `run-${context.runId}`;
const title = '[debian-daily-stable] package archive failure';
const body = [
`Automated Debian daily stable failed for \`${version}\`.`,
'',
`Workflow run: ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`,
`Workflow commit: ${context.sha}`,
'',
'Job results:',
`- build: ${process.env.BUILD_RESULT}`,
`- publish: ${process.env.PUBLISH_RESULT}`,
`- verify: ${process.env.VERIFY_RESULT}`,
'',
'Inspect the workflow artifacts and logs, then rerun the workflow manually after fixing the cause.'
].join('\n');
const {owner, repo} = context.repo;
const existing = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: 'open',
per_page: 100
});
const issue = existing.find(item => item.title === title && !item.pull_request);
if (issue) {
await github.rest.issues.createComment({
owner,
repo,
issue_number: issue.number,
body
});
return;
}
const created = await github.rest.issues.create({
owner,
repo,
title,
body
});
try {
await github.rest.issues.addLabels({
owner,
repo,
issue_number: created.data.number,
labels: ['release', 'packaging', 'debian', 'daily-stable', 'automated']
});
} catch (error) {
core.warning(`Could not add labels: ${error.message}`);
}

444
.github/workflows/debian_daily_staging.yml vendored
View File

@ -1,444 +0,0 @@
# Copyright 2026 Rainer Gerhards and Others
#
# https://github.com/rsyslog/rsyslog
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
name: debian daily staging
on:
workflow_dispatch:
inputs:
publish_to_pages:
description: Publish the generated APT repository to the staging Pages repository
required: false
default: false
type: boolean
schedule:
- cron: '23 3 * * *'
concurrency:
group: debian-daily-staging
cancel-in-progress: false
env:
DEBIAN_CI_POLICY_FILE: .github/debian-ci-policy.yml
DEBIAN_CI_HELPER: .github/scripts/debian_package_build.sh
DEBIAN_STAGING_HELPER: devtools/release/debian-daily-staging.sh
DEBIAN_PACKAGING_REPO: https://salsa.debian.org/debian/rsyslog.git
DEBIAN_PACKAGING_BRANCH: debian/latest
DEBIAN_STAGING_BUILD_ROOT: /tmp/rsyslog-debian-daily-staging-build
DEBIAN_STAGING_SUITE: trixie
DEBIAN_STAGING_COMPONENT: main
DEBIAN_STAGING_ARCH: amd64
jobs:
preflight:
name: preflight
runs-on: ubuntu-latest
if: github.repository == 'rsyslog/rsyslog'
permissions:
contents: read
outputs:
should_run: ${{ steps.decision.outputs.should_run }}
should_publish: ${{ steps.decision.outputs.should_publish }}
steps:
- name: Decide whether this run is active
id: decision
env:
EVENT_NAME: ${{ github.event_name }}
SCHEDULE_ENABLED: ${{ vars.DEBIAN_DAILY_STAGING_ENABLED }}
MANUAL_PUBLISH: ${{ github.event.inputs.publish_to_pages }}
run: |
set -euo pipefail
should_run=false
should_publish=false
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
should_run=true
if [ "${MANUAL_PUBLISH:-false}" = "true" ]; then
should_publish=true
fi
elif [ "${SCHEDULE_ENABLED:-false}" = "true" ]; then
should_run=true
should_publish=true
fi
{
echo "should_run=$should_run"
echo "should_publish=$should_publish"
} >> "$GITHUB_OUTPUT"
build:
name: build Debian staging packages
needs: preflight
if: needs.preflight.outputs.should_run == 'true'
runs-on: ubuntu-latest
timeout-minutes: 120
container:
image: debian:trixie
options: --user root
permissions:
contents: read
outputs:
version: ${{ steps.version.outputs.version }}
upstream_version: ${{ steps.version.outputs.upstream_version }}
base_version: ${{ steps.version.outputs.base_version }}
steps:
- name: Checkout rsyslog source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Install prerequisites
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" install_prereqs
apt-get install -y --no-install-recommends curl xz-utils
- name: Generate staging version
id: version
run: |
"$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" version
- name: Load Debian CI policy
run: |
mkdir -p "$DEBIAN_STAGING_BUILD_ROOT"
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" load_policy \
"$GITHUB_WORKSPACE/$DEBIAN_CI_POLICY_FILE" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy"
- name: Fetch Debian packaging baseline
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" fetch_debian_packaging \
"$DEBIAN_PACKAGING_REPO" \
"$DEBIAN_PACKAGING_BRANCH" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-packaging"
- name: Apply packaging baseline policy
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" apply_control_replacements \
"$DEBIAN_STAGING_BUILD_ROOT/debian-packaging/control" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/control_replacements.tsv" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/control_replacements.reasons.txt"
- name: Install Debian build dependencies
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" install_build_deps \
"$DEBIAN_STAGING_BUILD_ROOT/debian-packaging/control" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/supplemental_build_deps.txt"
- name: Generate rsyslog dist tarball
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" run_dist_build \
"$GITHUB_WORKSPACE"
- name: Locate rsyslog dist tarball
run: |
dist_tarball="$(
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" find_dist_tarball \
"$GITHUB_WORKSPACE"
)"
echo "DIST_TARBALL=$dist_tarball" >> "$GITHUB_ENV"
- name: Unpack tarball and inject Debian packaging
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" unpack_source_tree \
"$GITHUB_WORKSPACE" \
"$DIST_TARBALL" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-packaging" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-src"
- name: Stamp staging changelog
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
"$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" stamp-changelog \
"$DEBIAN_STAGING_BUILD_ROOT/debian-src" \
"$VERSION"
- name: Apply Debian CI policy
run: |
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" apply_not_installed_policy \
"$DEBIAN_STAGING_BUILD_ROOT/debian-src" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/not_installed_paths.txt" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/not_installed_paths.reasons.txt"
"$GITHUB_WORKSPACE/$DEBIAN_CI_HELPER" resolve_patch_policy \
"$DEBIAN_STAGING_BUILD_ROOT/debian-src" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/allowed_patch_skips.txt" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-ci-policy/allowed_patch_skips.reasons.txt" \
strict
- name: Build source and binary packages
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
"$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" build-package \
"$GITHUB_WORKSPACE" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-src" \
"$DIST_TARBALL" \
"$GITHUB_WORKSPACE/debian-staging-artifacts" \
"$VERSION" \
"$DEBIAN_STAGING_BUILD_ROOT/debian-daily-staging-build.log"
- name: Generate artifact manifest
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
"$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" manifest \
"$GITHUB_WORKSPACE/debian-staging-artifacts" \
"$VERSION" \
"$DEBIAN_STAGING_SUITE" \
"$DEBIAN_STAGING_ARCH"
- name: Upload Debian staging artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: debian-staging-${{ steps.version.outputs.version }}
path: debian-staging-artifacts/
if-no-files-found: error
- name: Summarize build
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
{
echo "### Debian daily staging build"
echo
echo "- Version: \`$VERSION\`"
echo "- Suite: \`$DEBIAN_STAGING_SUITE\`"
echo "- Architecture: \`$DEBIAN_STAGING_ARCH\`"
echo
echo "Artifacts:"
find debian-staging-artifacts -maxdepth 1 -type f -printf '%f\n' | sort |
while IFS= read -r artifact; do
printf -- "- \`%s\`\n" "$artifact"
done
} >> "$GITHUB_STEP_SUMMARY"
publish:
name: publish GitHub Pages APT repo
needs:
- preflight
- build
if: needs.preflight.outputs.should_publish == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
environment: debian-daily-staging
permissions:
contents: read
env:
STAGING_REPO_OWNER: ${{ vars.DEBIAN_STAGING_REPO_OWNER }}
STAGING_REPO_NAME: ${{ vars.DEBIAN_STAGING_REPO_NAME }}
STAGING_PAGES_TOKEN: ${{ secrets.DEBIAN_STAGING_PAGES_TOKEN }}
DEBIAN_STAGING_GPG_PRIVATE_KEY: ${{ secrets.DEBIAN_STAGING_GPG_PRIVATE_KEY }}
DEBIAN_STAGING_GPG_PASSPHRASE: ${{ secrets.DEBIAN_STAGING_GPG_PASSPHRASE }}
steps:
- name: Checkout rsyslog source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install repository generation tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends apt-utils gnupg xz-utils
- name: Download Debian staging artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: debian-staging-${{ needs.build.outputs.version }}
path: debian-staging-artifacts
- name: Verify artifact checksums
run: |
cd debian-staging-artifacts
sha256sum -c SHA256SUMS
- name: Import staging signing key
run: |
set -euo pipefail
install -m 700 -d "$HOME/.gnupg"
printf '%s\n' "$DEBIAN_STAGING_GPG_PRIVATE_KEY" | gpg --batch --import
gpg --batch --list-secret-keys
- name: Checkout staging Pages repository
run: |
set -euo pipefail
[ -n "$STAGING_REPO_OWNER" ] || { echo "DEBIAN_STAGING_REPO_OWNER is empty" >&2; exit 1; }
[ -n "$STAGING_REPO_NAME" ] || { echo "DEBIAN_STAGING_REPO_NAME is empty" >&2; exit 1; }
[ -n "$STAGING_PAGES_TOKEN" ] || { echo "DEBIAN_STAGING_PAGES_TOKEN is empty" >&2; exit 1; }
repo_url="https://x-access-token:${STAGING_PAGES_TOKEN}@github.com/${STAGING_REPO_OWNER}/${STAGING_REPO_NAME}.git"
if git ls-remote --exit-code --heads "$repo_url" gh-pages >/dev/null; then
git clone --depth=1 --branch gh-pages "$repo_url" staging-pages
else
git ls-remote "$repo_url" >/dev/null
mkdir staging-pages
git -C staging-pages init
git -C staging-pages checkout -b gh-pages
git -C staging-pages remote add origin "$repo_url"
fi
- name: Generate signed APT repository
run: |
"$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" generate-repo \
"$GITHUB_WORKSPACE/debian-staging-artifacts" \
"$GITHUB_WORKSPACE/staging-pages" \
"$DEBIAN_STAGING_SUITE" \
"$DEBIAN_STAGING_COMPONENT"
- name: Commit and push staging repository
env:
VERSION: ${{ needs.build.outputs.version }}
run: |
set -euo pipefail
cd staging-pages
git config user.name "rsyslog release bot"
git config user.email "release-bot@adiscon.com"
git add .
if git diff --cached --quiet; then
echo "No staging repository changes to publish."
exit 0
fi
git commit -m "publish rsyslog Debian staging $VERSION"
git push origin HEAD:gh-pages
verify:
name: verify Pages repository
needs:
- preflight
- build
- publish
if: needs.preflight.outputs.should_publish == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
environment: debian-daily-staging
permissions:
contents: read
env:
DEBIAN_STAGING_REPO_URL: ${{ vars.DEBIAN_STAGING_REPO_URL }}
DEBIAN_STAGING_GPG_FINGERPRINT: ${{ vars.DEBIAN_STAGING_GPG_FINGERPRINT }}
steps:
- name: Checkout rsyslog source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install verification tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends curl gnupg xz-utils
- name: Wait for Pages and verify package metadata
env:
VERSION: ${{ needs.build.outputs.version }}
run: |
set -euo pipefail
[ -n "$DEBIAN_STAGING_REPO_URL" ] || { echo "DEBIAN_STAGING_REPO_URL is empty" >&2; exit 1; }
[ -n "$DEBIAN_STAGING_GPG_FINGERPRINT" ] || { echo "DEBIAN_STAGING_GPG_FINGERPRINT is empty" >&2; exit 1; }
for attempt in $(seq 1 20); do
if "$GITHUB_WORKSPACE/$DEBIAN_STAGING_HELPER" verify-repo \
"$DEBIAN_STAGING_REPO_URL" \
"$DEBIAN_STAGING_SUITE" \
"$VERSION" \
"$DEBIAN_STAGING_GPG_FINGERPRINT"; then
exit 0
fi
echo "Repository not ready yet, retrying ($attempt/20)..."
if [ "$attempt" -lt 20 ]; then
sleep 30
fi
done
echo "Repository verification did not succeed before timeout." >&2
exit 1
report_failure:
name: report failure
needs:
- preflight
- build
- publish
- verify
if: >-
always() &&
needs.preflight.outputs.should_run == 'true' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
environment: debian-daily-staging
permissions:
issues: write
contents: read
steps:
- name: Create or update failure issue
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
VERSION: ${{ needs.build.outputs.version }}
REPO_URL: ${{ vars.DEBIAN_STAGING_REPO_URL }}
BUILD_RESULT: ${{ needs.build.result }}
PUBLISH_RESULT: ${{ needs.publish.result }}
VERIFY_RESULT: ${{ needs.verify.result }}
with:
script: |
const version = process.env.VERSION || `run-${context.runId}`;
const title = `[debian-daily-staging] rsyslog ${version} failed`;
const body = [
`Automated Debian daily staging failed for \`${version}\`.`,
'',
`Workflow run: ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`,
`Commit: ${context.sha}`,
`Repository URL: ${process.env.REPO_URL || '(not configured)'}`,
'',
'Job results:',
`- build: ${process.env.BUILD_RESULT}`,
`- publish: ${process.env.PUBLISH_RESULT}`,
`- verify: ${process.env.VERIFY_RESULT}`,
'',
'Inspect the workflow artifacts and logs, then rerun the workflow manually after fixing the cause.'
].join('\n');
const {owner, repo} = context.repo;
const existing = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: 'open',
per_page: 100
});
const issue = existing.find(item => item.title === title && !item.pull_request);
if (issue) {
await github.rest.issues.createComment({
owner,
repo,
issue_number: issue.number,
body
});
return;
}
const created = await github.rest.issues.create({
owner,
repo,
title,
body
});
try {
await github.rest.issues.addLabels({
owner,
repo,
issue_number: created.data.number,
labels: ['release', 'packaging', 'debian', 'daily-stable', 'automated']
});
} catch (error) {
core.warning(`Could not add labels: ${error.message}`);
}

1
devtools/check-flake-evidence-coverage.py
View File

@ -33,6 +33,7 @@ EXPECTED_UPLOADS = {
"imbeats.yml": 1,
"imhttp_prometheus_scrape.yml": 1,
"impstats_push_victoriametrics.yml": 1,
"debian_daily_stable.yml": 2,
}
TEST_COMMAND_RE = re.compile(r"run-ci\.sh|make\s+[^\n]*\b(?:check|distcheck)\b|devtools/test-[^\s]+\.sh")
UPLOAD_RE = re.compile(

523
devtools/release/debian-daily-stable.sh Executable file
View File

@ -0,0 +1,523 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat >&2 <<'EOF'
Usage: debian-daily-stable.sh <command> [args...]
Commands:
version
stamp-changelog <source-dir> <version>
build-package <workspace> <source-dir> <dist-tarball> <artifact-dir> <version> <build-log>
manifest <artifact-dir> <version> <suite> <arch> <channel> <distro> <distro-version>
generate-repo <artifact-dir> <repo-dir> <suite> <component> <arch>
verify-repo <repo-url> <suite> <component> <arch> <version> <expected-key-fingerprint>
self-test
EOF
}
die() {
echo "ERROR: $*" >&2
exit 1
}
base_version() {
local configure_file script_dir
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
configure_file="${RSYSLOG_SOURCE_DIR:-$script_dir/../..}/configure.ac"
sed -n 's/AC_INIT(\[rsyslog\],\[\([^]]*\)\].*/\1/p' "$configure_file" |
sed 's/\.daily$//'
}
short_commit_sha() {
local candidate="${SOURCE_GIT_SHA:-${GITHUB_SHA:-}}"
if [ -n "$candidate" ] &&
printf '%s\n' "$candidate" | grep -Eq '^[0-9a-fA-F]{12,64}$'; then
printf '%.12s\n' "$candidate" | tr '[:upper:]' '[:lower:]'
return
fi
git rev-parse --verify HEAD >/dev/null 2>&1 ||
die "could not determine git commit from SOURCE_GIT_SHA, GITHUB_SHA, or local HEAD"
git rev-parse --short=12 HEAD
}
cmd_version() {
local base date short_sha run attempt version
base="$(base_version)"
[ -n "$base" ] || die "could not determine base version from configure.ac"
date="${RSYSLOG_BUILD_DATE:-$(date -u +%Y%m%d)}"
printf '%s\n' "$date" | grep -Eq '^[0-9]{8}$' ||
die "RSYSLOG_BUILD_DATE must use YYYYMMDD"
short_sha="$(short_commit_sha)"
run="${GITHUB_RUN_NUMBER:-0}"
attempt="${GITHUB_RUN_ATTEMPT:-1}"
version="${base}~daily${date}.${run}.${attempt}+git${short_sha}-1adiscon1"
printf '%s\n' "$version"
if [ -n "${GITHUB_OUTPUT:-}" ]; then
{
printf 'version=%s\n' "$version"
printf 'upstream_version=%s\n' "${version%%-*}"
printf 'base_version=%s\n' "$base"
printf 'archive_date=%s-%s-%s\n' \
"${date:0:4}" "${date:4:2}" "${date:6:2}"
} >> "$GITHUB_OUTPUT"
fi
}
cmd_stamp_changelog() {
local source_dir="$1"
local version="$2"
[ -d "$source_dir/debian" ] ||
die "missing debian directory: $source_dir/debian"
cd "$source_dir"
export DEBEMAIL="${DEBEMAIL:-release-bot@adiscon.com}"
export DEBFULLNAME="${DEBFULLNAME:-Adiscon package maintainers}"
dch --force-distribution --distribution trixie --newversion "$version" \
"Automated rsyslog Debian daily stable build."
}
cmd_build_package() {
local workspace="$1"
local source_dir="$2"
local dist_tarball="$3"
local artifact_dir="$4"
local version="$5"
local build_log="$6"
local upstream_version parent parent_real workspace_real rc
upstream_version="${version%%-*}"
parent="$(dirname "$source_dir")"
parent_real="$(readlink -f "$parent")"
workspace_real="$(readlink -f "$workspace")"
[ -f "$workspace/$dist_tarball" ] ||
die "missing dist tarball: $workspace/$dist_tarball"
[ -d "$source_dir/debian" ] ||
die "missing Debian source tree: $source_dir"
case "$parent_real" in
/|/tmp|"$workspace_real")
die "refusing to collect package artifacts from unsafe build parent: $parent_real"
;;
esac
rm -rf "$artifact_dir"
mkdir -p "$artifact_dir"
find "$parent" -maxdepth 1 -type f \
\( -name '*.deb' -o -name '*.dsc' -o -name '*.changes' -o -name '*.buildinfo' \
-o -name '*.orig.tar.*' -o -name '*.debian.tar.*' \) \
-delete
cp "$workspace/$dist_tarball" "$parent/rsyslog_${upstream_version}.orig.tar.gz"
cd "$source_dir"
export DEB_BUILD_OPTIONS="${DEB_BUILD_OPTIONS:-nocheck}"
set +e
dpkg-buildpackage -us -uc -j"$(nproc)" 2>&1 | tee "$build_log"
rc=${PIPESTATUS[0]}
set -e
if [ "$rc" -ne 0 ]; then
return "$rc"
fi
if grep -Eq '(^|/)(doc/source|source)/[^:]+:[0-9]+: (CRITICAL|ERROR):|^Sphinx error:' "$build_log"; then
die "Debian docs build emitted fatal Sphinx/docutils diagnostics"
fi
cp "$build_log" "$artifact_dir/build.log"
find "$parent" -maxdepth 1 -type f \
\( -name "*_${version}_*.deb" -o -name "*_${version}.dsc" \
-o -name "*_${version}_*.changes" -o -name "*_${version}_*.buildinfo" \
-o -name "rsyslog_${upstream_version}.orig.tar.*" \
-o -name "rsyslog_${version}.debian.tar.*" \) \
-exec cp -a {} "$artifact_dir/" \;
find "$artifact_dir" -maxdepth 1 -type f -name '*.deb' | grep -q . ||
die "no .deb artifacts collected"
find "$artifact_dir" -maxdepth 1 -type f -name '*.dsc' | grep -q . ||
die "no .dsc artifact collected"
find "$artifact_dir" -maxdepth 1 -type f -name '*.changes' | grep -q . ||
die "no .changes artifact collected"
}
cmd_manifest() {
local artifact_dir="$1"
local version="$2"
local suite="$3"
local arch="$4"
local channel="$5"
local distro="$6"
local distro_version="$7"
[ -d "$artifact_dir" ] ||
die "missing artifact directory: $artifact_dir"
(
cd "$artifact_dir"
find . -maxdepth 1 -type f \
\( -name '*.deb' -o -name '*.dsc' -o -name '*.changes' -o -name '*.buildinfo' \
-o -name '*.orig.tar.*' -o -name '*.debian.tar.*' \) \
-printf '%P\0' | sort -z | xargs -0 -r sha256sum > SHA256SUMS
)
python3 - "$artifact_dir" "$version" "$suite" "$arch" \
"$channel" "$distro" "$distro_version" <<'PY'
import hashlib
import json
import os
import sys
artifact_dir, version, suite, arch, channel, distro, distro_version = sys.argv[1:8]
files = []
release_suffixes = (
".deb",
".dsc",
".changes",
".buildinfo",
".orig.tar.gz",
".orig.tar.xz",
".orig.tar.bz2",
".debian.tar.gz",
".debian.tar.xz",
".debian.tar.bz2",
)
for name in sorted(os.listdir(artifact_dir)):
path = os.path.join(artifact_dir, name)
if not os.path.isfile(path) or not name.endswith(release_suffixes):
continue
with open(path, "rb") as fh:
digest = hashlib.file_digest(fh, "sha256").hexdigest()
files.append({"name": name, "sha256": digest, "size": os.path.getsize(path)})
manifest = {
"schema": 1,
"package": "rsyslog",
"version": version,
"channel": channel,
"distribution": distro,
"distribution_version": distro_version,
"suite": suite,
"architecture": arch,
"retention": "minimum-five-years-from-publication",
"git_sha": os.environ.get("SOURCE_GIT_SHA", os.environ.get("GITHUB_SHA", "")),
"github_run_id": os.environ.get("GITHUB_RUN_ID", ""),
"github_run_attempt": os.environ.get("GITHUB_RUN_ATTEMPT", ""),
"github_run_number": os.environ.get("GITHUB_RUN_NUMBER", ""),
"files": files,
}
with open(os.path.join(artifact_dir, "manifest.json"), "w", encoding="utf-8") as fh:
json.dump(manifest, fh, indent=2, sort_keys=True)
fh.write("\n")
PY
}
secret_key_fingerprint() {
gpg --batch --list-secret-keys --with-colons |
awk -F: '$1 == "fpr" {print $10; exit}'
}
gpg_sign_args() {
if [ -n "${DEBIAN_DAILY_STABLE_GPG_PASSPHRASE:-}" ]; then
printf '%s\0' --pinentry-mode loopback \
--passphrase "$DEBIAN_DAILY_STABLE_GPG_PASSPHRASE"
fi
}
restore_plain_index() {
local index_path="$1"
if [ ! -f "$index_path" ] && [ -f "$index_path.xz" ]; then
xz -dc "$index_path.xz" > "$index_path"
elif [ ! -f "$index_path" ]; then
: > "$index_path"
fi
}
merge_deb822_indexes() {
local existing_file="$1"
local new_file="$2"
local output_file="$3"
local kind="$4"
python3 - "$existing_file" "$new_file" "$output_file" "$kind" <<'PY'
import sys
existing_path, new_path, output_path, kind = sys.argv[1:5]
def paragraphs(path):
with open(path, "r", encoding="utf-8") as stream:
text = stream.read().strip()
return [] if not text else text.split("\n\n")
def fields(paragraph):
parsed = {}
current = None
for line in paragraph.splitlines():
if line.startswith((" ", "\t")) and current:
parsed[current] += "\n" + line
continue
if ":" not in line:
continue
current, value = line.split(":", 1)
parsed[current] = value.strip()
return parsed
def key(paragraph):
data = fields(paragraph)
if kind == "binary":
names = ("Package", "Version", "Architecture")
else:
names = ("Package", "Version")
values = tuple(data.get(name, "") for name in names)
if not all(values):
raise SystemExit(f"incomplete {kind} index paragraph: {values}")
return values
merged = {}
order = []
for path in (existing_path, new_path):
for paragraph in paragraphs(path):
paragraph_key = key(paragraph)
if paragraph_key not in merged:
order.append(paragraph_key)
merged[paragraph_key] = paragraph
with open(output_path, "w", encoding="utf-8") as stream:
for paragraph_key in order:
stream.write(merged[paragraph_key].rstrip() + "\n\n")
PY
}
compress_index() {
local index_path="$1"
gzip -9 -n -c "$index_path" > "$index_path.gz"
xz -9 -c "$index_path" > "$index_path.xz"
}
add_by_hash() {
local index_path="$1"
local hash_dir="$2"
local candidate digest
mkdir -p "$hash_dir"
for candidate in "$index_path" "$index_path.gz" "$index_path.xz"; do
digest="$(sha256sum "$candidate" | awk '{print $1}')"
cp "$candidate" "$hash_dir/$digest"
done
}
copy_pool_artifacts() {
local artifact_dir="$1"
local pool_dir="$2"
mkdir -p "$pool_dir"
find "$artifact_dir" -maxdepth 1 -type f \
\( -name '*.deb' -o -name '*.dsc' -o -name '*.orig.tar.*' \
-o -name '*.debian.tar.*' \) \
-exec cp -a {} "$pool_dir/" \;
}
cmd_generate_repo() {
local artifact_dir="$1"
local repo_dir="$2"
local suite="$3"
local component="$4"
local arch="$5"
local key_fpr dist_dir pool_dir binary_dir source_dir
local packages_file sources_file
[ -d "$artifact_dir" ] ||
die "missing artifact directory: $artifact_dir"
find "$artifact_dir" -maxdepth 1 -type f -name '*.deb' | grep -q . ||
die "no .deb artifacts in $artifact_dir"
command -v apt-ftparchive >/dev/null 2>&1 ||
die "apt-ftparchive not found; install apt-utils"
pool_dir="$repo_dir/pool/$component/r/rsyslog"
dist_dir="$repo_dir/dists/$suite"
binary_dir="$dist_dir/$component/binary-$arch"
source_dir="$dist_dir/$component/source"
packages_file="$binary_dir/Packages"
sources_file="$source_dir/Sources"
mkdir -p "$binary_dir" "$source_dir"
restore_plain_index "$packages_file"
restore_plain_index "$sources_file"
copy_pool_artifacts "$artifact_dir" "$pool_dir"
(
cd "$repo_dir"
apt-ftparchive packages "pool/$component/r/rsyslog" > "$packages_file.new"
if find "pool/$component/r/rsyslog" -maxdepth 1 -type f -name '*.dsc' |
grep -q .; then
apt-ftparchive sources "pool/$component/r/rsyslog" > "$sources_file.new"
else
: > "$sources_file.new"
fi
)
merge_deb822_indexes "$packages_file" "$packages_file.new" \
"$packages_file.merged" binary
merge_deb822_indexes "$sources_file" "$sources_file.new" \
"$sources_file.merged" source
mv "$packages_file.merged" "$packages_file"
mv "$sources_file.merged" "$sources_file"
rm -f "$packages_file.new" "$sources_file.new"
compress_index "$packages_file"
compress_index "$sources_file"
(
cd "$repo_dir"
apt-ftparchive \
-o "APT::FTPArchive::Release::Origin=Adiscon" \
-o "APT::FTPArchive::Release::Label=rsyslog daily stable" \
-o "APT::FTPArchive::Release::Suite=$suite" \
-o "APT::FTPArchive::Release::Codename=$suite" \
-o "APT::FTPArchive::Release::Architectures=$arch" \
-o "APT::FTPArchive::Release::Components=$component" \
-o "APT::FTPArchive::Release::Acquire-By-Hash=yes" \
-o "APT::FTPArchive::Release::Description=rsyslog Debian daily stable packages" \
release "dists/$suite" > "dists/$suite/Release"
)
add_by_hash "$packages_file" "$binary_dir/by-hash/SHA256"
add_by_hash "$sources_file" "$source_dir/by-hash/SHA256"
key_fpr="$(secret_key_fingerprint)"
[ -n "$key_fpr" ] ||
die "no GPG secret key available for repository signing"
gpg --batch --armor --export "$key_fpr" > "$repo_dir/rsyslog-archive-keyring.asc"
mapfile -d '' -t extra_gpg_args < <(gpg_sign_args)
gpg --batch --yes "${extra_gpg_args[@]}" --local-user "$key_fpr" \
--clearsign --output "$dist_dir/InRelease" "$dist_dir/Release"
gpg --batch --yes "${extra_gpg_args[@]}" --local-user "$key_fpr" \
--detach-sign --armor --output "$dist_dir/Release.gpg" "$dist_dir/Release"
}
cmd_verify_repo() {
local repo_url="$1"
local suite="$2"
local component="$3"
local arch="$4"
local version="$5"
local expected_key_fpr="$6"
local tmp_dir packages_url_path packages_release_path
local packages_hash packages_size actual_key_fpr
command -v curl >/dev/null 2>&1 || die "curl not found"
command -v gpg >/dev/null 2>&1 || die "gpg not found"
command -v gpgv >/dev/null 2>&1 || die "gpgv not found"
command -v xz >/dev/null 2>&1 || die "xz not found"
[ -n "$expected_key_fpr" ] ||
die "missing expected signing key fingerprint"
repo_url="${repo_url%/}"
packages_url_path="dists/$suite/$component/binary-$arch/Packages.xz"
packages_release_path="$component/binary-$arch/Packages.xz"
tmp_dir="$(mktemp -d)"
curl -fsSL "$repo_url/dists/$suite/InRelease" -o "$tmp_dir/InRelease"
curl -fsSL "$repo_url/rsyslog-archive-keyring.asc" -o "$tmp_dir/repo-key.asc"
gpg --batch --dearmor -o "$tmp_dir/repo-key.gpg" "$tmp_dir/repo-key.asc"
actual_key_fpr="$(
gpg --batch --show-keys --with-colons "$tmp_dir/repo-key.asc" |
awk -F: '$1 == "fpr" {print $10; exit}'
)"
[ "${actual_key_fpr^^}" = "${expected_key_fpr^^}" ] ||
die "published signing key fingerprint $actual_key_fpr does not match expected $expected_key_fpr"
gpgv --keyring "$tmp_dir/repo-key.gpg" "$tmp_dir/InRelease"
curl -fsSL "$repo_url/$packages_url_path" -o "$tmp_dir/Packages.xz"
packages_hash="$(sha256sum "$tmp_dir/Packages.xz" | awk '{print $1}')"
packages_size="$(wc -c < "$tmp_dir/Packages.xz" | tr -d ' ')"
awk -v hash="$packages_hash" -v size="$packages_size" \
-v path="$packages_release_path" \
'$1 == hash && $2 == size && $3 == path { found = 1 } END { exit !found }' \
"$tmp_dir/InRelease" ||
die "Packages.xz checksum is not present in signed InRelease"
xz -dc "$tmp_dir/Packages.xz" > "$tmp_dir/Packages"
grep -Fxq "Package: rsyslog" "$tmp_dir/Packages" ||
die "rsyslog package not found in Packages"
grep -Fxq "Version: $version" "$tmp_dir/Packages" ||
die "version $version not found in Packages"
rm -rf "$tmp_dir"
}
build_self_test_deb() {
local root="$1"
local version="$2"
local package_dir="$root/package-$version"
install -m 755 -d "$package_dir/DEBIAN"
{
echo "Package: rsyslog"
echo "Version: $version"
echo "Architecture: amd64"
echo "Maintainer: rsyslog test <test@example.invalid>"
echo "Description: synthetic daily archive test package"
} > "$package_dir/DEBIAN/control"
dpkg-deb --build "$package_dir" "$root/rsyslog_${version}_amd64.deb" >/dev/null
}
cmd_self_test() {
local tmp_dir repo_dir first_artifacts second_artifacts fingerprint
for tool in apt-ftparchive curl dpkg-deb gpg gpgv xz; do
command -v "$tool" >/dev/null 2>&1 ||
die "$tool is required for self-test"
done
tmp_dir="$(mktemp -d)"
repo_dir="$tmp_dir/repo"
first_artifacts="$tmp_dir/artifacts-1"
second_artifacts="$tmp_dir/artifacts-2"
mkdir -p "$repo_dir" "$first_artifacts" "$second_artifacts"
export GNUPGHOME="$tmp_dir/gnupg"
install -m 700 -d "$GNUPGHOME"
gpg --batch --passphrase '' --quick-generate-key \
"rsyslog archive self-test <test@example.invalid>" rsa2048 sign 0 >/dev/null 2>&1
fingerprint="$(secret_key_fingerprint)"
build_self_test_deb "$first_artifacts" "1.0~daily1"
cmd_generate_repo "$first_artifacts" "$repo_dir" trixie main amd64
cmd_verify_repo "file://$repo_dir" trixie main amd64 \
"1.0~daily1" "$fingerprint"
build_self_test_deb "$second_artifacts" "1.0~daily2"
cmd_generate_repo "$second_artifacts" "$repo_dir" trixie main amd64
cmd_verify_repo "file://$repo_dir" trixie main amd64 \
"1.0~daily1" "$fingerprint"
cmd_verify_repo "file://$repo_dir" trixie main amd64 \
"1.0~daily2" "$fingerprint"
rm -rf "$tmp_dir"
echo "Debian daily stable archive self-test passed."
}
command="${1:-}"
case "$command" in
version) shift; cmd_version "$@" ;;
stamp-changelog) shift; cmd_stamp_changelog "$@" ;;
build-package) shift; cmd_build_package "$@" ;;
manifest) shift; cmd_manifest "$@" ;;
generate-repo) shift; cmd_generate_repo "$@" ;;
verify-repo) shift; cmd_verify_repo "$@" ;;
self-test) shift; cmd_self_test "$@" ;;
*) usage; exit 2 ;;
esac

308
devtools/release/debian-daily-staging.sh
View File

@ -1,308 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat >&2 <<'EOF'
Usage: debian-daily-staging.sh <command> [args...]
Commands:
version
stamp-changelog <source-dir> <version>
build-package <workspace> <source-dir> <dist-tarball> <artifact-dir> <version> <build-log>
manifest <artifact-dir> <version> <suite> <arch>
generate-repo <artifact-dir> <repo-dir> <suite> <component>
verify-repo <repo-url> <suite> <version> <expected-key-fingerprint>
EOF
}
die() {
echo "ERROR: $*" >&2
exit 1
}
base_version() {
local script_dir
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
sed -n 's/AC_INIT(\[rsyslog\],\[\([^]]*\)\].*/\1/p' "$script_dir/../../configure.ac" | sed 's/\.daily$//'
}
short_commit_sha() {
if [ -n "${GITHUB_SHA:-}" ]; then
if printf '%s\n' "$GITHUB_SHA" | grep -Eq '^[0-9a-fA-F]{12,64}$'; then
printf '%.12s\n' "$GITHUB_SHA" | tr '[:upper:]' '[:lower:]'
return
fi
fi
git rev-parse --verify HEAD >/dev/null 2>&1 ||
die "could not determine git commit from GITHUB_SHA or local HEAD"
git rev-parse --short=12 HEAD
}
cmd_version() {
local base date short_sha run version
base="$(base_version)"
[ -n "$base" ] || die "could not determine base version from configure.ac"
date="$(date -u +%Y%m%d)"
short_sha="$(short_commit_sha)"
run="${GITHUB_RUN_NUMBER:-0}"
version="${base}~daily${date}.${run}+git${short_sha}-1adiscon1"
printf '%s\n' "$version"
if [ -n "${GITHUB_OUTPUT:-}" ]; then
{
printf 'version=%s\n' "$version"
printf 'upstream_version=%s\n' "${version%%-*}"
printf 'base_version=%s\n' "$base"
} >> "$GITHUB_OUTPUT"
fi
}
cmd_stamp_changelog() {
local source_dir="$1"
local version="$2"
[ -d "$source_dir/debian" ] || die "missing debian directory: $source_dir/debian"
cd "$source_dir"
export DEBEMAIL="${DEBEMAIL:-release-bot@adiscon.com}"
export DEBFULLNAME="${DEBFULLNAME:-Adiscon package maintainers}"
dch --force-distribution --distribution trixie --newversion "$version" \
"Automated rsyslog Debian daily staging build."
}
cmd_build_package() {
local workspace="$1"
local source_dir="$2"
local dist_tarball="$3"
local artifact_dir="$4"
local version="$5"
local build_log="$6"
local upstream_version parent parent_real workspace_real rc
upstream_version="${version%%-*}"
parent="$(dirname "$source_dir")"
parent_real="$(readlink -f "$parent")"
workspace_real="$(readlink -f "$workspace")"
[ -f "$workspace/$dist_tarball" ] || die "missing dist tarball: $workspace/$dist_tarball"
[ -d "$source_dir/debian" ] || die "missing Debian source tree: $source_dir"
case "$parent_real" in
/|/tmp|"$workspace_real")
die "refusing to collect package artifacts from unsafe build parent: $parent_real"
;;
esac
rm -rf "$artifact_dir"
mkdir -p "$artifact_dir"
find "$parent" -maxdepth 1 -type f \
\( -name '*.deb' -o -name '*.dsc' -o -name '*.changes' -o -name '*.buildinfo' \
-o -name '*.orig.tar.*' -o -name '*.debian.tar.*' \) \
-delete
cp "$workspace/$dist_tarball" "$parent/rsyslog_${upstream_version}.orig.tar.gz"
cd "$source_dir"
export DEB_BUILD_OPTIONS="${DEB_BUILD_OPTIONS:-nocheck}"
set +e
dpkg-buildpackage -us -uc -j"$(nproc)" 2>&1 | tee "$build_log"
rc=${PIPESTATUS[0]}
set -e
if [ "$rc" -ne 0 ]; then
return "$rc"
fi
if grep -Eq '(^|/)(doc/source|source)/[^:]+:[0-9]+: (CRITICAL|ERROR):|^Sphinx error:' "$build_log"; then
die "Debian docs build emitted fatal Sphinx/docutils diagnostics"
fi
cp "$build_log" "$artifact_dir/build.log"
find "$parent" -maxdepth 1 -type f \
\( -name "*_${version}_*.deb" -o -name "*_${version}.dsc" \
-o -name "*_${version}_*.changes" -o -name "*_${version}_*.buildinfo" \
-o -name "rsyslog_${upstream_version}.orig.tar.*" \
-o -name "rsyslog_${version}.debian.tar.*" \) \
-exec cp -a {} "$artifact_dir/" \;
find "$artifact_dir" -maxdepth 1 -type f -name '*.deb' | grep -q . || die "no .deb artifacts collected"
find "$artifact_dir" -maxdepth 1 -type f -name '*.dsc' | grep -q . || die "no .dsc artifact collected"
find "$artifact_dir" -maxdepth 1 -type f -name '*.changes' | grep -q . || die "no .changes artifact collected"
}
cmd_manifest() {
local artifact_dir="$1"
local version="$2"
local suite="$3"
local arch="$4"
[ -d "$artifact_dir" ] || die "missing artifact directory: $artifact_dir"
(
cd "$artifact_dir"
find . -maxdepth 1 -type f \
\( -name '*.deb' -o -name '*.dsc' -o -name '*.changes' -o -name '*.buildinfo' \
-o -name '*.orig.tar.*' -o -name '*.debian.tar.*' \) \
-printf '%P\0' | sort -z | xargs -0 -r sha256sum > SHA256SUMS
)
python3 - "$artifact_dir" "$version" "$suite" "$arch" <<'PY'
import hashlib
import json
import os
import sys
artifact_dir, version, suite, arch = sys.argv[1:5]
files = []
for name in sorted(os.listdir(artifact_dir)):
path = os.path.join(artifact_dir, name)
if not os.path.isfile(path):
continue
release_suffixes = (
".deb",
".dsc",
".changes",
".buildinfo",
".orig.tar.gz",
".orig.tar.xz",
".orig.tar.bz2",
".debian.tar.gz",
".debian.tar.xz",
".debian.tar.bz2",
)
if not name.endswith(release_suffixes):
continue
with open(path, "rb") as fh:
h = hashlib.sha256()
for chunk in iter(lambda: fh.read(65536), b""):
h.update(chunk)
digest = h.hexdigest()
files.append({"name": name, "sha256": digest, "size": os.path.getsize(path)})
manifest = {
"package": "rsyslog",
"version": version,
"suite": suite,
"architecture": arch,
"git_sha": os.environ.get("GITHUB_SHA", ""),
"github_run_id": os.environ.get("GITHUB_RUN_ID", ""),
"github_run_number": os.environ.get("GITHUB_RUN_NUMBER", ""),
"files": files,
}
with open(os.path.join(artifact_dir, "manifest.json"), "w", encoding="utf-8") as fh:
json.dump(manifest, fh, indent=2, sort_keys=True)
fh.write("\n")
PY
}
secret_key_fingerprint() {
gpg --batch --list-secret-keys --with-colons | awk -F: '$1 == "fpr" {print $10; exit}'
}
gpg_sign_args() {
if [ -n "${DEBIAN_STAGING_GPG_PASSPHRASE:-}" ]; then
printf '%s\0' --pinentry-mode loopback --passphrase "$DEBIAN_STAGING_GPG_PASSPHRASE"
fi
}
cmd_generate_repo() {
local artifact_dir="$1"
local repo_dir="$2"
local suite="$3"
local component="$4"
local arch="amd64"
local key_fpr dist_dir packages_file
[ -d "$artifact_dir" ] || die "missing artifact directory: $artifact_dir"
find "$artifact_dir" -maxdepth 1 -type f -name '*.deb' | grep -q . || die "no .deb artifacts in $artifact_dir"
command -v apt-ftparchive >/dev/null 2>&1 || die "apt-ftparchive not found; install apt-utils"
mkdir -p "$repo_dir/pool/main/r/rsyslog"
cp -a "$artifact_dir"/*.deb "$repo_dir/pool/main/r/rsyslog/"
cp -a "$artifact_dir"/manifest.json "$artifact_dir"/SHA256SUMS "$repo_dir/pool/main/r/rsyslog/"
key_fpr="$(secret_key_fingerprint)"
[ -n "$key_fpr" ] || die "no GPG secret key available for repository signing"
gpg --batch --armor --export "$key_fpr" > "$repo_dir/rsyslog-debian-staging.asc"
dist_dir="$repo_dir/dists/$suite"
packages_file="$dist_dir/$component/binary-$arch/Packages"
mkdir -p "$(dirname "$packages_file")"
(
cd "$repo_dir"
apt-ftparchive packages "pool/main/r/rsyslog" > "dists/$suite/$component/binary-$arch/Packages"
xz -9 -k -f "dists/$suite/$component/binary-$arch/Packages"
apt-ftparchive \
-o "APT::FTPArchive::Release::Origin=Adiscon" \
-o "APT::FTPArchive::Release::Label=rsyslog Debian daily staging" \
-o "APT::FTPArchive::Release::Suite=$suite" \
-o "APT::FTPArchive::Release::Codename=$suite" \
-o "APT::FTPArchive::Release::Architectures=$arch" \
-o "APT::FTPArchive::Release::Components=$component" \
-o "APT::FTPArchive::Release::Description=rsyslog Debian daily staging packages" \
release "dists/$suite" > "dists/$suite/Release"
)
mapfile -d '' -t extra_gpg_args < <(gpg_sign_args)
gpg --batch --yes "${extra_gpg_args[@]}" --local-user "$key_fpr" --clearsign \
--output "$dist_dir/InRelease" "$dist_dir/Release"
gpg --batch --yes "${extra_gpg_args[@]}" --local-user "$key_fpr" --detach-sign --armor \
--output "$dist_dir/Release.gpg" "$dist_dir/Release"
}
cmd_verify_repo() {
local repo_url="$1"
local suite="$2"
local version="$3"
local expected_key_fpr="$4"
local tmp_dir
local packages_url_path="dists/$suite/main/binary-amd64/Packages.xz"
local packages_release_path="main/binary-amd64/Packages.xz"
local packages_hash packages_size actual_key_fpr
command -v curl >/dev/null 2>&1 || die "curl not found"
command -v gpg >/dev/null 2>&1 || die "gpg not found"
command -v gpgv >/dev/null 2>&1 || die "gpgv not found"
command -v xz >/dev/null 2>&1 || die "xz not found"
[ -n "$expected_key_fpr" ] || die "missing expected signing key fingerprint"
repo_url="${repo_url%/}"
tmp_dir="$(mktemp -d)"
trap 'rm -rf "${tmp_dir:-}"' EXIT
curl -fsSL "$repo_url/dists/$suite/InRelease" -o "$tmp_dir/InRelease"
curl -fsSL "$repo_url/rsyslog-debian-staging.asc" -o "$tmp_dir/repo-key.asc"
gpg --batch --dearmor -o "$tmp_dir/repo-key.gpg" "$tmp_dir/repo-key.asc"
actual_key_fpr="$(gpg --batch --show-keys --with-colons "$tmp_dir/repo-key.asc" |
awk -F: '$1 == "fpr" {print $10; exit}')"
[ "${actual_key_fpr^^}" = "${expected_key_fpr^^}" ] ||
die "published signing key fingerprint $actual_key_fpr does not match expected $expected_key_fpr"
gpgv --keyring "$tmp_dir/repo-key.gpg" "$tmp_dir/InRelease"
curl -fsSL "$repo_url/$packages_url_path" -o "$tmp_dir/Packages.xz"
packages_hash="$(sha256sum "$tmp_dir/Packages.xz" | awk '{print $1}')"
packages_size="$(wc -c < "$tmp_dir/Packages.xz" | tr -d ' ')"
awk -v hash="$packages_hash" -v size="$packages_size" -v path="$packages_release_path" \
'$1 == hash && $2 == size && $3 == path { found = 1 } END { exit !found }' \
"$tmp_dir/InRelease" || die "Packages.xz checksum is not present in signed InRelease"
xz -dc "$tmp_dir/Packages.xz" > "$tmp_dir/Packages"
grep -Fxq "Package: rsyslog" "$tmp_dir/Packages" || die "rsyslog package not found in Packages"
grep -Fxq "Version: $version" "$tmp_dir/Packages" || die "version $version not found in Packages"
}
command="${1:-}"
case "$command" in
version) shift; cmd_version "$@" ;;
stamp-changelog) shift; cmd_stamp_changelog "$@" ;;
build-package) shift; cmd_build_package "$@" ;;
manifest) shift; cmd_manifest "$@" ;;
generate-repo) shift; cmd_generate_repo "$@" ;;
verify-repo) shift; cmd_verify_repo "$@" ;;
*) usage; exit 2 ;;
esac