mirror of
https://github.com/rsyslog/rsyslog.git
synced 2025-12-17 07:00:42 +01:00
ab1bd8c01b
This commit greatly refactors imfile internal workings. It changes the handling of inotify, FEN, and polling modes. Mostly unchanged is the processing of the way a file is read and state files are kept. This is about a 50% rewrite of the module. Polling, inotify, and FEN modes now use greatly unified code. Some differences still exists and may be changed with further commits. The internal handling of wildcards and file detection has been completely re-written from scratch. For example, previously when multi-level wildcards were used these were not reliably detected. The code also now provides much of the same functionality in all modes, most importantly wildcards are now also supported in polling mode. The refactoring sets ground for further enhancements and smaller refactorings. This commit provides the same feature set that imfile had previously and all existing CI tests pass, as do some newly created tests. Some specific changes: - bugfix: module parameter "sortfiles" ignored This parameter only works in Solaris FEN mode, but is otherwise ignored. Most importantly it is ignored under Linux. fixes https://github.com/rsyslog/rsyslog/issues/2528 - bugfix: imfile did not pick up all files when not present at startup fixes https://github.com/rsyslog/rsyslog/issues/2241 fixes https://github.com/rsyslog/rsyslog/issues/2230 fixes https://github.com/rsyslog/rsyslog/issues/2354 - bugfix: directories only support "*" wildcard, no others fixes https://github.com/rsyslog/rsyslog/issues/2303 - bugfix: parameter "sortfiles" did only work in FEN mode fixes https://github.com/rsyslog/rsyslog/issues/2528 - provides the ability to dynamically add and remove files via multi-level wildcards see also https://github.com/rsyslog/rsyslog/issues/1280 - the state file name currently has been changed to inode number This will further be worked on in upcoming PRs see also https://github.com/rsyslog/rsyslog/issues/2231 - some enhancements were also done to CI tests, most importantly they were made more compatibile with BSD Note that most of the mentioned bug fixes cannot be applied to older versions, as they fix design issues which are solved by the refactoring. Thus there are not separate commits for them. Distro maintainers: you need to decide to apply this patch as whole or not. Believe me, it is not worth the effort to try to extract specific patches from this commit. There is a good reason we do not have multiple commits. closes https://github.com/rsyslog/rsyslog/issues/2359
77 lines
2.8 KiB
Bash
Executable File
77 lines
2.8 KiB
Bash
Executable File
#!/bin/bash
|
|
# This is part of the rsyslog testbench, licensed under GPLv3
|
|
export IMFILEINPUTFILES="10"
|
|
echo [imfile-wildcards.sh]
|
|
. $srcdir/diag.sh check-inotify
|
|
. $srcdir/diag.sh init
|
|
# generate input files first. Note that rsyslog processes it as
|
|
# soon as it start up (so the file should exist at that point).
|
|
|
|
imfilebefore="rsyslog.input.1.log"
|
|
./inputfilegen -m 1 > $imfilebefore
|
|
|
|
. $srcdir/diag.sh generate-conf
|
|
. $srcdir/diag.sh add-conf '
|
|
# comment out if you need more debug info:
|
|
global( debug.whitelist="on"
|
|
debug.files=["imfile.c"])
|
|
|
|
module(load="../plugins/imfile/.libs/imfile"
|
|
mode="inotify" normalizePath="off")
|
|
|
|
input(type="imfile" File="./rsyslog.input.*.log" Tag="file:"
|
|
Severity="error" Facility="local7" addMetadata="on")
|
|
|
|
input(type="imfile" File="/does/not/exist/*.log" Tag="file:"
|
|
Severity="error" Facility="local7" addMetadata="on")
|
|
|
|
template(name="outfmt" type="list") {
|
|
constant(value="HEADER ")
|
|
property(name="msg" format="json")
|
|
constant(value=", filename: ")
|
|
property(name="$!metadata!filename")
|
|
constant(value=", fileoffset: ")
|
|
property(name="$!metadata!fileoffset")
|
|
constant(value="\n")
|
|
}
|
|
|
|
if $msg contains "msgnum:" then
|
|
action( type="omfile" file="rsyslog.out.log" template="outfmt")
|
|
'
|
|
# Start rsyslog now before adding more files
|
|
. $srcdir/diag.sh startup
|
|
|
|
for i in `seq 2 $IMFILEINPUTFILES`;
|
|
do
|
|
cp $imfilebefore rsyslog.input.$i.log
|
|
imfilebefore="rsyslog.input.$i.log"
|
|
# Wait little for correct timing
|
|
./msleep 50
|
|
done
|
|
./inputfilegen -m 3 > rsyslog.input.$((IMFILEINPUTFILES + 1)).log
|
|
ls -l rsyslog.input.*
|
|
|
|
. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages
|
|
. $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished!
|
|
|
|
printf 'HEADER msgnum:00000000:, filename: ./rsyslog.input.1.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.2.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.3.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.4.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.5.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.6.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.7.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.8.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.9.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.10.log, fileoffset: 0
|
|
HEADER msgnum:00000000:, filename: ./rsyslog.input.11.log, fileoffset: 0
|
|
HEADER msgnum:00000001:, filename: ./rsyslog.input.11.log, fileoffset: 17
|
|
HEADER msgnum:00000002:, filename: ./rsyslog.input.11.log, fileoffset: 34\n' | cmp - rsyslog.out.log
|
|
if [ ! $? -eq 0 ]; then
|
|
echo "invalid output generated, rsyslog.out.log is:"
|
|
cat rsyslog.out.log
|
|
exit 1
|
|
fi;
|
|
|
|
. $srcdir/diag.sh exit
|