mirror of
https://github.com/rsyslog/rsyslog.git
synced 2025-12-17 07:00:42 +01:00
ab1bd8c01b
This commit greatly refactors imfile internal workings. It changes the handling of inotify, FEN, and polling modes. Mostly unchanged is the processing of the way a file is read and state files are kept. This is about a 50% rewrite of the module. Polling, inotify, and FEN modes now use greatly unified code. Some differences still exists and may be changed with further commits. The internal handling of wildcards and file detection has been completely re-written from scratch. For example, previously when multi-level wildcards were used these were not reliably detected. The code also now provides much of the same functionality in all modes, most importantly wildcards are now also supported in polling mode. The refactoring sets ground for further enhancements and smaller refactorings. This commit provides the same feature set that imfile had previously and all existing CI tests pass, as do some newly created tests. Some specific changes: - bugfix: module parameter "sortfiles" ignored This parameter only works in Solaris FEN mode, but is otherwise ignored. Most importantly it is ignored under Linux. fixes https://github.com/rsyslog/rsyslog/issues/2528 - bugfix: imfile did not pick up all files when not present at startup fixes https://github.com/rsyslog/rsyslog/issues/2241 fixes https://github.com/rsyslog/rsyslog/issues/2230 fixes https://github.com/rsyslog/rsyslog/issues/2354 - bugfix: directories only support "*" wildcard, no others fixes https://github.com/rsyslog/rsyslog/issues/2303 - bugfix: parameter "sortfiles" did only work in FEN mode fixes https://github.com/rsyslog/rsyslog/issues/2528 - provides the ability to dynamically add and remove files via multi-level wildcards see also https://github.com/rsyslog/rsyslog/issues/1280 - the state file name currently has been changed to inode number This will further be worked on in upcoming PRs see also https://github.com/rsyslog/rsyslog/issues/2231 - some enhancements were also done to CI tests, most importantly they were made more compatibile with BSD Note that most of the mentioned bug fixes cannot be applied to older versions, as they fix design issues which are solved by the refactoring. Thus there are not separate commits for them. Distro maintainers: you need to decide to apply this patch as whole or not. Believe me, it is not worth the effort to try to extract specific patches from this commit. There is a good reason we do not have multiple commits. closes https://github.com/rsyslog/rsyslog/issues/2359
75 lines
2.1 KiB
Bash
Executable File
75 lines
2.1 KiB
Bash
Executable File
#!/bin/bash
|
|
# This is part of the rsyslog testbench, licensed under ASL 2.0
|
|
echo ======================================================================
|
|
echo [imfile-readmode2-with-persists.sh]
|
|
. $srcdir/diag.sh check-inotify
|
|
. $srcdir/diag.sh init
|
|
. $srcdir/diag.sh startup imfile-readmode2-with-persists.conf
|
|
|
|
# write the beginning of the file
|
|
echo 'msgnum:0
|
|
msgnum:1' > rsyslog.input
|
|
echo 'msgnum:2' >> rsyslog.input
|
|
|
|
# sleep a little to give rsyslog a chance to begin processing
|
|
sleep 1
|
|
|
|
# now stop and restart rsyslog so that the file info must be
|
|
# persisted and read again on startup. Results should still be
|
|
# correct ;)
|
|
echo stopping rsyslog
|
|
. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages
|
|
. $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished!
|
|
|
|
echo spool:
|
|
ls -l test-spool
|
|
|
|
echo restarting rsyslog
|
|
. $srcdir/diag.sh startup imfile-readmode2-with-persists.conf
|
|
echo restarted rsyslog, continuing with test
|
|
|
|
# write some more lines (see https://github.com/rsyslog/rsyslog/issues/144)
|
|
echo 'msgnum:3
|
|
msgnum:4' >> rsyslog.input
|
|
echo 'msgnum:5' >> rsyslog.input # this one shouldn't be written to the output file because of ReadMode 2
|
|
|
|
# give it time to finish
|
|
sleep 1
|
|
|
|
. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages
|
|
. $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished!
|
|
|
|
# give it time to write the output file
|
|
sleep 1
|
|
|
|
## check if we have the correct number of messages
|
|
|
|
NUMLINES=$(grep -c HEADER rsyslog.out.log 2>/dev/null)
|
|
|
|
if [ -z $NUMLINES ]; then
|
|
echo "ERROR: expecting at least a match for HEADER, maybe rsyslog.out.log wasn't even written?"
|
|
cat ./rsyslog.out.log
|
|
exit 1
|
|
else
|
|
if [ ! $NUMLINES -eq 3 ]; then
|
|
echo "ERROR: expecting 3 headers, got $NUMLINES"
|
|
cat ./rsyslog.out.log
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
## check if all the data we expect to get in the file is there
|
|
|
|
for i in {1..4}; do
|
|
grep msgnum:$i rsyslog.out.log > /dev/null 2>&1
|
|
if [ ! $? -eq 0 ]; then
|
|
echo "ERROR: expecting the string 'msgnum:$i', it's not there"
|
|
cat ./rsyslog.out.log
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
## if we got here, all is good :)
|
|
|
|
. $srcdir/diag.sh exit
|