rsyslog/tests/imfile-wildcards-dirs-multi.sh
Rainer Gerhards ab1bd8c01b imfile: large refactoring of complete module
This commit greatly refactors imfile internal workings. It changes the
handling of inotify, FEN, and polling modes. Mostly unchanged is the
processing of the way a file is read and state files are kept.

This is about a 50% rewrite of the module.

Polling, inotify, and FEN modes now use greatly unified code. Some
differences still exists and may be changed with further commits. The
internal handling of wildcards and file detection has been completely
re-written from scratch. For example, previously when multi-level
wildcards were used these were not reliably detected. The code also
now provides much of the same functionality in all modes, most importantly
wildcards are now also supported in polling mode.

The refactoring sets ground for further enhancements and smaller
refactorings. This commit provides the same feature set that imfile
had previously and all existing CI tests pass, as do some newly
created tests.

Some specific changes:
- bugfix: module parameter "sortfiles" ignored
  This parameter only works in Solaris FEN mode, but is otherwise
  ignored.  Most importantly it is ignored under Linux.
  fixes https://github.com/rsyslog/rsyslog/issues/2528
- bugfix: imfile did not pick up all files when not present
  at startup
  fixes https://github.com/rsyslog/rsyslog/issues/2241
  fixes https://github.com/rsyslog/rsyslog/issues/2230
  fixes https://github.com/rsyslog/rsyslog/issues/2354
- bugfix: directories only support "*" wildcard, no others
  fixes https://github.com/rsyslog/rsyslog/issues/2303
- bugfix: parameter "sortfiles" did only work in FEN mode
  fixes https://github.com/rsyslog/rsyslog/issues/2528
- provides the ability to dynamically add and remove files via
  multi-level wildcards
  see also https://github.com/rsyslog/rsyslog/issues/1280
- the state file name currently has been changed to inode number
  This will further be worked on in upcoming PRs
  see also https://github.com/rsyslog/rsyslog/issues/2231
- some enhancements were also done to CI tests, most importantly
  they were made more compatibile with BSD

Note that most of the mentioned bug fixes cannot be applied to older
versions, as they fix design issues which are solved by the refactoring.
Thus there are not separate commits for them.

Distro maintainers: you need to decide to apply this patch as whole
or not. Believe me, it is not worth the effort to try to extract
specific patches from this commit. There is a good reason we do
not have multiple commits.

closes https://github.com/rsyslog/rsyslog/issues/2359
2018-03-22 08:25:47 +01:00

44 lines
1.3 KiB
Bash
Executable File

#!/bin/bash
# This is part of the rsyslog testbench, licensed under GPLv3
export IMFILEINPUTFILES="10"
export IMFILEINPUTFILESSTEPS="5"
#export IMFILEINPUTFILESALL=$(($IMFILEINPUTFILES * $IMFILEINPUTFILESSTEPS))
export IMFILECHECKTIMEOUT="20"
. $srcdir/diag.sh init
. $srcdir/diag.sh check-inotify-only
# generate input files first. Note that rsyslog processes it as
# soon as it start up (so the file should exist at that point).
# Start rsyslog now before adding more files
. $srcdir/diag.sh startup imfile-wildcards-dirs-multi.conf
for j in `seq 1 $IMFILEINPUTFILESSTEPS`;
do
echo "Loop Num $j"
for i in `seq 1 $IMFILEINPUTFILES`;
do
mkdir rsyslog.input.dir$i
mkdir rsyslog.input.dir$i/dir$i
./inputfilegen -m 1 > rsyslog.input.dir$i/dir$i/file.logfile
done
ls -d rsyslog.input.*
# Check correct amount of input files each time
let IMFILEINPUTFILESALL=$(($IMFILEINPUTFILES * $j))
. $srcdir/diag.sh content-check-with-count "HEADER msgnum:00000000:" $IMFILEINPUTFILESALL $IMFILECHECKTIMEOUT
# Delete all but first!
for i in `seq 1 $IMFILEINPUTFILES`;
do
rm -rf rsyslog.input.dir$i/dir$i/file.logfile
rm -rf rsyslog.input.dir$i
done
done
. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages
. $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished!
. $srcdir/diag.sh exit