This commit greatly refactors imfile internal workings. It changes the
handling of inotify, FEN, and polling modes. Mostly unchanged is the
processing of the way a file is read and state files are kept.
This is about a 50% rewrite of the module.
Polling, inotify, and FEN modes now use greatly unified code. Some
differences still exists and may be changed with further commits. The
internal handling of wildcards and file detection has been completely
re-written from scratch. For example, previously when multi-level
wildcards were used these were not reliably detected. The code also
now provides much of the same functionality in all modes, most importantly
wildcards are now also supported in polling mode.
The refactoring sets ground for further enhancements and smaller
refactorings. This commit provides the same feature set that imfile
had previously and all existing CI tests pass, as do some newly
created tests.
Some specific changes:
- bugfix: module parameter "sortfiles" ignored
This parameter only works in Solaris FEN mode, but is otherwise
ignored. Most importantly it is ignored under Linux.
fixes https://github.com/rsyslog/rsyslog/issues/2528
- bugfix: imfile did not pick up all files when not present
at startup
fixes https://github.com/rsyslog/rsyslog/issues/2241
fixes https://github.com/rsyslog/rsyslog/issues/2230
fixes https://github.com/rsyslog/rsyslog/issues/2354
- bugfix: directories only support "*" wildcard, no others
fixes https://github.com/rsyslog/rsyslog/issues/2303
- bugfix: parameter "sortfiles" did only work in FEN mode
fixes https://github.com/rsyslog/rsyslog/issues/2528
- provides the ability to dynamically add and remove files via
multi-level wildcards
see also https://github.com/rsyslog/rsyslog/issues/1280
- the state file name currently has been changed to inode number
This will further be worked on in upcoming PRs
see also https://github.com/rsyslog/rsyslog/issues/2231
- some enhancements were also done to CI tests, most importantly
they were made more compatibile with BSD
Note that most of the mentioned bug fixes cannot be applied to older
versions, as they fix design issues which are solved by the refactoring.
Thus there are not separate commits for them.
Distro maintainers: you need to decide to apply this patch as whole
or not. Believe me, it is not worth the effort to try to extract
specific patches from this commit. There is a good reason we do
not have multiple commits.
closes https://github.com/rsyslog/rsyslog/issues/2359
rsyslog uses unsigned short for configuration setting
SystemLogRateLimitBurst. Being just 16 bits, unsigned short cannot
hold values bigger than 65535. On our LDAP server rsyslog misbehaved with
SystemLogRateLimitBurst being bigger than 65535. Make it unsigned int to
accept bigger values.
function modules add functions to rainerscript dynamically,
change http_request into such a module (enabled by default).
This module can be used as a sample for future function modules.
these test are still instable and some concerns about their
validity at all have been raised. We now disable them to make
the testbench more reliable. Root cause needs to be investigated
and tests finally fixed or removed.
see also https://github.com/rsyslog/rsyslog/issues/2403
some tests use rsyslog.out.* file names, which were so far not cleaned up.
This can lead to all sorts of problems, including failure of
"make distcheck" due to leftover files. We now delete all of these files
on cleanup. This should not have any undesired side-effects.
Thanks to Kasumi Hanazuki for spotting this problem.
see also https://github.com/rsyslog/rsyslog/pull/2527
this can happen if imfile reads a state file. On each open, memory for the
file name can be lost.
We detected this while working on imfile refactoring, there is no related
bug report. No specific test has been crafted, as the refactored imfile
tests catch it (as soon as they are merged).
