Rainer Gerhards ab1bd8c01b imfile: large refactoring of complete module
This commit greatly refactors imfile internal workings. It changes the
handling of inotify, FEN, and polling modes. Mostly unchanged is the
processing of the way a file is read and state files are kept.

This is about a 50% rewrite of the module.

Polling, inotify, and FEN modes now use greatly unified code. Some
differences still exists and may be changed with further commits. The
internal handling of wildcards and file detection has been completely
re-written from scratch. For example, previously when multi-level
wildcards were used these were not reliably detected. The code also
now provides much of the same functionality in all modes, most importantly
wildcards are now also supported in polling mode.

The refactoring sets ground for further enhancements and smaller
refactorings. This commit provides the same feature set that imfile
had previously and all existing CI tests pass, as do some newly
created tests.

Some specific changes:
- bugfix: module parameter "sortfiles" ignored
  This parameter only works in Solaris FEN mode, but is otherwise
  ignored.  Most importantly it is ignored under Linux.
  fixes https://github.com/rsyslog/rsyslog/issues/2528
- bugfix: imfile did not pick up all files when not present
  at startup
  fixes https://github.com/rsyslog/rsyslog/issues/2241
  fixes https://github.com/rsyslog/rsyslog/issues/2230
  fixes https://github.com/rsyslog/rsyslog/issues/2354
- bugfix: directories only support "*" wildcard, no others
  fixes https://github.com/rsyslog/rsyslog/issues/2303
- bugfix: parameter "sortfiles" did only work in FEN mode
  fixes https://github.com/rsyslog/rsyslog/issues/2528
- provides the ability to dynamically add and remove files via
  multi-level wildcards
  see also https://github.com/rsyslog/rsyslog/issues/1280
- the state file name currently has been changed to inode number
  This will further be worked on in upcoming PRs
  see also https://github.com/rsyslog/rsyslog/issues/2231
- some enhancements were also done to CI tests, most importantly
  they were made more compatibile with BSD

Note that most of the mentioned bug fixes cannot be applied to older
versions, as they fix design issues which are solved by the refactoring.
Thus there are not separate commits for them.

Distro maintainers: you need to decide to apply this patch as whole
or not. Believe me, it is not worth the effort to try to extract
specific patches from this commit. There is a good reason we do
not have multiple commits.

closes https://github.com/rsyslog/rsyslog/issues/2359
2018-03-22 08:25:47 +01:00
..
2016-06-21 14:19:57 +02:00
2017-12-21 17:09:32 +01:00
2017-04-13 14:07:38 +02:00
2016-06-21 11:28:09 +02:00
2017-01-25 13:20:07 -05:00
2017-05-09 17:21:20 +02:00
2016-10-11 14:49:58 +02:00
2016-10-11 14:49:58 +02:00
2017-12-24 11:46:57 +01:00
2017-12-02 18:26:39 +01:00
2017-12-02 18:26:39 +01:00
2017-12-21 17:09:32 +01:00
2016-07-26 17:11:51 +08:00

This directory contains the rsyslog testbench. It is slowly
evolving. New tests are always welcome. So far, most tests check
out the functionality of a single module. More complex tests are
welcome.

For a simple sample, see rtinit.c, which does a simple
init/deinit check of the runtime system.

Test Naming
===========

Test that use valgrind shall end in "-vg.sh".
Test that use valgrind's helgrind thread debugger shall end in "-vgthread.sh".

Setting up Test Environments
============================

Setting up MariaDB/MySQL
------------------------
to create the necessary user:

echo "create user 'rsyslog'@'localhost' identified by 'testbench';" | mysql -u root
mysql -u root < ../plugins/ommysql/createDB.sql
echo "grant all on Syslog.* to 'rsyslog'@'localhost';" | mysql -u root

openSUSE
--------
To configure system properties like hostname and firewall, use the
graphical "yast2" administration tool. Note the ssh-access by default
is disable in the firewall!

Core Dump Analysis
==================
The testbench contains some limited (yet useful) support for automatically
anaylzing core dumps. In order for this to work, obviously core files need
to be generated. This often doesn't work as intended. If you hit this problem,
check

1. ulimit -c unlimited (or a reasonable limit)
   Note that root may need to increase a system-wide limit, which is
   usually recorded in /etc/security/limits.conf
   You need:
   *     soft    core      unlimited

2. cat  /proc/sys/kernel/core_pattern"
   On systemd systems (and some others), the pattern is changed to save
   core files so that systemd can import them -- with the result that the
   testbench doesn't see them any longer. We require classic format, which
   can be set via
   $ sudo bash -c "echo \"core\" > /proc/sys/kernel/core_pattern"

Note that you probably want to do neither of these changes to a production
system.