Rsyslog may leave some dangling disk queue files under the following
conditions:
- batch sizes and/or messages are large
- queue files are comparatively small
- a batch spans more than two queue files (from n to n+m with m>1)
In this case, queue files n+1 to (n+m-1) are not deleted. This can
lead to problems when the queue is re-opened again. In extreme cases
this can also lead to stalled processing when the max disk space is
used up by such left-over queue files.
Using defaults this scenario is very unlikely, but it can happen,
especially when large messages are being processed.
The test was timing-sensitive as we did not properly check all data
was output to the output file - we just relied on sleep periods.
This has been changed. Also, we made some changes to the testing
framework to fully support sequence checking of multiple ZIP files.
miscellaneous bug fixes in improg:
- properly truncate string after an input event is submitted
- set msgoffset to 0.
- tests added to check above fixes
The test frequently failed in a way that suggests rsyslog terminated too
early. We try if we can improve the situation by prolonging the empty
check. While there are better solutions for this special case, there are
some cases where we do no have them. So it is good to know if this
method works.
remove trailing whitespace before checking the status string. This is
most important as a line usually ends with \n, which is considered
trailing whitespace. Accepting this increases usability.
We have added new capabilities to the testbench plumbing to automatically
deal with gzip-compressed files. This also permits to use the wait_seq_check
function to work for gzip tests as well. The known-timing-sensitive
gzipwr_large test now makes use of the new capabilities. This enables us
to more reliably detect when we can savely shutdown the tested instance.
This commit also adds an ability to "abort" the full testbench run on
first test failure. This is especially useful during CI.
when multiple interfaces and/or protocols could be bound, each of
them used a different listener ports were assigned. While this is
basically correct, it makes things unusable, especially as
listenPortFileName will only contain the port number used for
the latest listener.
This patch now follows the model of nsd_ptcp.c to assign only
the first port randomly and then use that port consistently.
imptcp counts the number of bytes received. However, receives
happen on different worker thread. The access to the counter
was not synchronized, which can cause loss of updates. Also,
thread debuggers validly flag this as an error, which creates
problems under CI.
This commit fixes the situation via atomic operations and
falls back to mutex calls if they are not available.
Detected by LLVM thread sanitizer.
closes https://github.com/rsyslog/rsyslog/issues/3798
