so that we get a better sense of execution time in failed tests
this is important to understand implications of slow test machines
and what we need to fix to handle that. Code is probably to be
extended if it proves useful.
We now no longer depend on a fixed 'sleep' command but rather
check the output file for what we expect. This is much more
robust on slow test machines.
We believe this closes the below-mentioned issue. If not, it
should be re-opened.
closes https://github.com/rsyslog/rsyslog/issues/3104
- the test now tries to detect unavailable http server, which
should not result in test failure
- equivalent valgrind test changed to new methode, removing code
duplication
- testbench supports
* new exit code 177, which indicates environment error, makes
test SKIP but still reports the failure
* new exitcode, logurl stats reporting fields
* report buildbot builder (if provided) in failure report
improving both style and reliability of journal tests; along that way
also improve testbench framework:
- do cleanup on error_exit and skip
- explicit skip handler (vs exit 77)
this permits us to do better cleanup
- new testbench functions for journal-specific functionality
reduce code duplication and make things easier to maintain in the
future
- provide a way to do valgrind and non-valgrind tests with a single
test file
see also https://github.com/rsyslog/rsyslog/issues/2564
working on removing potential race between journal forward latency
and rsyslog shutdown. Also adding additional debug information to
improve understanding of potentially upcoming new test failures.
see also https://github.com/rsyslog/rsyslog/issues/2564
- duplicate diag.sh init call was not detected due to typo
- queue-persits test did not work correctly
- some general testbench framework improvements
issues found be shellcheck, fixes brought up other work to do
... and also fix some issue in test setup. This means the test has been
rewritten and now also support current script syntax. Bufferred i/o was
probably only used because of where it was copied from, and it was used
in a way that does not seem correct (iotimeout was set, but background
writing not activated).
zookeeper startup was not actually checked, it was just hoped
that it started within 2 seconds (sleep). On busy systems, this
may be too short, thus results in an improper start and thus can
result in follow-on problems during the test.
We have now corrected it so that we wait until we see a pid. It's
still questionable if this is sufficient to conclude zookeeper is
really ready for processing. But let's take one step after the
other.
see also https://github.com/rsyslog/rsyslog/issues/3057
This permits to flag some tests as being unreliable. If done so, CI
can treat them specially. This is meant to get rid of time-consuming
issues with known problematic tests, but still retain the ability to
gather trouble-shooting information. Later commits may extend the
functionality.
The test often misses messages. One idea is that rsyslog is actually
terminated too early, and thus messages are missing. To proove that idea,
we now explicitely wait for all messages to be processed.
see also https://github.com/rsyslog/rsyslog/issues/3057
