https://github.com/rsyslog/rsyslog/issues/3573
Previously, when constructing the message to submit for a retry
for an original request, if the original request did not contain
the field `message`, the system property `rawmsg` was set to
the entire metadata + data from the original request. This was
causing problems with Elasticsearch. This patch changes
the code so that the `rawmsg` will be set to only the data part
of the original request if there is no `message` field.
We now no longer depend on a fixed 'sleep' command but rather
check the output file for what we expect. This is much more
robust on slow test machines.
We believe this closes the below-mentioned issue. If not, it
should be re-opened.
closes https://github.com/rsyslog/rsyslog/issues/3104
Also now permit interactivly running tests without explicitly setting
$srcdir. This now works if we are inside ./tests and fails, as before,
when we are in a different directory.
Detected by shellcheck via CodeFactor.io
This is required to support parallel test runs.
Among others, make thise files dynamic:
* test-spool
* rsyslog.input
* rsyslog.out*.log
* tmp.in
Also:
* convert presort test statement to function
* cleanup imfile truncation test
* cleanup imfile-growing-file-id test
some cruft was left due to copy and paste error
* serialize mysql tests
- DEAD_PORT now uses unassigned IANA port unlike to be used on the system
(dynamic port querying is racy and we had at least once an issue, so we
can remove ambiguity here easily)
- replace some diag.sh commands by bash functions
Write all of the original request metadata fields to $.omes for
the retry, if present. This may include all of the following:
_index, _type, _id, _parent, pipeline
This is in addition to the fields from the response. If the same
field name exists in the request metadata and the response, the
field from the request will be used, in order to facilitate
retrying the exact same request.
Have to set a tag with `MsgSetTAG` to avoid a UBSAN error.
changes some of the test commands to use bash functions
includes some small bug fixes to tests where bugs were
previously not seen due to different plumbing.
This makes the ES tests more universally available (they should now
also support running in containers). It also simplifies the tests
as fewer support files are needed.
Add support for a 'create' write operation type in addition to
the default 'index'. Using create allows specifying a unique id
for each record, and allows duplicate document detection.
Add support for checking each record returned in a bulk index
request response. Allow specifying a ruleset to send each failed
record to. Add a local variable `omes` which contains the
information in the error response, so that users can control how
to handle responses e.g. retry, or send to an error file.
Add support for response stats - count successes, duplicates, and
different types of failures.
Add testing for bulk index rejections.
