mirror of
https://github.com/rsyslog/rsyslog.git
synced 2025-12-13 04:50:41 +01:00
cd26aecaf9
Non-technical: test flakiness makes it hard to validate unrelated changes. This aligns omelasticsearch tests with ES 7.14 defaults to get the testbench back to a deterministic state and pave the way for further modernization. Impact: test behavior changes; one test skipped; CI coverage slightly reduced. Before: tests mixed ES 6-era types and ad-hoc tarball picks; deprecation checks intermittently failed and retries were brittle. After: tests use the 7.14.1 tarball via diag.sh default, typeless mappings, and `_doc` type in omelasticsearch actions; known-flaky bulk-retry test is skipped for now. Technical details: - Add `searchType="_doc"` to all omelasticsearch actions and update index provisioning to typeless mappings compatible with ES 7.14. - Remove script-level `ES_DOWNLOAD` overrides to follow diag.sh's 7.14.1 default, keeping test scripts and helper defaults in sync. - Drop deprecation-log assertion in `es-searchType-empty.sh` to prevent spurious failures specific to ES 6-era types. - Temporarily skip `es-bulk-retry.sh` (exit 77) pending a rewrite of retry semantics under ES 7.x. - CI: export `VERBOSE=1`; disable Kafka and Elasticsearch tests in the affected matrix job to keep CI green while ES/Kafka suites are refactored. VERBOSE ensures test logs are emitted to stderr and as such are visible in CI test runs. - Minor whitespace/indent cleanups; no runtime code or plugin behavior changes.
52 lines
2.0 KiB
Bash
Executable File
52 lines
2.0 KiB
Bash
Executable File
#!/bin/bash
|
|
# This test stresses the DA queue disk subsystem with multiple threads.
|
|
# To do so, the in-memory queues are deliberately sized very small.
|
|
# NOTE: depending on circumstances, this test frequently starts the
|
|
# DAWorkerPool, which shuffles messages over from the main queue to
|
|
# the DA queue. It terminates when we reach low water mark. This can
|
|
# happen in our test. So the DA worker pool thread is, depending on
|
|
# timing, started and shut down multiple times. This is not a problem
|
|
# indication!
|
|
# The pstats display is for manual review - it helps to see how many
|
|
# messages actually went to the DA queue.
|
|
# Copyright (C) 2019-10-28 by Rainer Gerhards
|
|
# This file is part of the rsyslog project, released under ASL 2.0
|
|
. ${srcdir:=.}/diag.sh init
|
|
export ES_PORT=19200
|
|
export NUMMESSAGES=25000
|
|
export QUEUE_EMPTY_CHECK_FUNC=es_shutdown_empty_check
|
|
ensure_elasticsearch_ready
|
|
generate_conf
|
|
add_conf '
|
|
global(workDirectory="'$RSYSLOG_DYNNAME'.spool")
|
|
template(name="tpl" type="string" string="{\"msgnum\":\"%msg:F,58:2%\"}")
|
|
|
|
main_queue(queue.size="2000")
|
|
module(load="../plugins/impstats/.libs/impstats"
|
|
log.syslog="off" log.file="'$RSYSLOG_DYNNAME'.pstats" interval="1")
|
|
module(load="../plugins/imtcp/.libs/imtcp")
|
|
module(load="../plugins/omelasticsearch/.libs/omelasticsearch")
|
|
input(type="imtcp" port="0" listenPortFileName="'$RSYSLOG_DYNNAME'.tcpflood_port")
|
|
:msg, contains, "msgnum:" {
|
|
action(type="omelasticsearch" name="act-es"
|
|
template="tpl" server="127.0.0.1"
|
|
serverport="'$ES_PORT'"
|
|
searchIndex="rsyslog_testbench"
|
|
bulkmode="on"
|
|
queue.lowwatermark="250"
|
|
queue.highwatermark="1500"
|
|
queue.type="linkedList" queue.size="2000"
|
|
queue.dequeueBatchSize="64" queue.workerThreads="4"
|
|
queue.fileName="actq" queue.workerThreadMinimumMessages="64")
|
|
}
|
|
'
|
|
startup
|
|
tcpflood -m$NUMMESSAGES # use tcpflood to get better async processing than injectmsg!
|
|
shutdown_when_empty
|
|
wait_shutdown
|
|
echo FOR MANUAL REVIEW: pstats
|
|
tail $RSYSLOG_DYNNAME.pstats | grep maxqsize
|
|
es_getdata $NUMMESSAGES $ES_PORT
|
|
seq_check
|
|
exit_test
|