2543 Commits

Author SHA1 Message Date
Rainer Gerhards
3ca969fae8
Merge pull request #4106 from rgerhards/tb-ci-runner-test
CI: add script to do a full CI run
2020-01-11 16:01:38 +01:00
Rainer Gerhards
0845ff8e02
CI: add script to do a full CI run
The core ID is to use this script together with pretty generic
buildbot workers. This provides a simple way to load-balance
across the worker pool.
2020-01-11 12:10:48 +01:00
Rainer Gerhards
0004185c48
testbench: modernize some tests
also make more robust against slow testbench machines
2020-01-10 10:52:06 +01:00
Rainer Gerhards
8dbb33cd03
testbench: fix races in old-style tests
these were introduced with commit fcffb063e just recently
2020-01-08 16:07:02 +01:00
Rainer Gerhards
6f74f7e7b4
Merge pull request #4091 from alorbach/pr-issue-4035
ossl tls: Added support to configure certificate verify depth
2020-01-08 15:44:57 +01:00
Rainer Gerhards
52b609869d
Merge pull request #4093 from rgerhards/tb-q-shutdown-to
testbench: increase main msg queue default shutdown timeout
2020-01-08 15:44:33 +01:00
Rainer Gerhards
97ab992f4f
Merge pull request #4103 from rgerhards/imtcp-legacy-listenportfile
imtcp: add legacy directive $inputtcpserverlistenportfile
2020-01-08 15:44:16 +01:00
fcffb063e3 tls: Added support to configure certificate verify depth
Support added in omfwd as instance parameter:
streamdriver.TlsVerifyDepth
Support added in imtcp as module parameter:
streamdriver.TlsVerifyDepth

Can be 2 or higher.

Support added into ossl driver
Support added into gtls driver
Added testcases for both drivers.

Parameter
streamdriver.TlsVerifyDepth

closes: https://github.com/rsyslog/rsyslog/issues/4035
2020-01-08 11:40:14 +01:00
Rainer Gerhards
2377bfa0e7
imtcp: add legacy directive $inputtcpserverlistenportfile
This is added for testbench use - we need it to harden tests that
need to test legacy syntax.

The new directive is NOT intended for users. Thus it is intentionally
NOT DOCUMENTED.
2020-01-08 11:38:20 +01:00
Rainer Gerhards
dffb2bb8f7
testbench: modernize tests
also make them more robust against slow testbench machines
2020-01-08 10:22:29 +01:00
Rainer Gerhards
f2560d5229
Merge pull request #4090 from rgerhards/tb-robust2
testbench: harden two tests against slow testbench machines
2020-01-08 09:22:06 +01:00
Rainer Gerhards
73941eaad2
Merge pull request #4094 from rgerhards/tb-timeout
testbench: do not create hanging tool instances
2020-01-08 09:20:52 +01:00
Rainer Gerhards
77ee19d9b0
testbench: increase main msg queue default shutdown timeout
this is necessary for very slow test environments
2020-01-08 08:53:36 +01:00
Rainer Gerhards
a03855765e
Merge pull request #4096 from rgerhards/tb-kafkacache
testbench: use local kafka download cache if present
2020-01-07 16:33:40 +01:00
Rainer Gerhards
bc71ab0738
testbench: use local kafka download cache if present 2020-01-07 11:13:17 +01:00
Rainer Gerhards
c8461990f6
testbench: do not create hanging tool instances
the omhttp test server hangs indefinitely if tests requiring
it fail. This is relevant if the environment ist not destructed
after CI run (the buildbot CI system does not do this on VMs).

To guard against this, we have added a timeout of 30 minutes to
ensure the tool is terminated.
2020-01-06 17:05:12 +01:00
Rainer Gerhards
92e0a75e99
testbench: harden two tests against slow testbench machines 2020-01-06 12:53:00 +01:00
Rainer Gerhards
77180c9b08
testbench: make imdocker tests more robust in regard to timing
some have problems on heavily loaded (CI) machines. This commit solves
that problem.
2020-01-06 12:52:18 +01:00
Rainer Gerhards
8be4e37df6
testbench: modernize some tests and make more robust
slow test machines are now better handled
2020-01-03 11:45:55 +01:00
Rainer Gerhards
58e49ae07b
testbench cleanup: remove debug info
mysqld-start script always did an SKIP, so that we could check it's outcome.
This is a left-over from previous debugging. Thus removing it.
2020-01-02 10:38:52 +01:00
Rainer Gerhards
3860122f41
testbench: improve speed of elasticsearch tests
we avoid unnecessary ElasticSearch restarts now. This also helps with flakes
on slow CI machines.
2020-01-01 12:23:50 +01:00
Rainer Gerhards
879fea33ff
testbench: rough fix for clickhouse 2019 test data problem
not a real fix, but to get us going: clickhouse test had hardcoded
year; needs to be removed soon. Right now, just pusing it to 2020.
2020-01-01 11:08:45 +01:00
Rainer Gerhards
4d69817808
config system: emit proper error message on $ in double-quoted string
closes https://github.com/rsyslog/rsyslog/issues/2869
2019-12-30 17:39:04 +01:00
Rainer Gerhards
394232f3a0
testbench: modernize some tests 2019-12-27 14:14:54 +01:00
Josh Soref
af19128573 fix spelling errors in testbench 2019-12-27 13:57:25 +01:00
Rainer Gerhards
40583f6d35
Merge pull request #4067 from rgerhards/i4043
core bugfix: APP-NAME fields could become empty
2019-12-27 11:57:00 +01:00
Rainer Gerhards
8d3da211c4
core bugfix: APP-NAME fields could become empty
RFC 5424 specifies that an empty APP-NAME needs to be indicated by
"-". Instead, the field could become empty under certain conditions.
If so, outgoing 5424 messages were invalidly formatted.

This happened under quite unusual conditions, but could be seen
in practice.

This commit also does some very light non-related code improvement
and also includes the testbench test to check the fixed error condition.

closes https://github.com/rsyslog/rsyslog/issues/4043
2019-12-27 10:30:27 +01:00
Josh Soref
9c67c4db9f fix spelling errors in text files 2019-12-26 17:37:14 +01:00
Rainer Gerhards
dd14b03d33
Merge pull request #4059 from rgerhards/tb-robust3
testbench: harden tests against port-assignment races
2019-12-23 11:14:11 +01:00
Rainer Gerhards
7b535d38ff
Merge pull request #4047 from rgerhards/travis-new-compilers
new default development container and update Travis to new compilers
2019-12-20 18:58:50 +01:00
Rainer Gerhards
d40484e225
testbench: harden tests against port-assignment races 2019-12-20 16:45:48 +01:00
Rainer Gerhards
e7541dcd0f
new default development container and update Travis to new compilers 2019-12-20 16:13:28 +01:00
Rainer Gerhards
079ce47f92
testbench: modernize test and make more robust in regard to timing 2019-12-19 12:59:53 +01:00
Rainer Gerhards
f557b5c538
testbench: test imtcp "addtlFrameDelimiter" module param
so far, only the legacy style equivalent was tested. We have changed this
to current style. Legacy is no longer tested to keep CI runtime low.

The new test also has been made more reliable than the previous one.
2019-12-18 19:28:38 +01:00
Rainer Gerhards
233fde2ec9
Merge pull request #3962 from alorbach/pr-issue-3959
relp: Add support setting openssl configuration commands.
2019-12-18 19:05:18 +01:00
Rainer Gerhards
c967622fdc
testbench: cleanup no longer needed debug instrumentation 2019-12-18 16:17:22 +01:00
Rainer Gerhards
7daed87a8a
testbench: add test for config param "config.enabled"
Make existing test more reliable and cover new bug scenario.

see also https://github.com/rsyslog/rsyslog/issues/4011
2019-12-09 17:43:53 +01:00
fd6c32c2f2 testbench: Add two real tests for omsnmp
Adds a python based SNMP Trap Receiver which outputs received
SNMP Traps by UDP onyl at the moment. Can be extended later.
Tests results are verified by content_check.

The tests have to be enabed with ./configure --enable-snmp-tests

Test sndrcv_omsnmpv1_udp_dynsource.sh checks the new snmpv1dynsource
option.

In order to work, the following python packages needs to be installed:
	pip install pysnmp

On Ubuntu 18, we need these packages to be installed:
	apt install snmp libsnmp-dev snmp-mibs-downloader

closes: https://github.com/rsyslog/rsyslog/issues/3985
2019-12-03 13:26:36 +01:00
2bd6c99cbe relp: Add support setting openssl configuration commands.
Add new configuration parameter tls.tlscfgcmd to omrelp and imrelp.
(Using relpSrvSetTlsConfigCmd and relpCltSetTlsConfigCmd)

OpenSSL Version 1.0.2 or higher is required for this feature.
A list of possible commands and their valid values can be found in the
documentation: https://www.openssl.org/docs/man1.0.2/man3/SSL_CONF_cmd.html

The setting can be single or multiline, each configuration command is
separated by linefeed (n). Command and value are separated by
equal sign (=). Here are a few samples:

tls.tlscfgcmd="Protocol=ALL,-SSLv2,-SSLv3,-TLSv1,-TLSv1.2"

tls.tlscfgcmd="Protocol=ALL,-SSLv2,-SSLv3,-TLSv1
MinProtocol=TLSv1.2"

Add to new testcases for librelp and tlscfgcmd.

closes https://github.com/rsyslog/rsyslog/issues/3959
2019-11-29 16:00:27 +01:00
Rainer Gerhards
8945dcfd02
Merge pull request #3999 from rgerhards/tb-python3-3
testbench: fix some issue finding correct python interpreter
2019-11-29 15:54:52 +01:00
Rainer Gerhards
0bb33d153e
testbench: fix some issue finding correct python interpreter
see also https://github.com/rsyslog/rsyslog/issues/3853
2019-11-29 14:06:41 +01:00
Rainer Gerhards
0421e3c34d
testbench: fix some python env-induced issues
see also https://github.com/rsyslog/rsyslog/issues/3853
2019-11-29 13:15:25 +01:00
Rainer Gerhards
aa0f2d6cfc
Merge pull request #3997 from rgerhards/python3
testbench: improve python3 support
2019-11-29 08:48:06 +01:00
Rainer Gerhards
2869f471d6
testbench: improve python3 support 2019-11-28 13:49:09 +01:00
Rainer Gerhards
a6d6101916
testbench: make some tls/ossl tests more robust
This commit makes them use the reliable method to obtain a
dynamic port. The previous method lead to unreliability (flakes)
especially on busy machines.
2019-11-27 10:15:26 +01:00
Rainer Gerhards
0307a0cc0a
testbench: improve mysql test framework plumbing
This is an initial try to make mysql tests able to run in parallel,
which also hardens them against some timing problems.
2019-11-26 10:56:33 +01:00
Rainer Gerhards
771fc80112
Merge pull request #3982 from dmolik/omhttp-loki
omhttp: add basic support for Loki Rest
2019-11-22 16:41:24 +01:00
Rainer Gerhards
35e3200566
Merge pull request #3984 from rgerhards/tb-am_path_python
testbench: obtain python binary path via AM_PATH_PYTHON
2019-11-22 16:11:33 +01:00
Dan Molik
8793a88987
omhttp: add basic support for Loki Rest
Loki is a new message indexer and querier from Grafana Labs. See
https://github.com/grafana/loki for details on Loki.

This change provides the initial message structure to send bulk message
payloads to the Loki Rest endpoint. omhttp, recieved a new bulk message
format called lokirest. Additionally, the plugin relies on the user to
provide the correct "stream" read message format.

A loki template must be json compatible and include a "stream" key of
key value tags, and a values key of an array of 2 element arrays, where
each 2 element array is the unix epoch in nanoseconds followed by an
unstrectured message.

An example:

    template(name="array_loki" type="string" string="{\"stream\":{\"host\":\"%HOSTNAME%\",\"facility\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"syslogtag\":\"%syslogtag%\"},\"values\": [[ \"%timegenerated:::date-unixtimestamp%000000000\", \"%msg%\" ]]}")
2019-11-22 09:03:10 -05:00
Rainer Gerhards
fd0bff3a88
Merge pull request #3977 from rgerhards/i3975
omprog: detect violation of interface protocol
2019-11-22 14:45:49 +01:00