2718 Commits

Author SHA1 Message Date
Rainer Gerhards
6e810d3f5e
CI: run testbench on ubuntu 20 with github action 2020-09-07 11:21:30 +02:00
Rainer Gerhards
e0b8634010
testbench fix: python 3 incompatibility 2020-09-03 12:58:57 +02:00
Rainer Gerhards
d23e73c782
Merge pull request #4394 from rgerhards/i4282
Queue: add test for segfault issue 4282
2020-09-02 10:15:06 +02:00
Rainer Gerhards
6763185783
core bugfix: segfault if disk-queue file cannot be created
When using Disk Queue and a queue.filename that can not be created
by rsyslog, the service does not switch to another queue type as
supposed to and crashes at a later step.

closes: https://github.com/rsyslog/rsyslog/issues/4282
2020-09-02 09:20:10 +02:00
Rainer Gerhards
72e91f6109
Merge pull request #4402 from rgerhards/TB-simplify
testbench: simplify test ID generation a bit
2020-09-01 15:05:13 +02:00
Rainer Gerhards
19e80e9e5c testbench: simplify test ID generation a bit 2020-09-01 14:33:41 +02:00
9513b7aa40 testbench: Add helpertool to check for minimum librelp version
- add testcases for imrelp / omrelp chained certificates
  for new librelp version 1.7.0
- add chained certificate based on existing certs

closes: https://github.com/rsyslog/rsyslog/issues/4388
2020-08-31 11:39:25 +02:00
Rainer Gerhards
89990c2a36
Merge pull request #4370 from splbio/imptcp_max_sessions
Add max sessions for imptcp.c similar to imtcp.c to prevent OOM
2020-08-25 10:30:16 +02:00
Nelson Yen
6102f6083c initial implemenation http input module
- uses http library to provide http input.
user would need to configure an 'endpoint' as input, along
with a ruleset, defining how the input should be routed in
rsyslog.

bugfix, free dynamic buf if created, don't echo to client data.

do data framing using newline character.
more input options support:
  - input name
  - flowcontrol

add support for gzip content support, and other options

- gzip content
- parse linefeeds by default, option to ignore linefeeds

add support for gzip content support, and other options

- gzip content
- parse linefeeds by default, option to ignore linefeeds

WIP - misc edits enable port, documentroot module parameter

fix overloaded stack issue

update tests to use available port instead of hard-coded one.

Add 'octet counted framing' support

- option is "SupportOctetCountedFraming", currently 'off' by
default.

update imhttp-getrequest-file.sh test to use $srcdir

imhttp - support multi-threaded connection contexts.

- tests for large data posts

- Add header data as metadata option

move mg_start into activatecnf instead of in runinput
2020-08-23 07:24:48 -07:00
Rainer Gerhards
5d78a8a5dd
Merge pull request #4380 from rgerhards/i3698
Increase timeouts in imfile-basic-2GB-file and imfile-truncate-2GB-file
2020-08-21 09:32:42 +02:00
Michael Biebl
effe40d4b4
Increase timeouts in imfile-basic-2GB-file, imfile-truncate-2GB-file
Those tests can take a long time, especially on slow architectures like
armhf, so bump the test timeouts considerably.

closes https://github.com/rsyslog/rsyslog/issues/3698
2020-08-20 15:19:02 +02:00
Rainer Gerhards
e366276aca
testbench: add additional debug support for hard to find issue
Some imfile tests have a very hard to find sporadic failure, which
is irreproducible on a regular workstation. This enhances the
testbench's debug support in the hope to provide better diagnostics
when the issue occurs in regular CI runs.
2020-08-14 15:20:54 +02:00
Alfred Perlstein
6c50426dbf Add max sessions for imptcp.c similar to imtcp.c
The max is per-instance, not global across all instances.

There is also a bugfix where if epoll failed I think we could leave a
session linked in the list of sessions, this code unlinks it.
2020-08-11 19:06:37 -07:00
Rainer Gerhards
99d950203b
Merge pull request #4343 from VultureProject/impcap-merge-upstream
IMPCAP::Fixes: segfault, memory and build corrections (resolves #4332)
2020-08-06 14:52:07 +02:00
4e5741807e testbench: set msg size to 64kb for sndrcv_omudpspoof-bigmsg.sh test 2020-07-20 08:35:57 +02:00
Rainer Gerhards
94c4a871d0
imklog: implement ruleset support
see also: https://github.com/rsyslog/rsyslog/issues/4344#issuecomment-658001854
see also: https://github.com/rsyslog/rsyslog/issues/106
2020-07-14 10:38:42 +02:00
frikilax
62dd906245 IMPCAP::Fixes: segfault, memory and build corrections (resolves #4332)
- fix bug in ethernet packets parsing
  - fix removes build error with gcc10: 'multiple definition of...'
- resolve memory leak during interface init failure (device not freed after post-create error)
- add test 'impcap_bug_ether' to prove ethernet parser fix is working
2020-07-06 10:49:47 +02:00
Rainer Gerhards
8ff2179304
Merge pull request #4321 from alorbach/pr-issue-4319
openssl: Fixed memory leak when tls handshake failed.
2020-06-22 10:45:07 +02:00
6d0fea9b69 openssl: Fixed memory leak when tls handshake failed.
Added testcase for ossl memory leak on failed handshake.

closes: https://github.com/rsyslog/rsyslog/issues/4319
2020-06-09 10:01:53 +02:00
frikilax
76318ad926 NEW INPUT MODULE:: impcap, network packets input parser + TESTS 2020-06-05 17:41:01 +02:00
Rainer Gerhards
d3f3cf54b4
testbench: add test case for cat non-existing-file
see also https://github.com/rsyslog/rsyslog/issues/4290
2020-05-22 10:13:30 +02:00
82621cdd14 imtcp: fix octet framing/stuffing problem with discardTruncatedMsg on
When "discardTruncatedMsg" is enabled in imtcp, messages were incorrectly
skipped if the last character before the truncation was the LFdelimiter.

Add two tests for octet stuffing and framing.

closes: https://github.com/rsyslog/rsyslog/issues/4281
2020-05-15 09:56:44 +02:00
Rainer Gerhards
0ae9a7a3d0
Merge pull request #4193 from jmarckel/bugfix/bug_2133
imjournal: remove strcat call
2020-05-12 12:51:10 +02:00
Rainer Gerhards
da7ca37c10 CI: fix problems with ElasticSearch download
ElasticSearch can no longer be reliably downloaded, we receive error
451 - prohibited for legal reasons. This causes false positive test
failures.

We have solved this by moving the required tarballs to rsyslog.com and
download them now from there.
2020-05-12 11:09:42 +02:00
Rainer Gerhards
6aed1b4975
Merge pull request #4219 from callmegar/master
omhttp: Adding multiple http headers capability
2020-05-11 12:12:55 +02:00
Rainer Gerhards
36725afe2c
Merge pull request #4256 from jsiwrk/feature/adapt_omprog_test
omprog: adapt test to testbench improvements
2020-04-29 17:35:00 +02:00
Rainer Gerhards
f0d71ce408
Merge pull request #4263 from alorbach/pr-issue-4131
SKIP librelp tlscmd tests if librelp was build without openssl support
2020-04-29 13:29:15 +02:00
Rainer Gerhards
589eac07b2
Merge pull request #4240 from alorbach/pr-issue-4005
testbench: Add test for imrelp to check broken session handling.
2020-04-29 12:03:16 +02:00
Rainer Gerhards
67934f1feb
Merge pull request #4254 from alorbach/pr-issue-4238
omudpspoof: Fixed issues with oversized messages (Exceeding the configured MTU setting).
2020-04-29 11:42:50 +02:00
15f5a1bda2 SKIP librelp tlscmd tests if librelp was build without openssl support
closes https://github.com/rsyslog/rsyslog/issues/4131
2020-04-29 09:26:25 +02:00
Joan Sala
0e49c8d1f8 omprog: fix assert failed on HUP with output flag
If the 'output' setting of omprog was used and rsyslog received a HUP
signal just after starting (and before the omprog action received the
first log to process), an internal assertion could fail, causing
rsyslog to terminate. The failure message was "rsyslogd: omprog.c:660:
closeOutputFile: Assertion `pCtx->bIsRunning' failed."

The failure could also occur if rsyslog received a HUP signal during
the shutdown sequence.

This bug was introduced in v8.2004 by PR #4255.

Although a test already existed that checked the interaction of HUPs
with the 'output' setting, it didn't always fail in this particular case
due to timing conditions. The test has been improved to cover this case
more reliably.
2020-04-29 01:18:25 +02:00
Joan Sala
abc2f83943 omprog: adapt test to testbench improvements
Adapt one of the omprog tests to the latest improvements in the
testbench framework, for consistency with other similar tests. The test
behaviour has not changed.

Add some temporary files created by the testbench to gitignore file.
2020-04-28 21:04:15 +02:00
Rainer Gerhards
686835c880
Merge pull request #4234 from alorbach/pr-issue-4233
stream.c: Moved doSizeLimitProcessing check to strmWrite
2020-04-28 10:26:57 +02:00
3c0f642740 omudpspoof: Fixed issues with oversized messages
First issue was an incorrect packet length in UDP Header. It has to be the FULL UDP Packet
regardless of the MTU Setting. As a result regardless of IP fragmentation, the MTU setting
also limited the siizmax size of the UDP message.

The second issue was incorrect calculation of the UDP Checksum with libnet if
IP fragmentation was used (Based on MTU Setting). As a result, the network packets were
dropped by the tcp stack before they even could reach there target. The workarround for this
problem is, that we set the UDP Checksum to 0x0000 which allows skipping of the checksum
test. Fixing the problem by calculating the correct UDP Checksum would require some
code changes in the libnet.

Also fixed the omudpspoof bigmsg test and increased the testing size to 16KB.
2020-04-27 11:13:24 +02:00
e2a9e3da2d omudpspoof: Added test for oversized UDP messages (above 1500 MTU) 2020-04-27 11:13:24 +02:00
baee0bd542 testbench: Add test for imrelp to check broken session handling.
For successfull test, the PR
https://github.com/rsyslog/librelp/pull/184
 for Issue
https://github.com/rsyslog/librelp/issues/183
is needed. Otherwise the test will trigger a double free inside
librelp which results in a segfault in some cases.

closes: https://github.com/rsyslog/rsyslog/issues/4184
closes: https://github.com/rsyslog/rsyslog/issues/4005
2020-04-27 10:55:50 +02:00
Rainer Gerhards
02e5cbee91
testbench tooling: update kafka components
those used so far are outdated, bump to newest version
2020-04-24 08:53:17 +02:00
b84c9debea stream.c: Moved doSizeLimitProcessing check to strmWrite
The check was done in strmPhysWrite before which caused syslog
messages to split in the middle if the syslog message batch exceeded
the default IO Buffer size.

closes: https://github.com/rsyslog/rsyslog/issues/4233
2020-03-31 17:27:01 +02:00
Rodriguez,German J
95e830cc22
omhttp: Adding multiple http headers capability 2020-03-17 20:12:10 -05:00
901eb09860 openssl: add check for LIBRESSL
Disable use of "@SECLEVEL" in default cipher string and
avoid SSL_CONF_CTX_set_flags() API when LIBRESSL is used.
This means tlscommands will not work.

closes: https://github.com/rsyslog/rsyslog/issues/4210
2020-03-11 08:17:01 +01:00
Rainer Gerhards
6ef5b3e1db
Merge pull request #4208 from rgerhards/testbench-complex1
testbench: adjust queue enqueue timeouts to current standards
2020-03-04 15:52:28 +01:00
Rainer Gerhards
9753d18f1f
testbench: adjust queue enqueue timeouts to current standards
Timeouts were too short to take care of slow CI environments.
2020-03-04 12:38:15 +01:00
Rainer Gerhards
b17fec0c71
testbench: add some diagnostic info to analyze false positives
We have some false positives with imfile checks and this commit
both improves the testbench framework slightly and adds debug
info.

The debug info should be remove when we finally find the cause
of the issue, but it does not hurt if it stays for a quite a
while. Thus we can analyze the false positives over an extended
period of time - what is what it looks like we need to do to
find the root cause.
2020-03-04 10:51:22 +01:00
Jeff Marckel
f604d5269b imjournal: remove strcat call 2020-02-27 11:52:32 -06:00
Rainer Gerhards
4f2394a9d1
Merge pull request #4174 from jwslater0823/master
testbench: modify es_response_get_msgnum.py for python3
2020-02-24 08:47:49 +01:00
Rainer Gerhards
4d1f7bd11a
Merge pull request #4182 from alorbach/pr-issue-4170
openssl nsd driver: Switch to SSL_CTX_use_certificate_chain_file API
2020-02-24 08:43:03 +01:00
Rainer Gerhards
822ac8df1c
Merge pull request #4177 from alorbach/pr-issue-4134
testbench: Fixed two minor issues in omkafkadynakey.sh test.
2020-02-20 17:36:22 +01:00
Rainer Gerhards
04599b9de8
Merge pull request #4168 from alorbach/pr-issue-4158
testbench: increased max extra data length in tcpflood from 256 to 512KB
2020-02-20 17:34:27 +01:00
493b04b846 openssl nsd driver: Switch to SSL_CTX_use_certificate_chain_file API
Switched from SSL_CTX_use_certificate_file to SSL_CTX_use_certificate_chain_file
API for loading the certificate into the certificate store.

According to the openssl doc, the SSL_CTX_use_certificate_chain_file API has the
advantage that it can load the main certificate (First in the PEM) and many chain
certificates after that automatically.

See notes section for more:
https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_use_certificate_chain_file.html

closes: https://github.com/rsyslog/rsyslog/issues/4170
2020-02-19 14:26:25 +01:00
608c4907ab testbench: Fixed two minor issues in omkafkadynakey.sh test.
- topic check was done on wrong (hardcoded topic)
- rsyslog_out_log was overwritten by kafkacat check which caused the
  seq_check to fail at the end.

closes: https://github.com/rsyslog/rsyslog/issues/4134
2020-02-17 11:00:03 +01:00