Mmdblookup module used to suspend after erroring on an IPv6 IP lookup on IPv4-only DBs
This commit replaces the suspension of the module by a simple log message, allowing it to keep working for future lookups
- Extracted basic OpenSSL helper functions into own module net_ossl.h/net_ossl.c
Both are compiled into lmnsd_ossl.
- Cleanup of OpenSSL code, fixed minor compiler and linking issues.
- Added DTLS Sender option DTLS into tcpflood for testbench.
- Add initial implementation of imdtls input module. Added to configure and makefile
- Add initial implementation of omdtls output module. Added to configure and makefile
- Add multiple basic tests for imdtls receiving data by using tcpflood.
- Add multiple send-receive test for imdtls and omdtls based on existing tls tests.
- Add timeout and sessionbreak tests for imdtls stress testing.
closes: https://github.com/rsyslog/rsyslog/issues/5211
The previous test did not always detect an abort of rsyslog/omusrmsg.
The detection method has now been improved, so it is far more
probable that an abort is detected.
While doing this, we noticed that the omusrmsg-noabort-legacy test was
now a 100% duplicate. There is no need any longer to check pure legacy
syntax, and so that test has been removed.
We also added a valgrind-based test ofr omusrmsg-noabort, which furthers
strengthens bug detection. Most importantly, it helps us to detect
potentially new memory leaks on all CI platforms (in case the lib
behaves differently depending on os/distro).
see also https://github.com/rsyslog/rsyslog/issues/5294
- Added test for connection interrupts (requires root)
- Corrected handling of PN_TRANSPORT_CLOSED.
- Make sure Connection is being reestablished trough tryResume
- Enhanced Debug log output
closes: https://github.com/rsyslog/rsyslog/issues/5269
This provides richer and easier to process logs for error and warning
cases. One goal is to enable automatic operations without the need
to consolidate multiple message to a single information.
This improves one situation in gtls driver and provides a more
generic approach in ossl driver for OpenSSL error reporting.
There is probably still room for improvement, however this patch
is at least a good starting point for further work. Please
provide feedback if you need more!
closes https://github.com/rsyslog/rsyslog/issues/5244
The error code when plain tcp sending failed was improperly returned,
resulting in no meaningful error message.
Note: tcpflood is a testbench tool, not part of production rsyslog.
There was a race between tcpflood and rsyslog in imptpc_maxsessions.sh.
We now use the new -A tcpflood option to make the timing more
predictable, hopefully fixing test flakiness.
Note: if that does not help, we need to introduce a wait on the number
of error messages and maybe a delay before tcpflood termination. The
theory behind the latter is that rsyslog possibly does not fully
iniaitlize session which are quickly aborted before rsyslog receives
the related OS notification! We just record this info in case we
need it and are positive that this change will fix the situation.
Note: tcpflood is a testbench tool, not meant for production use
We add the -A option. If set, it does NOT abort tcpflood if
sending messages fails.
This is required for some tests which test connection closes.
If tcpflood terminates quickly before rsyslog could fully
initializes all sessions, the expected error message count is
not necessarily reached, which results in flakes. When tcpflood
contines to run, rsyslog has sufficient time (at least a race
is far less likely).
In commit 5050249a1e tcpflood was "fixed" to work properly when senden
data to a TCP stream socket. Unfortunately, there was an undetected
copy&past problem, which this commit corrects.
The issue did unfortunatley not surfcase with the previous commit as
the situation where this can happen occurs infrequently and randomly
(has to do with timing on the system in question). So the original PR
went fine, but later PRs brought the issue up.
This is not a real regression, as the original fix was for the same
symptom, but with different cause. Now everything should indeed be
fixed.
NOTE: same day correction on a testbench tool
In OpenSSL 1.1.0 and higher, SSLv23_method causes some errors
in TLS handshake from time to time. As this method is depreceated
since 1.1.0, I have replaced it with the follow up method
TLS_method which is the most generic one.
It fixes the random test failures in tests like
- sndrcv_tls_ossl_anon_rebind.sh
Also added some debug output in OpenSSL error handling, which is
useful when analysing debug files.
closes: ./sndrcv_tls_ossl_anon_rebind.sh
Not all imfile tests have state file directories or a global working
directory defined. This results in usage of the default location.
While state file names should be sufficiently different, there is still
some riks of using the same name in different tests. That becomes
problematic if tests are run in parallel (and they are run in
parallel inside the regular CI).
Note: tcpflood is a testbench tool. This bug could lead to testbench
false positives. No way it can affect production deployments.
The tcpflood tool did improperly assume that a TCP sendto() call
would send messages of any size in a single shot. This is not the
case. It has now been corrected to proper behavior.
As a side-activity, some int variables which acutally needed to be
size_t have been fixed as well.
The previous approach was more or less delay based. We have now
changed the code to enable imdiag to detect if HUP is underway
and wait until it is completed. The new method still employs some
kind of timeout, but is now quite reliable. Most importantly,
it works great with long-running HUP processing, which can happen
e.g. when querying the system name takes long or some actions need
longer time to persist their HUP processing.
The new approach will most likely reduce CI flakes and also speed
up testbench runs. The speedup happens from not having to wait a
full delay in cases where we detect HUP is completed (plus reduced
timeout when we cannot clearly detect this - see code comments why
the new method is still considered more reliable than the old one).
Code note: we needed to slightly re-structure the way actual HUP
processing and the "HUP mutex" is handled. After best analysis,
this does not affect the reliability or speed in production
settings.
closes https://github.com/rsyslog/rsyslog/issues/5192
- Add TLS CRL support tp GnuTLS driver using gnutls_certificate_set_x509_crl_file.
- Add code in OpenSSL driver that works with OpenSSL 1.0.2 and higher.
Disable feature on older features with error message.
- Some cosmetic changes
- testbench: Add revoked certificate for testing (Including CRL PEM and other files)
- testbench: Add testcase for gtls and ossl testing revoked certificates
The output module uses Apache "Qpid Proton C API" which is a solid
AMQP protocol library implementation that can be integrated
very well into the rsyslog dev environment.
- Implemented Delivery with submitted and accepted state checking
- impstatscounter used in testcases
- saving of failed messages in a failed list with support of saving
and restoring.
- Add testcases (requires ENV variables) to testbench
- Using application/octect-stream (binary) to send messages based on
Microsoft Code Sample:
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-c-getstarted-send
* Note original Microsoft Samplecode is not working anymore, we are using
* QPID Proton Proactor based on
https://github.com/apache/qpid-proton/blob/main/c/examples/send.c
- requires QPID-PROTON Version 0.13 or higher because of the proactor API
- Add EventProperties configuration parameters
- using internal array instead of linkedlist for better performance
- using single byte helper to store message status (ubsubmit/submitted...)
- trigger wake up if messages need to be resubmitted (rejected)
- Slow down when sender credit reaches zero (10ns).
- Add enhanced performance stress test omazureeventhubs-stress.sh
- Add support for static library linking of qpid-proton
This is needed to build the module from source and remove
library package dependencies.
- adjusted valgrind suppressions
FileCreateMode allows to set the default file mode bits
when creating new files. As of now, it has only impact on the state file.
Add test suite as well.
Minor indentation fix in run_journal.yml
This changes the current behaviour from honouring the double quotes in
any part of the argument but leaving them in place and passing to the
executed binary to requiring quotes exactly at the beginning and at the
end of a multi-word argument, and not including them in the actual call.
Testcases added to ensure the expected results.
Fixes#4249.
### ADDED
- [OMHIREDIS] module is now able to insert entries to a Redis Stream
- [OMHIREDIS] in 'stream' mode, module can insert the message to a custom field in the entry ('msg' by default)
- [OMHIREDIS] in 'stream' mode, module can acknowledge an entry coming from imhiredis (if entry was claimed but not ACK'ed)
- [OMHIREDIS] in 'stream' mode, acknowledgements can be made from dynamic templates or static values
- [OMHIREDIS] in 'stream' mode, module can approximately cap the size of the output stream
- [OMHIREDIS] in 'stream' mode, module can delete an entry while inserting its message (useful to remove entry coming from another stream with imhiredis)
- [OMHIREDIS] new tests for 'stream' mode
### FIXED
- [CONFIGURE.AC] Missing line to give omhiredis compilation status
- [IMHIREDIS] Add support for simple XREADs from Redis Streams (Redis >= 5.0 required)
- [IMHIREDIS] Add support for XREADGROUP from Redis Streams, allowing for user to define workers to dequeue logs in a stream
- [IMHIREDIS] stream mode can select fields to extract and insert in custom keys
- [IMHIREDIS] Add tests for the new 'stream' modes
- changed diag.sh to be able to start/stop/clean a redis server
- added helper functions in diag.sh to be able to query a redis server instance
- added new tests for imhiredis module to check
- that the queue mode works, with both lpop and rpop
- that the module is capable of handling a redis server going down
- that the module is capable of handling a redis server that appears afterwards
- that the subscribe mode works
