3550 Commits

Author SHA1 Message Date
Rainer Gerhards
a08591be5d
imtcp: add stream driver parameter to input() configuration
This permits to have different inputs use different stream drivers
and stream driver parameters.

closes https://github.com/rsyslog/rsyslog/issues/3727
2021-05-05 12:57:09 +02:00
Rainer Gerhards
89be1d86d5
imtcp: permit to run multiple inputs in parallel
Previously, a single server was used to run all imtcp inputs. This
had a couple of drawsbacks. First and foremost, we could not use
different stream drivers in the varios inputs. This patch now
provides a baseline to do that, but does still not implement the
capability (in this sense it is a staging patch).

Secondly, we now ensure that each input has at least one exclusive
thread for processing, untangling the performance of multiple
inputs from each other.

This is part of the patch series to enable different driver
configurations for imtcp.

see also: https://github.com/rsyslog/rsyslog/issues/3727
2021-04-27 12:30:28 +02:00
Rainer Gerhards
b160813f82
tcpsrv bugfix: potential sluggishnes and hang on shutdown
tcpsrv is used by multiple other modules (imtcp, imdiag, imgssapi, and,
in theory, also others - even ones we do not know about). However, the
internal synchornization did not properly take multiple tcpsrv users
in consideration.

As such, a single user could hang under some circumstances. This was
caused by improperly awaking all users from a pthread condition wait.
That in turn could lead to some sluggish behaviour and, in rare cases,
a hang at shutdown.

Note: it was highly unlikely to experience real problems with the
officially provided modules.

This patch corrects the situation.
2021-04-27 08:31:24 +02:00
Rainer Gerhards
3470728d44
further refactoring of tcpsrv parameter passing
Some further stream-lining and cleanup of paramter passing. This
levels ground for more substantial changes to the imtcp/tcpsrv
interaction.

see also https://github.com/rsyslog/rsyslog/issues/3727#issuecomment-525705318
2021-04-26 09:05:01 +02:00
Rainer Gerhards
926e207575
phase 1: consolidate current imtcp input params in a single structure
see also https://github.com/rsyslog/rsyslog/issues/3727
2021-04-22 08:44:55 +02:00
Rainer Gerhards
ab3bd4609b
Merge pull request #4537 from VultureProject/fix_empty_foreach
RainerScript::FIX:: handle foreach with empty arrays/objects
2021-04-19 10:42:21 +02:00
Rainer Gerhards
1d74aa7064
Merge pull request #4543 from VultureProject/imhiredis_upstream
NEW CONTRIB::IMHIREDIS:: input redis module
2021-04-19 10:18:02 +02:00
Rainer Gerhards
2381890467
Merge pull request #4551 from julthomas/dev/jth/fmunflatten
fmunflatten: rscript function to unflatten keys in a JSON object
2021-04-06 19:14:23 +02:00
Rainer Gerhards
55b68be792
Merge pull request #4509 from Anusha-Pai95/adding_tag_to_tls_handshake_failures
Improving nsd ossl logs for better debugging
2021-03-29 13:12:00 +02:00
Julien Thomas
454088780a msg: expose function jsonDeepCopy()
This function will be used by rainerscript function module
fmunflatten in next commit. Let's expose it to make it public.
2021-03-09 21:33:26 +01:00
b0b6320c2a testbench: Enhanced error detection in imrelp-tls-cfgcmd.sh
This solves test failures on some platforms like solaris.

Also added _attribute__((unused)) into SetTlsVerifyDepth in
nsd_ptcp.c to fix compiler errors with gcc9.

closes: https://github.com/rsyslog/rsyslog/issues/4544
2021-03-09 09:18:08 +01:00
Anusha Pai G
9e8614a6f2
Improving nsd ossl logs for better debugging
when tls handshake fails, function osslLastSSLErrorMsg is called to
print the logs saved on error stack, adding the tag "nsd_ossl:" for the
better identification of these logs in such failure scenarios.
2021-02-24 10:22:31 +01:00
Jérémie Jourdin
5823abf0f8 NEW CONTRIB::IMHIREDIS:: input redis module 2021-02-23 19:06:44 +01:00
frikilax
65d34853eb RAINERSCRIPT::FOREACH:: HANDLE EMPTY ARRAYS/OBJECTS 2021-02-16 22:41:14 +01:00
Tim Gates
e3247d4557
docs: fix simple typo, charcters -> characters
There is a small typo in runtime/msg.h.

Should read `characters` rather than `charcters`.
2020-11-29 10:06:29 +11:00
Rainer Gerhards
1c85aa8a59
add new system property $now-unixtimestamp
Among others, this may be used as a monotonic counter
for doing load-balancing and other things.

Thanks to Nicholas Brown for suggesting this feature.
2020-11-04 13:02:22 +01:00
Rainer Gerhards
0716d8983b
Merge pull request #4440 from Dinesh-Ramakrishnan/master
omfwd: adding new rate limit option
2020-11-04 12:38:06 +01:00
b5356f8db7 tls: Allow calling PermitExpiredCerts with NULL parameter
In order to set the default PermitExpiredCerts handling (Denied),
we need to call PermitExpiredCerts with NULL parameter.

testbench: Add test to check expired handling in omfwd

debug: Fix dbgprintf calls with possible NULL character parameters

closes: https://github.com/rsyslog/rsyslog/issues/4425
2020-10-28 09:18:25 +01:00
Rainer Gerhards
1ecb54b600
Merge pull request #4439 from alorbach/pr-issue-4288
gnutls: Added handshake error handling into doRetry handler.
2020-10-13 09:19:58 +02:00
Rainer Gerhards
11b58367dd
Merge pull request #4443 from julthomas/dev/jth/json-set-segfault
msg: Fix segfault and memory leak on set operation
2020-10-13 09:08:04 +02:00
Rainer Gerhards
d3c3c68213
Merge pull request #4433 from Anusha-Pai95/adding_tag
Improving nsd ossl logs for better debugging
2020-10-13 09:03:39 +02:00
Julien Thomas
c54d3d5e8c msg: memory leak in msgAddJSON() if jsonPathFindParent() failed
There is a missing call to json_object_put(json) if the call to
jsonPathFindParent() failed. It's leaking memory.
2020-10-12 10:09:58 +02:00
Julien Thomas
e5dc93e076 msg: segfault in jsonPathFindNext() when <root> is not an object
The segfault gets happens when <bCreate> is 1 and when the <root>
container where to insert the <namebuf> key is not an object.

Here is simple reproducible test case:

// ensure we start fresh
// unnecessary if there was no previous set
unset $!;

set $! = "";
set $!event!created = 123;
2020-10-12 10:09:58 +02:00
Dinesh-Ramakrishnan
72fecfe2c2 omfwd: adding new rate limit option
Adding new rate limit option to omfwd for rate limiting
syslog messages sent to the remote server

ratelimit.interval:
	Specifies the rate-limiting interval in seconds.
    Default value is 0, which turns off rate limiting.

ratelimit.burst
	Specifies the rate-limiting burst in number of messages.

fixes #4423
https://github.com/rsyslog/rsyslog/issues/4423
2020-10-07 09:03:06 +05:30
1cf08f0a26 gnutls: Added handshake error handling into doRetry handler.
If the tls handshake does not immediatelly finish, gnutls_handShake is called in
doRetry handler again. However the error handling was not
complete in the doRetry handler. A failed gnutls_handShake call
did not abort the connection and properly caused unexpected
problems like in issues:

https://github.com/rsyslog/rsyslog/issues/4270
https://github.com/rsyslog/rsyslog/issues/4288
2020-10-06 14:48:32 +02:00
Anusha Pai G
82ec2a77e1 Improving nsd ossl logs for better debugging
Adding error logs at the ssl handshake failure scenarios.
Adding the header "nsd_ossl:" tag to these logs to identify
the origin module from which logs are generated.
2020-10-06 04:12:24 -07:00
Rainer Gerhards
d247c31888
add 'exists()' script function to check if variable exists
This implements a way to check if rsyslog variables (e.g. '$!path!var') is
currently set of not.

Sample: if exists($!somevar) then ...

closes https://github.com/rsyslog/rsyslog/issues/4385
2020-10-05 12:38:58 +02:00
Julien Thomas
1a1117c735 Do not create empty objects when accessing non-existent keys
This is a proposal for Github issue rsyslog/rsyslog#4430:
accessing a non-existing key creates an empty parent object
https://github.com/rsyslog/rsyslog/issues/4430

When looking up an object property, the tree of intermediate
object containers was ceated by get and del functions. The
patch is an attempt to fix that behavior by passing 0 to the
bCreate argument of jsonPathFindParent().

There is also one case where the return value of
jsonPathFindParent() was not checked, in the recurssive call
of jsonPathFindParent() itself. This was leading to infinite
loops if bCreate was 0.
2020-10-02 18:12:32 +02:00
Rainer Gerhards
c8b1e306f8
Merge pull request #4424 from rmetrich/GNUTLS_SHUT_WR
Replace GNUTLS_SHUT_RDWR by GNUTLS_SHUT_WR when ending TLS connections
2020-09-28 10:29:23 +02:00
Kalle Kankare
de0750c6c2 core/network: obey net.enableDNS=off when querying local hostname 2020-09-21 11:28:21 +03:00
Renaud Métrich
4054573746 Replace GNUTLS_SHUT_RDWR by GNUTLS_SHUT_WR when ending TLS connections
Some TLS servers don't reply to graceful shutdown requests "for
optimization". This results in rsyslog's omfwd+gtls client to wait
forever for a reply of the TLS server which never comes, due to shutting
down the connection with gnutls_bye(GNUTLS_SHUT_RDWR).

On systemd systems, commands such as "systemctl restart rsyslog" just
hang for 1m30 and rsyslogd gets killed upon timeout by systemd.

This patch replaces call to gnutls_bye(GNUTLS_SHUT_RDWR) by calls to
gnutls_bye(GNUTLS_SHUT_WR) which is sufficient and doesn't wait for a
server reply.

A Red Hat customer reproduces the hang reliably when sending the logs to
his Kiwi Syslog server, which apparently doesn't send the TLS reply upon
connection termination request.
2020-09-18 16:27:08 +02:00
Rainer Gerhards
57a15d81d8
Merge pull request #4420 from aaronmaxlevy/issue_4408_fix
Replaced eCmdHdlrPositiveInt with eCmdHdlrNonNegInt where default is 0
2020-09-14 11:24:17 +02:00
Rainer Gerhards
2f8046d34d
Merge pull request #4418 from rgerhards/i4300b-replace
core bugfix: potential segfault on querey of PROGRAMNAME property
2020-09-14 09:30:40 +02:00
Aaron Levy
2e3b767d03 Replaced eCmdHdlrPositiveInt with eCmdHdlrNonNegInt where default is 0 2020-09-13 15:56:07 -07:00
Rainer Gerhards
6cffa83af3
core bugfix: potential segfault on querey of PROGRAMNAME property
A data race can happen on variable iLenProgram as it is not guarded
by the message mutex at time of query. This can lead to it being
non -1 while the buffer has not yet properly set up.

Thanks to github user wsp1991 for alerting us and a related
patch proposal.

replaces https://github.com/rsyslog/rsyslog/pull/4300
2020-09-11 13:14:50 +02:00
Leo Fang
29afbafcd5 imtcp bugfix: broken connection not necessariy detected
Due to an invalid return code check, broken TCP sessions could not
necessarily be detected "right in time". This can result is the loss
of one message.

closes https://github.com/rsyslog/rsyslog/issues/4227
2020-09-11 12:33:44 +02:00
Rainer Gerhards
6763185783
core bugfix: segfault if disk-queue file cannot be created
When using Disk Queue and a queue.filename that can not be created
by rsyslog, the service does not switch to another queue type as
supposed to and crashes at a later step.

closes: https://github.com/rsyslog/rsyslog/issues/4282
2020-09-02 09:20:10 +02:00
Daiki Ueno
37a19fb899 gnutls: Propagate CheckExtendedKeyPurpose when accepting connection
Previously, when the server accepts a new connection, it doesn't
properly set the dataTypeCheck field based on the listening socket.
That results in skipping ExtendedKeyUsage (EKU) check on the client
certificates.
2020-08-10 16:37:43 +02:00
Rainer Gerhards
879a645bfb
Merge pull request #4069 from rgerhards/i4020
queue: permit ability to double size at shutdown
2020-06-22 12:46:37 +02:00
Rainer Gerhards
8ff2179304
Merge pull request #4321 from alorbach/pr-issue-4319
openssl: Fixed memory leak when tls handshake failed.
2020-06-22 10:45:07 +02:00
Rainer Gerhards
2b309b69aa
Merge pull request #4318 from taavi-valjaots/optimization-fix
ksi bugfix: Optimized code in KSI module initialization fixed.
2020-06-22 09:35:51 +02:00
6d0fea9b69 openssl: Fixed memory leak when tls handshake failed.
Added testcase for ossl memory leak on failed handshake.

closes: https://github.com/rsyslog/rsyslog/issues/4319
2020-06-09 10:01:53 +02:00
Rainer Gerhards
89943dabf0
operatingstatefile bugfix: month was given too low
The month was printed with the range 0 (January) to 11 (December).
This commit corrects it.

closes https://github.com/rsyslog/rsyslog/issues/4292
2020-06-05 09:04:43 +02:00
taavi.valjaots
5906968120 ksi bugfix: Optimized code in KSI module initialization fixed.
KSI module initialization will not stuck in infinite loop when code is
built with optimization -O2.
2020-06-03 15:21:50 +03:00
Rainer Gerhards
7f79730e49
Merge pull request #4285 from mbiebl/drop-leftover-sd_daemon
build: Drop update-systemd target
2020-05-18 09:39:56 +02:00
Michael Biebl
545418d9b0 build: Drop update-systemd target
This is no longer necessary, as we no longer use an embedded copy of
sd-daemon but the libsystemd system library.
2020-05-16 16:14:38 +02:00
82621cdd14 imtcp: fix octet framing/stuffing problem with discardTruncatedMsg on
When "discardTruncatedMsg" is enabled in imtcp, messages were incorrectly
skipped if the last character before the truncation was the LFdelimiter.

Add two tests for octet stuffing and framing.

closes: https://github.com/rsyslog/rsyslog/issues/4281
2020-05-15 09:56:44 +02:00
Rainer Gerhards
4982146186
Merge pull request #4215 from M1cha/fix-ossl-ctx-leak
openssl tls: stop leaking ctx
2020-05-11 12:59:37 +02:00
Rainer Gerhards
6b56e0418e Merge branch 'sudden-ksi-module-closure-fix' 2020-04-28 13:11:22 +02:00
Rainer Gerhards
80c6be0c47
Merge pull request #4211 from taavi-valjaots/default-hash-algo-fix
Hardcoded default hash algorithm KSI_HASHALG_SHA2_256 replaced with
2020-04-28 12:28:52 +02:00