3523 Commits

Author SHA1 Message Date
Julien Thomas
1a1117c735 Do not create empty objects when accessing non-existent keys
This is a proposal for Github issue rsyslog/rsyslog#4430:
accessing a non-existing key creates an empty parent object
https://github.com/rsyslog/rsyslog/issues/4430

When looking up an object property, the tree of intermediate
object containers was ceated by get and del functions. The
patch is an attempt to fix that behavior by passing 0 to the
bCreate argument of jsonPathFindParent().

There is also one case where the return value of
jsonPathFindParent() was not checked, in the recurssive call
of jsonPathFindParent() itself. This was leading to infinite
loops if bCreate was 0.
2020-10-02 18:12:32 +02:00
Rainer Gerhards
c8b1e306f8
Merge pull request #4424 from rmetrich/GNUTLS_SHUT_WR
Replace GNUTLS_SHUT_RDWR by GNUTLS_SHUT_WR when ending TLS connections
2020-09-28 10:29:23 +02:00
Kalle Kankare
de0750c6c2 core/network: obey net.enableDNS=off when querying local hostname 2020-09-21 11:28:21 +03:00
Renaud Métrich
4054573746 Replace GNUTLS_SHUT_RDWR by GNUTLS_SHUT_WR when ending TLS connections
Some TLS servers don't reply to graceful shutdown requests "for
optimization". This results in rsyslog's omfwd+gtls client to wait
forever for a reply of the TLS server which never comes, due to shutting
down the connection with gnutls_bye(GNUTLS_SHUT_RDWR).

On systemd systems, commands such as "systemctl restart rsyslog" just
hang for 1m30 and rsyslogd gets killed upon timeout by systemd.

This patch replaces call to gnutls_bye(GNUTLS_SHUT_RDWR) by calls to
gnutls_bye(GNUTLS_SHUT_WR) which is sufficient and doesn't wait for a
server reply.

A Red Hat customer reproduces the hang reliably when sending the logs to
his Kiwi Syslog server, which apparently doesn't send the TLS reply upon
connection termination request.
2020-09-18 16:27:08 +02:00
Rainer Gerhards
57a15d81d8
Merge pull request #4420 from aaronmaxlevy/issue_4408_fix
Replaced eCmdHdlrPositiveInt with eCmdHdlrNonNegInt where default is 0
2020-09-14 11:24:17 +02:00
Rainer Gerhards
2f8046d34d
Merge pull request #4418 from rgerhards/i4300b-replace
core bugfix: potential segfault on querey of PROGRAMNAME property
2020-09-14 09:30:40 +02:00
Aaron Levy
2e3b767d03 Replaced eCmdHdlrPositiveInt with eCmdHdlrNonNegInt where default is 0 2020-09-13 15:56:07 -07:00
Rainer Gerhards
6cffa83af3
core bugfix: potential segfault on querey of PROGRAMNAME property
A data race can happen on variable iLenProgram as it is not guarded
by the message mutex at time of query. This can lead to it being
non -1 while the buffer has not yet properly set up.

Thanks to github user wsp1991 for alerting us and a related
patch proposal.

replaces https://github.com/rsyslog/rsyslog/pull/4300
2020-09-11 13:14:50 +02:00
Leo Fang
29afbafcd5 imtcp bugfix: broken connection not necessariy detected
Due to an invalid return code check, broken TCP sessions could not
necessarily be detected "right in time". This can result is the loss
of one message.

closes https://github.com/rsyslog/rsyslog/issues/4227
2020-09-11 12:33:44 +02:00
Rainer Gerhards
6763185783
core bugfix: segfault if disk-queue file cannot be created
When using Disk Queue and a queue.filename that can not be created
by rsyslog, the service does not switch to another queue type as
supposed to and crashes at a later step.

closes: https://github.com/rsyslog/rsyslog/issues/4282
2020-09-02 09:20:10 +02:00
Daiki Ueno
37a19fb899 gnutls: Propagate CheckExtendedKeyPurpose when accepting connection
Previously, when the server accepts a new connection, it doesn't
properly set the dataTypeCheck field based on the listening socket.
That results in skipping ExtendedKeyUsage (EKU) check on the client
certificates.
2020-08-10 16:37:43 +02:00
Rainer Gerhards
879a645bfb
Merge pull request #4069 from rgerhards/i4020
queue: permit ability to double size at shutdown
2020-06-22 12:46:37 +02:00
Rainer Gerhards
8ff2179304
Merge pull request #4321 from alorbach/pr-issue-4319
openssl: Fixed memory leak when tls handshake failed.
2020-06-22 10:45:07 +02:00
Rainer Gerhards
2b309b69aa
Merge pull request #4318 from taavi-valjaots/optimization-fix
ksi bugfix: Optimized code in KSI module initialization fixed.
2020-06-22 09:35:51 +02:00
6d0fea9b69 openssl: Fixed memory leak when tls handshake failed.
Added testcase for ossl memory leak on failed handshake.

closes: https://github.com/rsyslog/rsyslog/issues/4319
2020-06-09 10:01:53 +02:00
Rainer Gerhards
89943dabf0
operatingstatefile bugfix: month was given too low
The month was printed with the range 0 (January) to 11 (December).
This commit corrects it.

closes https://github.com/rsyslog/rsyslog/issues/4292
2020-06-05 09:04:43 +02:00
taavi.valjaots
5906968120 ksi bugfix: Optimized code in KSI module initialization fixed.
KSI module initialization will not stuck in infinite loop when code is
built with optimization -O2.
2020-06-03 15:21:50 +03:00
Rainer Gerhards
7f79730e49
Merge pull request #4285 from mbiebl/drop-leftover-sd_daemon
build: Drop update-systemd target
2020-05-18 09:39:56 +02:00
Michael Biebl
545418d9b0 build: Drop update-systemd target
This is no longer necessary, as we no longer use an embedded copy of
sd-daemon but the libsystemd system library.
2020-05-16 16:14:38 +02:00
82621cdd14 imtcp: fix octet framing/stuffing problem with discardTruncatedMsg on
When "discardTruncatedMsg" is enabled in imtcp, messages were incorrectly
skipped if the last character before the truncation was the LFdelimiter.

Add two tests for octet stuffing and framing.

closes: https://github.com/rsyslog/rsyslog/issues/4281
2020-05-15 09:56:44 +02:00
Rainer Gerhards
4982146186
Merge pull request #4215 from M1cha/fix-ossl-ctx-leak
openssl tls: stop leaking ctx
2020-05-11 12:59:37 +02:00
Rainer Gerhards
6b56e0418e Merge branch 'sudden-ksi-module-closure-fix' 2020-04-28 13:11:22 +02:00
Rainer Gerhards
80c6be0c47
Merge pull request #4211 from taavi-valjaots/default-hash-algo-fix
Hardcoded default hash algorithm KSI_HASHALG_SHA2_256 replaced with
2020-04-28 12:28:52 +02:00
Rainer Gerhards
b74a14a9cd
Merge pull request #4218 from taavi-valjaots/signer-thread-init-fix
ksi bugfix: Signer thread initialization is verified before usage.
2020-04-28 12:03:47 +02:00
Rainer Gerhards
38861e838a
Merge pull request #4228 from jaankit/rsyslog-stream
stream: Fix for segfault on imfile read
2020-04-28 10:48:42 +02:00
Rainer Gerhards
30dc2546be
Merge pull request #4225 from alorbach/pr-issue-4224
ossl: setting SSL_MODE_AUTO_RETRY on ssl objects returned by SSL_new
2020-04-28 10:38:06 +02:00
Rainer Gerhards
686835c880
Merge pull request #4234 from alorbach/pr-issue-4233
stream.c: Moved doSizeLimitProcessing check to strmWrite
2020-04-28 10:26:57 +02:00
b84c9debea stream.c: Moved doSizeLimitProcessing check to strmWrite
The check was done in strmPhysWrite before which caused syslog
messages to split in the middle if the syslog message batch exceeded
the default IO Buffer size.

closes: https://github.com/rsyslog/rsyslog/issues/4233
2020-03-31 17:27:01 +02:00
993569353a ossl: setting SSL_MODE_AUTO_RETRY on ssl objects returned by SSL_new
Older OpenSSL Versions do not have SSL_MODE_AUTO_RETRY enabeld by default.
If the setting is set to OFF, SSL_read can return an error with
SSL_ERROR_WANT_READ when a non-application data record has been processed.

For more see: https://www.openssl.org/docs/man1.1.1/man3/SSL_read.html

closes: https://github.com/rsyslog/rsyslog/issues/4224
2020-03-19 13:34:14 +01:00
Ankit Jain
ea96f4c99a stream: Fix for segfault on imfile read
- if cstrLen(pThis->prevMsgSegment) > maxMsgSize then len calculation
  become negative if cstrLen(thisLine) < cstrLen(pThis->prevMsgSegment)
  This causes illegal access to memory location and thus causing segfault.
- assigning len = 0 if cstrLen(pThis->prevMsgSegment) > maxMsgSize so that
  it access the correct memory location.

Signed-off-by: Ankit Jain <ankitja@vmware.com>
2020-03-19 12:01:53 +05:30
taavi.valjaots
7211b6aac4 ksi bugfix: Signer thread initialization is verified before usage.
When signer thread is created in rsksiInitModule thread successful
initialization is verified before returning the function. This will
prevent adding records to not initialized module and in case of an
error signature files opened will contain only magic bytes.

Thread flags replaced with thread state.

When init module fails, module is disabled.
2020-03-12 15:21:02 +02:00
901eb09860 openssl: add check for LIBRESSL
Disable use of "@SECLEVEL" in default cipher string and
avoid SSL_CONF_CTX_set_flags() API when LIBRESSL is used.
This means tlscommands will not work.

closes: https://github.com/rsyslog/rsyslog/issues/4210
2020-03-11 08:17:01 +01:00
Michael Zimmermann
b50d72b051 openssl tls: stop leaking ctx 2020-03-10 14:38:33 +01:00
taavi.valjaots
f219fa3003 ksi bugfix: Hardcoded default hash algorithm replaced with 'default'.
Instead of hardcoded SHA-256 KSI_getHashAlgorithmByName("default")
is used to get default hash function.
Function rsksiSetHashFunction and SetCnfParam updated.
2020-03-10 13:07:35 +02:00
taavi.valjaots
8b3c24a768 ksi bugfix: When KSI module is suddenly closed, files are finalized.
In async. mode all pending signature requests are closed immediately and
unsigned block marker is attached with message about sudden closure.
Similar approach is used for blocks that already contain some records.
Empty blocks are just closed without any metadata.
2020-03-10 12:59:17 +02:00
Rainer Gerhards
2e7207e3a5
queue subsystem: cap max queue size to 2^31-1
closes https://github.com/rsyslog/rsyslog/issues/4192
2020-03-04 10:33:55 +01:00
Rainer Gerhards
a9dd12b967
queue: permit ability to double size at shutdown
This prevents message loss due to "queue full" when re-enqueueing data
under quite exotic settings.

see also https://github.com/rsyslog/rsyslog/issues/3941#issuecomment-549765813
closes https://github.com/rsyslog/rsyslog/issues/4020
2020-03-03 16:14:31 +01:00
Rainer Gerhards
f2afcb45b2
Merge pull request #4166 from rgerhards/i4164
core: add global parameter "security.abortOnIDResolutionFail"
2020-02-24 09:47:25 +01:00
Rainer Gerhards
4d1f7bd11a
Merge pull request #4182 from alorbach/pr-issue-4170
openssl nsd driver: Switch to SSL_CTX_use_certificate_chain_file API
2020-02-24 08:43:03 +01:00
Rainer Gerhards
ce106d3086
Merge pull request #4188 from rgerhards/i4187
core bugfix: too early parsing of incoming messages
2020-02-24 08:40:51 +01:00
Rainer Gerhards
c4a9d637da
core bugfix: too early parsing of incoming messages
In theory, rsyslog should call parsers on the queue worker threads whenever
possible. This enables the parsers to be executed in parallel. There are
some cases where parsers needs to be called earlier, namely when parsed
data is needed for rate-limiting.

The logic to do this previously did not work correctly and was fixed six
years ago (!) by b51dd22. Unfortunately, b51dd22 was overly agressive:
it actually makes the early parser call now mandatory, effectively moving
parsing to the input side where there is no to little concurrency.

We still do not need to call the parser when all messages, regardless of
severity, need to be rate-limited. This is the default and very frequent
case. This patch introduces support for this and as such makes parsers
able to run in parallel in the frequent case again.

closes https://github.com/rsyslog/rsyslog/issues/4187
2020-02-20 13:15:57 +01:00
493b04b846 openssl nsd driver: Switch to SSL_CTX_use_certificate_chain_file API
Switched from SSL_CTX_use_certificate_file to SSL_CTX_use_certificate_chain_file
API for loading the certificate into the certificate store.

According to the openssl doc, the SSL_CTX_use_certificate_chain_file API has the
advantage that it can load the main certificate (First in the PEM) and many chain
certificates after that automatically.

See notes section for more:
https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_use_certificate_chain_file.html

closes: https://github.com/rsyslog/rsyslog/issues/4170
2020-02-19 14:26:25 +01:00
Jiri Vymazal
ba5b68be84 Fixed processing of 'cofig.enabled' directive
Previously the directive was processed way too late which caused
false errors whenever it was set to 'off' and possibly other
problems.
2020-02-12 10:14:51 +01:00
Rainer Gerhards
cbcaf2c7e5
core: add global parameter "security.abortOnIDResolutionFail"
This parameter controls whether or not rsyslog aborts when a name ID
lookup fails (for user and group names). This is necessary as a security
measure, as otherwise the wrong permissions can be assigned or privileges
are not dropped.

CHANGE OF BEHAVIOR
The default for this parameter is "on". In previous versions, the default
was "off" (by virtue) of this parameter not existing. As such, existing
configurations may now error out.

We have decided to accept this change of behavior because of the potential
security implications.

closes https://github.com/rsyslog/rsyslog/issues/4164
2020-02-11 12:54:19 +01:00
Rainer Gerhards
7870f248f3
oversize message log bugfix: do not close fd -1
The oversizie message log fd is always closed on HUP, even if it never
was opened (and thus has -1 value). This patch corrects the issue.

The bug had no know-bad effect in practice other than getting an
(ignored) error status from close(). However, it introduced warnings
in test runs (e.g. when running under valgrind).
2020-01-22 17:15:40 +01:00
Rainer Gerhards
e00580534b
core bugfix: potential race during HUP
when rsyslog is HUPed immediately after startup and before it is fully
initialized, there is a potential race with the list of loaded modules.

This patch ensures no bad things can happen in that case.

Detected by LLVM TSAN, not seen in practice.

Signed-off-by: Rainer Gerhards <rgerhards@adiscon.com>
2020-01-20 08:22:15 +01:00
Rainer Gerhards
58dfe006a8
testbench: improve tests
modernize them, reduce robustness against slow machines, provide some
test framework functional enhancements and optimizie some tests
2020-01-19 16:09:44 +01:00
Rainer Gerhards
694eb79c39
tcpsrv bugfix: build problem regression 2020-01-18 10:49:45 +01:00
Rainer Gerhards
9b56191970
Merge pull request #3924 from Frogging101/ciena-fixes-for-upstream-20191023
Various bug fixes relating to shutdown handling
2020-01-17 12:56:17 +01:00
Rainer Gerhards
4ab8ffa561
Merge pull request #4042 from miztake/prototype
Fix race condition related to libfastjson when using DA queue
2020-01-12 12:58:48 +01:00