2820 Commits

Author SHA1 Message Date
Rainer Gerhards
5b20e4ccc1
Merge pull request #2388 from rgerhards/statan-container
add reproducible static analyzer builds
2018-01-14 16:55:00 +01:00
Rainer Gerhards
815c2cd99f
Merge pull request #2387 from mbiebl/no-libcurl
build system: Don't link core against libcurl if explicitly requested
2018-01-14 16:50:48 +01:00
Michael Biebl
50205ced0f build system: Don't link core against libcurl if explicitly requested
When using --disable-libcurl, libgrammar should not be linked against
libcurl unconditionally as CURL_LIBS might be set if one of the modules
using libcurl, like elasticsearch, is enabled.
2018-01-14 12:21:42 +01:00
Rainer Gerhards
0d1658d26f omkafka: fix build problem with old librdkafka 2018-01-14 11:00:26 +01:00
Rainer Gerhards
d7aea382fa fix build warning on Alpine Linux
due to old-style header names -- now using the new ones, which are also
supported on the other platforms.
2018-01-12 13:15:55 +01:00
Rainer Gerhards
53930f31f2
Merge pull request #2364 from rgerhards/merge-2181
Merge PR #2181
2018-01-08 13:06:42 +01:00
Rainer Gerhards
e134b545b3 mmpstrucdata: make case preservation of SD_IDs optional
commit 5cee06a60 introduced case-perservation, but did not make
it optional. This broke existing behavior and thus potentially
existing configurations. This commit introduces a new parameter
"sd_name.lowercase" which now controls this behavior.

Note that 5cee06a60 is merged to master branch together with this
commit, so no released code ever had the potential regression.

see also https://github.com/rsyslog/rsyslog/pull/2181
2018-01-07 10:18:57 +01:00
Rainer Gerhards
48a49644d5 Merge branch 'master' of https://github.com/alanrobson/rsyslog into merge-2181 2018-01-06 11:28:01 +01:00
Rainer Gerhards
09aaf0a1d5 im3195: fix compile issues on modern compilers
... namely gcc 7 and clang 5. We do the most minimal change to
get to a clean build as this module AFAIK is not used by anyone
for quite some years now (RFC3195 is a total failure).

closes https://github.com/rsyslog/rsyslog/issues/2094
2018-01-05 17:49:55 +01:00
Rainer Gerhards
23ae3018bf
Merge pull request #2358 from jgerhards/mmanon-error
mmanon bugfix: fix wrong ipv6 embedded recognition
2018-01-05 17:20:04 +01:00
af09d33b49 imfile: Partially fixed issues not detecting files in directory when wildcards are used.
When directories and files are created at the same time,
imfile may missed subdirs or file if the machine is on high load.

The handling has been enhanced to scan newly created directories ALWAYS for
matching files.

This should fix following issues:
closes https://github.com/rsyslog/rsyslog/issues/2271

However we still have a problem when a multilevel directory configurations.
Details are discussed here https://github.com/rsyslog/rsyslog/issues/2354
2018-01-05 12:59:19 +01:00
Jan Gerhards
782982d94a mmanon bugfix: fix wrong ipv6 embedded recognition
mmanon recognizes IPv6 with embedded IPv4 that have too few (16 bit) fields.

example: 13:abd:45:0.0.0.0

closes https://github.com/rsyslog/rsyslog/issues/2357
2018-01-05 12:33:36 +01:00
Jan Gerhards
b8bb66eb81 imfile: add sortfiles parameter
if set, files will be input in a sorted order

closes https://github.com/rsyslog/rsyslog/issues/1489
2018-01-03 11:00:25 +01:00
Peter Portante
eba65ad483
Update imfile.c 2018-01-02 23:31:47 -05:00
Rainer Gerhards
259e9b12c4
Merge pull request #2337 from rgerhards/i2336
omelasticsearch: better error messages and mild refactoring
2018-01-02 08:56:46 +01:00
Rainer Gerhards
eac214eb0b
Merge pull request #2334 from rgerhards/i2314
omkafka: expose operational status to user where useful
2018-01-01 18:30:02 +01:00
Rainer Gerhards
3acf848db1 omelasticsearch bugfix: operational error messages not passed to user
closes https://github.com/rsyslog/rsyslog/issues/2336
2018-01-01 18:28:02 +01:00
Rainer Gerhards
92ea253d4e omelasticsearch: theoretical NULL pointer access
commit 0fbe873438e introduced an unchecked realloc, which under
extreme conditions (total out-of-memory condition) could lead to
a NULL pointer access. This commit fixes the issue be avoiding the
need for realloc() in the first place.

Note that commit 0fbe873438e was merged today, so there is no
problem in any released code.
2018-01-01 18:13:52 +01:00
Rainer Gerhards
05ca64c368 omelasticsearch: code style cleanup, minor compile-time hardening 2018-01-01 18:06:57 +01:00
Rainer Gerhards
c6f26e9d3d
Merge pull request #2306 from bayaro/master
omelasticsearch: add eol to every elastic response for error file
2018-01-01 17:58:34 +01:00
Rainer Gerhards
37ad53b177 omkafka: expose operational status to user where useful
omkafka emits many useful operational status messages only to the debug
log. After careful review, we have exposed many of these as user error
and warning message (ex: librdkafka queue full, so user knows why we
suspend the plugin temporarily). This may have made the module too
chatty. If so, one can always filter out messages via configuration. And
if we really went overboard, we can undo those parts with the next
release. So IMHO it's better to give a bit more than less, as this
definitely eases troubleshooting for users.

closes https://github.com/rsyslog/rsyslog/issues/2314
2018-01-01 13:58:50 +01:00
Rainer Gerhards
48c248d626 impstats: fix invalid counter definitions for getrusage() reporting
some of the counters were defined as  int (32 bit) vs. intctr_t (64 bit).
On some platforms "long" seems to be 64bit, and getrusage() provides
what we store as int via long. So this caused truncation and/or overflow.
This had undefined effects. Most often, everything worked fine
for values smaller than 2^31 but sometimes we got negative values.

closes https://github.com/rsyslog/rsyslog/issues/1517
2017-12-31 17:12:51 +01:00
Rainer Gerhards
8956f35841 omkafka: fix multithreading
omkafka has several issue if multiple worker instances are used. This commit
actually make the module use a single worker thread at max. Reasoning:
Librdkafka creates background threads itself. So omkafka basically needs to move
memory buffers over to librdkafka, which then does the heavy hauling. As such, we
think that it is best to run max one wrkr instance of omkafka -- otherwise we just
get additional locking (contention) overhead without any real gain. As such,
we use a global mutex for doAction which ensures only one worker can be active
at any given time. That mutex is also used to guard utility functions (like
tryResume) which may also be accessed by multiple workers in parallel.
Note: shall this method be changed, the kafka connection/suspension handling needs
to be refactored. The current code assumes that all workers share state information
including librdkafka handles.

closes https://github.com/rsyslog/rsyslog/issues/2313
2017-12-30 10:58:14 +01:00
Rainer Gerhards
c9543c6543 omkafka: fix potential memory leak
if kafka produce fails when resubmitting messages, the message object
is duplicated. This potentially leads to a mem leak or message duplication
(not fully checked yet).
2017-12-30 10:58:14 +01:00
Rainer Gerhards
b1c6ba7122
Merge pull request #2312 from rgerhards/i2084
imkafka: fix potential small ressource leak
2017-12-29 12:41:35 +01:00
Rainer Gerhards
1548fb6d20
Merge pull request #2311 from rgerhards/imfile-refactor2
imfile: remove unnedeed code
2017-12-29 09:40:10 +01:00
Rainer Gerhards
5522ea2af4 imkafka: fix potential small ressource leak
if rdkafka handle cannot fully populated, cleanup is added. Previously, we
could leak a handle if e.g. no brokers were avaiable. Note that this was
a cosmetic leak, as proper processing is not possible in any case and the
leak is once per listener, so not growing. But we now also ensure that
proper error reporting and handling is done in any case. Previously, we
may have some misleading error messages if the defunct kafka handle was
used.

closes https://github.com/rsyslog/rsyslog/issues/2084
2017-12-28 18:11:46 +01:00
Rainer Gerhards
7d6f6ad63f imfile: remove unnedeed code 2017-12-28 18:03:57 +01:00
Rainer Gerhards
99deab378b imuxsock: permit undefined behaviour where introduced by system lib
in particular, bind() and helpers. It's not actually undefined in this
case, because the rest of the system obviously defines the semantics
(plus, we cannot really solve it ;-)).
2017-12-28 17:40:03 +01:00
Rainer Gerhards
047a84eb32
Merge pull request #2308 from rgerhards/imptcp-asan1
imptcp bugfix: access to free'ed memory
2017-12-28 17:36:28 +01:00
Rainer Gerhards
4770750828 imptcp bugfix: access to free'ed memory
When notifyconnectionclose was on, a string buffer was accessed immediately
after it was freed (as part of the connection close operation). This commit
changes the order of operations to fix that.

Detected by LLVM ASAN.
2017-12-28 15:20:13 +01:00
baya
0fbe873438 omelasticsearch: add eol to every elastic response for error file 2017-12-28 01:44:36 +02:00
Rainer Gerhards
ff2bddd232 imfile: some basic refactoring and simplification
simplify and streamline code, remove (kind of) duplicate code,
add TODO items for things that are not OK
2017-12-27 13:18:27 +01:00
Rainer Gerhards
accf1fcf17 imfile: add a bit compile-time hardening
helps generating better code, protects against coding glitches (as it
enables compiler to alert against unintensional interface violations)
and helps doing better in the testbench (especially via UBSan)
2017-12-26 13:04:59 +01:00
Rainer Gerhards
6ea7254903 imfile: cosmetic fixes (code style and very long lines) 2017-12-26 12:36:18 +01:00
Rainer Gerhards
cd7718c82c
Merge pull request #2291 from jgerhards/errmsg-imfile
imfile: use new errmsg interface
2017-12-25 16:06:56 +01:00
Jan Gerhards
5388e6c9fd mmanon: fix undefined behavior
closes https://github.com/rsyslog/rsyslog/issues/2243
2017-12-24 11:56:43 +01:00
Jan Gerhards
e5baa34bfb imfile: use new errmsg interface
see also https://github.com/rsyslog/rsyslog/issues/1684
2017-12-24 11:53:09 +01:00
Rainer Gerhards
fbbfbdcba2 imkafka bugfix: do not emit error message on regular state
This was misunderstood as an error state and could spam the system
log considerably.

It is a recent regression.
2017-12-22 12:46:34 +01:00
Rainer Gerhards
ff2bad87d0 omkafka bugfix: potential misadressing
The failed message list is improperly cleaned. This is a regression
from recent commit 4eae19e089b5a83da679fe29398c6b2c10003793, which
was introduced in 8.31.0.

This problem is more likely to happen under heavy load or bad
connectivity, when the local librdkafka queue overruns or message
delivery times out.

closes https://github.com/rsyslog/rsyslog/issues/2184
closes https://github.com/rsyslog/rsyslog/issues/2067
2017-12-21 19:19:10 +01:00
Rainer Gerhards
2af5c24d2d
Merge pull request #2285 from PascalWithopf/line_length_correction
codestyle: partially reduce line length to 120
2017-12-21 18:54:10 +01:00
Rainer Gerhards
fdb71684a0
Merge pull request #2281 from rgerhards/i2276
imklog: local host IP was hardcoded to 127.0.0.1
2017-12-21 17:12:05 +01:00
PascalWithopf
29c9fb1f9a codestyle: reduce line length to 120
lines are still checked for a length of 125 because
these are just some of the lines
2017-12-21 17:09:32 +01:00
Rainer Gerhards
19282d4810
Merge pull request #2267 from rgerhards/i2249
imfile: let user know if file was not found during FEN event notifica…
2017-12-21 14:35:28 +01:00
Rainer Gerhards
48ffcacc14 imklog: local host IP was hardcoded to 127.0.0.1
This is now taken from the global localHostIP setting, which is used
consistent accross all modules.

Also, the removed (2012?) directive $klogLocalIPIF has been added
again but directly marked as removed. That way, an informative error
message is generated if someone tries to use it.

closes https://github.com/rsyslog/rsyslog/issues/2276
2017-12-21 14:33:46 +01:00
Rainer Gerhards
bc8e88be63
Merge pull request #2277 from rgerhards/i1976
mmexternal: refactor and enhanced testbench test
2017-12-21 12:39:01 +01:00
PascalWithopf
79de51e4fc codestyle: reduce max line length to 125 2017-12-20 16:16:57 +01:00
Rainer Gerhards
bcb2df3ff2 omprog: some cleanup and fix theoretical ressource leak
If pipe fds could not be obtained, all ones before the failure were
lost.
2017-12-20 15:58:49 +01:00
Rainer Gerhards
147584c19b mmexternal: refactor stdin/out/err assignment
use more reliable and portable way

closes https://github.com/rsyslog/rsyslog/issues/2275
2017-12-20 15:58:26 +01:00
Rainer Gerhards
27db69da13 omprog refactor: make use of new core fct split_binary_parameters
This is code shared between omprog and mmexternal and has been moved
to srutils -- we now pick it from here.
2017-12-20 15:58:26 +01:00