1426 Commits

Author SHA1 Message Date
Rainer Gerhards
e134b545b3 mmpstrucdata: make case preservation of SD_IDs optional
commit 5cee06a60 introduced case-perservation, but did not make
it optional. This broke existing behavior and thus potentially
existing configurations. This commit introduces a new parameter
"sd_name.lowercase" which now controls this behavior.

Note that 5cee06a60 is merged to master branch together with this
commit, so no released code ever had the potential regression.

see also https://github.com/rsyslog/rsyslog/pull/2181
2018-01-07 10:18:57 +01:00
Rainer Gerhards
48a49644d5 Merge branch 'master' of https://github.com/alanrobson/rsyslog into merge-2181 2018-01-06 11:28:01 +01:00
Rainer Gerhards
23ae3018bf
Merge pull request #2358 from jgerhards/mmanon-error
mmanon bugfix: fix wrong ipv6 embedded recognition
2018-01-05 17:20:04 +01:00
Rainer Gerhards
2b2678a72a
Merge pull request #2353 from alorbach/imfile-i2271
imfile: Fixed issues not detecting files in directory when wildcards …
2018-01-05 17:14:39 +01:00
Rainer Gerhards
9a7481aa43
Merge pull request #2335 from rgerhards/i326
config processing: bare variable root access not permitted
2018-01-05 17:12:27 +01:00
af09d33b49 imfile: Partially fixed issues not detecting files in directory when wildcards are used.
When directories and files are created at the same time,
imfile may missed subdirs or file if the machine is on high load.

The handling has been enhanced to scan newly created directories ALWAYS for
matching files.

This should fix following issues:
closes https://github.com/rsyslog/rsyslog/issues/2271

However we still have a problem when a multilevel directory configurations.
Details are discussed here https://github.com/rsyslog/rsyslog/issues/2354
2018-01-05 12:59:19 +01:00
Jan Gerhards
782982d94a mmanon bugfix: fix wrong ipv6 embedded recognition
mmanon recognizes IPv6 with embedded IPv4 that have too few (16 bit) fields.

example: 13:abd:45:0.0.0.0

closes https://github.com/rsyslog/rsyslog/issues/2357
2018-01-05 12:33:36 +01:00
Rainer Gerhards
31cc4355bd testbench: check if external URLs can be accessed, skip if not
They may not be accessible because they (or parts of the
network) can be down or access is not permitted. In this
case, the affected tests skip themselves.

Special thanks to Thomas D. (Whissi) for providing the check
function.

closes https://github.com/rsyslog/rsyslog/issues/2339
2018-01-03 17:25:12 +01:00
Rainer Gerhards
22df64124c testbench: add test case for bare empty var root access 2018-01-03 16:20:32 +01:00
Rainer Gerhards
ff7e123872 testbench: add test for bare variable root script access 2018-01-03 16:20:32 +01:00
Rainer Gerhards
03531cb4ab testbench: temporarily disable another imfile tests
this is known to fail due to known bug in imfile (which
will be worked on soon).

Thanks to whissi for mentioning this test.

see also https://github.com/rsyslog/rsyslog/issues/2325
2018-01-03 09:34:44 +01:00
Rainer Gerhards
b83647fc97 testbench: try to provide minimal locaction info on aborts
While we try to keep the testbench output brief, we try to do a single
stacktrace if we find a core file.
2017-12-31 18:31:02 +01:00
Rainer Gerhards
046ce2ac82
Merge pull request #2328 from rgerhards/better-q-shutdown-msgs
core/queue: emit better status messages at rsyslog shutdown
2017-12-31 10:37:32 +01:00
Rainer Gerhards
79eb210891
Merge pull request #2322 from rgerhards/travis-reenable-statan
travis: reenable static analyzer
2017-12-30 17:31:39 +01:00
Rainer Gerhards
49e13d5555 core/queue: emit better status messages at rsyslog shutdown
this helps to diagnose issue - unfortunately we need more
work to ensure that the messages always make it to the user. This
is a start and hopefully useful at least for the testbench, possibly
more.
2017-12-30 17:08:34 +01:00
Rainer Gerhards
f679ac6fd2
Merge pull request #2317 from rgerhards/i2313alt
omkafka: refactor and fixes
2017-12-30 17:08:18 +01:00
Rainer Gerhards
0fcc9dc609 travis: reenable static analyzer
LLVM has changed packaging structure, this fix adapts to it. So we
now have scan-build-5.0 again.
2017-12-30 12:46:45 +01:00
Rainer Gerhards
0fb47e346c testbench: TEMPORARILY disable tests to permit "normal" CI runs
... because we know they sometimes fail due to a confirmed bug
inside imfile. These need to be re-enabled once the bug is fixed.

see also https://github.com/rsyslog/rsyslog/issues/2271
2017-12-30 12:25:28 +01:00
Rainer Gerhards
a872d1fc73 testbench/kafka: increase queue timeouts
on slow testbench machines, we may cancel processing where this is not
desired. Most importantly, this could lead to small memorey leaks
due to the thread cancellation (which are then expected!) that then
lead to false test failures.
2017-12-30 10:58:14 +01:00
Rainer Gerhards
02761dd0d8 testbench: clean up "normal" error message in kafka test plumbing
The error message regularly occurs if no kafka/zk instance is running,
which should be the case at start of test. However, the message
tends to make think reviewers that it is a real error. So we now
hide it.
2017-12-30 10:58:14 +01:00
Jan Gerhards
f5eef34361 rscript: add int2Hex
closes https://github.com/rsyslog/rsyslog/issues/2224
2017-12-24 11:46:57 +01:00
Rainer Gerhards
27f3cc4e04
Merge pull request #2286 from rgerhards/imkafka-msg
Imkafka: some smaller improvements
2017-12-24 10:47:19 +01:00
Rainer Gerhards
44eacd7261 Travis: line length check was not run, re-enabling
This was a side-effect of disabling static analyzer, which we were forced to
due to missing LLVM packages.
2017-12-23 14:06:48 +01:00
Rainer Gerhards
49f9b26601
Merge pull request #2246 from mrworkman/rscript-is-timestamp
Add is_time() RainerScript function.
2017-12-22 12:49:02 +01:00
Rainer Gerhards
a5f51eb366 testbench: make kafka tests not spam the test log
closes https://github.com/rsyslog/rsyslog/issues/2086
2017-12-22 12:46:34 +01:00
PascalWithopf
29c9fb1f9a codestyle: reduce line length to 120
lines are still checked for a length of 125 because
these are just some of the lines
2017-12-21 17:09:32 +01:00
Rainer Gerhards
bc8e88be63
Merge pull request #2277 from rgerhards/i1976
mmexternal: refactor and enhanced testbench test
2017-12-21 12:39:01 +01:00
Rainer Gerhards
049d750079
Merge pull request #2279 from PascalWithopf/line_length_correction
codestyle: reduce max line length to 125
2017-12-21 11:39:17 +01:00
PascalWithopf
79de51e4fc codestyle: reduce max line length to 125 2017-12-20 16:16:57 +01:00
Rainer Gerhards
ac209cd471 testbench: update and activate mmexternal test for invalid program
closes https://github.com/rsyslog/rsyslog/issues/1976
2017-12-20 15:58:25 +01:00
Rainer Gerhards
57837d0dd7 Travis: enable omgssapi and usertools 2017-12-20 15:53:15 +01:00
Rainer Gerhards
cb309b22a2 testbench/mmexternal: add testcase for invalid external script
this currently leads to misadressing; not activated yet in testbench,
will happen after the bugfix

see also https://github.com/rsyslog/rsyslog/issues/1976
2017-12-19 19:10:58 +01:00
Stephen Workman
cdc779760b Add is_time() RainerScript function. 2017-12-19 09:12:09 -05:00
4486cc0131 imfile: implemented wildcard support into FEN mode (solaris)
Partitionally moved inotify code to global imfile code as needed
for FEN Mode.
Enabled all wildcard tests that are supported by FEN API.
2017-12-19 11:03:34 +01:00
Rainer Gerhards
6a140a2cb3 script engine: improper string-to-number conversion for negative numbers
Thanks to Stephen Workman for providing a very detailled hint of the
problem and root cause. This patch bases on his recommendation for a
fix.

closes https://github.com/rsyslog/rsyslog/issues/2255
2017-12-18 17:50:32 +01:00
Rainer Gerhards
a395a79879
Merge pull request #2146 from rgerhards/i-1977
rainerscript: new function "http_request"
2017-12-18 13:08:00 +01:00
Stephen Workman
d068170818 Fix test so it will run on FreeBSD. 2017-12-16 18:49:12 -05:00
Rainer Gerhards
49efb53985 rainerscript: new function "http_request"
closes https://github.com/rsyslog/rsyslog/issues/1977
2017-12-16 17:47:08 +01:00
Rainer Gerhards
0d6b3d47f7 testbench: fix new --enable-libfaketime configure option
some tests requiring faketime were executed even when this was
disabled
2017-12-16 16:26:07 +01:00
Rainer Gerhards
835d6b104b
Merge pull request #2233 from rgerhards/i-2212
testbench: name valgrind tests consitently (end with -vg.sh)
2017-12-16 16:19:00 +01:00
Rainer Gerhards
78356cdf1b testbench: name valgrind tests consitently (end with -vg.sh)
closes https://github.com/rsyslog/rsyslog/issues/2212
2017-12-16 12:19:28 +01:00
Rainer Gerhards
47c7d824db testbench: add a capability to turn off libfaketime tests via configure
Unfortunately, libfaketime becomes more and more problematic in newer
versions and causes aborts on some platforms. This provides the ability
to turn it off via --disable-libfaketime.

In the longer term, we should consider writing our own replacement.
2017-12-16 11:39:15 +01:00
Rainer Gerhards
a9601ebaec pmrfc3164: try to detected headerless message
headerless is indicated by whitespace followed by either '{' or '['.
This follows the trend to send JSON messages via syslog without any
header. We use default header values in this case.

This right now is a bit experimental; we may roll it back if
problems show up in practice.

closes https://github.com/rsyslog/rsyslog/issues/2030
2017-12-12 18:25:16 +01:00
Rainer Gerhards
f0f173325e
Merge pull request #2201 from rgerhards/testbench-timeout
testbench: reduce startup timeout for tests
2017-12-12 16:43:01 +01:00
Rainer Gerhards
23a41c87d4
Merge pull request #2198 from rgerhards/i-1979
config: implement function parse_json()
2017-12-12 16:42:43 +01:00
Rainer Gerhards
f23a1d3959 testbench: reduce startup timeout for tests
previous timeout caused very long runtime in cases where rsyslog
segfaulted on startup, which made the testbench failure harder to
diagnose than necessary.
2017-12-12 11:39:54 +01:00
Rainer Gerhards
0d0ff11991 config: implement function parse_json()
It is useful to be able to parse arbitrary JSON strings without the need
of an external message modification module. For example, together with
    https://github.com/rsyslog/rsyslog/issues/1977
this can be used to parse out specific fields from Amazon AWS
configuration info. There are also other ample uses for such functionality.

closes https://github.com/rsyslog/rsyslog/issues/1979
2017-12-12 11:35:29 +01:00
Rainer Gerhards
4d1e10ad46
Merge pull request #2191 from jgerhards/rscript_substring
rainerscript: implement substring function
2017-12-12 11:07:25 +01:00
Rainer Gerhards
a256b9ab77
Merge pull request #2195 from rgerhards/i-2192
testbench: do not run tests that require root permissions by default
2017-12-11 13:45:35 +01:00
Rainer Gerhards
4e04e2ee55 testbench: provide capability to split testbench
This is necessary as we hit the Travis max runtime limit per VM,
so we need to duplicate the tests. This is done via
--enable-testbench1 and --enable-testbench2 which we than use
in different VMs.

This PR also includes updates to the Travis scripts so that we
use the new capability. We have only duplicated Travis VMs where
acutally necessary -- we may need to do more of this in the future.

closes https://github.com/rsyslog/rsyslog/issues/2196
2017-12-11 11:22:14 +01:00