1409 Commits

Author SHA1 Message Date
Rainer Gerhards
f679ac6fd2
Merge pull request #2317 from rgerhards/i2313alt
omkafka: refactor and fixes
2017-12-30 17:08:18 +01:00
Rainer Gerhards
0fb47e346c testbench: TEMPORARILY disable tests to permit "normal" CI runs
... because we know they sometimes fail due to a confirmed bug
inside imfile. These need to be re-enabled once the bug is fixed.

see also https://github.com/rsyslog/rsyslog/issues/2271
2017-12-30 12:25:28 +01:00
Rainer Gerhards
a872d1fc73 testbench/kafka: increase queue timeouts
on slow testbench machines, we may cancel processing where this is not
desired. Most importantly, this could lead to small memorey leaks
due to the thread cancellation (which are then expected!) that then
lead to false test failures.
2017-12-30 10:58:14 +01:00
Rainer Gerhards
02761dd0d8 testbench: clean up "normal" error message in kafka test plumbing
The error message regularly occurs if no kafka/zk instance is running,
which should be the case at start of test. However, the message
tends to make think reviewers that it is a real error. So we now
hide it.
2017-12-30 10:58:14 +01:00
Jan Gerhards
f5eef34361 rscript: add int2Hex
closes https://github.com/rsyslog/rsyslog/issues/2224
2017-12-24 11:46:57 +01:00
Rainer Gerhards
27f3cc4e04
Merge pull request #2286 from rgerhards/imkafka-msg
Imkafka: some smaller improvements
2017-12-24 10:47:19 +01:00
Rainer Gerhards
44eacd7261 Travis: line length check was not run, re-enabling
This was a side-effect of disabling static analyzer, which we were forced to
due to missing LLVM packages.
2017-12-23 14:06:48 +01:00
Rainer Gerhards
49f9b26601
Merge pull request #2246 from mrworkman/rscript-is-timestamp
Add is_time() RainerScript function.
2017-12-22 12:49:02 +01:00
Rainer Gerhards
a5f51eb366 testbench: make kafka tests not spam the test log
closes https://github.com/rsyslog/rsyslog/issues/2086
2017-12-22 12:46:34 +01:00
PascalWithopf
29c9fb1f9a codestyle: reduce line length to 120
lines are still checked for a length of 125 because
these are just some of the lines
2017-12-21 17:09:32 +01:00
Rainer Gerhards
bc8e88be63
Merge pull request #2277 from rgerhards/i1976
mmexternal: refactor and enhanced testbench test
2017-12-21 12:39:01 +01:00
Rainer Gerhards
049d750079
Merge pull request #2279 from PascalWithopf/line_length_correction
codestyle: reduce max line length to 125
2017-12-21 11:39:17 +01:00
PascalWithopf
79de51e4fc codestyle: reduce max line length to 125 2017-12-20 16:16:57 +01:00
Rainer Gerhards
ac209cd471 testbench: update and activate mmexternal test for invalid program
closes https://github.com/rsyslog/rsyslog/issues/1976
2017-12-20 15:58:25 +01:00
Rainer Gerhards
57837d0dd7 Travis: enable omgssapi and usertools 2017-12-20 15:53:15 +01:00
Rainer Gerhards
cb309b22a2 testbench/mmexternal: add testcase for invalid external script
this currently leads to misadressing; not activated yet in testbench,
will happen after the bugfix

see also https://github.com/rsyslog/rsyslog/issues/1976
2017-12-19 19:10:58 +01:00
Stephen Workman
cdc779760b Add is_time() RainerScript function. 2017-12-19 09:12:09 -05:00
4486cc0131 imfile: implemented wildcard support into FEN mode (solaris)
Partitionally moved inotify code to global imfile code as needed
for FEN Mode.
Enabled all wildcard tests that are supported by FEN API.
2017-12-19 11:03:34 +01:00
Rainer Gerhards
6a140a2cb3 script engine: improper string-to-number conversion for negative numbers
Thanks to Stephen Workman for providing a very detailled hint of the
problem and root cause. This patch bases on his recommendation for a
fix.

closes https://github.com/rsyslog/rsyslog/issues/2255
2017-12-18 17:50:32 +01:00
Rainer Gerhards
a395a79879
Merge pull request #2146 from rgerhards/i-1977
rainerscript: new function "http_request"
2017-12-18 13:08:00 +01:00
Stephen Workman
d068170818 Fix test so it will run on FreeBSD. 2017-12-16 18:49:12 -05:00
Rainer Gerhards
49efb53985 rainerscript: new function "http_request"
closes https://github.com/rsyslog/rsyslog/issues/1977
2017-12-16 17:47:08 +01:00
Rainer Gerhards
0d6b3d47f7 testbench: fix new --enable-libfaketime configure option
some tests requiring faketime were executed even when this was
disabled
2017-12-16 16:26:07 +01:00
Rainer Gerhards
835d6b104b
Merge pull request #2233 from rgerhards/i-2212
testbench: name valgrind tests consitently (end with -vg.sh)
2017-12-16 16:19:00 +01:00
Rainer Gerhards
78356cdf1b testbench: name valgrind tests consitently (end with -vg.sh)
closes https://github.com/rsyslog/rsyslog/issues/2212
2017-12-16 12:19:28 +01:00
Rainer Gerhards
47c7d824db testbench: add a capability to turn off libfaketime tests via configure
Unfortunately, libfaketime becomes more and more problematic in newer
versions and causes aborts on some platforms. This provides the ability
to turn it off via --disable-libfaketime.

In the longer term, we should consider writing our own replacement.
2017-12-16 11:39:15 +01:00
Rainer Gerhards
a9601ebaec pmrfc3164: try to detected headerless message
headerless is indicated by whitespace followed by either '{' or '['.
This follows the trend to send JSON messages via syslog without any
header. We use default header values in this case.

This right now is a bit experimental; we may roll it back if
problems show up in practice.

closes https://github.com/rsyslog/rsyslog/issues/2030
2017-12-12 18:25:16 +01:00
Rainer Gerhards
f0f173325e
Merge pull request #2201 from rgerhards/testbench-timeout
testbench: reduce startup timeout for tests
2017-12-12 16:43:01 +01:00
Rainer Gerhards
23a41c87d4
Merge pull request #2198 from rgerhards/i-1979
config: implement function parse_json()
2017-12-12 16:42:43 +01:00
Rainer Gerhards
f23a1d3959 testbench: reduce startup timeout for tests
previous timeout caused very long runtime in cases where rsyslog
segfaulted on startup, which made the testbench failure harder to
diagnose than necessary.
2017-12-12 11:39:54 +01:00
Rainer Gerhards
0d0ff11991 config: implement function parse_json()
It is useful to be able to parse arbitrary JSON strings without the need
of an external message modification module. For example, together with
    https://github.com/rsyslog/rsyslog/issues/1977
this can be used to parse out specific fields from Amazon AWS
configuration info. There are also other ample uses for such functionality.

closes https://github.com/rsyslog/rsyslog/issues/1979
2017-12-12 11:35:29 +01:00
Rainer Gerhards
4d1e10ad46
Merge pull request #2191 from jgerhards/rscript_substring
rainerscript: implement substring function
2017-12-12 11:07:25 +01:00
Rainer Gerhards
a256b9ab77
Merge pull request #2195 from rgerhards/i-2192
testbench: do not run tests that require root permissions by default
2017-12-11 13:45:35 +01:00
Rainer Gerhards
4e04e2ee55 testbench: provide capability to split testbench
This is necessary as we hit the Travis max runtime limit per VM,
so we need to duplicate the tests. This is done via
--enable-testbench1 and --enable-testbench2 which we than use
in different VMs.

This PR also includes updates to the Travis scripts so that we
use the new capability. We have only duplicated Travis VMs where
acutally necessary -- we may need to do more of this in the future.

closes https://github.com/rsyslog/rsyslog/issues/2196
2017-12-11 11:22:14 +01:00
Rainer Gerhards
5099df4319
Merge pull request #2188 from rgerhards/i-1939
RainerScript: implement previous_action_suspended() function
2017-12-11 10:26:23 +01:00
Rainer Gerhards
0a5ec439e7 testbench: do not run tests that require root permissions by default
Root permissions are uncommon in the CI environmen (requires very special
setup). Test skip if run under non-root, but that pollutes the
testbench result. So the default now is not to do that. The can still be
activated via --enable-root-tests configure switch.

closes https://github.com/rsyslog/rsyslog/issues/2192
2017-12-10 18:43:29 +01:00
Jan Gerhards
effee21a81 rainerscript: implement substring function 2017-12-10 14:24:46 +01:00
Rainer Gerhards
ce06bba15c RainerScript: implement previous_action_suspended() function
This function returns a boolean indicating if the previous action is
suspended (0 - no, 1 - yes). This is useful if arbitrary processing
(other than calling actions) should be made depending on that state.
A prime example for this is probably calling a ruleset.

closes https://github.com/rsyslog/rsyslog/issues/1939
2017-12-09 17:48:28 +01:00
Rainer Gerhards
2614b1803c
Merge pull request #2172 from GrayTShirt/pg
update postgres output module
2017-12-09 17:25:22 +01:00
Rainer Gerhards
4af883651e
Merge pull request #2185 from rgerhards/i-1978
RainerScript: add function script_error() and error-reporting support
2017-12-09 17:13:15 +01:00
Rainer Gerhards
b63d3051bc RainerScript: add function script_error() and error-reporting support
This permits script functions that could fail to report errors back, so
that the user can handle them if desired. We use an errno-style of
interface. That means script_error() needs to be called after functions
that supports it. It will return either 0 (all OK) or something else
(an error condition).

The commit also modifies the parse_time() function to use the new
interface. First of all, this is useful for users, and secondly we
needed a capability to craft a testbench test.

closes https://github.com/rsyslog/rsyslog/issues/1978
2017-12-09 16:08:36 +01:00
Dan Molik
f334e2b250 ompgsql plugin - update postgres module
The PostgreSQL output module was woefully out-of-date the following
list is changes made to update the module to current Rsyslog standards.

  - allow for v6 configuration syntax
  - configurable ports
  - support transactional interface
  - push db connection into workers (libpq is threadsafe)
  - enable module testing on travis
  - ensure configuration syntax backwards compatibility
  - formatting around postgres core templating
  - use new test conventions
  - add new configuration syntax test
  - add valgrind tests for new and old syntax
  - add threading tests
  - add action queue long running tests
  - add action queue valgrind test
2017-12-09 09:04:46 -05:00
Rainer Gerhards
ba89fd38ab testbench cleanup
closes https://github.com/rsyslog/rsyslog/pull/2170
2017-12-09 13:39:30 +01:00
ed7ca31ce6 imkafka: Fixed memory leak in msgConsume
When rkmessage->err was set, rkmessage was not proberly destroyed with
rd_kafka_message_destroy.

Closes https://github.com/edenhill/librdkafka/issues/1536
2017-12-06 09:42:38 +01:00
Jan Gerhards
38d92ff814 rainerscript: bugfix
fix a bug causing a crash if rtrim or ltrim were called
with a string containing a blank and one single different character.

closes https://github.com/rsyslog/rsyslog/issues/2127
2017-12-02 18:26:39 +01:00
Rainer Gerhards
e6ea9a1ac6 testbench: enable more tests on BSD
... by fixing command lines that are not valid under BSD.
2017-12-02 10:32:01 +01:00
7508010750 Basic Imfile FEN API support for Solaris (#2141)
imfile: Initial implementation of solaris FEN API (without wildcards)

FEN API is used to implement simular event based file
monitoring like with inotify.

This commit provides partial functionality. It supports event-driven
processing of files but does not yet provide wildcard functionality.
Wildscards will be provided by a later commit.

This can be committed as-is because the previous code did also
not provide wildcard support, so this is an improvement for
Solaris in any case.

see also https://github.com/rsyslog/rsyslog/issues/1954
2017-12-01 12:49:38 +01:00
Rainer Gerhards
8d8abf2d9b travis: enable gcc-7 with high warning levels
see also https://github.com/rsyslog/rsyslog/pull/2121
2017-11-29 12:39:07 +01:00
Rainer Gerhards
da01aae879
Merge pull request #2126 from mrworkman/rscript-parse-time-fix
Fix to work on systems where Python 3.x is the default.
2017-11-28 08:07:39 +01:00
Stephen Workman
04c903a05d Fix to work on systems where Python 3.x is the default. 2017-11-27 16:44:26 -05:00