1076 Commits

Author SHA1 Message Date
Rainer Gerhards
e8627a4ec5 remove obsolete pre-KSI GuardTime signature interface
this is no longer functional starting Jan 2018 as it
reached end of life.

closes https://github.com/rsyslog/rsyslog/issues/2128
2017-12-19 09:42:44 +01:00
Rainer Gerhards
46f4f43b8f build: permit to disable libcurl for rsyslog core
currently, this disables the http_request() script function

see also https://github.com/rsyslog/rsyslog/pull/2146#issuecomment-352412933
see also https://github.com/rsyslog/rsyslog/pull/2146#issuecomment-352413248
2017-12-18 18:56:37 +01:00
Rainer Gerhards
49efb53985 rainerscript: new function "http_request"
closes https://github.com/rsyslog/rsyslog/issues/1977
2017-12-16 17:47:08 +01:00
Rainer Gerhards
47c7d824db testbench: add a capability to turn off libfaketime tests via configure
Unfortunately, libfaketime becomes more and more problematic in newer
versions and causes aborts on some platforms. This provides the ability
to turn it off via --disable-libfaketime.

In the longer term, we should consider writing our own replacement.
2017-12-16 11:39:15 +01:00
Rainer Gerhards
ab7677108e cleanup: remove unused runtime instrumentation code
The --enable-rtinst configure switch and associated instrumentation
code dates back to the time pre-(valgrind, address sanitizer, ...).
It has not been actively used by rsyslog developers for years and is
inferior to the new debugging tools.

With this commit, we remove the --enable-rtinst option and do some
mild refactoring of the code itself. As it turned out, fully removing
the debug code requires a bit more of time, and we push this as a
follow-on activity. It's not as urgent as getting us to build
properly under all configure switches.

closes https://github.com/rsyslog/rsyslog/issues/2048
2017-12-12 16:39:36 +01:00
Rainer Gerhards
a256b9ab77
Merge pull request #2195 from rgerhards/i-2192
testbench: do not run tests that require root permissions by default
2017-12-11 13:45:35 +01:00
Rainer Gerhards
4e04e2ee55 testbench: provide capability to split testbench
This is necessary as we hit the Travis max runtime limit per VM,
so we need to duplicate the tests. This is done via
--enable-testbench1 and --enable-testbench2 which we than use
in different VMs.

This PR also includes updates to the Travis scripts so that we
use the new capability. We have only duplicated Travis VMs where
acutally necessary -- we may need to do more of this in the future.

closes https://github.com/rsyslog/rsyslog/issues/2196
2017-12-11 11:22:14 +01:00
Rainer Gerhards
0a5ec439e7 testbench: do not run tests that require root permissions by default
Root permissions are uncommon in the CI environmen (requires very special
setup). Test skip if run under non-root, but that pollutes the
testbench result. So the default now is not to do that. The can still be
activated via --enable-root-tests configure switch.

closes https://github.com/rsyslog/rsyslog/issues/2192
2017-12-10 18:43:29 +01:00
7508010750 Basic Imfile FEN API support for Solaris (#2141)
imfile: Initial implementation of solaris FEN API (without wildcards)

FEN API is used to implement simular event based file
monitoring like with inotify.

This commit provides partial functionality. It supports event-driven
processing of files but does not yet provide wildcard functionality.
Wildscards will be provided by a later commit.

This can be committed as-is because the previous code did also
not provide wildcard support, so this is an improvement for
Solaris in any case.

see also https://github.com/rsyslog/rsyslog/issues/1954
2017-12-01 12:49:38 +01:00
Rainer Gerhards
80bd3666fc build system: add capability to required libsystemd
and fail if it cannot be found. This is needed to ensure that builds
for platforms that actually need it succeed.

see also https://github.com/rsyslog/rsyslog/issues/2134
2017-11-30 09:12:57 +01:00
Rainer Gerhards
7bf280e9aa prepare for new development cycle 2017-11-28 10:06:09 +01:00
Thomas Deutschmann
659c15a6d4
configure: Fix "]: command not found"
Due to an error in the configure script, on systems without systemd support
an error like

  ./configure: line 17627: ]: command not found

is shown.

This commit fixes the error in the configure script.
2017-11-27 05:14:17 +01:00
Thomas Deutschmann
53941d0408
configure: Remove unnecessary message regarding man page existence
Since commit 5f5eca03a09671b5df609f9ee6f885d63bbedaca we no longer check for
the existence of the man page(s), so it is time to get rid of the message
as well.
2017-11-27 04:59:56 +01:00
Thomas Deutschmann
0fa12355bd
configure: Fix have_to_generate_man_pages value assignment
Closes: https://github.com/rsyslog/rsyslog/issues/1651
2017-11-27 04:44:40 +01:00
Rainer Gerhards
03d99f1238
Merge pull request #2091 from allanpark/Issue-2090-lmsig_ksils12_libksi_async_mode
Issue 2090: Add possibility to use the asyncronous mode of the libksi (>=3.16)
2017-11-24 12:59:23 +01:00
Rainer Gerhards
dfe89f29e0
Merge pull request #2071 from rgerhards/es-testbench-additions
testbench: improvements in elasticsearch testing
2017-11-24 10:43:15 +01:00
Rainer Gerhards
cf425b2c36 testbench/omkafka: update valgrind test to more flexible test framework 2017-11-24 08:14:32 +01:00
Rainer Gerhards
0fe2de8f24
Merge pull request #1917 from jjourdin/master
Improvement on ommongodb: mongo-c-driver with TLS and replicaset
2017-11-23 15:31:48 +01:00
Allan Park
a8a754ee40 Add possibility to use the asyncronous mode of the libksi (>=3.16) (Issue 2090) 2017-11-23 13:11:41 +02:00
Rainer Gerhards
acb62dfdab remove systemd embedded code, use library instead
Since the early days rsyslog uses the orginal systemd embedded
interface code. This commit changes it to libsystemd where available.
If we do not find libsystemd, we assume the system does not use
systemd, which is a safe assumption nowadays. This ensures we use the
fresh interface code and also removes bloat from our project.

closes https://github.com/rsyslog/rsyslog/issues/1933
2017-11-21 09:12:39 +01:00
Hugo SOSZYNSKI
60ac93f939 Updated the usertool logctl to use mongo-c-drvier instead of mongo-client. 2017-11-17 15:23:08 +01:00
Hugo SOSZYNSKI
54c8ef8e8f Added verification if ommongodb is enabled in config.ac for logctl 2017-11-16 14:10:57 +01:00
Hugo SOSZYNSKI
3eecc1a4c7 Adding libmongo-client for the user tools 2017-11-10 11:30:24 +01:00
Hugo SOSZYNSKI
cdeb6d3685 Going back with the configure.ac, removing the lib prefix to install in the default directories 2017-11-10 11:10:58 +01:00
Hugo SOSZYNSKI
5c221e3342 Changing syntax to detect libmongoc in the configure.ac 2017-11-10 10:53:42 +01:00
Hugo SOSZYNSKI
29e93a4a2c Updated the configure.ac to search libmongoc-1.0 instead of libmongo-client. 2017-10-27 16:34:14 +02:00
Rainer Gerhards
1813e7cc40 maintain ChangeLog 2017-10-18 08:03:49 +02:00
Rainer Gerhards
b53c48a910 prepare for 8.30.0 release 2017-10-16 13:44:44 +02:00
Rainer Gerhards
ad040017e8 Merge pull request #1807 from rgerhards/case-sensitive
handle (JSON) variables in case-insensitive way
2017-10-16 11:09:56 +02:00
Rainer Gerhards
6fe8153be9 handle (JSON) variables in case-insensitive way
The variable system inside rsyslog is JSON based (for easy consumption
of JSON input, the prime source of structured data). In JSON, keys
("variable names") are case-sensitive. This causes constant problems
inside rsyslog configurations. A major nit is that case-insensitivity
option inside templates (even if turned on) does not work with JSON
keys because they of inner workings*1.

It is much more natural to treat keys in a case-INsensitive way (e.g.
"$!Key" and "$!key" are the same). We do not expect any real problems
out of this, as key names only differing in case is highly unlikely.
However, as it is possible we provide a way to enable case-sensitivity
via the global(variables.casesensitve="on") global config object.

Note that the default is to do case-insensitive matches. The reason
is that this is the right thing to do in almost all cases, and we do
not envision any problems at all in existing deployments.

*1 Note: case-insensitivity in rsyslog is achieved by converting all
names to lower case. So that the higher speed of strcmp() can be used.
The template option does actually that, convert the template keys to
lower case. Unfortunately, this does not work with JSON, as JSON keys
are NOT converted to lower case.

closes https://github.com/rsyslog/rsyslog/issues/1805
2017-10-13 09:39:56 +02:00
Michael Biebl
775705e1d9 configure: move message modification modules to right spot in summary
closes https://github.com/rsyslog/rsyslog/issues/1648
2017-10-05 13:03:08 +02:00
Jiri Vymazal
83388eabe9 enabling falback to older systemd version without restart-less
persistent journal switch
2017-09-07 12:19:18 +02:00
Jiri Vymazal
92ac8016e4 Made switching to persistent journal in runtime possible,
now you can continue logging after switch to persistent
journal without need to restart rsyslog service.
Bumped correspondidng systemd version dependency.
2017-09-05 14:12:31 +02:00
Rainer Gerhards
5ffa5114a1 bump version for next release cycle 2017-08-08 17:15:55 +02:00
Rainer Gerhards
c4ba96f621 prepare for 8.29.0 release 2017-08-08 14:39:34 +02:00
Rainer Gerhards
96d11c43b2 Merge pull request #1465 from PascalWithopf/pmnormalize
pmnormalize: new parser module
2017-08-01 17:12:29 +02:00
Rainer Gerhards
cf2197d740 testbench bugfix: hang in tests if omstdout is not present
Many tests depend on omstdout. Given the fact that omstdout
is really only useful for the testbench (at least that's the intent),
we now require --enable-omstdout if --enable-testbench is given.

The alternative would have been to disable all those tests that
need it, which would have lead to considerable less testbench
coverage.

closes https://github.com/rsyslog/rsyslog/issues/1649
2017-07-20 14:31:08 +02:00
PascalWithopf
d4cc699129 pmnormalize: add new parser module pmnormalize 2017-07-19 08:58:20 +02:00
Rainer Gerhards
b5096fa6ee ksi subsystem: remove no longer needed code
The KSI subsystem has been replaced by a newer Guardtime-provided
subsystem. Note that the old KSI subsystem does no longer work
to to Guardtime backend changes.

Libgt still continous to work.

closes https://github.com/rsyslog/rsyslog/issues/1590
2017-07-03 08:12:03 +02:00
Rainer Gerhards
e9617cf5f2 bump version number for new development cycle 2017-06-27 16:07:28 +02:00
Rainer Gerhards
5f91d47283 prepare for 8.28.0 release 2017-06-27 15:40:40 +02:00
Rainer Gerhards
d060be7920 Merge pull request #1609 from alorbach/kafka-extension-omkafka
Kafka extension omkafka
2017-06-21 08:16:53 +02:00
8ecf0aeb34 kafka: Added option enable static linking of librdkafa (disabled by default).
Also saving failed msgs now when kafka produce fails. No message loss should
be possible anymore.

Also removed old kafka tests and readded new kafka tests.
If kafka instances cannot be started (happens from time to time when
kafka/zookepper sync fails), tests will be skipped instead of failing.

Closes https://github.com/rsyslog/rsyslog/issues/1559
Closes https://github.com/rsyslog/rsyslog/issues/1584
Closes https://github.com/rsyslog/rsyslog/issues/1515
May fix https://github.com/rsyslog/rsyslog/issues/1230
Closes https://github.com/rsyslog/rsyslog/issues/1052
2017-06-19 17:14:35 +02:00
732d0e21a7 omkafka: Enhanced support to detect kafka broker problems.
Also fixed tryresume and doAction handling when action is
in suspend state.

Calling for Callbacks more often.

Adjusted kafka fail test settings.

diag.sh script detects of kafka server comes up proberly

Added liblz4 to dependencies for omkafka/imkafka needed
for static linking.

This addresses the omkafka queue problem from issue:
https://github.com/rsyslog/rsyslog/issues/1052
Yet it does not fully solve the message problem yet,
but I am working on this.
2017-06-08 15:35:45 +02:00
Allan Park
f11a911add bugfix build system: add libksi only to those binaries that need it
Previously, libksi was added to all binaries when --enable-ksi-ls12
was activated.

closes https://github.com/rsyslog/rsyslog/issues/1600
2017-06-04 11:30:41 +02:00
Rainer Gerhards
97f1c3f67f Merge pull request #1589 from rgerhards/mmdb-enh
mmdblookup: enhancements and fixes
2017-06-01 14:26:28 +02:00
Rainer Gerhards
aecdbdf360 mmdblookup: upgrade from contrib module to fully supported one 2017-05-31 13:43:05 +02:00
Nate Brown
b323f53ca7 relp version bump 2017-05-30 20:44:21 -07:00
Rainer Gerhards
24780ef71d Merge pull request #1569 from rgerhards/nitfixes
cosmetic: build system still used JSON_C name
2017-05-19 10:17:33 +02:00
Thomas Deutschmann
15e2e8fc70
configure: imkafka: Only check for rdkafka library when package config check has failed 2017-05-17 23:56:03 +02:00