318 Commits

Author SHA1 Message Date
Rainer Gerhards
370b86133b
imfile: add new input parameter escapeLF.replacement
The new parameter permits to specify a replacement to be configured
when "escapeLF" is set to "on". Previously, a fixed replacement string
was used ("#012"/"\n") depending on circumstances. If the parameter is
set to an empty string, the LF is simply discarded.

closes https://github.com/rsyslog/rsyslog/issues/3889
2019-11-14 14:31:24 +01:00
Rainer Gerhards
51aad39e4a
Merge pull request #3932 from VultureProject/mmdarwin
MMDARWIN:: Optimisations, new parameters, update to protocol header
2019-11-05 09:01:04 +01:00
frikilax
578f74161a MMDARWIN:: Optimisations, new parameters, update to protocol header
- use permanent worker-dependent buffers to avoid malloc/free for each entry
- move socket structures to worker data, remove global mutex
- add log lines for parameters and general workflow
- don't send body if empty/incomplete (see new parameters)
- don't close/reopen socket every time -> let session open or create new every X entry (see new parameters)
- clean up code

- added 'send_partial', to let mmdarwin send body if not all fields were retrieved, or not
  default false = only send complete bodies
- added 'socket_max_use' to open new session every X packet, useful for some versions of Darwin (prior to 1.1)
  default is 0 = do not open new session/keep only one

- added 'evt_id' to the darwin header (Darwin v1+ compatibility)
2019-11-03 16:10:00 +01:00
Rainer Gerhards
fb246cef83
Merge pull request #3906 from rgerhards/calloc
mmkubernetes bugfix: improper use of calloc()
2019-11-01 16:19:44 +01:00
Rainer Gerhards
36231cb020
Merge pull request #3882 from n2yen/improg-fix
bug fix: allow improg to handle multi-line inputs
2019-10-30 18:48:08 +01:00
Rainer Gerhards
b4e625f074
mmkubernetes bugfix: improper use of calloc()
could cause problems under extreme memory shortage - very unlikely
credits to LGTM.COM for detecting this
2019-10-30 18:46:06 +01:00
Rainer Gerhards
1aea234ac7
mmkubernetes bugfix: improper use of calloc()
can cause problems under extreme memory shortage - very unlikely

credits to LGTM.COM for detecting this
2019-10-16 17:39:42 +02:00
Nelson Yen
cc0537c808 bug fix: allow improg to handle multi-line inputs
miscellaneous bug fixes in improg:
- properly truncate string after an input event is submitted
- set msgoffset to 0.
- tests added to check above fixes
2019-10-02 10:46:13 -07:00
Rainer Gerhards
513d7f9e24
Merge pull request #3740 from rgerhards/suse-fix
core bugfix: incorrect error message on duplicate module load
2019-10-01 09:53:47 +02:00
Rainer Gerhards
7092b225f1
Merge pull request #3883 from rgerhards/fix-pmcisco
pmcisconames bugfix: potential misadressing
2019-10-01 08:27:14 +02:00
Rainer Gerhards
10549ba915
pmaixforwardedfrom bugfix: potential misadressing 2019-09-30 13:22:37 +02:00
Rainer Gerhards
abc0960a75 pmcisconames bugfix: potential misadressing 2019-09-30 11:34:09 +02:00
Philippe Duveau
b0894088b6
Out of bounds issue
Add a new sanity check after determining the level len.
2019-09-24 20:45:25 +02:00
Nelson Yen
12ba857bdf Enable checkpath option in omhttp
omhttp, 'checkpath' option, was not configurable in the past.
- add 'checkpath' to the cnfparamdescr table.
- fix issue with checkpath passing extra garbage characters in string.
- add 'checkpath' into unit test - omhttp-retry.sh
2019-07-29 09:03:23 -07:00
Rainer Gerhards
2a33b2877a
maintain ChangeLog
plus a very minor formatting change, which I think is OK to submit
together with the ChangeLog.
2019-07-24 15:24:03 +02:00
major
43d7365f20 mmdarwin: add new module
This is a contributed module.
2019-07-24 13:30:09 +02:00
Rainer Gerhards
2e7ae2ccf0
imczmq nitfix: potential NULL ptr in printf on out-of-memory condition
very unlikely to happen but if it does without any real issue on
most platforms.
2019-07-05 09:42:20 +02:00
Rich Megginson
7c46fc0112 omamqp1 - port to latest api, add tests
This brings omamqp1 up-to-date with the latest qpid-proton-c
api version.  This also adds a test for the plugin, to test
the basic functionality.  The test requires the user to
install qdrouterd and the python qpid-proton library in order
to use the simple_recv.py test program.
2019-06-11 08:04:17 -06:00
Rainer Gerhards
fcb83fb247
Merge pull request #3476 from pduveau/omrabbitmq_featured
Featuring omrabbitmq
2019-04-09 12:53:43 +02:00
Rainer Gerhards
2ba3924863
Merge pull request #3603 from n2yen/rm-imdocker-unittest
remove redundant compiled imdocker-unit-tests
2019-04-09 12:34:35 +02:00
Nelson Yen
47bfa582c0 fix coverity errors
- remove logically dead code.
- remove unnecessary NULL check
2019-04-08 14:33:46 -07:00
Nelson Yen
f72dbe4009 remove unneeded imdocker-unit-tests
Remove the compile time tests which are now redundant,
due to the variety of imdocker testbench tests.
2019-04-08 14:22:11 -07:00
Philippe Duveau
3f673be483 improg : tests (traces) + patch + distcheck 2019-04-08 20:33:40 +02:00
Philippe Duveau
b2600c2aa6 Featuring omrabbitmq + tests + LGTM alert 2019-04-08 12:11:33 +02:00
Philippe Duveau
969c337ac0 Create contrib pmdb2diag + alignment for clang + test 2019-04-04 04:12:36 +02:00
Rainer Gerhards
f4ed61e9ce
Merge pull request #3539 from richm/mmkubernetes-cache-expiration
mmkubernetes - support for metadata cache expiration
2019-04-03 12:52:57 +02:00
Nelson Yen
82142c7c73 imdocker input plugin - enables pulling container logs from docker
- plugin will get (docker) container logs from a host as well as filling out some
    basic container metadata as id, name, image, labels.
  - requirements: curl-7.40.0+, for unix_domain_socket option (Docker API listens on a
    local socket). Additionally, plugin will handle the issue of log lines larger than
    16KB begin split by Docker.
  - include imdocker-unittests in testbench
  - enable imdocker tests in osx travis tests
  - use curl master branch for imdocker-tests
  - fix cflags for pthread
  - enforce minimum curl version during configuration - to 7.40.0 for imdocker
  - container polls after the initial one are automatically filtered utilizing the 'since' option as per docker api
  - imdocker tests refactored.
  - add escapeLF config option

Test bench test added:
  - imdocker-basic: checks for completeness.
  - imdocker-basic-vg: same as basic with valgrind enabled.
  - imdocker-long-logline: checks imdocker can handle 16K+ log lines.
  - imdocker-long-logline-vg: same as long-logline test with valgrind enabled.
  - add option 'retrieveNewLogsFromStart' which will cause imdocker to ignore the 'tail'
  option when retrieving container logs for newly activated containers. Containers that
  were already active when imdocker first starts still uses the tail option.
  - imdocker-new-logs-from-start: checks that new containers will ignore the tail option
  - imdocker-new-logs-from-start-vg: valgrind enabled
  - multi-line support via docker label: imdocker.startregex
  added multi-line tests for testbench

Travis test bench related history and notes:
 - Travis tests, enable imdocker
 - imdocker travis tests not enabled in precise or trusty ubuntu versions of travis-test.
   Assuming travis tests will run in ubuntu 16
 - tests: harden container name against strange file system pathes
   RSYSLOG_DYNNAME contains the file system path, which is NOT guarnateed
   to be suitable for things other than file names.
 - tests: bugfix and guard against false negative
 - content_check_with count was done before rsyslog was terminated, so
   it was uncertain how many messages were written at that spot
   (especially on slow machines)
 - so far, only the number of lines was checked, not their content. So
   any problems with the actual messages would not have been detected.
 - improved support for listcontainer options, fix racy imdocker tests
2019-04-02 08:29:05 +02:00
Rich Megginson
8b92ea1bd9 mmkubernetes - support for metadata cache expiration
New parameters for mmkubernetes (module and action):

* `cacheexpireinterval`
If `cacheexpireinterval` is -1, then do not check for cache expiration.
If `cacheexpireinterval` is 0, then check for cache expiration.
If `cacheexpireinterval` is greater than 0, check for cache expiration
if the last time we checked was more than this many seconds ago.

* `cacheentryttl` - maximum age in seconds for cache entries

New statistics counters:

* `podcachenumentries` - the number of entries in the pod metadata cache.
* `namespacecachenumentries` - the number of entries in the namespace
  metadata cache.
* `podcachehits` - the number of times a requested entry was found in the
  pod metadata cache.
* `namespacecachehits` - the number of times a requested entry was found
  in the namespace metadata cache.
* `podcachemisses` - the number of times a requested entry was not found
  in the pod metadata cache, and had to be requested from Kubernetes.
* `namespacecachemisses` - the number of times a requested entry was not
  found in the namespace metadata cache, and had to be requested from
  Kubernetes.
2019-04-01 11:53:29 -06:00
Rainer Gerhards
e338098d7b
Merge pull request #3558 from richm/add-skipverifyhost
add support for skipverifyhost
2019-04-01 10:49:16 +02:00
Rainer Gerhards
23af460e5f
Merge remote-tracking branch 'pduveau/contrib_imbatchreport' 2019-04-01 10:42:32 +02:00
Philippe Duveau
ff384385d1 Create module mmtaghostname with tests 2019-03-30 16:51:06 +01:00
Philippe Duveau
dff0e393f6 Create imbatchreport input module + parallel tests 2019-03-30 16:40:32 +01:00
Rainer Gerhards
4c48d167a0
Merge pull request #3482 from pduveau/contrib_imtuxedoulog
contrib_imtuxedoulog
2019-03-27 17:21:56 +01:00
Rainer Gerhards
4c9bece8e8
omamqp: fix build errors
They occur on some, newer, platforms. We do not really fix them, but rather
make the compiler ignore them. This is not really good, but the module is
contributed and so that's for now the best thing we can do.
2019-03-22 13:17:36 +01:00
Rich Megginson
6e0ccf89c2 add support for skipverifyhost
Add ability to specify the libcurl CURLOPT_SSL_VERIFYHOST
option to skip verification of the hostname in the peer cert.
WARNING: This option is insecure, and should only be used
for testing. The default value is off, meaning, the hostname
will be verified by default.
2019-03-20 20:32:09 -06:00
Philippe Duveau
9b1cc52152 contrib_imtuxedoulog + tests 2019-03-10 13:27:51 +01:00
Rainer Gerhards
e66bd10fe5
improg: add new module 2019-03-04 10:23:04 +01:00
Nate Brown
ea87fa9fd8 Allow header values to have spaces 2019-02-28 16:04:11 -08:00
Rainer Gerhards
7be46e575c
Merge pull request #3454 from mg964/imczmq-free-received-zframe
Release zframe following read from socket
2019-02-22 16:04:28 +01:00
Philippe Duveau
bc0d78c25a AIX_port: Core PR 2019-02-15 10:21:35 +01:00
Philippe Duveau
2d5991a98b AIX_port: style and compatibility issues (10) 2019-02-14 19:17:05 +01:00
Philippe Duveau
8fa84a82db AIX_port: style and compatibility issues (9) 2019-02-14 19:07:07 +01:00
Philippe Duveau
6ed9f9e601 AIX_port: style and compatibility issues (8) 2019-02-14 18:56:45 +01:00
Philippe Duveau
0fa6019423 AIX_port: style and compatibility issues (7) 2019-02-14 18:38:26 +01:00
Philippe Duveau
04b7e6be70 AIX_port: style and compatibility issues (6) 2019-02-14 18:22:57 +01:00
Philippe Duveau
abae82cebb AIX_port: corrects style and compatibility issues (5) 2019-02-14 18:06:17 +01:00
Philippe Duveau
3ec965f8b3 AIX_port: corrects style and compatibility issues (4) 2019-02-14 17:50:00 +01:00
Philippe Duveau
11c6365eba AIX_port: corrects style and compatibility issues (3) 2019-02-14 17:17:37 +01:00
Philippe Duveau
4dd23cffc1 AIX_port: corrects style and compatibility issues 2019-02-14 16:35:59 +01:00
Philippe Duveau
dda425dc91 AIX_port: corrects style and compatibility issues 2019-02-14 16:15:07 +01:00