We so far tried to ensure a value is really an IPv6 address, in order
to avoid to mangle with just similar-looking information elements.
However, this lead to misdetection for unusual formats, e.g. when a
port is appended to a numerical IPv6 adress given without braces [].
This has been changed now. In a sense, we now prefer to err on the
side of privacy.
BEHAVIOR CHANGE:
Previously, a suspect value was not anonymized, and thus some other
elements (like some MAC addresses) preserved. Now the opposite is
true, and we anonymize anything that looks close enough to be an
IPv6 address. This improves anonymization.
closes https://github.com/rsyslog/rsyslog/issues/4725
The call rscript statement is able to call a rule set either synchronously or
asynchronously. We did this, because practice showed that both modes
are needed. For various reasons we decided to make async
calls if the ruleset has a queue assigned and sync if not.
To know if a "queue is assigned" we just checked if queue parameters were
given. It was overlookeded the case of someone explicitly specifying a
"direct queue", aka "no queue". As such, queue="direct" triggered async
calls. That in turn meant that when a write operation to a variable was
made inside that rule set, other rulesets could or could not see the
write. While if was often not seen, this was a data race where the
change could also be seen by the outside.
This is now fixed. No matter if queue.type="direct" is specified or
left out, the call will always by synchronous. Any values written to
variables will also be seen by the "outside world" in later processing
stages.
Note that this has some potential to BREAK EXISTING CONFIGURATIONS.
We deem this acceptable because:
1. this was racy at all, so unexpected behaviour could alwas occur
2. it is actually unlikely that someone used the triggering conditions
in practice. But we can not outrule this, especially when the
configuration was auto-generated.
Potential compatibility issues can be solved by defining a small
array-memory queue on the ruleset in question instead of specifying
direct type.
Again, we expect that almost all users will never experience any
problems. If you do, however, please let us know: we may add an
option to re-enable the bug.
It looks like the current version of clickhouse does not error out
on some types of errors. So tests need either to be redesigned
or removed.
As such, we temporarily skip them for now.
- Only apply default anon ciphers if gnutlsPriorityString is NULL and
Authentication Mode is set to anon. Otherwise we do not set them
as they overwrite custom Ciphers.
- Added two tests for custom cipher configuration (anon/certvalid mode).
- Add call for applyGnutlsPriorityString if gnutlsPriorityString changes.
- Merged openssl init code from Connect into osslInitSession
closes: https://github.com/rsyslog/rsyslog/issues/4686
The testbench framework does not properly wait until ES has created
its pid file, which probably means it did basic initializiation.
This can cause test synchronization issues and ultimately failures.
ElasticSearch 7 is now the major version. Replacing old-style
ES with it by default.
also add basic test for ElasticSearch 7.14, to make sure we will
test ES 7 once we move up to next higher version.
Note: ES 5 is no longer beeing tested as it looks outphased.
see also: https://github.com/rsyslog/rsyslog/pull/4566
- Basic Authentication support & tests
* configured via imhttp option "basicAuthFile". This option should be configured
to point to your htpasswd file generated via a standard htpasswd tool.
tests:
* imhttp-post-payload-basic-auth.sh
* imhttp-post-payload-basic-auth-vg.sh
- Query parameter ingestion capability & tests
use t `addmetadata` option to inject query parameters into
metadata for imhttp input.
Signed-off-by: Nelson Yen <nyen@salesforce.com>
Rate limiting may not have worked if the considered message had already
been parsed (not having NEEDS_PARSING in msgFlags).
This affects also imuxsock in its default configuration
(useSpecialParser="true" and ratelimit.severity="1")
A test for imuxsock is included which requires
./configure --enable-liblogging-stdlog
When running under root, the privdrop tests did not properly work. This
patch fixes the issue and skips test where necessary.
This also includes some modernization of the related tests.
closes https://github.com/rsyslog/rsyslog/issues/4619
Submit on behalf of the primary author @yanjunli76
instructs imfile not to ingest a file that has not been modified in the
specified number of seconds.
Practice has shown that environment-induced error messages may
appear during test runs. For example, incorrect resolver settings
may cause name resolution warnings or errors. These are unrelated
to the test itself.
This patch enables tests that are otherwise unaffected by the faillures
to continue and provide "pass" test result.
see also https://github.com/rsyslog/rsyslog/issues/4619
Test should have been added to commit 3d23c7ac8aea but was forgotten.
This test also showed an issue on Solaris, where NULL values cannot
be used in printf functions. This has also been addressed.
see also: https://github.com/rsyslog/rsyslog/pull/4627
submit on behalf of @abwaheed
- Added graceful handling of apiserver errors with unexpected responses,
i.e., anything other than 200, 404, or 429. Idea is that apiserver
transient error state will recover. We don't want mmkubernetes to miss
metadata resolution for containers that don't have cached metadata.
During these transient error states, mmkubernetes will provide basic
container file path based resolution of namespace and pod metadata for
new pods whose metadata is not yet cached. After this error state
recovers, mmkubernetes is expected to resume its metadata resolution as
expected.
- Added a unit test case for apiserver return 500 with changes to mock server
- Fixed existing unit test that was failing due to missing expected results file
- Added mmkubernetes unit tests to testbench
Brief overview:
TO configure tracking percentile metrics in rainerscript:
User would need to define:
- which percentile to track, such as [p50, p99, etc.]
- window size - note, this correlates directly with memory usage to
track the percentiles.
To track a value, user would call built-in function `percentile_observe()` in their configurations to
record an integer value, and percentile metrics would be emitted every
impstats interval.
When batchmode was used, the templates could not be used to
expand dynrestpath. We are now storing the restpath param
within the batch data if we are in batch mode.
- testbench: Added tests for omhttp dynrestpath param
Testing setting the restpath by template - one with batch mode
and one without batch mode
- When we are in batch mode, and the restpath value changes, the
batch is submitted and reinitialized
closes: https://github.com/rsyslog/rsyslog/issues/4567
Provides ability to evaluate a rsyslog variable using dynamically
evaluated parameters.
1st param is the rsyslog param, 2nd param is a key, can be an array
index or key string.
Useful for accessing json sub-objects, where a key
needs to be evaluated at runtime. Can be used to access arrays as well.
see tests for examples
remove unnecessary escape char
