submit on behalf of @abwaheed
- Added graceful handling of apiserver errors with unexpected responses,
i.e., anything other than 200, 404, or 429. Idea is that apiserver
transient error state will recover. We don't want mmkubernetes to miss
metadata resolution for containers that don't have cached metadata.
During these transient error states, mmkubernetes will provide basic
container file path based resolution of namespace and pod metadata for
new pods whose metadata is not yet cached. After this error state
recovers, mmkubernetes is expected to resume its metadata resolution as
expected.
- Added a unit test case for apiserver return 500 with changes to mock server
- Fixed existing unit test that was failing due to missing expected results file
- Added mmkubernetes unit tests to testbench
New parameters for mmkubernetes (module and action):
* `cacheexpireinterval`
If `cacheexpireinterval` is -1, then do not check for cache expiration.
If `cacheexpireinterval` is 0, then check for cache expiration.
If `cacheexpireinterval` is greater than 0, check for cache expiration
if the last time we checked was more than this many seconds ago.
* `cacheentryttl` - maximum age in seconds for cache entries
New statistics counters:
* `podcachenumentries` - the number of entries in the pod metadata cache.
* `namespacecachenumentries` - the number of entries in the namespace
metadata cache.
* `podcachehits` - the number of times a requested entry was found in the
pod metadata cache.
* `namespacecachehits` - the number of times a requested entry was found
in the namespace metadata cache.
* `podcachemisses` - the number of times a requested entry was not found
in the pod metadata cache, and had to be requested from Kubernetes.
* `namespacecachemisses` - the number of times a requested entry was not
found in the namespace metadata cache, and had to be requested from
Kubernetes.
Also now permit interactivly running tests without explicitly setting
$srcdir. This now works if we are inside ./tests and fails, as before,
when we are in a different directory.
Detected by shellcheck via CodeFactor.io
The plugin was not handling 404 Not Found correctly when looking
up pods and namespaces. In this case, we assume the pod/namespace
was deleted, annotate the record with whatever metadata we have,
and cache the fact that the pod/namespace is missing so we don't
attempt to look it up again.
In addition, the plugin was not handling error 429 Busy correctly.
In this case, it should also annotate the record with whatever
metadata it has, and _not_ cache anything. By default the plugin
will retry every 5 seconds to connect to Kubernetes. This
behavior is controlled by the new config param `busyretryinterval`.
This commit also adds impstats counters so that admins can
view the state of the plugin to see if the lookups are working
or are returning errors. The stats are reported per-instance
or per-action to facilitate using multiple different actions
for different Kubernetes servers.
This commit also adds support for client cert auth to
Kubernetes via the two new config params `tls.mycert` and
`tls.myprivkey`.
to wait for process startup, the rsyslog startup calles were (ab)used. This
caused issues whenever they were updated to new rsyslog needs. Now a dedicated
"API" for process startup has been added and the tests been modified to use it.
When mmkubernetes encounters a record with a CONTAINER_NAME field,
but the value does not match the rulebase, mmkubernetes returns
an error, and mmkubernetes does not do any further processing
of any records.
The fix is to check the return value of ln_normalize to see if
it is a "hard" error or a "does not match" error.
This also adds a test for pod names with dots in them.
changes some of the test commands to use bash functions
includes some small bug fixes to tests where bugs were
previously not seen due to different plumbing.
Fix lnrules for CONTAINER_NAME
Add pkg check for lognorm >= 2.0.3 so we can set the macro
to enable ln_loadSamplesFromString
Add some reasonable default values for parameters, such as
kubernetesurl https://kubernetes.default.svc.cluster.local:443
Clean up sample.conf configuration file
Add test for mmkubernetes, including mock kubernetes service
