FileCreateMode allows to set the default file mode bits
when creating new files. As of now, it has only impact on the state file.
Add test suite as well.
Minor indentation fix in run_journal.yml
If SYSLOG_IDENTIFIER is not present in the journal message,
then lookup the _COMM field, which stands for the name
of the process the journal entry originates from. This is
needed in order to be in compliance with the journalctl
output.
Flush the FILE* buffer before rename & fsync in order
to not end up syncing an empty file.
Also, close WorkDir on fsync in order to prevent
file descriptor leakage.
Signed-off-by: Gerd Rausch <gerd.rausch@oracle.com>
Signed-off-by: Venu Busireddy <venu.busireddy@oracle.com>
When we have optional components (like imjournal) a dummy module
is used. It's sole purpose is to emit "this module is not available".
During init, the module emitted an invalid module name into the debug
log. This has now been replaced by the generic term "dummy".
Note: it is highly unlikely that someone will ever see that message
at all, as it is unlikely for the dummy modules to be build.
see also: 84a7e3d80b (r41782830)
Thanks to Thomas D. (whissi) for the patch.
Commit f604d5269bd31c09a661e9e4a20dd5044f4750da introduced strlen("Constant")
what we prefer not to do. This is now changed to sizeof("Constant") - 1,
what we usally use.
The journal API will sometimes issue an SD_JOURNAL_INVALIDATE
which results in a repositioning of the cursor. From the man page:
Programs only interested in a strictly sequential stream
of log data may treat SD_JOURNAL_INVALIDATE the same way as
SD_JOURNAL_APPEND, thus ignoring any changes to the log view
earlier than the old end of the log stream.
If we have configured IgnorePreviousMessages, we want to treat the
invalidate event as an append, and wind to the end of the message
stream.
The burst parameter in the ratelimit was increased to an unsigned int
but the interval remained an unsigned short. While it may be unusual,
there is possibly a chance to need to represent an interval longer than
about 3/4 of a day.
While here, go through and normalize all the various incarnations of
rate limiting to be explicitly unsigned int for the burst and interval.
When trying to recover journal errors, imjournal waited a hardcoded
period of 10s between tries. This was pretty long and could lead to
loss of journal data.
This commit adjust it to 100ms, which should still be fully sufficient
to prevent the journal from "hammering" the CPU.
It may be worth considering to make this setting configurable - but
let's first see if there is real demand to actually do that.
closes https://github.com/rsyslog/rsyslog/issues/3969
When data is read from the journal using sd_journal_get_data it may be
truncated to a certain threshold (64K by default).
If the rsyslog MaxMessageSize is larger than the threshold, there's a
chance rsyslog will receive incomplete messages from the journal.
Empirically, this appears to happen reliably when XZ compression is
used by journald. Systems where journald uses LZ4 compression do not
appear to suffer this issue reliably--if at all.
This change sets the threshold to the MaxMessageSize when the
journal is opened.
commit 78976a9bc059 introduced a regression that caused writing
the journal state file to fail. This happens when the state file
is given as relative file name and the working directory is also
a relative path. This situation is very uncommon. So most deployments
will never experience it. We discovered the issue during CI runs
where the trigger condition is given. Note that it also takes
multiple times of loading the journal to actually see the bug.
see also https://github.com/rsyslog/rsyslog/pull/3878
Decomposed ReadJournal() a bit, also now coupling journald
variables in one struct, added few warning messages and debug
prints to help with bug hunts in future, also got rid of two
needless journald calls. WorkAroundJournalBug now deprecated.
Added option to pull journald records from outside local machine.
The new option makes possible to force physical write of stateFile
to persistent storage, ensuring we do not lose/duplicate messages
in case of hard crash or power loss.
This fixes situation when journal was preloaded from previously
saved cursor but was not advanced to next entry so reading begun
from last message which was therefore duplicated.
The sd_journal_get_cursor() got called regradless of previous
retcodes from other jorunal calls which flooded logs with journald
errors. Now skipping the call in case of previous journal call
non-zero result. Fixed success checking of get_cursor() call
to eliminate double-free possibility.
Also, making WorkAroundJournalBug true by default, as there were no
confirmed performance regressions for a quite long time.
This is workaround for possible imjournal interaction with systemd
where journal invalidate fix is not present. The code tries to
detect SD_JOURNAL_INVALIDATE loop and not reload after each call.
Fetching journal cursor in persistJournal could cause us to save
invalid cursor leading to duplicating messages further on, when new
WorkAroundJournalBug option is set we are saving it on each
readJournal() where we now that the state is good.
pollJournal() is now cleaner and faster, correctly handles INVALIDATE
status from journald and is able to continue polling after journal
flush. Also reduced POLL_TIMEOUT a bit as it caused rsyslog to exit
with error in corner cases for some ppc when left at full second.
re-factored imjournal CI tests with journal_print tool to have more
detailed error reporting
When starting rsyslog with imjournal for first time it outputs
an error and plugin does not run because no state file exists yet.
Now it skips the loading and creates state file on first persist.
Also fixed imjournal tests to actually test plugin functionality.
The fix is to immediately setup the inotify file descriptor via
`sd_journal_get_fd()` right after a journal open, and then
periodically call `sd_journal_process()` to give the client API
library a chance to detect deleted journal files on disk that need to
be closed so they can be properly erased by the file system.
We remove the open/close dance and simplify that code as a result.
Fixes issue #2436.
Remove "time of check time of use" Coverity scan warning by refactoring
the code to directly try out things prior to just checking if we could.
Ultimately, we need to handle the same error (and already did so), so
there was little point in doing a check first.
Coverity scan CID 185500
If some functions calls inside the main loop failed, imjournal exited
with an error code, actually disabling all logging from the journal.
This was probably never intended.
This patch makes imjournal recover the situation instead.
closes https://github.com/rsyslog/rsyslog/issues/1895
The return state was invalidly handled. Errno is not used by libjournal,
we need to use the return value to obtain error message.
see also https://github.com/rsyslog/rsyslog/issues/1867
now you can continue logging after switch to persistent
journal without need to restart rsyslog service.
Bumped correspondidng systemd version dependency.
- added fallback for _PID proprety when SYSLOG_PID is not available
- introduced new option "usepid" which sets which property should
rsyslog use, it has 3 states system|syslog|both, default is both
- deprecated "usepidfromsystem" option, still can be used
and override the "usepid"
- it is possible to revert previous default with usepid="syslog"
This is resolving the situation when system is after reboot and boot_id doesn't match
so cursor pointing into "future".
Usually sd_journal_next jump to head of journal due to journal aproximation,
but when system time goes backwards and cursor is still invalid, rsyslog stops logging.
We use sd_journal_get_cursor to validate our cursor.
When cursor is invalid we are trying to jump to the head of journal
This problem with time should not affect persistent journal,
but if cursor has been intentionally compromised it could stop logging even
with persistent journal.
