rsyslog/rsyslog
1
0
Fork 0
mirror of https://github.com/rsyslog/rsyslog.git synced 2025-12-19 22:00:42 +01:00
Code Issues Packages Projects Releases Wiki Activity

imfile bugfix: if freshStartTail is set some initial file lines missing

Browse Source 
When the option is set and a new file is created after rsyslog startup,
freshStartTail is also applied to it. That is data written quickly to it
(before rsyslog can process it) will potentially be discarded. If so,
and how much, depends on the timing between rsyslog and the logging process.
This problem is most likely to be seen in polling mode, where a relatively
long time may be required for rsyslog to find the new file.

This is changed so that now freshStartTail only applies to files that
are already-existing during rsyslog's initial processing of the file
monitors. HOWEVER, depending on the number and location (network?) of
existing files, this initial startup processing may take some time as
well. If another process creates a new file at exactly the time of
startup processing and writes data to it, rsyslog might detect this
file and it's data as prexisting and may skip it. This race is inevitable.
So when freshStartTail is used, some risk of data loss exists. The same
holds true if between the last shutdown of rsyslog and its restart log
file content has been added. This is no rsyslog bug if it occurs.

As such, the rsyslog team advises against activating the freshStartTail
option.

closes https://github.com/rsyslog/rsyslog/issues/2464
This commit is contained in:
Rainer Gerhards 2018-05-17 12:30:44 +02:00
parent c7824fa522
commit 99d351d4aa
1 changed files with 26 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
plugins/imfile/imfile.c
View File

@ -1894,11 +1894,35 @@ CODESTARTfreeCnf
ENDfreeCnf
/* initial poll run, to be used for all modes. Depending on mode, it does some
* further initializations (e.g. watches in inotify mode). Most importantly,
* it processes already-existing files, which would not otherwise be picked
* up in notifcation modes (inotfiy, FEN). Also, when freshStartTail is set,
* this run assumes that all previous existing data exists and needs not
* to be considered.
* Note: there is a race on files created *during* the run, but that race is
* inevitable (and thus freshStartTail is actually broken, but users still seem
* to want it...).
* rgerhards, 2018-05-17
*/
static void
do_initial_poll_run(void)
{
fs_node_walk(runModConf->conf_tree, poll_tree);
/* fresh start done, so disable freshStartTail for files that now will be created */
for(instanceConf_t *inst = runModConf->root ; inst != NULL ; inst = inst->next) {
inst->freshStartTail = 0;
}
}
/* Monitor files in polling mode. */
static rsRetVal
doPolling(void)
{
DEFiRet;
do_initial_poll_run();
while(glbl.GetGlobalInputTermState() == 0) {
DBGPRINTF("doPolling: new poll run\n");
do {
@ -2063,8 +2087,7 @@ do_inotify(void)
}
DBGPRINTF("inotify fd %d\n", ino_fd);
/* do watch initialization */
fs_node_walk(runModConf->conf_tree, poll_tree);
do_initial_poll_run();
while(glbl.GetGlobalInputTermState() == 0) {
if(runModConf->haveReadTimeouts) {
@ -2182,8 +2205,7 @@ do_fen(void)
return RS_RET_FEN_INIT_FAILED;
}
/* do watch initialization */
fs_node_walk(runModConf->conf_tree, poll_tree);
do_initial_poll_run();
DBGPRINTF("do_fen ENTER monitoring loop \n");
while(glbl.GetGlobalInputTermState() == 0) {