messages could get lost or be duplicated due to non-proper sync
of transactions. This is a notable slowdown again, but we know
how to get back concurrency, it just takes "some" more programming.
It is important now to come back to correct code, so that we can
base further improvements on that.
This, in default mode, caused buffered writing to be used, what
means that it looked like no output were written or partial
lines. Thanks to Michael Biebl for pointing out this bug.
Further testing turned out that the rsyslog core works correctly and
this fix is not needed. The concurrency we saw was actually caused by
other actions (even processes) during directory creation. See commit
9e5b31fc44136dbcc1e443cfe7714e9daf97d844 for further details.
This bug was triggered by an open failure. The the cache was full and
a new entry needed to be placed inside it, a victim for eviction was
selected. That victim was freed, then the open of the new file tried. If
the open failed, the victim entry was still freed, and the function
exited. However, on next invocation and cache search, the victim entry
was used as if it were populated, most probably resulting in a segfault.
when dynaCache is enabled, the cache is full, a new entry needs to
be allocated, thus the LRU discarded, then a new entry is opend and that
fails. In that case, it looks like the discarded stream may be reused
improperly (based on code analysis, test case and confirmation pending)
In theory, the rsyslog core should never call in parallel into an output
module for the same instance. However, it looks like this seems to happen
under (strange?) circumstances. I have now enhanced omfile so that it guards
itself against being called in parallel on the same instance data. This is
done to help troubleshooting and may stay as an interim solution if it
proves to solve an anomaly we see in at least one installation (to trigger
this problem, an extremely large traffic volume is needed).
The previous fix fixed an issue with on/off bying used in the exact wrong
semantic. It corrected the situation, but failed to fix one spot where the
wrong semantics were used. This is done with this commit.
Note that this is NOT a bug seen in any released version.
When a write error occured in stream.c, variable iWritten had the error
code but this was handled as if it were the actual number of bytes
written. That was used in pointer arithmetic later on, and thus could
lead to all sorts of problems. However, this could only happen if the
error was EINTR or the file in question was a tty. All other cases were
handled properly. Now, iWritten is reset to zero in such cases, resulting
in proper retries.
- bugfix: potential segfault in omfile when a dynafile open failed
In that case, a partial cache entry was written, and some internal
pointers (iCurrElt) not correctly updated. In the next iteration, that
could lead to a segfault, especially if iCurrElt then points to the
then-partial record. Not very likely, but could happen in practice.
- bugfix (theoretical): potential segfault in omfile under low memory
condition. This is only a theoretical bug, because it would only
happen when strdup() fails to allocate memory - which is highly
unlikely and will probably lead to all other sorts of errors.
This was not honored by the new ompipe module, because it is a local
file directive (it was applied to pipes as a side-effect of using the
same module for pipes and files...). I now made this a global, so that
semantics are the same as previously. Not really nice, but probably
the best thing to do in the current situation (everything else would
involve much more overhead --- leave that for the new config system).
... based on old omfile. Michael Biebl reported that xconsole seems
to have some issues with the new pipe code, so it was best to use
the old code for pipes. The optimizations were done to speed up file
access, so it doesn't really matter pipes do not receive them.
...under some circumstances (this smells like a gcc problem, but a simple
solution was available). Thanks to Kenneth Marshall for some advice.
[backported from 5.5.x branch]
