This is kind of a bugfix, but a very intrusive one, thus it goes
into the devel version first. Right now, "file not found" is
handled and leads to the new emergency mode, in which disk action
is stopped and the queue run in direct mode. An error message is
emited if this happens.
at least in important cases (not for non-direct action queues and some
other minor things). This version is definitely buggy, but may be tried
with success on a non-production system. I will continue to work on the
correctness, but needed to commit now to get a baseline.
We now manage to cancel threads that block inside a retry loop to
terminate without the need to cancel the thread. Avoiding cancellation
helps keep the system complexity minimal and thus provides for better
stability. This also solves some issues with improper shutdown when
inside an action retry loop.
replaced atomic operation emulation with new code. The previous code
seemed to have some issue and also limited concurrency severely. The
whole atomic operation emulation has been rewritten.
Problems could happen if the queue worker needed to be cancelled
and this cancellation happened inside queue-code (including
wtp, wti). We have now solved this by disabling cancellation while
in this code and only enabling it when working inside the user consumer.
This exactly matches the use case for which cancellation may be needed.
these occured in very unusual scenarios where we had a DA-queue running
in parallel and very lengthy actions. Then, in some situations, the
shutdown could hang. The code needs some addition lab time, but
is believed to be much better than any previous version.
support for enhancing probability of memory addressing failure by
using non-NULL default value for malloced memory (optional, only if
requested by configure option). This helps to track down some
otherwise undetected issues within the testbench and is expected
to be very useful in the future.
simplified and thus speeded up the queue engine, also fixed some
potential race conditions (in very unusual shutdown conditions)
along the way. The threading model has seriously changes, so there may
be some regressions.
NOTE: the code passed basic tests, but there is still more work
and testing to be done. This commit should be treated with care.
- bugfix: solved potential (temporary) stall of messages when the queue was
almost empty and few new data added (caused testbench to sometimes hang!)
- fixed some race condition in testbench
- added more elaborate diagnostics to parts of the testbench
- solved a potential race inside the queue engine
code review brought up some few places where we may have run into a race.
They have most probably been introduced during the recent set of changes. But
I do not look at older versions because of the changed architecture, one can
not simply backport this patch.
...if not running in direct mode. Previous versions could run without
any active workers. This simplifies the code at a very small expense.
See v5 compatibility note document for more in-depth discussion.
... could even remove one mutex by using a better algorithm. I think I also
spotted some situation in which a hang could have happened. As I can't fix it
in v4 and less without moving to the new engine, I make no effort in testing
this out. Hangs occur during shutdown, only (if at all). The code changes
should also result in some mild performance improvement. Some bug potential,
but overall the bug potential should have been greatly reduced.
... first commit in a series of more. Makes worker threads detached. Needs more
testing (will be done soon) and if it works as expected, we can further reduce
code.
- bugfix: subtle (and usually irrelevant) issue in timout processing
timeout could be one second too early if nanoseconds wrapped
- set a more sensible timeout for shutdow, now 1.5 seconds to complete
processing (this also removes those cases where the shutdown message
was not written because the termination happened before it)
- bugfix: subtle (and usually irrelevant) issue in timout processing
timeout could be one second too early if nanoseconds wrapped
- set a more sensible timeout for shutdow, now 1.5 seconds to complete
processing (this also removes those cases where the shutdown message
was not written because the termination happened before it)
This was a complex manual merge, especially in action.c. So if
there occur some problems, this would be a good point to start
troubleshooting. I run a couple of tests before commiting and
they all went well.
Conflicts:
action.c
action.h
runtime/queue.c
runtime/queue.h
runtime/wti.c
runtime/wti.h
we usually stay long enough inside the actions, so there should be
no problem with reaching a cancellation point. Actually, if we
really need to cancel, the thread is in an output action (otherwise
it would have willingly terminated).
