... this is necessary in preparation for the final solution (we need
to have a "unified" writer). If it causes worse performance to have the
zip writher togehter with the synchronous write, we may do an async write...
some things inside the message can be used over a large number of
messages and need to to be allocated and re-written every time. I now
begin to implement this as a "prop_t" object, first use for the inputName.
Some input modules are already converted, some others to go. Will do
a little performance check on the new method before I go further.
Also, this commit has some cleanup and a few bug fixes that prevented
compiliation in debug mode (I overlooked this as I did not compile
for debug, what I normally do, and the automatted test also does not
do that)
This was a complex manual merge, especially in action.c. So if
there occur some problems, this would be a good point to start
troubleshooting. I run a couple of tests before commiting and
they all went well.
Conflicts:
action.c
action.h
runtime/queue.c
runtime/queue.h
runtime/wti.c
runtime/wti.h
Note that this was NOT a trivial merge, and there may be
some issues. This needs to be seen when we continue developing.
Conflicts:
runtime/msg.h
runtime/obj.h
runtime/queue.c
runtime/srUtils.h
runtime/stream.c
runtime/stream.h
runtime/wti.c
tests/Makefile.am
tools/omfile.c
tools/syslogd.c
... actually, it was not broken, but just very slow. I have now
reduced the number of test messages so that make check will not be
held for an extended period of time.
... this was long overdue, and I finlly tackeld it. It turned out to
be more complex than I initially thought. The next step now probably is
to actually implement multiple rule sets and the beauty that comes
with them.
also adds speed, because you do no longer need to run the whole file
system in sync mode. New testbench and new config directives:
- $MainMsgQueueSyncQueueFiles
- $ActionQueueSyncQueueFiles
now only stream class is utilized. ttys, pipes and outchannel functionality
is currently disabled. But the testbench worked again. Cleanup needed, will
do this with next commit (it may break things and I like to have this
milestone here).
this permits us to keep a persistent test environment between
v4 and v5, most importantly using the same tools. As far as the
actual tests are concerned, some had issues. I had no time to check
if that was an issue with the test or an actual issue with the
v3/4 engine. Will do that at some later stage.
This change is considered important but small enough
to apply it directly to the stable version. [But it is a border case,
the change requires more code than I had hoped. Thus I have NOT tried
to actually catch all cases, this is left for the current devel
releases, if necessary]
... in preparation for some larger changes - I need to apply some
serious design changes, as the current system does not play well
at all with ultra-reliable queues. Will do that in a totally new version.
The imdiag module now can very effectively inject messages, which also
frees us from uncertainties of tcp reception and processing. All shell
script based tests have been modularized, what makes it far easier to
create new tests. Also, the test bench now executes more reliable and
much faster, because we can now rely on actual engine information where
we previously did just a dumb sleep.
... and also improved the test suite. There is a design issue in the
v3 queue engine that manifested to some serious problems with the new
processing mode. However, in v3 shutdown may take eternally if a queue
runs in DA mode, is configured to preserve data AND the action fails and
retries immediately. There is no cure available for v3, it would
require doing much of the work we have done on the new engine. The window
of exposure, as one might guess from the description, is very small. That
is probably the reason why we have not seen it in practice.
