53 Commits

Author SHA1 Message Date
PascalWithopf
8ceaddde35 imfile: add posibility to truncate msg
closes https://github.com/rsyslog/rsyslog/issues/1552
2017-05-19 12:04:02 +02:00
PascalWithopf
a0a6e5bbba imfile: fix problem that fileNotFoundError was reported repeatedly in polling mode
fix https://github.com/rsyslog/rsyslog/issues/1534
2017-05-03 10:00:28 +02:00
PascalWithopf
3f789a4458 imfile: add error message when file is not found and parameter to disable this message 2017-05-02 10:54:56 +02:00
Pascal Withopf
b63a6b9010 codestyle: line length adjusted 2017-01-17 08:54:37 +01:00
kordex
ed37e8b6de core: fix potential negative file numbers for queue files 2016-10-18 13:04:23 +02:00
Rainer Gerhards
8eea909367 imfile: add "readTimeout" parameter
closes https://github.com/rsyslog/rsyslog/issues/1133
2016-10-10 15:45:45 +02:00
Rainer Gerhards
51c03c66e0 make inline functions C99-compliant
As it seems, different C compilers have different rules/interpretations
on inline functions. The current code base did not properly obey all C99
rules. We fix this by converting some functions to macros and others to
include the necessary C99 plumbing. We also remove some inline attributes
for functions where this seems to be to agressive (aka "function to big").

This fixes build problems in some environments and is a general code
cleanup measure.
2016-09-05 16:47:14 +02:00
Rainer Gerhards
6a387869fa queue subsystem: improve debug messages 2016-08-22 16:20:02 +02:00
Rainer Gerhards
3b0222c284 bugfix omfile: async output file writing does not respect flushing
neither parameter flushInterval nor flushOnTXEnd="on" was respected.

closes https://github.com/rsyslog/rsyslog/issues/1054
2016-05-27 14:49:01 +02:00
Rainer Gerhards
42155cd5b4 Merge branch 'trim-line-over-bytes' of https://github.com/JindongChen/rsyslog into merge-664
Conflicts:
	runtime/stream.h
2016-02-15 15:45:40 +01:00
Chen Jindong
7da93520ca Trim line when length of line over specified bytes 2015-12-11 13:58:37 +08:00
Matthew Wang
ca79a68f04 Reopen truncated file with opt-in flag 'reopenOnTruncate'
Attempt to fix #511.  This is an enhanced version based on
https://github.com/rsyslog/rsyslog/pull/514, works fine for us so far.
As discussed before, I added an opt-in flag 'reopenOnTruncate' and
default is "off".
2015-12-03 20:24:35 +08:00
Rainer Gerhards
483522a117 imfile: add support for multi-line msg termination via regex 2015-05-19 09:32:35 +02:00
Rainer Gerhards
6e511e8e8b imfile: preliminary patch for msg loss in readmode != 0
This is a preliminary patch, which needs to undergo further testing
in cases where rsyslogd is shut down with partial messages read.

see also https://github.com/rsyslog/rsyslog/issues/144
2015-03-29 14:38:59 +02:00
Rainer Gerhards
4bcccb64fd refactor: another round of qualifiers... 2013-12-04 12:10:07 +01:00
Rainer Gerhards
ea9c81ce3c imfile: support LF escaping in read mode 1 2013-09-10 11:34:31 +02:00
Rainer Gerhards
c3784aab3c Merge branch 'v6-stable' into tmp 2013-07-19 14:59:49 +02:00
Rainer Gerhards
ea4625e8a4 Merge branch 'v5-stable' into tmp 2013-07-19 14:59:09 +02:00
Tomas Heinrich
cbf7ab6f35 bugfix: correction to the interface of stream obj
The datatype of the argument to SetiMaxFileSize() was too small and
could overflow.
The SetMaxFileSize() interface wasn't used at all (possibly confused
with SetiMaxFileSize()) and was dropped.
2013-07-19 14:53:00 +02:00
Rainer Gerhards
0cee769fcd log encryption: initial PoC implementation
rough baseline, needs to be extended for actual use.
2013-04-05 12:08:49 +02:00
Rainer Gerhards
35bafbef04 imfile: now detects file change when rsyslog was inactive
Previosly, this case could not be detected, so if a file was overwritten
or rotated away while rsyslog was stopped, some data was missing. This
is now detected and the new file being forwarded right from the
beginning.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=228
2013-03-20 17:05:18 +01:00
Rainer Gerhards
42cd1fa7d7 Merge branch 'v7-stable'
Conflicts:
	configure.ac
	doc/manual.html
	plugins/imfile/imfile.c
	plugins/imklog/bsd.c
	plugins/imklog/imklog.c
	plugins/imklog/imklog.h
2012-11-22 11:05:45 +01:00
Rainer Gerhards
c563914d6f queue: file delete stream does no longer do real io
This stream is primarily used for state tracking, and has been
modified to do just that. This results in considerable less io
being done and the respective speedup.
2012-11-19 16:46:41 +01:00
Rainer Gerhards
3475caa394 refactor stream.h; fix some issues with last patch
Most importantly, the last patch for imfile contained a number
of glitches, which are fixed by this commit (a memory leak under
unusual conditions, partial message loss when rsyslog was
terminated in the interim & mixing file data to the wrong files
when multiple monitors were used [due to static variable]). The
commit is actually a re-write of the patch, based on its core
idea.

Also some other minor cleanup was done.
2012-11-15 10:08:26 +01:00
Rainer Gerhards
5a59687119 Merge branch 'c1-streamzip' into master-streamzip
Conflicts:
	ChangeLog
	configure.ac
	doc/rsyslog_conf_global.html
	runtime/stream.c
	tools/omfile.c
2012-10-05 12:45:14 +02:00
Rainer Gerhards
5df935249e improved omfile zip writer to increase compression
This was achieved by somewhat reducing the robustness of the zip
archive. This is controlled by the new setting $OMFileVeryReliableZip.
2012-09-14 18:41:30 +02:00
Rainer Gerhards
46605923a2 refactor: removing dead apc code
another concept that did not prove valuable enough to be kept active. For
the last couple of releases, the dead code was kept inside the project as
we thought we could probably reuse it. Doesn't look so...
2012-01-19 15:37:56 +01:00
David Lang
9813a12789 enhanced imfile to support multi-line messages
Signed-off-by: Rainer Gerhards <rgerhards@adiscon.com>
2011-01-25 14:24:50 +01:00
Rainer Gerhards
8f0a8076cc Merge branch 'beta' into master
Conflicts:
	ChangeLog
	configure.ac
	plugins/imudp/imudp.c
	runtime/stream.h
	tests/Makefile.am
	tests/diag.sh
	tools/omfile.c
2010-04-09 13:00:56 +02:00
Rainer Gerhards
8b9cef552d Merge branch 'v4-stable' into v5-stable
Conflicts:
	ChangeLog
	configure.ac
	doc/manual.html
	runtime/debug.c
	runtime/stream.c
	tests/Makefile.am
	tests/diskqueue.sh
	tests/nettester.c
	tools/omfile.c
2010-03-29 11:07:15 +02:00
Rainer Gerhards
6c43f93022 fixed regression introduced with previous commit
disk queue mode did no longer work correctly. A side-effect of
this commit here is slightly cleaned-up (and more elegant) code
for circular files.
2010-03-10 09:52:49 +01:00
Rainer Gerhards
38cb392672 replaced data type "bool" by "sbool" because this created some portability issues 2010-02-02 15:51:01 +01:00
Rainer Gerhards
9901996b5b Merge branch 'v4-beta' into beta
Conflicts:
	ChangeLog
2010-01-15 17:03:51 +01:00
Rainer Gerhards
5b4e06fc28 bugfix: rsyslog hangs when writing to a named pipe which nobody was reading.
Thanks to Michael Biebl for reporting this bug.
2010-01-15 17:01:10 +01:00
Rainer Gerhards
7d9ad61884 Merge branch 'beta' 2009-08-19 13:04:35 +02:00
Rainer Gerhards
bfc3eaf23c bugfix: potential segfault in output file writer (omfile)
In async write mode, we use modular arithmetic to index the output
buffer array. However, the counter variables accidently were signed,
thus resulting in negative indizes after integer overflow. That in turn
could lead to segfaults, but was depending on the memory layout of
the instance in question (which in turn depended on a number of
variables, like compile settings but also configuration). The counters
are now unsigned (as they always should have been) and so the dangling
mis-indexing does no longer happen. This bug potentially affected all
installations, even if only some may actually have seen a segfault.
2009-08-18 18:48:18 +02:00
Rainer Gerhards
cf906392c3 Merge branch 'v4-devel'
Conflicts:
	runtime/debug.h
	runtime/stream.c
2009-07-08 15:23:25 +02:00
Rainer Gerhards
5221a1e42e added capability to write incomplete buffers after an inactivity timeout
for the stream class and thus finally activating omfile's timeout
capability in a useful way without polling and too-high performance
overhead.
2009-07-07 18:33:00 +02:00
Rainer Gerhards
26227091fa fixed a bug introduced today that lead to an abort in queue disk mode 2009-07-07 17:18:51 +02:00
Rainer Gerhards
e3040285db first shot at asynchronous stream writer with timeout capability
... seems to work on quick testing, but needs a far more testing
and improvement. Good milestone commit.
2009-07-06 16:38:09 +02:00
Rainer Gerhards
a3e512f65f Merge branch 'omfile' into v5-devel
Conflicts:
	runtime/rsyslog.h
2009-06-23 17:23:28 +02:00
Rainer Gerhards
b1f2e53921 prevented unneccessary apc calls 2009-06-23 15:17:55 +02:00
Rainer Gerhards
b5f3387357 Merge branch 'omfile' into v5-devel
Note that this was NOT a trivial merge, and there may be
some issues. This needs to be seen when we continue developing.

Conflicts:
	runtime/msg.h
	runtime/obj.h
	runtime/queue.c
	runtime/srUtils.h
	runtime/stream.c
	runtime/stream.h
	runtime/wti.c
	tests/Makefile.am
	tools/omfile.c
	tools/syslogd.c
2009-06-16 13:59:09 +02:00
Rainer Gerhards
16ecb90c3a omfile buffers are now synchronized after inactivity
This is the first shot at this functionality. Currently, we run off a fixed
counter in the rsyslogd mainloop, which needs to be restructured. But this
code works, so it is a good time for a commit.
2009-06-15 13:44:51 +02:00
Rainer Gerhards
e3d9843c85 re-enabled pipe, tty and console in omfile
... by moving code to stream.c. Thanks to the new design, new cases are
not really needed, resulting in cleaner code.

I also did a cleanup of header file usage as a side-activity.
2009-06-12 11:47:00 +02:00
Rainer Gerhards
0917edf26d re-enabled outchannel functionality 2009-06-12 09:47:44 +02:00
Rainer Gerhards
8d1e2e496c improved error handling and added fsync() support
... restoring missing functionality after the restructuring of imfile. As
a side-effect, this also lays the foundation for even more reliable queue
engine operations (but this is not yet done).
2009-06-08 18:39:06 +02:00
Rainer Gerhards
84b7867537 cleanup in preparation of next steps
now cleand up omfile and straighted out some things. The only commented-out
code left is code that must be moved/merged to the stream class, my next target.
2009-06-05 18:41:28 +02:00
Rainer Gerhards
76da7f9f4e added $OMFileIOBufferSize config directive and plumbing 2009-06-04 15:59:37 +02:00
Rainer Gerhards
f2800ba261 modified stream class and omfile to work with it
now some basic operations are carried out via the stream class.
2009-06-04 15:10:24 +02:00