added doc: howto set up a reliable connection to remote server

via queued mode (and plain tcp protocol)
2025-12-20 05:00:42 +01:00 · 2008-06-27 16:11:49 +02:00 · 2008-06-27 16:11:49 +02:00 · 2f405ef14a
commit 2f405ef14a
parent a3a5924d9e
3 changed files with 160 additions and 9 deletions
--- a/2
+++ b/2
@ -1,5 +1,7 @@
 ---------------------------------------------------------------------------
 Version 3.17.5 (rgerhards), 2008-06-??
 - added doc: howto set up a reliable connection to remote server via
  queued mode (and plain tcp protocol)
 - bugfix: comments after actions were not properly treated. For some
  actions (e.g. forwarding), this could also lead to invalid configuration
 ---------------------------------------------------------------------------
--- a/doc/manual.html
+++ b/doc/manual.html
@ -50,14 +50,11 @@ modules</a></li><li><a href="man_rsyslogd.html">rsyslogd man page</a>
 <p><b>We have some in-depth papers on</b></p>
 <ul>
 <li><a href="install.html">installing rsyslog</a></li>
-<li><a href="ipv6.html">rsyslog and IPv6</a>
+<li><a href="ipv6.html">rsyslog and IPv6</a> (which is fully supported)</li>
-(which is fully supported)</li>
+<li><a href="rsyslog_stunnel.html">ssl-encrypting syslog with stunnel</a></li>
-<li><a href="rsyslog_stunnel.html">ssl-encrypting
+<li><a href="rsyslog_mysql.html">writing syslog messages to MySQL (and other databases as well)</a></li>
-syslog with stunnel</a></li>
+<li><a href="rsyslog_high_database_rate.html">writing massive amounts of syslog messages to a database</a></li>
-<li><a href="rsyslog_mysql.html">writing syslog
+<li><a href="rsyslog_reliable_forwarding.html">reliable forwarding to a remote server</a></li>
 messages to MySQL (and other databases as well)</a></li>
 <li><a href="rsyslog_high_database_rate.html">writing
 massive amounts of syslog messages to a database</a></li>
 <li><a href="rsyslog_php_syslog_ng.html">using
 php-syslog-ng with rsyslog</a></li>
 <li><a href="rsyslog_recording_pri.html">recording
@ -66,7 +63,7 @@ the syslog priority (severity and facility) to the log file</a></li>
 syslog sender over NAT</a> (online only)</li>
 <li><a href="gssapi.html">an overview and howto of rsyslog gssapi support</a></li>
 <li><a href="debug.html">debug support in rsyslog</a></li>
-<li><a href="dev_queue.html">the rsyslog message queue object</a></li>
+<li><a href="dev_queue.html">the rsyslog message queue object (developer's view)</a></li>
 </ul>
 <p>Our <a href="history.html">rsyslog history</a>
 page is for you if you would like to learn a little more
--- a/doc/rsyslog_reliable_forwarding.html
+++ b/doc/rsyslog_reliable_forwarding.html
@ -0,0 +1,152 @@
 <html><head>
 <title>Reliable Forwarding of syslog Messages (via plain TCP syslog)</title>
 </head>
 <body>
 <h1>Reliable Forwarding of syslog Messages with Rsyslog</h1>
 		<P><small><i>Written by
 		<a href="http://www.gerhards.net/rainer">Rainer 
 		Gerhards</a> (2008-06-27)</i></small></P>
 <h2>Abstract</h2>
 <p><i><b>In this paper, I describe how to forward
 <a href="http://www.monitorware.com/en/topics/syslog/">syslog</a>
 messages (quite) reliable to a central rsyslog server.</b>
 This depends on rsyslog being installed on the client system and
 it is recommended to have it installed on the server system. Please note
 that industry-standard
 <a href="http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html">plain TCP syslog protocol is not fully reliable</a>
 (thus the "quite reliable"). If you need a truely reliable solution, you need
 to look into RELP (natively supported by rsyslog).</i></p>
 <h2>The Intention</h2>
 <p>Whenever two systems talk over a network, something can go wrong.
 For example, the communications link may go down, or a client or server may abort.
 Even in regular cases, the server may be offline for a short period of time
 because of routine maintenance.
 <p>A logging system should be capable of avoiding message loss in situations where the
 server is not reachable. To do so, unsent data needs to be buffered at the client while the
 server is offline. Then, once the server is up again, this data is to be sent.
 <p>This can easily be acomplished by rsyslog. In rsyslog, every action runs on its own queue
 and each queue can be set to buffer data if the action is not ready. Of course,
 you must be able to detect that "the action is not ready", which means the remote
 server is offline. This can be detected with plain TCP syslog and RELP, but not with UDP.
 So you need to use either of the two. In this howto, we use plain TCP syslog.
 <p>Please note that we are using rsyslog-specific features. The are required on the
 client, but not on the server. So the client system must run rsyslog (at least version 3.12.0), while on the
 server another syslogd may be running, as long as it supports plain tcp syslog.
 <p><b>The rsyslog queueing subsystem tries to buffer to memory. So even if the
 remote server goes
 offline, no disk file is generated.</b> File on disk are created only if there is
 need to, for example if rsyslog runs out of (configured) memory queue space or needs
 to shutdown (and thus persist yet unsent messages). Using main memory and going to the
 disk when needed is a huge performance benefit. You do not need to care about it,
 because, all of it is handled automatically and transparently by rsyslog.</p>
 <h2>How To Setup</h2>
 <p>First, you need to create a working directory for rsyslog. This is where it
 stores its queue files (should need arise). You may use any location on your 
 local system.
 <p>Next, you need to do is instruct rsyslog to use a 
 disk queue and then configure your action. There is nothing else to do. With the 
 following simple config file, you forward anything you receive to a remote server
 and have buffering applied automatically when it goes down. This must be done on the
 client machine.</p>
 <textarea rows="9" cols="80">
 $ModLoad imuxsock             # local message reception
 $WorkDirectory /rsyslog/work  # default location for work (spool) files
 $ActionQueueType LinkedList   # use asynchronous processing
 $ActionQueueFileName srvrfwd  # set file name, also enables disk mode
 $ActionResumeRetryCount -1    # infinite retries on insert failure
 $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
 *.*       @@server:port
 </textarea>
 <p>The port given above is optional. It may not be specified, in which case you only
 provide the server name. The "$ActionQueueFileName" is used to create queue files, should need
 arise. This value must be unique inside rsyslog.conf. No two rules must use the same queue file.
 Also, for obvious reasons, it must only contain those characters that can be used inside a
 valid file name. Rsyslog possibly adds some characters in front and/or at the end of that name
 when it creates files. So that name should not be at the file size name length limit (which
 should not be a problem these days).
 <p>Please note that actual spool files are only created if the remote server is down
 <b>and</b> there is no more space in the in-memory queue. By default, a short failure
 of the remote server will never result in the creation of a disk file as a couple of
 hundered messages can be held in memory by default. [These parameters can be fine-tuned. However,
 then you need to either fully understand how the queue works
 (<a href="http://www.rsyslog.com/doc-queues.html">read elaborate doc</a>) or
 use <a href="http://www.rsyslog.com/doc-professional_support.html">professional services</a>
 to have it done based on
 your specs ;) - what that means is that fine-tuning queue parameters is far from
 being trivial...]
 <p>If you would like to test if your buffering scenario works, you need to 
 stop, wait a while and restart you central server. Do <b>not</b> watch for files being created,
 as this usually does not happen and never happens immediately.
 <h3>Forwarding to More than One Server</h3>
 <p>If you have more than one server you would like to forward to, that's quickly done.
 Rsyslog has no limit on the number or type of actions, so you can define as many targets
 as you like. What is important to know, however, is that the full set of directives make
 up an action. So you can not simply add (just) a second forwarding rule, but need to
 duplicate the rule configuration as well. Be careful that you use different queue
 file names for the second action, else you will mess up your system.
 <p>A sample for forwarding to two hosts looks like this:
 <p>
 <textarea rows="20" cols="80">
 $ModLoad imuxsock.so # local message reception
 $WorkDirectory /rsyslog/work # default location for work (spool) files
 # start forwarding rule 1
 $ActionQueueType LinkedList   # use asynchronous processing
 $ActionQueueFileName srvrfwd1 # set file name, also enables disk mode
 $ActionResumeRetryCount -1    # infinite retries on insert failure
 $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
 *.*       @@server1:port
 # end forwarding rule 1
 # start forwarding rule 2
 $ActionQueueType LinkedList   # use asynchronous processing
 $ActionQueueFileName srvrfwd2 # set file name, also enables disk mode
 $ActionResumeRetryCount -1    # infinite retries on insert failure
 $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
 *.*       @@server2
 # end forwarding rule 2
 </textarea>
 <p>Note the filename used for the first rule it is "srvrfwd1" and for the second it
 is "srvrfwd2". I have used a server without port name in the second forwarding rule.
 This was just to illustrate how this can be done. You can also specify a port there
 (or drop the port from server1).
 <p>When there are multiple action queues, they all work independently. Thus, if server1 
 goes down, server2 still receives data in real-time. The client will <b>not</b> block
 and wait for server1 to come back online. Similarily, server1's operation will not
 be affected by server2's state.
 <h2>Some Final Words on Reliability ...</h2>
 <p>Using plain TCP syslog provides a lot of reliability over UDP syslog. However,
 plain TCP syslog is <b>not</b> a fully reliable transport. In order to get full reliability, 
 you need to use the RELP protocol.
 <p>Folow the next link to learn more about 
 <a href="http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html">the
 problems you may encounter with plain tcp syslog</a>.
 <h3>Feedback requested</h3>
 <P>I would appreciate feedback on this tutorial. If you have additional ideas, 
 comments or find bugs (I *do* bugs - no way... ;)), please
 <a href="mailto:rgerhards@adiscon.com">let me know</a>.</P>
 <h2>Revision History</h2>
 <ul>
 	<li>2008-06-27 * 
 	<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> * Initial Version created</li>
 </ul>
 <h2>Copyright</h2>
 <p>Copyright (c) 2008 
 <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
 <a href="http://www.adiscon.com/en/">Adiscon</a>.</p>
 <p>      Permission is granted to copy, distribute and/or modify this document
      under the terms of the GNU Free Documentation License, Version 1.2
      or any later version published by the Free Software Foundation;
      with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
 	 Texts.  A copy of the license can be viewed at
 <a href="http://www.gnu.org/copyleft/fdl.html">
 http://www.gnu.org/copyleft/fdl.html</a>.</p>
 </body>
 </html>