mirror of
https://github.com/rsyslog/rsyslog.git
synced 2025-12-16 02:50:40 +01:00
doc: omkafka SASL-from-env example + backticks doc overhaul
- omkafka.rst: add “Set SASL password from an environment variable”
section with two paths:
* v8.2508.0+: inline `${VAR}` with adjacency via backticks
(`echo sasl.password=${KAFKA_PASSWORD}`)
* pre-8.2508.0: pre-compose `SASL_PWDPARAM` and echo that
- Clarify security notes (env files, /proc visibility) and add anchor
label :ref:`omkafka-sasl-password-from-env`
- constant_strings.rst: restructure and expand
* document `${VAR}` support, unbraced `$VAR` termination rules, and
adjacency to static text (v8.2508.0)
* keep scope explicit: only `echo`/`cat`, no `$(...)`
* add clear examples and use `rsyslog` lexer for code blocks
* add compatibility guidance for older versions
* add note: backticks can toggle boolean flags such as `config.enabled`
- Motivation: addresses confusion seen in omkafka SASL cases (e.g. #5827)
No behavioral changes to code; documentation only.
This commit is contained in:
Rainer Gerhards
parent
5c461921a6
commit
135f2c08b3
@ -258,3 +258,78 @@ comma-delimited list of values as shown here:
|
||||
)
|
||||
|
||||
|
||||
.. _omkafka-sasl-password-from-env:
|
||||
|
||||
Set SASL password from an environment variable
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. versionchanged:: 8.2508.0
|
||||
|
||||
Backticks in RainerScript support the ``${VAR}`` form and adjacent text.
|
||||
This enables a simpler inline configuration such as::
|
||||
|
||||
`echo sasl.password=${KAFKA_PASSWORD}`
|
||||
|
||||
**Recommended (rsyslog v8.2508.0 and later)**
|
||||
|
||||
Keep the secret out of *rsyslog.conf* and inject it via environment.
|
||||
Then build the ``key=value`` pair inline with backticks:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# set by your service manager or a secure env file
|
||||
export KAFKA_PASSWORD='supersecret'
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
action(
|
||||
type="omkafka"
|
||||
broker=["kafka.example.com:9093"]
|
||||
confParam=[
|
||||
"security.protocol=SASL_SSL",
|
||||
"sasl.mechanism=SCRAM-SHA-512",
|
||||
"sasl.username=myuser",
|
||||
`echo sasl.password=${KAFKA_PASSWORD}`
|
||||
]
|
||||
)
|
||||
|
||||
Notes:
|
||||
|
||||
- This relies on the enhanced backtick handling; it is **not** a general shell.
|
||||
Only the documented backtick subset (notably ``echo`` and ``cat``) is supported.
|
||||
- The variable expansion happens at rsyslog parse time, using the process
|
||||
environment of the rsyslog daemon.
|
||||
|
||||
**Older rsyslog versions (before v8.2508.0)**
|
||||
|
||||
Backticks did **not** understand ``${VAR}`` or adjacency. Inline forms like
|
||||
`` `echo sasl.password=$KAFKA_PASSWORD` `` could cause errors such as
|
||||
“missing equal sign in parameter”. Use a pre-composed environment variable that
|
||||
already contains the full ``key=value`` pair and echo **that**:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export KAFKA_PASSWORD='supersecret'
|
||||
# Pre-compose the full key=value (done *outside* rsyslog)
|
||||
export SASL_PWDPARAM="sasl.password=${KAFKA_PASSWORD}"
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
action(
|
||||
type="omkafka"
|
||||
broker=["kafka.example.com:9093"]
|
||||
confParam=[
|
||||
"security.protocol=SASL_SSL",
|
||||
"sasl.mechanism=SCRAM-SHA-512",
|
||||
"sasl.username=myuser",
|
||||
`echo $SASL_PWDPARAM`
|
||||
]
|
||||
)
|
||||
|
||||
Security guidance
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
- Prefer environment files or service manager mechanisms with strict permissions
|
||||
over embedding secrets directly in *rsyslog.conf*.
|
||||
- Process environments may be visible to privileged users (e.g., via ``/proc``);
|
||||
secure host access accordingly.
|
||||
|
||||
@ -1,92 +1,255 @@
|
||||
String Constants
|
||||
================
|
||||
Rsyslog Parameter String Constants
|
||||
==================================
|
||||
|
||||
String constants are necessary in any scripting language. They provide
|
||||
values that are evaluated at startup and never change during rsyslog's
|
||||
run.
|
||||
String constants provide values that are evaluated at startup and remain unchanged
|
||||
for the lifetime of the rsyslog process.
|
||||
|
||||
Uses
|
||||
----
|
||||
String constants are necessary in many places: comparisons,
|
||||
configuration parameter values and function arguments, to name a few
|
||||
important ones.
|
||||
|
||||
In string constants, special characters are escaped by prepending a
|
||||
backslash in front of them -- just in the same way this is done in the C
|
||||
programming language or PHP.
|
||||
String constants are used in comparisons, configuration parameter values, and
|
||||
function arguments, among other places.
|
||||
|
||||
If in doubt how to properly escape, use the `RainerScript String Escape
|
||||
Online
|
||||
Tool <http://www.rsyslog.com/rainerscript-constant-string-escaper/>`_.
|
||||
Escaping
|
||||
--------
|
||||
|
||||
In string constants, special characters are escaped by prepending a backslash,
|
||||
similar to C or PHP.
|
||||
|
||||
If in doubt how to escape properly, use the
|
||||
`RainerScript String Escape Online Tool
|
||||
<http://www.rsyslog.com/rainerscript-constant-string-escaper/>`_.
|
||||
|
||||
Types
|
||||
-----
|
||||
|
||||
Rsyslog provides different types of string constants, closely inspired
|
||||
by the shell:
|
||||
Rsyslog provides different types of string constants, inspired by common shell
|
||||
semantics.
|
||||
|
||||
- single quotes
|
||||
Single quotes
|
||||
.............
|
||||
|
||||
Values are used unaltered, except for escape sequences, which are
|
||||
escaped.
|
||||
Values are used unaltered, except that escape sequences are processed.
|
||||
|
||||
- double quotes
|
||||
Double quotes
|
||||
.............
|
||||
|
||||
Right now, equivalent to single quotes, but $ signs need to be escaped.
|
||||
If not escaped, a syntax error will be generated and rsyslog startup
|
||||
be aborted in most cases.
|
||||
The idea is to support environment variables just like the shell does
|
||||
in later releases.
|
||||
Equivalent to single quotes; a literal dollar sign must be escaped (``\$``).
|
||||
If ``$`` is not escaped, a syntax error is raised and rsyslog usually refuses
|
||||
to start.
|
||||
|
||||
- backticks
|
||||
Backticks
|
||||
.........
|
||||
|
||||
This was added in 8.33.0. The idea is to provide a useful subset of
|
||||
what the shell does. Right now, only the following is supported:
|
||||
.. versionadded:: 8.33.0
|
||||
Initial support for backticks (limited subset).
|
||||
|
||||
- `echo $VARNAME` - It will evaluate the environment variable and use
|
||||
it as string constant. If the variable is not found, an empty string
|
||||
is generated (this is **not** an error).
|
||||
.. versionchanged:: 8.37.0
|
||||
Multiple environment variable expansions and constant text between them.
|
||||
|
||||
Starting with 8.37.0, the `echo` case has been enhanced. It is now
|
||||
more along the lines of what bash does. It supports multiple
|
||||
environment variable expansions as well as constant text
|
||||
between them.
|
||||
.. versionchanged:: 8.2508.0
|
||||
Support for brace-style variables (``${VAR}``), correct termination of
|
||||
unbraced ``$VAR`` at the first non-``[A-Za-z0-9_]`` character, and support
|
||||
for variable text **adjacent** to static text (e.g., key/value pairs).
|
||||
|
||||
An example:
|
||||
Backticks intentionally implement a **small, safe subset** of shell-like
|
||||
behavior. Currently, only the following forms are supported:
|
||||
|
||||
* env SOMEPATH is set to "/var/log/custompath"
|
||||
* config is: param=echo $SOMEPATH/myfile
|
||||
* param than is expanded to "/var/log/custompath/myfile"
|
||||
- ``echo …`` — evaluate simple text with environment variable expansion.
|
||||
- ``cat <filename>`` — include the contents of a single file.
|
||||
If the file is unreadable, the result is an empty string.
|
||||
|
||||
Note, however, that some common bash features are not supported.
|
||||
Environment variables are terminated once a character not belonging
|
||||
to the set ``[A-Za-z0-9_]`` is encountered or by the closing brace in
|
||||
``${VAR}`` syntax. Constructs like ``$(pwd)`` remain unsupported. The idea
|
||||
of this parameter is not to provide a
|
||||
full-blown bash-equivalent, but provide some functionality that is
|
||||
usually considered useful for customizing rsyslog configuration with
|
||||
outside data. For example, an expression like::
|
||||
Any other construct results in a parse error. There must be **exactly one space**
|
||||
between ``echo`` or ``cat`` and the following argument.
|
||||
|
||||
`echo Log-$HOSTNAME!`
|
||||
Environment variable expansion rules
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
expands to ``Log-<yourhost>!`` where ``$HOSTNAME`` is replaced and the
|
||||
exclamation mark follows as literal text. The braces form ``${HOSTNAME}``
|
||||
can also be used, e.g. `` `echo {${HOSTNAME}}` ``, which results in
|
||||
``{<yourhost>}``. That said, we are still
|
||||
interested in extending the coverage if clear need and reasoning is
|
||||
provided.
|
||||
- Both ``$VAR`` and ``${VAR}`` are supported.
|
||||
- For unbraced ``$VAR``, the variable name ends at the first character
|
||||
**not** in ``[A-Za-z0-9_]``; that character is emitted literally.
|
||||
This makes expressions like ``$VAR!`` or ``$VAR.suffix`` work as expected.
|
||||
- For braced ``${VAR}``, expansion ends at the matching ``}``, which naturally
|
||||
supports **adjacent** static text (e.g., ``prefix${VAR}suffix``).
|
||||
- Unknown variables expand to the empty string (not an error).
|
||||
- Command substitutions and other shell features (e.g., ``$(pwd)``) are
|
||||
**not supported** by design.
|
||||
|
||||
- `cat filename` - It will evaluate to the content of the given file.
|
||||
Only a single file name is supported. If the file is not readable,
|
||||
it will evaluate to an empty string.
|
||||
Examples
|
||||
========
|
||||
|
||||
Any other construct will currently lead to an error message.
|
||||
Note that there must be exactly one space between "echo" or "cat" and
|
||||
the other parameter.
|
||||
Simple expansion
|
||||
----------------
|
||||
|
||||
Backticks are especially useful for configuration files that are
|
||||
auto-generated but need to contain a small set of special functionality.
|
||||
Given ``SOMEPATH=/var/log/custompath``:
|
||||
|
||||
For an example of this in action, have a look at the rsyslog docker
|
||||
appliance available at
|
||||
https://github.com/rsyslog/rsyslog-docker/tree/master/appliance/alpine.
|
||||
.. code-block:: rsyslog
|
||||
|
||||
param = `echo $SOMEPATH/myfile`
|
||||
# expands to: /var/log/custompath/myfile
|
||||
|
||||
Brace form and adjacent text
|
||||
----------------------------
|
||||
|
||||
Given ``HOSTNAME=myhost``:
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
title = `echo Log-${HOSTNAME}!`
|
||||
# expands to: Log-myhost!
|
||||
|
||||
Key/value pairs (common with module parameters)
|
||||
-----------------------------------------------
|
||||
|
||||
Given ``KAFKA_PASSWORD=supersecret`` and rsyslog >= 8.2508.0:
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
action(
|
||||
type="omkafka"
|
||||
broker=["kafka.example.com:9093"]
|
||||
confParam=[
|
||||
"security.protocol=SASL_SSL",
|
||||
"sasl.mechanism=SCRAM-SHA-512",
|
||||
"sasl.username=myuser",
|
||||
`echo sasl.password=${KAFKA_PASSWORD}`
|
||||
]
|
||||
)
|
||||
|
||||
.. index::
|
||||
single: config.enabled
|
||||
single: environment variables; config.enabled
|
||||
single: systemd; environment variables
|
||||
single: imtcp; conditional loading
|
||||
pair: containers; environment-style configuration
|
||||
|
||||
Conditional enablement of configuration objects via ``config.enabled``
|
||||
----------------------------------------------------------------------
|
||||
|
||||
You can toggle modules, inputs, actions, etc. based on environment variables by
|
||||
setting the generic boolean parameter ``config.enabled``. The example below loads
|
||||
``imtcp`` and starts a TCP listener only if ``LOAD_IMTCP`` evaluates to a
|
||||
truthy value.
|
||||
|
||||
**systemd service (recommended on most distributions)**
|
||||
|
||||
Use a drop-in to set the environment variable in rsyslog’s service context:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo mkdir -p /etc/systemd/system/rsyslog.service.d
|
||||
sudo tee /etc/systemd/system/rsyslog.service.d/10-imtcp.conf >/dev/null <<'EOF'
|
||||
[Service]
|
||||
# Enable
|
||||
Environment=LOAD_IMTCP=on
|
||||
EOF
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart rsyslog
|
||||
|
||||
Alternatively, you can point to an ``EnvironmentFile=`` that contains
|
||||
``LOAD_IMTCP=…``.
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
# /etc/systemd/system/rsyslog.service.d/10-imtcp.conf
|
||||
[Service]
|
||||
EnvironmentFile=-/etc/default/rsyslog
|
||||
|
||||
# /etc/default/rsyslog
|
||||
LOAD_IMTCP=on
|
||||
|
||||
**Bash / non-systemd service environment**
|
||||
|
||||
If rsyslog is started from a shell wrapper (e.g., SysV init or container entrypoint):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# enable (any of 1, true, on, yes will do)
|
||||
export LOAD_IMTCP=on
|
||||
|
||||
# or disable
|
||||
# export LOAD_IMTCP=off
|
||||
|
||||
**RainerScript (rsyslog.conf)**
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
# Load the module conditionally. If LOAD_IMTCP is unset/empty/0/false,
|
||||
# the statement is disabled and the module is not loaded.
|
||||
module(
|
||||
load="imtcp"
|
||||
config.enabled=`echo ${LOAD_IMTCP}`
|
||||
)
|
||||
|
||||
# Start a TCP input only when enabled. This also works if you prefer
|
||||
# to always load the module but gate the input itself.
|
||||
input(
|
||||
type="imtcp"
|
||||
port="514"
|
||||
ruleset="in_tcp"
|
||||
config.enabled=`echo ${LOAD_IMTCP}`
|
||||
)
|
||||
|
||||
ruleset(name="in_tcp") {
|
||||
action(type="omfile" file="/var/log/remote/tcp.log")
|
||||
}
|
||||
|
||||
Accepted truthy value for ``config.enabled`` is ``on``. Any other value,
|
||||
including no value, is treated as false.
|
||||
|
||||
.. note::
|
||||
|
||||
In containerized deployments, this pattern is often the simplest way to pass
|
||||
environment-style toggles into a static configuration. For example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --rm \
|
||||
-e LOAD_IMTCP=on \
|
||||
-v /path/to/rsyslog.conf:/etc/rsyslog.conf:ro \
|
||||
my-rsyslog-image
|
||||
|
||||
|
||||
Including a file’s content
|
||||
--------------------------
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
token = `cat /etc/rsyslog.d/token.txt`
|
||||
|
||||
Compatibility note for older releases
|
||||
-------------------------------------
|
||||
|
||||
On versions **before 8.2508.0**, backticks did **not** support ``${VAR}`` or
|
||||
adjacent text reliably. If you need to build a ``key=value`` pair, pre-compose
|
||||
it outside rsyslog and expand the already-complete string, e.g.:
|
||||
|
||||
In bash or init script:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export KAFKA_PASSWORD='supersecret'
|
||||
export SASL_PWDPARAM="sasl.password=${KAFKA_PASSWORD}"
|
||||
|
||||
In rsyslog conf:
|
||||
|
||||
.. code-block:: rsyslog
|
||||
|
||||
action(
|
||||
type="omkafka"
|
||||
broker=["kafka.example.com:9093"]
|
||||
confParam=[
|
||||
"security.protocol=SASL_SSL",
|
||||
"sasl.mechanism=SCRAM-SHA-512",
|
||||
"sasl.username=myuser",
|
||||
`echo $SASL_PWDPARAM`
|
||||
]
|
||||
)
|
||||
|
||||
Rationale
|
||||
---------
|
||||
|
||||
Backticks are intended to provide just enough functionality to customize
|
||||
configuration with external data while avoiding the complexity and risks of a
|
||||
general shell interpreter.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user