Restore ability for our syslogd to collect pre-RFC3164 formatted
messages from remote hosts that was broken with r326573.
For example, the line from Cisco SCE8000 splitted for readability:
1130: 03:37:57: %USER-6-PORT_OPERSTATUS_CHANGE_TRAP: CPU#000 trap:link
down EntityAdminState: 4 EntityAlarmStatus: 32
Such line was collected and stored before mentioned change
but silently dropped after that. Now syslogd saves it again.
Note that parsing of RFC5424 format not changed.
MFC after: 1 month
Since 2d82b47 syslogd can't be built with `WITHOUT_INET` or
`WITHOUT_INET6` build variables set, because `iovlist_truncate` is not
defined but used.
This change wraps the problematic `iovlist_truncate` call within ifdef
directive. It's compiled out in this situation...
Pull Request: https://github.com/freebsd/freebsd-src/pull/475
Reviewed by: imp@ (commit message slightly tweaked)
Add a -M option to control the maximum length of forwarded messages.
syslogd(8) used to truncate forwarded messages to 1024 bytes, but after
commit 1a874a126a ("Add RFC 5424 syslog message output to syslogd.")
applies a more conservative limit of 480 bytes for IPv4 per RFC 5426
section 3.2. Restore the old default behaviour of truncating to 1024
bytes. RFC 5424 specifies no upper limit on the length of forwarded
messages, while for RFC 3164 the limit is 1024 bytes.
Increase MAXLINE to 8192 bytes to correspond to commit 672ef817a192.
Replaced bootfile[] size for MAXPATHLEN used in getbootfile(3) as a
returned value. Using (MAXLINE+1) as a size for bootfile[] is excessive.
PR: 241937
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D27206
Specifically this was causing the /dev/klog fd and the signal pipe
handling fd to get a sendmsg(2) called on them and always returned
[ENOTSOCK].
r310350 combined these sockets into the main socket list and properly
skipped AF_UNSPEC at the sendmsg(2) call but later in r344739 it was
broken such that these special sockets were no longer excluded since
the AF_UNSPEC check specifically excluded these special sockets. Only
these special sockets have sl_sa = NULL. The sl_family checks should
be redundant now but are left in case of future changes so the intent
is clearer.
MFC after: 2 weeks
This behavior is most relevant for ipfw(4) as documented in syslog.conf(5).
The recent addition of property-based regex filters in r359327 is a
fine workaround for this but the behavior was present since 1997 and
documented.
This only fixes local matching of the "kernel program". It does not
change the forwarded format at all. On the remote side it will still
be "kernel: ipfw:" and not be parsed as a kernel message. This matches
old behavior.
MFC after: 2 weeks
Reviewed by: markj
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D24286
Property-based filters allow substring and regular expressions
(see re_format(7)) matching against various message attributes.
Filter specification starts with '#:' or ':' followed by three
comma-separated fields property, operator, "value". Value must be
double-quoted. A double quote and backslash must be escaped by a
blackslash.
Following properties are supported as test value:
o msg - body of the message received;
o programname - program name sent the message;
o hostname - hostname of message's originator;
o source - an alias for hostname.
Supported operators:
o contains - true if filter value is found as a substring of property;
o isequal - true if filter value is equal to property;
o startswith - true if property starts with filter value;
o regex - true if property matches basic regular expression defined
in filter value;
o ereregex - true if property matches extended regular expression
defined in filter value;
Operator may be prefixed by '!' to invert compare logic or by
'icase_' to make comparison function case insensitive.
Submitted by: Boris N. Lytochkin <lytboris at gmail com>
MFC after: 2 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D23468
Buildworld failed when both WITHOUT_INET6_SUPPORT and INET equivalent were set.
Fix netstat and syslogd by applying appropriate #ifdef INET/INET6 to make world
compile again.
Reviewed by: ngie, hrs, ume
Welcomed by: Michael Dexter (D17040)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D19418
Tools such as Postfix use slashes in process names for hierarchy
(postfix/qmgr). By allowing these slashes, syslogd is able to extract
the process name and process ID nicely, so that they can be stored in
RFC 5424 message fields.
MFC after: 1 week
The refactoring of the syslogd code to format messages using iovecs
slightly altered the output of syslogd by placing the facility/priority
after the hostname, as opposed to printing it right before. This change
reverts the behaviour to be consistent with how it was before.
PR: 229457
Reported by: Andre Albsmeier
MFC after: 1 week
The changes made in r326573 required that messages always start with an
RFC 3164 timestamp. It looks like certain devices, but also certain
logging libraries (Python 3's "logging" package) simply don't generate
RFC 3164 formatted messages containing a timestamp.
Make timestamps optional again. When the timestamp is missing, also
assume that the message contains no hostname. The first word of the
message likely already belongs to the message payload.
PR: 229236
Reported by: Michael Grimm & Marek Zarychta
Reviewed by: glebius (cursory)
MFC after: 1 week
To conform to RFC 5426, this function is intended to truncate messages
if they exceed the message size limits. Unfortunately, the amount of
space was computed the wrong way around, causing messages to be
truncated entirely.
Reported by: Michael Grimm on stable@
MFC after: 3 days
- Move all of the code responsible for transmitting log messages into a
separate function, fprintlog_write().
- Instead of manually modifying a list of iovecs, add a structure
iovlist with some helper functions.
- Alter the F_FORW (UDP message forwarding) case to also use iovecs like
the other cases. Use sendmsg() instead of sendto().
- In the case of F_FORW, truncate the message to a size dependent on the
address family (AF_INET, AF_INET6), as proposed by RFC 5426.
- Move all traditional message formatting into fprintlog_bsd(). Get rid
of some of the string copying and snprintf()'ing. Simply emit more
iovecs to get the job done.
- Increase ttymsg()'s limit of 7 iovecs to 32. Add a definition for this
limit, so it can be reused by iovlist.
- Add fprintlog_rfc5424() to emit RFC 5424 formatted log entries.
- Add a "-O" command line option to enable RFC 5424 formatting. It would
have been nicer if we supported "-o rfc5424", just like on NetBSD.
Unfortunately, the "-o" flag is already used for a different purpose
on FreeBSD.
- Don't truncate hostnames in the RFC 5424 case, as suggested by that
specific RFC.
For people interested in using this, this feature can be enabled by
adding the following line to /etc/rc.conf:
syslogd_flags="-s -O rfc5424"
Differential Revision: https://reviews.freebsd.org/D15011
Now that all of parsemsg() parses both RFC 3164 and 5424 messages and
hands them to logmsg(), alter the latter to properly forward all RFC
5424 message attributes to fprintlog(). While there, make some minor
cleanups to this code:
- Instead of extending the existing code that compares hostnames and
message bodies for deduplication, print all of the relevant message
fields into a single string that we can compare ('saved').
- No longer let the behaviour of fprintflog() depend on whether
'msg == NULL' to print repetition messages, Simply decompose this
function into fprintlog_first() and fprintlog_successive(). This
makes the interpretation of function arguments less magical and also
allows us to get consistent behaviour across RFC 3164 and 5424 when
adding support for the RFC 5424 output format.
- As RFC 5424 syslog messages have a dedicated application name field,
alter the repetition messages to be printed on behalf of syslogd on
the current system. Change these messages to use the local hostname,
so that it's obvious which syslogd instance detected the repetition.
Remove f_prevhost, as it has now become unnecessary.
- Remove a useless strdup(). Deconsting the message string is safe in
this specific case.
Syslogd currently uses the RFC 3164 format for its log messages.One
limitation of RFC 3164 is that it cannot be used to log entries with
sub-second precision timestamps. One of our users has expressed a desire
for doing this for doing some basic performance measurements.
This change attempts to make a first cut at switching to RFC 5424 based
logging. The first step is to alter syslogd's input path to properly
parse such messages. It alters the logmsg() prototype to match the
fields of RFC 5424. The parsemsg() function is extended to parse both
RFC 3164 and 5424 messages and call into logmsg() accordingly.
Additional changes include:
- Introducing proper parsing of timestamps, so that they can be printed
in any desired output format. This means we need to infer the year and
timezone for RFC 3164 timestamps.
- Removing ISKERNEL. This can now be realised by simply providing an
APP-NAME (== "kernel").
- Extending RFC 3164 parsing to trim off the TAG prefix and using that
to derive APP-NAME and PROCID.
- Increase MAXLINE. RFC 5424 mentions we should support 2k messages.
Differential Revision: https://reviews.freebsd.org/D14926
A memory leak in syslogd for processing of forward actions was
reported. This modification adapts the patch submitted with that bug
to fix the leak. While testing the modification, another leak was also
found and fixed.
PR: 198385
Submitted by: Sreeram <sreeramabs@yahoo.com>
Reported by: Sreeram <sreeramabs@yahoo.com>
Reviewed by: hrs
MFC after: 1 week
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D14510
field, and support properly parse out the hostname as described by RFC3164,
which wasn't done before. However, don't discard message if it doesn't
have hostname, for compatibility.
Enable logging of the message supplied hostname instead of real hostname
with -H switch.
PR: 200933
Reported by: Konstantin Pavlov <thresh nginx.com>
MFC after: 2 months
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
the r316874: don't call shutdown(2) on all sockets, but only net ones, which seems
to be the behaviour existed before that refactoring. Also don't call listen(2)
in datagram sockets and fix misplaced debug messages while I am here.
Reported by: peter
don't bother to select/recv on that socket. This prevents syslogd(8)
from spinning endlessly when started with the -s option (default).
Reported by: peter
getaddrinfo fails
If the asprintf call fails, fall back to the old code (as a last ditch effort
to provide the end-user with helpful output).
Sponsored by: Dell EMC Isilon
- Make the explanation more complete
- Correct a minor grammar nit with verb tense.
- Don't emit the message if `pe->pe_name` is NULL (it doesn't
have much value).
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Free `f` if an unknown priority or facility is parsed with the function.
MFC after: 1 week
Reported by: Coverity
CID: 1368068
Sponsored by: Dell EMC Isilon
- main(..): free memory assigned to fdsr before calling die(..).
- allowaddr(..): free memory assigned to ap before returning from the
function early. Add a `err` goto label to reduce freeaddrinfo/free(ap)
logic duplication.
MFC after: 1 week
X-MFC notes: some of this is dependent on refactoring not MFCed
Reported by: clang static analyzer, Coverity
CID: 1367750 (ap leakage in allowaddr(..))
Submitted by: Tom Rix <trix@juniper.net>
Reviewed by: ngie
Sponsored by: Dell EMC Isilon, Juniper
Differential Revision: D10004
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
This removes the hardcoded value for the field (16) and the equivalent
hardcoded lengths in logmsg(..).
This change is being done to help stage future work to add RFC5424/RFC5434
support to syslogd(8).
Obtained from: Isilon OneFS (dcd33d13da) (as part of a larger change)
Submitted by: John Bauman <john.bauman@isilon.com>
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
- Set O_CLOEXEC to the signal pipe and /dev/klog.
- Use a single signal handler to catch both SIGHUP and SIGCHLD.
- Fix a bug which did FD_SET() the writer-end of the pipe.
the main I/O multiplex loop. select() now watches
a pipe which is written by the new skinny signal
handlers and the received signals are handled inside
the loop sequencially.
This eliminates a complex signal mask to guarantee
async-signal safety.
