Currently newsyslog supports <include> directive that is used
in our default /etc/newsyslog.conf in the following form:
<include> /usr/local/etc/newsyslog.conf.d/*
While this is suitable for ports installing their own rules
for logs rotation, this also makes newsyslog break entire
processing of all files if it encounters single line it cannot parse.
This includes lines referring to nonexistent username/group for log
ownership, so newsyslog stops calling errx() function in the parser.
With this fix, newsyslog uses warnx() instead of errx() in such cases
to print a warning, recover gracefully and continue with execution.
Among other cases, this unbreaks initial creation of log files
having flag "C" at boot time (newsyslog -CN). This is most important
for systems having RAM-based /var file system like nanobsd(8)-based
that rely on newsyslog to bring system log files into existence.
MFC after: 1 month
This happens when compressing a previously uncompressed already-rotated
file, as happens when handling the 'p' flag in newsyslog.conf. The file
name is stored in a flexible array member, so these structures cannot be
stack allocated.
Also make sure that we call change_attrs() and do_zipwork() in dry-run
mode; they handle this properly, contrary to the commit log message for
r327451.
CID: 1008168
Github PR: https://github.com/freebsd/freebsd/pull/427
MFC after: 2 weeks
Submitted by: Radek Brich (original version)
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19105
It was pointed out that a couple of the "memory leak" CIDs that I
fixed were arguably Coverity errors rather than errors in the
newsyslog code and the cure was worse than the disease. Revert both
changes. The first change, which included fixes for other Coverity
errors, will be re-worked to omit the troublesome changes and then
re-committed with the remaining fixes.
Reported by: bde
Sponsored by: Dell EMC Isilon
The result of a strdup() was stored in a global variable and not freed
before program exit. This is a follow-up to r343906. That change
attempted to plug these resource leaks but managed to miss a code path
on which the leak still occurs. Plug the leak on that path, too.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007454, CID 1007453: Resource leak - The result of a strdup()
was stored in a global variable and not freed before program exit.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
We can no longer use sizeof() to get the path buffer's size. Apply
a straightforward fix for now with the aim of MFCing soon.
PR: 233633
Submitted by: Katsuyuki Miyoshi <katsu@miyoshi.matsuyama.ehime.jp>
MFC after: 3 days
Prevent some classes of foot-shooting that may result in permissions
problems.
Reviewed by: dab, delphij, vangyzen (earlier version)
Relnotes: yes (behavior change)
Sponsored by: Dell EMC Isilon
Differential Revision: D16831
- Remove the compression suffix macros and move them directly into the
compress_type array.
- Remove the hardcoded sizes on the suffix and compression args arrays.
- Simplify the compression args arrays at the expense of a __DECONST
when calling execv().
- Rewrite do_zipwork. The COMPRESS_* macros can directly index the
compress_types array, so the outer loop is not needed. Convert
fixed-length strings into asprintf or sbuf calls.
Submitted by: Dan Nelson <dnelson_1901@yahoo.com>
Reviewed by: gad
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16518
The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities. However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.
Reuse of the index variable in two nested loops resulted in only the first
argument in the list being used (fine for gzip, not fine for zstd). Also
add tests for xz and zstd, and fix the COMPRESS_SUFFIX_MAXLEN macro.
Submitted by: dnelson_1901_yahoo.com
Differential Revision: https://reviews.freebsd.org/D16509
The RFC 5424 spec mentions that logging FQDNs over short hostnames is
preferred. Alter this code, so that the hostname doesn't get truncated
on startup. Keep track of the length of the short hostname, so that
fprintf() can do the truncation where necessary.
MFC after: 1 month
Implement the 'p' flag for newsyslog from NetBSD. This flag results in
the first log file for a given file to not be compressed.
While here, don't change file attributes during a no-op run
PR: 162798
Submitted by: heas@shrubbery.net
MFC After: 1 month
When building the command to execute for compression, newsyslog was modifying
the generic arguments array instead of its own copy.
Meaning on the second file to compress with the same arguments, the command line
was not the one expected.
Fix it by creating one copy of the arguments per execution and modifying that
copy.
While here, print the command line executed in verbose mode.
Reported by: many
have a semantic different than the traditional gzip(1)
This is done to allow to use zstd(1) as a compression tool without
having to patch it to change its default behavior.
This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.
Reviewed by: vangyzen, ngie
Approved by: vangyzen (mentor)
MFC after: 2 months
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10253
This makes newsyslog use zstandard to compress log files.
Given Z is already taken for gzip and zstandard compression level stands in
between gzip and xz (which has the X flag) chosing Y sounds ok :)
It turns out that we had a couple of more calls to dirname()/basename()
in newsyslog(8) that assume the input isn't clobbered. This is bad,
because it apparently breaks log rotation now that the new dirname()
implementation has been merged.
Fix this by first copying the input and then calling
dirname()/basename(). While there, improve the naming of variables in
this function a bit.
Reported by: Ryan Steinmetz, gjb
Reviewed by: bdrewery, allanjude
Differential Revision: https://reviews.freebsd.org/D7838
Pull copies of the input pathname string before calling basename() and
dirname() to make this comply to POSIX. Free these copies at the end of
this function. While there, remove the duplication of the 's' ->
'logfname' string. There is no need for this.
After going through the signal work list, during which do_sigwork()
is called and essentially does nothing because -s and -R were
specified on the command line, newsyslog will sleep for 10 seconds
as the (verbose) code says: "Pause 10 seconds to allow daemon(s)
to close log file(s)".
However, the man page verbiage for -R (and -s) seems quite clear
that this sleep() is unnecessary because the daemon was expected
to have already closed the log file before calling newsyslog.
PR: 210020
Submitted by: David A. Bright <david_a_bright@dell.com>
MFC after: 1 week
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6727
When -C was introduced in r114137 the plan was to have -C and -c being used for
"create" due to a typo in FreeBSD <= 4.8 a temporary compatibility hack has been
added to make -c being like -G aka GLOB and a warning was issued for the user to
be aware of the futur change for -c.
12 years later it is more than time to remove that hack and finish the what was
intent in r114137
Submitted by: Alexandre Perrin <alex@kaworu.ch>
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4000
returning directory entries through readdir(3). In this case we need to
obtain the file type ourselves; otherwise newsyslog -t will not be able to
find archived log files and will fail to both delete old log files and to
do interval-based rotations properly.
Reported by: jilles
Reviewed by: jilles
MFC after: 2 weeks
the most-recently archived logfile and use its mtime to determine whether
or not to rotate, as in the non-timestamped case.
Previously we would just try to use the mtime of <logfile>.0, which always
results in a rotation since it generally doesn't exist in the -t case.
PR: bin/166448
Approved by: emaste (co-mentor)
Tested by: Marco Steinbach <coco executive-computing.de>
MFC after: 2 weeks
the corresponding struct sigwork_entry were left uninitialized,
potentially causing an early return from do_sigwork(). Ensure that these
fields are initialized, and handle the 'R' flag properly in
do_sigwork().
PR: bin/175330
Reviewed by: gad
Approved by: rstone (co-mentor)
MFC after: 1 week
ensures that the next rotation happens at the correct time when using
interval-based rotations.
PR: bin/174438
Reviewed by: gad
Approved by: rstone (co-mentor)
MFC after: 1 week
In addition to adding missing `static' keywords:
- bin/dd: Pull in `extern.h' to guarantee consistency with source file.
- libexec/rpc.rusersd: Move shared globals into an extern.h.
- libexec/talkd: Move `debug' and `hostname' into extern.h.
- usr.bin/cksum: Put counters in extern.h, as they are used by ckdist/mtree.
- usr.bin/m4: Move `end_result' into extern.h.
- usr.sbin/services_mkdb: Move shared globals into an extern.h.
Introduce dirfd() libc exported symbol replacing macro with same name,
preserve _dirfd() macro for internal use.
Replace dirp->dd_fd with dirfd() call. Avoid using dirfd as variable
name to prevent shadowing global symbol.
Sponsored by: Google Summer Of Code 2011
The index() and rindex() functions were marked LEGACY in the 2001
revision of POSIX and were subsequently removed from the 2008 revision.
The strchr() and strrchr() functions are part of the C standard.
This makes the source code a lot more consistent, as most of these C
files also call into other str*() routines. In fact, about a dozen
already perform strchr() calls.
will be considered as a path to a binary or a shell script to be executed
after rotation has been completed instead of sending signal to the process
id in that file.
Sponsored by: Sippy Software, Inc.
From the: FreeBSD hacking lounge at BSDCan
