Currently newsyslog supports <include> directive that is used
in our default /etc/newsyslog.conf in the following form:
<include> /usr/local/etc/newsyslog.conf.d/*
While this is suitable for ports installing their own rules
for logs rotation, this also makes newsyslog break entire
processing of all files if it encounters single line it cannot parse.
This includes lines referring to nonexistent username/group for log
ownership, so newsyslog stops calling errx() function in the parser.
With this fix, newsyslog uses warnx() instead of errx() in such cases
to print a warning, recover gracefully and continue with execution.
Among other cases, this unbreaks initial creation of log files
having flag "C" at boot time (newsyslog -CN). This is most important
for systems having RAM-based /var file system like nanobsd(8)-based
that rely on newsyslog to bring system log files into existence.
MFC after: 1 month
While here do a bit of cleanup:
- declare local variables as such,
- make tmpdir_create() clean up logfile directories, to handle a
previously interrupt test run more gracefully.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
This happens when compressing a previously uncompressed already-rotated
file, as happens when handling the 'p' flag in newsyslog.conf. The file
name is stored in a flexible array member, so these structures cannot be
stack allocated.
Also make sure that we call change_attrs() and do_zipwork() in dry-run
mode; they handle this properly, contrary to the commit log message for
r327451.
CID: 1008168
Github PR: https://github.com/freebsd/freebsd/pull/427
MFC after: 2 weeks
Submitted by: Radek Brich (original version)
autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems. As of r296194 the amd man page claimed that it
is deprecated. Remove it from base now; the sysutils/am-utils port is
still available if necessary.
Discussed with: cy
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
log daemon facility now that daemon(8) has syslog support which defaults to
daemon facility, info priority
Reviewed by: bapt
Approved by: bapt
Differential Revision: https://reviews.freebsd.org/D21561
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19105
It was pointed out that a couple of the "memory leak" CIDs that I
fixed were arguably Coverity errors rather than errors in the
newsyslog code and the cure was worse than the disease. Revert both
changes. The first change, which included fixes for other Coverity
errors, will be re-worked to omit the troublesome changes and then
re-committed with the remaining fixes.
Reported by: bde
Sponsored by: Dell EMC Isilon
The result of a strdup() was stored in a global variable and not freed
before program exit. This is a follow-up to r343906. That change
attempted to plug these resource leaks but managed to miss a code path
on which the leak still occurs. Plug the leak on that path, too.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007454, CID 1007453: Resource leak - The result of a strdup()
was stored in a global variable and not freed before program exit.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
We can no longer use sizeof() to get the path buffer's size. Apply
a straightforward fix for now with the aim of MFCing soon.
PR: 233633
Submitted by: Katsuyuki Miyoshi <katsu@miyoshi.matsuyama.ehime.jp>
MFC after: 3 days
The new default config will only include files from the following
directories which end with '.conf' and do not beginning with a '.'
character:
- /etc/newsyslog.conf.d/
- /usr/local/etc/newsyslog.conf.d/
This matches the syslog.conf(5) functionality, and also prevents '.sample' or
'.pkgnew' files being included. This is important for ports which install files
in /usr/local/etc/newsyslog.conf.d/ and also for pkgbase.
Approved by: eadler
Approved by: bapt
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17086
Prevent some classes of foot-shooting that may result in permissions
problems.
Reviewed by: dab, delphij, vangyzen (earlier version)
Relnotes: yes (behavior change)
Sponsored by: Dell EMC Isilon
Differential Revision: D16831
This is related to pkgbase and changes these to use CONFS so that these are
tagged as config files.
Approved by: AllanJude (mentor)
Sponsored by: Essen Hackathon
Differential Revision: https://reviews.freebsd.org/D16694
- Remove the compression suffix macros and move them directly into the
compress_type array.
- Remove the hardcoded sizes on the suffix and compression args arrays.
- Simplify the compression args arrays at the expense of a __DECONST
when calling execv().
- Rewrite do_zipwork. The COMPRESS_* macros can directly index the
compress_types array, so the outer loop is not needed. Convert
fixed-length strings into asprintf or sbuf calls.
Submitted by: Dan Nelson <dnelson_1901@yahoo.com>
Reviewed by: gad
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16518
The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities. However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.
Reuse of the index variable in two nested loops resulted in only the first
argument in the list being used (fine for gzip, not fine for zstd). Also
add tests for xz and zstd, and fix the COMPRESS_SUFFIX_MAXLEN macro.
Submitted by: dnelson_1901_yahoo.com
Differential Revision: https://reviews.freebsd.org/D16509
The RFC 5424 spec mentions that logging FQDNs over short hostnames is
preferred. Alter this code, so that the hostname doesn't get truncated
on startup. Keep track of the length of the short hostname, so that
fprintf() can do the truncation where necessary.
MFC after: 1 month
Move the 'X' and 'Y' entries into their sorted location in the list
of flags just above 'Z'.
Reviewed by: bcr
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13904
Implement the 'p' flag for newsyslog from NetBSD. This flag results in
the first log file for a given file to not be compressed.
While here, don't change file attributes during a no-op run
PR: 162798
Submitted by: heas@shrubbery.net
MFC After: 1 month
When building the command to execute for compression, newsyslog was modifying
the generic arguments array instead of its own copy.
Meaning on the second file to compress with the same arguments, the command line
was not the one expected.
Fix it by creating one copy of the arguments per execution and modifying that
copy.
While here, print the command line executed in verbose mode.
Reported by: many
have a semantic different than the traditional gzip(1)
This is done to allow to use zstd(1) as a compression tool without
having to patch it to change its default behavior.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
directories to SUBDIR.${MK_TESTS} idiom
This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .
No functional change intended.
MFC after: 1 weeks
This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.
Reviewed by: vangyzen, ngie
Approved by: vangyzen (mentor)
MFC after: 2 months
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10253
This makes newsyslog use zstandard to compress log files.
Given Z is already taken for gzip and zstandard compression level stands in
between gzip and xz (which has the X flag) chosing Y sounds ok :)
It turns out that we had a couple of more calls to dirname()/basename()
in newsyslog(8) that assume the input isn't clobbered. This is bad,
because it apparently breaks log rotation now that the new dirname()
implementation has been merged.
Fix this by first copying the input and then calling
dirname()/basename(). While there, improve the naming of variables in
this function a bit.
Reported by: Ryan Steinmetz, gjb
Reviewed by: bdrewery, allanjude
Differential Revision: https://reviews.freebsd.org/D7838
to newsyslog, etc made in the past month.
The issue is being root-caused as part of the bug noted below. This commit
will need to be partially reverted once the issue has been found/fixed
PR: 212160
Reported by: Jenkins
Sponsored by: EMC / Isilon Storage Division
Pull copies of the input pathname string before calling basename() and
dirname() to make this comply to POSIX. Free these copies at the end of
this function. While there, remove the duplication of the 's' ->
'logfname' string. There is no need for this.
After going through the signal work list, during which do_sigwork()
is called and essentially does nothing because -s and -R were
specified on the command line, newsyslog will sleep for 10 seconds
as the (verbose) code says: "Pause 10 seconds to allow daemon(s)
to close log file(s)".
However, the man page verbiage for -R (and -s) seems quite clear
that this sleep() is unnecessary because the daemon was expected
to have already closed the log file before calling newsyslog.
PR: 210020
Submitted by: David A. Bright <david_a_bright@dell.com>
MFC after: 1 week
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6727
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
