There are a couple of places in the tree that directly parse the newvers.sh
script looking for the BRANCH variable. I found two locations, one in
release/Makefile and the other in bin/freebsd-version/Makefile.
While there is a good argument that BRANCH_OVERRIDE should properly
propagate in those circumstances and the new behavior is thus better, the
reality is this change broke freebsd-update's ability to find timestamps in
binaries and resulted in a large number of gratuitous changes.
Reported by: freebsd-update
Discussed with: cperciva
MFC after: 1 day
freebsd-update(8) builds, where BRANCH is suffixed with -p0 for
builds.
Noticed by: gordon
With help from: cperciva
MFC after: 3 days
MFC note: before 12.2-BETA2
Sponsored by: Rubicon Communications, LLC (netgate.com)
It appears that the macOS /bin/sh echo now defaults to -e and therefore the
`#define VERSTR` included newline characters instead of \n. This caused compiler
errors due to unterminated strings. Fix by using printf instead of echo.
A less fragile solution might be to bootstrap the in-tree /bin/sh but that
requires more changes.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D24136
In a git world this provides a facsimile of a monotonically increasing
version number. This might be refined further, but this provides a
starting point for investigation.
Reviewed by: cem
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20462
With the reorg r348175, we now look at modified before it is
set. Rearrange things so that we can set include_metadata to either
yes, no or if-modified. This should fix the -R flag that was broken in
r348175, which broke WITH_REPRODUCIBLE_BUILD for kernels.
Feedback From: emaste@
Differential Revision: https://reviews.freebsd.org/D20480
Add -v to print TYPE REVISION BRANCH RELEASE VERSION RELDATE variables
Add -V var to print var's value
Both of these in ${var}="${val}" format suitable for
eval $(sh newvers.sh -v)
in shell scripts / makefiles.
Add -c to print the copyright / license comment text only.
Document these, and remove soon-to-be obsolete comment.
Minor code motion as well bunded here to put functions after
VARS_ONLY and command line argument parsing.
Differential Revision: https://reviews.freebsd.org/D19849
$() is more modern and also nests. Convert the mix of styles to using
only the former (although the latter was more common). It's the more
dominant style in other shell scripts these days as well.
Differential Revision: https://reviews.freebsd.org/D19840
All variable assignments that start in column 1 have to be on a single
line for amd to build due to as weird dependency there (most likely it
can be fixed to use the new VARS_ONLY feature, but it isn't
today). usr.sbin/amd/include/Makefile calls
usr.sbin/amd/include/newvers.sh which does:
eval `LC_ALL=C egrep '^[A-Z]+=' $1 | grep -v COPYRIGHT`
which is where that requirement comes from. It handles COPYRIGHT since
that's an exception. Rather than add additional exceptions, cope with
the long line in newvers.sh instead. Note: it no longer needs to
filter COPYRIGHT because the assignment doesn't start in column 1
anymore.
I had done a universe when I had an earlier version of r346018 that
had it as one line. When I changed it to multi-line as suggested in
the review, I only built kernels on a couple of architectures to make
sure it didn't break anything.
Add comment to newvers.sh noting this.
Obviously, this unbreaks the amd build.
Remove the phrase from boilerplate copyright we stick on vers.c when
we can't find the template file. In practice, this won't change a
thing, except for the case of compiling the kernel standalone w/o the
rest of a tree on a system that doesn't have
/usr/share/examples/etc/bsd-copyright installed.
newvers.sh takes upwards of 4-5 seconds to complete on trees checked
out from github, due to searching the entire history for non-existent
git-svn metadata. Similarly, if one does not check out notes, we
again search the entire history for notes. That makes newvers.sh very
slow for many github users.
To fix this in a fair way, limit the history search to the last 10K
commits: if you're more than 10K commits out of sync, then you've
forked the project, and our SVN rev is no longer very important to you.
Due to how git implements --grep in conjunction with -n, --grep has been
removed for performance reasons (git does not seem to limit its search
to the -n limit in this case, and takes just as long as it did with no
limit).
Reviewed by: emaste, imp
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18745
Perforce no longer offers a FreeBSD client and it not a viable VCS for
FreeBSD development. Remove p4 version logic to simplify newvers.sh in
advance of other changes.
Sponsored by: The FreeBSD Foundation
Consider the case where FreeBSD is checked out via Subversion with a
(perhaps unrelated) .git or .hg directory at a higher level - for
example,
.../.git
.../src/freebsd
Previously newvers obtained the SVN revision information via svnversion,
and then tried to obtain the SVN revision corresponding to the git or hg
commit, overwriting the existing information.
As a short term fix use a different variable for hg-svn or git-svn
information, setting $svn from hg or git info only if not empty.
Reported by: Matthias Apitz
Sponsored by: The FreeBSD Foundation
newvers.sh runs mkfifo which did not exist before this change.
However, I didn't notice before because it is run from a function
where a missing command does cause a noticeable failure.
Reviewed By: emaste, markj
Differential Revision: https://reviews.freebsd.org/D18377
When reproducible build mode is enabled vers.c may be unchanged between
successive builds. In this case avoid changing the file's metadata so
that it does not cause dependent targets to be rebuilt.
Sponsored by: The FreeBSD Foundation
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D17892
Assuming that any output from `git diff-index --name-only` implies
changes in the working tree results in false positives: files with
metadata, but not content, changes are also listed.
Check that content differences exist before adding the -dirty tag to
the git hash.
PR: 229230
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D15968
cycle.
This is expected to be the final ALPHA build of this release
cycle, prior to branching stable/12.
Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation
Building the kernel in Git repositories when git-svn is not available and
the "help.autocorrect" Git parameter is enabled results in Git trying to
replace the "svn" command (it does not know) with "serve". As a result the
output of the "git server" command is appended to the value of the
environmental variable VERINFO, which causes the auto generated vers.c
file to contain invalid C syntax (missing newline escapes):
#define "@(#)FreeBSD 12.0-ALPHA7 r000eversion 2
0015agent=git/2.19.0
000cls-refs
0012fetch=shallow
0012server-option
0000=5e2272613fa(splash-vt)"
#define VERSTR "FreeBSD 12.0-ALPHA7 r000eversion 2
0015agent=git/2.19.0
000cls-refs
0012fetch=shallow
0012server-option
0000=5e2272613fa(splash-vt)\n"
Using `-c help.autocorrect=0` seems to be a good solution as it does not
modify user's environment. I am not sure, however, if we should use
programs (or Git commands), which we are not sure exist (we never check if
git-svn is available on the host), as there may be more unexpected
behaviors like this one.
Reviewed by: eadler, emaste, krion
Approved by: re (gjb), krion (mentor)
Sponsored by: Bally Wulff Games & Entertainment GmbH
Differential Revision: https://reviews.freebsd.org/D17271
In non-reproducible mode we have the kernel ident as a side effect of
including the build directory. Explicitly add it to the ident string in
reproducible mode.
Reported by: mjg
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation
cycle. The i386 build failure appears to be transient, and
now becoming more difficult to reliably reproduce to identify
the cause. I will continue to investigate this, however.
Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation
Previously newvers.sh passed --work-tree=${VCSDIR}/.. when invoking git.
When using git worktree .git is actually a file, not a directory, and
.git/.. is not a valid path. Although it appears git handles this
internally (perhaps it normalizes the path first), it is simple enough
for the script to store both the working tree top-level directory and
the VCS (.git) directory, so do so.
Sponsored by: The FreeBSD Foundation
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
In the case of running newvers.sh on a git tree w/o git-svn-id notes we
previously piped the entire 'git log' to grep. Add --grep to the log
invocation to avoid processing log entries of no interest.
This saves about 2-3 seconds of newvers.sh run time on my SSD laptop.
Later changes will bring further speedups.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
This prevents incorrect subversion revision detection when "git svn" is
not being used to get the sources but git is available. Previously old
subversion revisions included in commit messages were favoured over the
more recent and correct revisions in git notes.
For example cf1f355747 represents r315395 but was treated as r313908
which is referenced in the commit message. Commits following
r315395/cf1f35574722 but before another commit with a git-svn-id
reference in the commit message would be treated as r313908 as well.
Patch from PR updated to accommodate the initial four space indent in
`git log` ouptut.
PR: 221848
Submitted by: Fabian Keil
Obtained from: ElectroBSD
MFC after: 2 weeks
newvers.sh looks for a .vcs subdirectory (e.g. .git, .svn) to determine
which vcs info tool to run (e.g., git rev-parse, svn info).
(As of r308789 if a .vcs subdirectory is not found at ${TOPDIR} then
newvers.sh walks up successive parent directories, testing for the .vcs
subdirectory at each step. This is done in case the FreeBSD source is
built in a subdirectory as part of some larger project, but either way
newvers.sh still tests for the .vcs subdirectory.)
However, when using git worktree there is no .git subdirectory but
rather a plain text .git file which contains a reference to the main
working tree.
Change findvcs() to test that the .vcs entry exists, regardless of type.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
The newvers -R option is intended to include build metadata (e.g. user,
host, time) if the build is from an unmodified VCS tree. For subversion
it considered a trailing 'M' as an indication of a modified tree, and
any other version string as modified.
Also include mixed revision checkouts (e.g. 123:126), switched (123S)
and partial (123P) working copies as modified: the revision number is
insufficient to uniquely determine which source was used for the build.
Reported by: gjb
Reviewed by: gjb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8853
Build metadata (username, hostname, etc.) prevents the FreeBSD kernel
from building reproducibly. Add an option to disable inclusion of that
metadata but retain the release information and SVN/git VCS details.
See https://reproducible-builds.org/ for additional background.
Reviewed by: bapt
Obtained from: NetBSD
MFC after: 1 month
Sponsored by: Reproducible Builds World Summit 2, Berlin
Differential Revision: https://reviews.freebsd.org/D4347
.hg may reside above FreeBSD sources root. Provide function findvcs()
that will climb up and seek for presence of a VCS directory.
Reviewed by: imp (earlier version of the patch)
