Dear all,
the FreeBSD project has switched their source of truth repository from
Subversion to Git on 2020-12-23. The previously published commit hashes
were missing required merge commits and were thus unsuitable to be the
basis of the source of truth repository.
They had to be changed (aka "force pushed"). We're sorry for the
inconvenience this causes.
The `master` branch will no longer see any updates and we will push to
`main` going forward. The `stable/X` and `releng/X` branches will see a
force-push directly.
To transplant your work, it is recommended to head over to the wiki at
https://github.com/freebsd/git_conv/wiki for instructions on how to
rebase or remerge your in-flight work. A short outline will be provided
below.
Please note that the actual git "trees" are of course identical between
the old and new conversions, so it's relatively easy to craft git
commits that merge or rebase the work w/o any merge conflicts. Please
reach out to git@FreeBSD.org or file an issue under
https://github.com/freebsd/git_conv/issues if you need assistance.
The mapping of the old to new commit hashes (for the same tree) are
given below. We have archived a copy of the legacy repo under
https://github.com/freebsd/freebsd-legacy which you can add as a remote
to always have a reference to the old `master` or `stable/X` branches
and names.
This is a merge commit that brings both histories together, giving you
common history ancestors, which should help with later merging.
You should be able to cleanly merge into this "legacy" master, and then
merge into "main" following from that. All the following commands assume
you've checked out your own workbranch.
-- Merge from master to main --
$ git remote add freebsd-legacy https://github.com/freebsd/freebsd-legacy.git
$ git fetch --all
$ git merge freebsd-legacy/master (this convenience commit)
$ git merge origin/main
-- Merge for stable/X or releng/Y branches --
$ git remote add freebsd-legacy https://github.com/freebsd/freebsd-legacy.git
$ git fetch --all
$ git merge freebsd-legacy/stable/X (to get to the legacy hashes from below)
$ git merge -s ours --allow-unrelated-histories <new-hash-from-below> (this is guaranteed conflict free)
$ git merge origin/stable/X
PLEASE NOTE: You'll end up with twice the history and git log output
will show old history twice and will likely confuse you. Please make an
effort to migrate your work over to a fresh branch based off of main.
A git replace --graft can be used to at least patch up git log output.
-- Rebase your work --
Only for folks that always rebase their local work on top of an origin
branch.
$ git remote add freebsd-legacy https://github.com/freebsd/freebsd-legacy.git
$ git fetch --all
$ git branch -u freebsd-legacy/master
$ git rebase --onto origin/main <your_rebased_work_branch>
-- Old to new hash mappings --
BRANCH LEGACY HASH NEW HASH
master / main 9e4440ca5b3cc0c0d66a
stable/10 10f4a6c225ee83ceae8013937d720abc6d55a42b 3ad46d0783
stable/11 a76e2a369a53c030aca4a52b694102e99aadbb2f 46062cb1a2
stable/12 f262e04c92d7b7a409e91c21a09ea09d54619099 f4d0bc6aa6
releng/10.0 fcf03644270b915aec5f010cd84202ad00ce746d 0099f6f89a
releng/10.1 a5b8a0cee842e12aa090449e042788b9eabc35da a9b4be2046
releng/10.2 14e67f3152c63dff2a97a5fcd9b3fa03a6625f31 2a34883424
releng/10.3 57fbb64699be1ef57f413a39f3d6920bb192ee7f 8411c9ac24
releng/10.4 976886d4ea7217ba54d00b10afc0c0a3e59cc552 d869f4ade2
releng/11.0 bc72ed658ec8c6cef0ceb206e8e29ed67a58af03 8d2708de5d
releng/11.1 d3b778b791f4e4d8afc69698b2097efdf2f79de7 3e9337c6b2
releng/11.2 24e863807ab509af9c38e84b4e5df34ff4fdf9e2 c0d2bc3b03
releng/11.3 b43bbc16b05f1b7e0770ac18bc2be0b3805a994e 9e63140d5a
releng/11.4 9332e027885d3d785a45dfcf3f7bd2f7c8181d0a 4adbf1f668
releng/12.0 58cdccde6ac7b22cc1ca42fbca6460a58304077f 12b769ed36
releng/12.1 c522014dbaadf4b1d7431093f91c52cbf1d3ca62 e30782bbda
releng/12.2 30ef47ef8fb0026951af8501fe30d33f3b4a7090 2120d07af0
--
Again, we're sorry for the inconvenience, please reach out via GitHub
issues and the mailing list to get further help in transplanting your
work onto the one true, canonical, source of truth.
Good luck in 2021!
Ulrich Spörlein, on behalf of the FreeBSD Git working group.
In order to efficiently serve web traffic on a NUMA
machine, one must avoid as many NUMA domain crossings as
possible. With SO_REUSEPORT_LB, a number of workers can share a
listen socket. However, even if a worker sets affinity to a core
or set of cores on a NUMA domain, it will receive connections
associated with all NUMA domains in the system. This will lead to
cross-domain traffic when the server writes to the socket or
calls sendfile(), and memory is allocated on the server's local
NUMA node, but transmitted on the NUMA node associated with the
TCP connection. Similarly, when the server reads from the socket,
he will likely be reading memory allocated on the NUMA domain
associated with the TCP connection.
This change provides a new socket ioctl, TCP_REUSPORT_LB_NUMA. A
server can now tell the kernel to filter traffic so that only
incoming connections associated with the desired NUMA domain are
given to the server. (Of course, in the case where there are no
servers sharing the listen socket on some domain, then as a
fallback, traffic will be hashed as normal to all servers sharing
the listen socket regardless of domain). This allows a server to
deal only with traffic that is local to its NUMA domain, and
avoids cross-domain traffic in most cases.
This patch, and a corresponding small patch to nginx to use
TCP_REUSPORT_LB_NUMA allows us to serve 190Gb/s of kTLS encrypted
https media content from dual-socket Xeons with only 13% (as
measured by pcm.x) cross domain traffic on the memory controller.
Reviewed by: jhb, bz (earlier version), bcr (man page)
Tested by: gonzo
Sponsored by: Netfix
Differential Revision: https://reviews.freebsd.org/D21636
In order to efficiently serve web traffic on a NUMA
machine, one must avoid as many NUMA domain crossings as
possible. With SO_REUSEPORT_LB, a number of workers can share a
listen socket. However, even if a worker sets affinity to a core
or set of cores on a NUMA domain, it will receive connections
associated with all NUMA domains in the system. This will lead to
cross-domain traffic when the server writes to the socket or
calls sendfile(), and memory is allocated on the server's local
NUMA node, but transmitted on the NUMA node associated with the
TCP connection. Similarly, when the server reads from the socket,
he will likely be reading memory allocated on the NUMA domain
associated with the TCP connection.
This change provides a new socket ioctl, TCP_REUSPORT_LB_NUMA. A
server can now tell the kernel to filter traffic so that only
incoming connections associated with the desired NUMA domain are
given to the server. (Of course, in the case where there are no
servers sharing the listen socket on some domain, then as a
fallback, traffic will be hashed as normal to all servers sharing
the listen socket regardless of domain). This allows a server to
deal only with traffic that is local to its NUMA domain, and
avoids cross-domain traffic in most cases.
This patch, and a corresponding small patch to nginx to use
TCP_REUSPORT_LB_NUMA allows us to serve 190Gb/s of kTLS encrypted
https media content from dual-socket Xeons with only 13% (as
measured by pcm.x) cross domain traffic on the memory controller.
Reviewed by: jhb, bz (earlier version), bcr (man page)
Tested by: gonzo
Sponsored by: Netfix
Differential Revision: https://reviews.freebsd.org/D21636
When ktls_bind_thread is 2, we pick a ktls worker thread that is
bound to the same domain as the TCP connection associated with
the socket. We use roughly the same code as netinet/tcp_hpts.c to
do this. This allows crypto to run on the same domain as the TCP
connection is associated with. Assuming TCP_REUSPORT_LB_NUMA
(D21636) is in place & in use, this ensures that the crypto source
and destination buffers are local to the same NUMA domain as we're
running crypto on.
This change (when TCP_REUSPORT_LB_NUMA, D21636, is used) reduces
cross-domain traffic from over 37% down to about 13% as measured
by pcm.x on a dual-socket Xeon using nginx and a Netflix workload.
Reviewed by: jhb
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21648
When ktls_bind_thread is 2, we pick a ktls worker thread that is
bound to the same domain as the TCP connection associated with
the socket. We use roughly the same code as netinet/tcp_hpts.c to
do this. This allows crypto to run on the same domain as the TCP
connection is associated with. Assuming TCP_REUSPORT_LB_NUMA
(D21636) is in place & in use, this ensures that the crypto source
and destination buffers are local to the same NUMA domain as we're
running crypto on.
This change (when TCP_REUSPORT_LB_NUMA, D21636, is used) reduces
cross-domain traffic from over 37% down to about 13% as measured
by pcm.x on a dual-socket Xeon using nginx and a Netflix workload.
Reviewed by: jhb
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21648
- varios "new sentence, new line" warnings
- varios "sections out of conventional order" warnings
- varios "unusual Xr order" warnings
- varios "missing section argument" warnings
- varios "no blank before trailing delimiter" warnings
- varios "normalizing date format" warnings
MFC after: 1 month
- varios "new sentence, new line" warnings
- varios "sections out of conventional order" warnings
- varios "unusual Xr order" warnings
- varios "missing section argument" warnings
- varios "no blank before trailing delimiter" warnings
- varios "normalizing date format" warnings
MFC after: 1 month
- Add missing quotation mark for a comment above the .Dd
- inserting missing end of block: Sh breaks Bd
- skipping paragraph macro: Pp before Bl
- skipping paragraph macro: Pp before Bd
- empty block: Bd
MFC after: 1 week
- Add missing quotation mark for a comment above the .Dd
- inserting missing end of block: Sh breaks Bd
- skipping paragraph macro: Pp before Bl
- skipping paragraph macro: Pp before Bd
- empty block: Bd
MFC after: 1 week
- new sentence, new line
- sections out of conventional order: Sh FILES
- unusual Xr order: bthost(1) after bthidd(8)
- no blank before trailing delimiter
- whitespace at end of input line
- sections out of conventional order: Sh EXIT STATUS
MFC after: 1 week
- new sentence, new line
- sections out of conventional order: Sh FILES
- unusual Xr order: bthost(1) after bthidd(8)
- no blank before trailing delimiter
- whitespace at end of input line
- sections out of conventional order: Sh EXIT STATUS
MFC after: 1 week
- no blank before trailing delimiter
- missing section argument: Xr inet_pton
- skipping paragraph macro: Pp before Ss
- unusual Xr order: syslogd after sysrc
- tab in filled text
There were a few multiline NAT examples which used the .Dl macro with
tabs. I converted them to .Bd, which is a more suitable macro for that case.
MFC after: 1 week
- no blank before trailing delimiter
- missing section argument: Xr inet_pton
- skipping paragraph macro: Pp before Ss
- unusual Xr order: syslogd after sysrc
- tab in filled text
There were a few multiline NAT examples which used the .Dl macro with
tabs. I converted them to .Bd, which is a more suitable macro for that case.
MFC after: 1 week
- inserting missing end of block: Ss breaks Bl
- skipping paragraph macro: Pp before Ss
- referenced manual not found: Xr nvme 4 (2 times)
- unknown standard specifier: St The
The macro .St can only be used for standards known by mdoc(7). So add a
SEE ALSO section and add a reference to the NVM Express Base Specification.
MFC after: 2 weeks
- inserting missing end of block: Ss breaks Bl
- skipping paragraph macro: Pp before Ss
- referenced manual not found: Xr nvme 4 (2 times)
- unknown standard specifier: St The
The macro .St can only be used for standards known by mdoc(7). So add a
SEE ALSO section and add a reference to the NVM Express Base Specification.
MFC after: 2 weeks
