Commit Graph

110 Commits

Author SHA1 Message Date
Colin Percival
2ef5995f49 portsnap: Remove superfluous semicolons
Submitted by:	des
2021-06-29 11:00:54 -07:00
Colin Percival
a9f5512f07 portsnap: Shorten 'Skipping' output lines
Portsnap uses patches opportunistically to reduce download bandwidth: It
attempts to fetch patches which could be useful, and then makes use of
whichever patches it actually gets.  (This solves the otherwise O(n^2)
issue for the server to build patches between every pair of versions.)

During the process of applying patches, portsnap prints lines of the
form "Skipping XXX-YYY (123 of 4567).\r", where the \r serves to allow
each of these (potentially many) lines to overwrite the previous one
on the console.  Unfortunately, XXX and YYY here are SHA256 hashes,
resulting in these lines wrapping on reasonable-width consoles.

Replace the hashes with abbreviations of the form "0123...cdef"
(cutting 64 characters down to 11) in order to keep lines to a
reasonable length.

The rather ugly shell code here is used to avoid forking additional
processes; it would be much cleaner using sed(1), but in my testing
the sed-based alternative increases CPU time consumption by 50%.

Requested by:	des
2021-06-29 11:00:54 -07:00
Xin LI
07d218f70c Now that the portsnap buildbox is generating the raw bits for INDEX-14,
add it to the set of INDEX files built by portsnap.

Switch to INDEX-14 for main.
2021-01-28 23:26:14 -08:00
Kyle Evans
6703731d6e phttpget: move out of portsnap
Currently, WITHOUT_PORTSNAP forces WITHOUT_FREEBSD_UPDATE because the
latter relies on phttpget, which lives inside the portsnap build bits.

Remove the dependency between these two options by moving phttpget out into
^/libexec and building/installing it if either WITH_PORTSNAP or
WITH_FREEBSD_UPDATE.

Future work could remove the conditional if it's decided that users will use
it independently of either the current in-base consumers.

Reported by:	swills
Reviewed by:	jilles, emaste
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D26255
2020-09-09 00:39:47 +00:00
Glen Barber
0135283d1b Remove INDEX-10 reference, as 10.x is now EoL.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-04-16 14:07:14 +00:00
Xin LI
afa89716f9 Now that the portsnap buildbox is generating the raw bits for INDEX-13,
add it to the set of INDEX files built by portsnap.

Switch to INDEX-13 for head/.
2018-10-25 08:05:53 +00:00
Brad Davis
99a84b826f Move portsnap.conf to head/usr.sbin/portsnap/portsnap/
This is prep for pkgbase to have config files tagged as such.

Approved by:	will (mentor)
Differential Revision:	https://reviews.freebsd.org/D16554
2018-08-04 22:31:29 +00:00
Edward Tomasz Napierala
775c9076c7 Add basic examples to portsnap(8).
MFC after:	2 weeks
2018-07-01 16:18:38 +00:00
Xin LI
b6f7731dba Remove "All rights reserved" from my files.
See r333391 for the rationale.

MFC after:	1 week
2018-05-10 06:41:08 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Colin Percival
5b3b877f89 Avoid division by zero in the rare case that portsnap needs to fetch
zero patches.  (This avoids two "dc: divide by zero" warnings.)

MFC after:	3 days
2016-12-16 22:37:16 +00:00
Ed Maste
9a99954958 portsnap: use lam on the known good hash list
This change is equivalent to the approach committed in r306417, but if
sed has a bug it could be exploited by the untrusted tar file. Instead,
generate the expected tar content and compare that with find's output.

convert the expected hash list to the expected tar content filesystem
layout, and compare that with find's output.

Submitted by:	cperciva (in review D8052)
Reviewed by:	oshogbo
MFC after:	2 weeks
2016-10-07 20:01:59 +00:00
Ed Maste
54786ab35e portsnap: only move expected snapshot contents from snap/ to files/
Previously it was possible to smuggle in addional files that would
be used by later portsnap runs. Now we only move those files expected
to be in the snapshot into files/ and require that there are no
unexpected files.

This was used by portsnap attacks 2, 3, and 4 in the "non-cryptanalytic
attacks against FreeBSD update components" anonymous gist.

Reported by:	anonymous gist
Reviewed by:	allanjude, delphij
MFC after:	ASAP
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8052
2016-09-28 21:22:51 +00:00
Xin LI
337b148e02 Ensure that we always open only files that is named by explicitly
using shell redirections instead of having gzip(1) to decide what
file to open.

Issue reported in the "non-cryptanalytic attacks against freebsd
update components" anonymous gist.

Reviewed by:	allanjude, emaste
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D7653
2016-09-06 08:52:00 +00:00
Pedro F. Giffuni
80c7cc1c8f Cleanup unnecessary semicolons from utilities we all love. 2016-04-15 22:31:22 +00:00
Jason Helfman
7bd680e19d - update portsnap(8) to direct users to pkg-version
PR:		202971
Submitted by:	adamw@FreeBSD.org
Reviewed by:	bdrewery@
Approved by:	wblock@
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D5429
2016-03-01 22:13:20 +00:00
Edward Tomasz Napierala
4b69ad25d0 Fix typo.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-02-29 17:25:13 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Xin LI
a9d9abe89e Add a manual page for phttpget(8).
MFC after:	1 month
2015-01-02 21:02:41 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Baptiste Daroussin
01c2b8ac0d use .Mt to mark up email addresses consistently (part2)
PR:		191174
Submitted by:	Franco Fichtner  <franco@lastsummer.de>
2014-06-20 09:57:27 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
3b8f084595 Merge head 2014-04-28 07:50:45 +00:00
Xin LI
236b00df63 Use case insensitive match in portsnap.
PR:		bin/186510
Submitted by:	olli
MFC after:	2 weeks
2014-04-21 19:33:27 +00:00
Warner Losh
3bdf775801 NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
2014-04-13 05:21:56 +00:00
Colin Percival
6fcfd7ab4e When verifying that files we need are present, also check that they don't
have zero length.  Filesystem corruption will tend to truncate files, and
since these are short that's likely to result in them becoming empty.

Suggested by:	Richard Clayton
Convinced by:	rwatson
MFC after:	3 weeks
2013-11-14 09:22:32 +00:00
Gavin Atkinson
b53e60d39a Rename the "alfred" command to "auto", and document it. Retain support
for "portsnap alfred" for now.

Discussed:	many times, most recently on svn-src-all
MFC after:	1 week
2013-10-22 18:55:52 +00:00
Simon J. Gerraty
d1d0158641 Merge from head 2013-09-05 20:18:59 +00:00
Colin Percival
e08aa295d0 Fix bug in deleting files: If two ports had the same tarball and one of
them changed (or was removed from the tree) then portsnap would delete
that file.  This happened earlier today when one of two empty port
directories was removed.  Uniquifying the lists of needed files fixes
this.

9.2-RELEASE candidate.

MFC after:	3 days
2013-07-11 22:19:18 +00:00
Xin LI
410bcf7cc7 Make 'portsnap alfred' overwrite ports tree if it's not created by a
portsnap.

Discussed with:	alfred
Reviewed by:	cperciva
2013-06-07 20:21:30 +00:00
Joel Dahl
37b983c09b mdoc sweep 2013-05-13 18:13:50 +00:00
Joel Dahl
2710751bc3 mdoc sweep. 2013-05-12 22:22:12 +00:00
Simon J. Gerraty
7cf3a1c6b2 Updated dependencies 2013-03-11 17:21:52 +00:00
Simon J. Gerraty
f5f7c05209 Updated dependencies 2013-02-16 01:23:54 +00:00
David E. O'Brien
d9a447559b Sync with HEAD. 2013-02-08 16:10:16 +00:00
Alfred Perlstein
7b4708028a Change progress indicator for downloading patches.
Use a percent style indicator instead of emitting one
long line of patch numbers.
2012-12-13 01:39:49 +00:00
Simon J. Gerraty
23090366f7 Sync from head 2012-11-04 02:52:03 +00:00
Eitan Adler
ba8999e989 Avoid changing IFS in a global context as this can sometimes cause
errors later on.

PR:		bin/172715
Submitted by:	Matthew D.Fuller <fullermd@over-yonder.net> (older
version)
Submitted by:	dteske
Approved by:	cperciva
MFC after:	1 week
2012-10-24 12:49:03 +00:00
Joel Dahl
a01d461b70 Remove cvs/cvsup reference. 2012-10-14 10:12:32 +00:00
Isabell Long
6e95460966 Change "only only" to "only" in portsnap(8).
Spotted by:	Ruslan Mahmatkhanov
Approved by:	gabor (mentor, implicit)
2012-09-28 11:40:59 +00:00
Isabell Long
eacee75a76 Add a note to portsnap(8) about the behaviour of the example cron command.
PR:		docs/171759
Submitted by:	Paul Hoffman (phoffman at proper dot com)
Approved by:	cperciva, gabor (mentor)
MFC after:	3 days
2012-09-28 11:11:42 +00:00
Marcel Moolenaar
7750ad47a9 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
Eitan Adler
a6c38148a0 Add interactive option to override portsnap's automagical detection of
a terminal.

Submitted by:	Hannes h2+fbsdports@fsfe.org
Approved by:	cperciva
MFC after:	1 week
2012-08-05 21:13:21 +00:00
Eitan Adler
94fd46b06b make it easier for users used to a VCS that allows up to be a shortcut
for update.

PR:		bin/152856
Approved by:	cperciva
MFC after:	3 days
2012-05-09 21:56:10 +00:00
Alfred Perlstein
c13b66c053 Fix spelling of extract.
Pointed out by: gcooper
2011-11-13 06:39:49 +00:00
Alfred Perlstein
d1982439b2 Utilize shell's IFS instead of forking ~6 processes to
handle splitting input files on a '|'.  This greatly
reduces the time taken to process several databases
during the update process.

Additionally add some more debug logging.
2011-11-13 03:18:57 +00:00