Commit Graph

397 Commits

Author SHA1 Message Date
Dag-Erling Smørgrav
08a49957b3 r308996 broke IP literals by assuming that a colon could only occur as
a separator between host and port, and using strchr() to search for it.
Rewrite fetch_resolve() so it handles bracketed literals correctly, and
remove similar code elsewhere to avoid passing unbracketed literals to
fetch_resolve().  Remove #ifdef INET6 so we still parse IP literals
correctly even if we do not have the ability to connect to them.

While there, fix an off-by-one error which caused HTTP 400 errors to be
misinterpreted as redirects.

PR:		217723
MFC after:	1 week
Reported by:	bapt, bz, cem, ngie
2017-03-17 14:18:52 +00:00
Dag-Erling Smørgrav
64c422355d Add a __printflike() that would have caught the bug fixed in r314396. 2017-03-12 11:54:35 +00:00
Pedro F. Giffuni
0012b66baf libfetch: extra bounds checking through reallocarray(3).
Reviewed by:	des
MFC after:	1 week
2017-03-06 15:38:03 +00:00
Dag-Erling Smørgrav
c8453e5bf4 Fix partial requests (used by fetch -r) when the requested file is
already complete.

Since 416 is an error code, any Content-Range header in the response
would refer to the error message, not the requested document, so
relying on the value of size when we know we got a 416 is wrong.
Instead, just verify that offset == 0 and assume that we've reached
the end of the document (if offset > 0, we did not request a range,
and the server is screwing with us).  Note that we cannot distinguish
between reaching the end and going past it, but that is a flaw in the
protocol, not in the code, so we just have to assume that the caller
knows what it's doing.  A smart caller would request an offset
slightly before what it believes is the end and compare the result to
what is already in the file.

PR:		212065
Reported by:	mandree
MFC after:	3 weeks
2017-03-05 12:06:45 +00:00
Baptiste Daroussin
5d1ce10bee Properly initialize netrcfd in fetchParseURL
This fixes ftp with fetch(1) which was broken after r313974

Submitted by:	dim
Reported by:	olivier
Pointyhat to:	bapt
2017-03-03 12:51:16 +00:00
Bjoern A. Zeeb
6d91604093 Properly indent a default: label and avoid crashing when running
under -v but cannot connect due to trying to print an int as %s [1].

Reported by:	andrew [1]
MFC after:	3 days
2017-02-28 18:10:03 +00:00
Baptiste Daroussin
d8713bf361 Add a file descriptor in struct url for netrc
When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
${HOME}/.netrc. Now a caller can prepare a fd to the said file and manually
assign it to the structure.

It is also a first step to allow to capsicumize libfetch applications

Reviewed by:	allanjude, des
Approved by:	des
Differential Revision:	https://reviews.freebsd.org/D9678
2017-02-20 00:14:31 +00:00
Dag-Erling Smørgrav
21ca0912c6 Fix inverted loop condition which broke multi-line responses to CONNECT.
PR:		194483
Submitted by:	Miłosz Kaniewski <milosz.kaniewski@gmail.com>
MFC after:	1 week
2016-12-30 14:54:54 +00:00
Dag-Erling Smørgrav
3af3efd115 More debugging code I missed in r309051.
Reported by:	jbeich, jkim
2016-12-09 16:14:55 +00:00
Dag-Erling Smørgrav
1da400983c Remove debugging code. 2016-11-23 10:52:19 +00:00
Dag-Erling Smørgrav
792ef1ae7b Refactor fetch_connect() and fetch_bind() to improve readability and avoid
repeating the same DNS lookups.

MFC after:	3 weeks
2016-11-22 13:30:07 +00:00
Dag-Erling Smørgrav
a5fc9a29bb r169386 (PR 112515) was incomplete: it treated 307 as an error except
in verbose mode, and did not handle 308 at all.

r241840 (PR 172451) added support for 308, but with the same bug.

Correctly handle both by recognizing them as redirects in all places
where we check the HTTP result code.

PR:		112515 173451 209546
Submitted by:	novel@
MFC after:	1 week
2016-05-31 08:27:39 +00:00
Don Lewis
65380b5f2f Call closedir() before returning from fetchListFile() to avoid a leak.
Reported by:	Coverity
CID:		1016697
MFC after:	1 week
2016-05-25 07:43:32 +00:00
Don Lewis
429bf952ae Don't leak addrinfo in fetch_bind()
Reported by:	Coverity
CID:		1225038
MFC after:	1 week
2016-05-25 07:39:48 +00:00
Don Lewis
77b822dbc0 Use strlcpy() instead of strncpy() to copy the string returned by
setlocale() so that static analyzers know that the string is NUL
terminated.  This was causing a false positive in Coverity even
though the longest string returned by setlocale() is ENCODING_LEN
(31) and we are copying into a 64 byte buffer.  This change is also
a bit of an optimization since we don't need the strncpy() feature
of padding the rest of the destination buffer with NUL characters.

Reported by:	Coverity
CID:		974654
2016-05-12 06:39:13 +00:00
Pedro F. Giffuni
75f46cf6c8 lib: minor spelling fixes in comments.
No functional change.
2016-05-01 19:37:33 +00:00
Glen Barber
876d357fa7 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-11 15:24:59 +00:00
Pedro F. Giffuni
3cd7f4295f libfetch: replace 0 with NULL for pointers.
Found with devel/coccinelle.

Reviewed by:	des
2016-04-09 18:08:21 +00:00
Glen Barber
d60840138f MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
Edward Tomasz Napierala
225636dccb Fix bunch of .Xrs.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-03-28 16:48:28 +00:00
Michael Gmelin
1ec923fd6d Update fetch.1 and fetch.3 to reflect libfetch's actual use of CA bundles
Reviewed by:	wblock
Approved by:	wblock
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D5558
2016-03-19 11:55:21 +00:00
Glen Barber
52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
Bryan Drewery
bd18fd57db DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
Glen Barber
0fe0fe112f MFH
Sponsored by:	The FreeBSD Foundation
2016-02-15 21:58:52 +00:00
Dag-Erling Smørgrav
a982c4c7f5 Fix double-free error: r289419 moved all error handling in http_connect()
to the end of the function, but did not remove a fetch_close() call which
was made redundant by the one in the shared error-handling code.

PR:		206774
Submitted by:	Christian Heckendorf <heckendorfc@gmail.com>
MFC after:	3 days
2016-02-11 17:48:15 +00:00
Glen Barber
a70cba9582 First pass through library packaging.
Sponsored by:	The FreeBSD Foundation
2016-02-04 21:16:35 +00:00
Baptiste Daroussin
f16550ad09 Test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH
Prior to this patch, unless SSL_CA_CERT_FILE is set in the environment,
libfetch will set the CA file to "/usr/local/etc/cert.pem" if it exists,
and to "/etc/ssl/cert.pem" otherwise. This has the consequence of
masking SSL_CA_CERT_PATH, because OpenSSL will ignore the CA path if a CA
file is set but fails to load (see X509_STORE_load_locations()).

While here, fall back to OpenSSL defaults if neither SSL_CA_CERT_FILE nor
SSL_CA_CERT_PATH are set in the environment, and if neither of the
libfetch default CA files exists.

PR:		193871
Submitted by:	John W. O'Brien <john@saltant.com>
Approved by:	des
MFC after:	1 week
2016-01-19 15:02:37 +00:00
Dag-Erling Smørgrav
adc1aa7a29 As a followup to r292330, standardize on size_t and add a few comments. 2015-12-16 09:20:45 +00:00
Dag-Erling Smørgrav
a568844c67 Reset bufpos to 0 immediately after refilling the buffer. Otherwise, we
risk leaving the connection in an indeterminate state if the server fails
to send a chunk delimiter.  Depending on the application and on the sizes
of the preceding chunks, the result can be anything from missing data to a
segfault.  With this patch, it will be reported as a protocol error.

PR:		204771
MFC after:	1 week
2015-12-16 09:17:07 +00:00
Dimitry Andric
a1b9b1743c Fix buildworld after r291453, similar to r284346: url->user and url->pwd
are arrays, so they can never be NULL.

Reported by:	many
Pointy hat to:	des
2015-11-29 22:37:48 +00:00
Dag-Erling Smørgrav
4d8b056ef1 Use .netrc for HTTP sites and proxies, not just FTP.
PR:		193740
Submitted by:	TEUBEL György <tgyurci@gmail.com>
MFC after:	1 week
2015-11-29 14:26:59 +00:00
Bryan Drewery
7b3ea376a2 META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
Dag-Erling Smørgrav
8819003234 Use fopen()'s newfangled "e" flag instead of explicit fcntl() calls.
PR:		199801
Submitted by:	Jukka Ukkonen <jau@iki.fi>
MFC after:	1 week
2015-10-16 12:53:22 +00:00
Dag-Erling Smørgrav
c3f9b93bd9 Fix two bugs in HTTPS tunnelling:
- If the proxy returns a non-200 result, set the error code accordingly
   so the caller / user gets a somewhat meaningful error message.
 - Consume and discard any HTTP response header following the result line.

PR:		194483
Tested by:	Fabian Keil <fk@fabiankeil.de>
MFC after:	1 week
2015-10-16 12:21:44 +00:00
Michael Gmelin
034fb271ae Fix non-POSIX-compliant use of getaddrinfo in libfetch
Submitted by:	Boris Kolpackov <boris@codesynthesis.com>
Reviewed by:	bapt
Approved by:	bapt
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D3724
2015-09-25 14:24:23 +00:00
Marcelo Araujo
ddcc2ecb3a Remove unused variable to silence clang warning.
Differential Revision:	D2683
Reviewed by:		rodrigc, bapt
2015-07-04 17:22:07 +00:00
Simon J. Gerraty
2ef6d5a7b9 new depends 2015-06-16 23:37:19 +00:00
Dimitry Andric
f2c41c554d Fix the following clang 3.7.0 warnings in lib/libfetch/http.c:
lib/libfetch/http.c:1628:26: error: address of array 'purl->user'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.user = purl->user ?
                                                   ~~~~~~^~~~ ~
    lib/libfetch/http.c:1630:30: error: address of array 'purl->pwd'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.password = purl->pwd?
                                                       ~~~~~~^~~~
    lib/libfetch/http.c:1657:25: error: address of array 'url->user'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.user = url->user ?
                                                   ~~~~~^~~~ ~
    lib/libfetch/http.c:1659:29: error: address of array 'url->pwd'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.password = url->pwd ?
                                                       ~~~~~^~~ ~
    lib/libfetch/http.c:1669:25: error: address of array 'url->user'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.user = url->user ?
                                                   ~~~~~^~~~ ~
    lib/libfetch/http.c:1671:29: error: address of array 'url->pwd'
    will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
                                    aparams.password = url->pwd ?
                                                       ~~~~~^~~ ~

Since url->user and url->pwd are arrays, they can never be NULL, so the
checks can be removed.

Reviewed by:	bapt
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D2673
2015-06-13 19:26:48 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Jung-uk Kim
01ed3ca3de Remove defunct SSLv2 support from fetch(1) and fetch(3). 2015-03-25 18:56:36 +00:00
Baptiste Daroussin
6b129086dc Convert libraries to use LIBADD
While here reduce a bit overlinking
2014-11-25 11:07:26 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Dag-Erling Smørgrav
767207650c As pointed out by several people, r273114 was incorrect: it unconditionally
disabled everything except TLS 1.0.  Replace it with a more carefully
wrought patch:

 - Switch the default for SSLv3 from on to off
 - Add environment variables to control TLS 1.1 and 1.2
 - In verbose mode, report which version is used
 - Update the man page to reflect these changes.

MFC after:	1 week
2014-10-15 07:35:50 +00:00
Dag-Erling Smørgrav
d75ca0c1fd Drop support for SSLv3.
MFC after:	1 week
2014-10-15 02:34:26 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Baptiste Daroussin
3674911c29 Add a comment to explain the EAGAIN is only there for POSIX compliance
Resquested by:	kib
Reviewed by:	des
2014-07-15 15:29:43 +00:00
Baptiste Daroussin
4472d6e1df Support EAGAIN in fetch_writev
Reviewed by:	des
Approved by:	des
2014-07-10 13:04:52 +00:00
Baptiste Daroussin
6cfac37b03 Use Mt macro to properly format mailto links (patch from Franco Fichtner <franco@lastsummer.de> for Dragonfly)
Reviewed by:	des
Approved by:	des
Obtained from:	Dragonfly
MFC after:	1 week
2014-06-11 20:20:14 +00:00