- If the proxy returns a non-200 result, set the error code accordingly
so the caller / user gets a somewhat meaningful error message.
- Consume and discard any HTTP response header following the result line.
PR: 194483
Tested by: Fabian Keil <fk@fabiankeil.de>
MFC after: 1 week
lib/libfetch/http.c:1628:26: error: address of array 'purl->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = purl->user ?
~~~~~~^~~~ ~
lib/libfetch/http.c:1630:30: error: address of array 'purl->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = purl->pwd?
~~~~~~^~~~
lib/libfetch/http.c:1657:25: error: address of array 'url->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = url->user ?
~~~~~^~~~ ~
lib/libfetch/http.c:1659:29: error: address of array 'url->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = url->pwd ?
~~~~~^~~ ~
lib/libfetch/http.c:1669:25: error: address of array 'url->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = url->user ?
~~~~~^~~~ ~
lib/libfetch/http.c:1671:29: error: address of array 'url->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = url->pwd ?
~~~~~^~~ ~
Since url->user and url->pwd are arrays, they can never be NULL, so the
checks can be removed.
Reviewed by: bapt
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D2673
The standard states that GMT must be used, but that UTC is equivalent. Still
parse UTC as otherwise this causes problems for pkg(8). It will refetch
the repository every time 'pkg update' or other remote operations
are used behind these proxies.
RFC2616: "All HTTP date/time stamps MUST be represented in Greenwich Mean
Time (GMT), without exception. For the purposes of HTTP, GMT is exactly equal
to UTC (Coordinated Universal Time).""
Approved by: bapt (mentor)
Reviewed by: des, peter
Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week
known in advance, or where the caller doesn't care and just keeps
reading until it hits EOF.
In fetch_read(): the socket is non-blocking, so read() will return 0
on EOF, and -1 (errno == EAGAIN) when the connection is still open but
there is no data waiting. In the first case, we should immediately
return 0. The EINTR case was also broken, although not in a way that
matters.
In fetch_writev(): use timersub() and timercmp() as in fetch_read().
In http_fillbuf(): set errno to a sensible value when an invalid chunk
header is encountered.
In http_readfn(): as in fetch_read(), a zero return from down the
stack indicates EOF, not an error. Furthermore, when io->error is
EINTR, clear it (but no errno) before returning so the caller can
retry after dealing with the interrupt.
MFC after: 3 days
simply not trying to return exactly what the caller asked for - just
return whatever we got and let the caller be the judge of whether it
was enough. If an error occurs or the connection times out after we
already received some data, return a short read, under the assumption
that the next call will fail or time out before we read anything.
As it turns out, none of the code that calls fetch_read() assumes an
all-or-nothing result anyway, except for a couple of lines where we
read the CR LF at the end of a hunk in HTTP hunked encoding, so the
changes outside of fetch_read() and http_readfn() are minimal.
While there, replace select(2) with poll(2).
MFC after: 3 days
request, RFC 2616 14.23 mandates the presence of the Host: header in
all HTTP 1.1 requests.
PR: kern/181445
Submitted by: Kimo <kimor79@yahoo.com>
MFC after: 3 days
the man page) [0]
While here add support for draft-reschke-http-status-308-07
PR: 172451 [0]
Submitted by: gcooper [0]
Reviewed by: des
Approved by: cperciva
MFC after: 1 week
progress information. The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal. The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.
PR: bin/153240
Submitted by: Mark <markjdb@gmail.com>
MFC after: 3 weeks
once, even if authentication is required, instead of retrying with the
proper credentials. Fix this by bumping the countdown if the origin or
proxy server requests authentication so that the initial unauthenticated
request does not count as an attempt.
PR: 148087
Submitted by: Tom Evans <tevans.uk@googlemail.com>
MFC after: 2 weeks
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.
libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.
PR: bin/87841
Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by: des, ru
MFC after: 3 weeks
lynx, curl etc. Note that this patch differs significantly from that
in the PR, as the submitter refined it after submitting the PR.
PR: 110388
Submitted by: Alexander Pohoyda <alexander.pohoyda@gmx.net>
MFC after: 3 weeks
correctly in the case of FTP_PROXY, because an empty FTP_PROXY has a
specific meaning ("don't use any proxy at all for ftp, even if HTTP_PROXY
is defined"), while an empty HTTP_PROXY has no meaning at all.
PR: bin/85185
Submitted by: Conall O'Brien <conallob=freebsd@maths.tcd.ie>
MFC after: 2 weeks
any pending HTTP request rather than calling shutdown(2) with SHUT_WR.
This makes libfetch (and thus fetch(1)) work again with Squid proxies
configured to not allow half-closed connections.
Reported by: Pawel Worach (pawel.worach AT telia DOT com)
reply with a 416 error code (requested range not satisfiable) because
we ask it to start at the end of the file. Handle this gracefully by
considering a 416 reply a success if the requested offset exactly
matches the length of the file and the requested length is zero.
