The standard states that GMT must be used, but that UTC is equivalent. Still
parse UTC as otherwise this causes problems for pkg(8). It will refetch
the repository every time 'pkg update' or other remote operations
are used behind these proxies.
RFC2616: "All HTTP date/time stamps MUST be represented in Greenwich Mean
Time (GMT), without exception. For the purposes of HTTP, GMT is exactly equal
to UTC (Coordinated Universal Time).""
Approved by: bapt (mentor)
Reviewed by: des, peter
Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week
known in advance, or where the caller doesn't care and just keeps
reading until it hits EOF.
In fetch_read(): the socket is non-blocking, so read() will return 0
on EOF, and -1 (errno == EAGAIN) when the connection is still open but
there is no data waiting. In the first case, we should immediately
return 0. The EINTR case was also broken, although not in a way that
matters.
In fetch_writev(): use timersub() and timercmp() as in fetch_read().
In http_fillbuf(): set errno to a sensible value when an invalid chunk
header is encountered.
In http_readfn(): as in fetch_read(), a zero return from down the
stack indicates EOF, not an error. Furthermore, when io->error is
EINTR, clear it (but no errno) before returning so the caller can
retry after dealing with the interrupt.
MFC after: 3 days
simply not trying to return exactly what the caller asked for - just
return whatever we got and let the caller be the judge of whether it
was enough. If an error occurs or the connection times out after we
already received some data, return a short read, under the assumption
that the next call will fail or time out before we read anything.
As it turns out, none of the code that calls fetch_read() assumes an
all-or-nothing result anyway, except for a couple of lines where we
read the CR LF at the end of a hunk in HTTP hunked encoding, so the
changes outside of fetch_read() and http_readfn() are minimal.
While there, replace select(2) with poll(2).
MFC after: 3 days
SSL_set_tlsext_host_name(3) internally does not modify the host buffer
pased to it. So it is safe to DECONST the struct url* here.
Reported by: gjb
Approved by: bapt (implicit)
MFC after: 1 week
X-MFC-With: r258347
SNI is Server Name Indentification which is a protocol for TLS that
indicates the host that is being connected to at the start of the
handshake. It allows to use Virtual Hosts on HTTPS.
Submitted by: sbz
Submitted by: Michael Gmelin <freebsd@grem.de> [1]
PR: kern/183583 [1]
Reviewed by: des
Approved by: bapt
MFC after: 1 week
request, RFC 2616 14.23 mandates the presence of the Host: header in
all HTTP 1.1 requests.
PR: kern/181445
Submitted by: Kimo <kimor79@yahoo.com>
MFC after: 3 days
the man page) [0]
While here add support for draft-reschke-http-status-308-07
PR: 172451 [0]
Submitted by: gcooper [0]
Reviewed by: des
Approved by: cperciva
MFC after: 1 week
To avoid unexpected process termination from SIGPIPE when writing to a
closed network connection, enable SO_NOSIGPIPE on all network connections.
The POSIX standard MSG_NOSIGNAL is not used since it requires modifying all
send calls to add this flag. This is particularly nasty for SSL connections.
Reviewed by: des
Tested by: bapt
MFC after: 5 days
when there is no timeout, because read(2) will return immediately if there
is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on
slow connections.
MFC after: 3 weeks
Noticed by: Yanhui Shen <shen.elf@gmail.com>
RFC 1738 specifies that any ":", "@", or "/" within a user name or
password in a URL is percent-encoded, to avoid ambiguity with the use
of those characters as URL component separators.
Reviewed by: rstone@
MFC after: 1 month
progress information. The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal. The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.
PR: bin/153240
Submitted by: Mark <markjdb@gmail.com>
MFC after: 3 weeks
1. Allow the caller to select active mode.
2. Fix the envar logic so it *always* overrides the caller's flags.
3. Document the change from active to passive.
avoid a hang in the SSL case if the server sends a close notification
before we are done reading. In the non-SSL case, it can provide a
minor (but probably not noticeable) performance improvement for small
transfers.
MFC after: 3 weeks
once, even if authentication is required, instead of retrying with the
proper credentials. Fix this by bumping the countdown if the origin or
proxy server requests authentication so that the initial unauthenticated
request does not count as an attempt.
PR: 148087
Submitted by: Tom Evans <tevans.uk@googlemail.com>
MFC after: 2 weeks
Mention some prominent past contributors: Hajimu Umemoto (ipv6), Henry
Whincup (https), Jukka Ukkonen (if-modified-since) and Jean-François
Dockes (digest auth)
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.
Reviewed by: kib
Approved by: re (rwatson)
rather than usually returning 1 but in a few instances using a sysexits(3)
return value.
2. Remove a few unused variables from libfetch.
PR: docs/122470 (1, only)
Reviewed by: des
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.
libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.
PR: bin/87841
Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by: des, ru
MFC after: 3 weeks
