Commit Graph

99 Commits

Author SHA1 Message Date
Dag-Erling Smørgrav
eab7a548ba Fix weird indentation. 2012-11-16 12:31:43 +00:00
Eitan Adler
8d049fb235 Implement HTTP 305 redirect handling.
PR:		172452
Submitted by:	gcooper
Reviewed by:	des
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:15 +00:00
Eitan Adler
c4fa1489ec Don't deny non-temporary redirects if the -A option is set (per
the man page) [0]

While here add support for draft-reschke-http-status-308-07

PR:		172451 [0]
Submitted by:	gcooper [0]
Reviewed by:	des
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:10 +00:00
Eitan Adler
e6c0e200f4 Be a bit more lenient in the maximum number of redirects allowed.
Chrome and Firefox have a limit of 20. IE has a limit of 8.

Reviewed by:	des
Approved by:	cperciva
MFC after:	3 days
2012-10-22 03:00:04 +00:00
Dag-Erling Smørgrav
0e50a83330 Use libmd if and only if OpenSSL is not available.
PR:		bin/171402
MFC after:	3 days
2012-09-14 13:00:43 +00:00
Dag-Erling Smørgrav
f51b84bcc4 Don't reuse credentials if redirected to a different host.
Submitted by:	Niels Heinen <heinenn@google.com>
MFC after:	3 weeks
2012-04-30 12:12:48 +00:00
Dag-Erling Smørgrav
2a7daafe67 Fix two issues related to the use of SIGINFO in fetch(1) to display
progress information.  The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal.  The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.

PR:		bin/153240
Submitted by:	Mark <markjdb@gmail.com>
MFC after:	3 weeks
2012-01-18 15:13:21 +00:00
Dag-Erling Smørgrav
578153f1ba latin1 -> utf8 2011-10-19 11:43:51 +00:00
Dag-Erling Smørgrav
6337341d81 Update copyright dates and strip my middle name. 2011-09-27 18:57:26 +00:00
Dag-Erling Smørgrav
eb9b80c30d Increase WARNS to 4. 2011-05-12 21:26:42 +00:00
Dag-Erling Smørgrav
c12c6e3cda Mechanical whitespace cleanup. 2011-05-12 21:18:55 +00:00
Dag-Erling Smørgrav
a42eecded0 Increase WARNS to 3. 2011-05-12 21:12:24 +00:00
Dag-Erling Smørgrav
c954ded250 Fix a couple of embarrassing mistakes in the previous commit.
Submitted by:	Dimitry Andric <dimitry@andric.com>
2010-07-28 15:29:18 +00:00
Dag-Erling Smørgrav
962cf29525 If the A flag is supplied, http_request() will attempt the request only
once, even if authentication is required, instead of retrying with the
proper credentials.  Fix this by bumping the countdown if the origin or
proxy server requests authentication so that the initial unauthenticated
request does not count as an attempt.

PR:		148087
Submitted by:	Tom Evans <tevans.uk@googlemail.com>
MFC after:	2 weeks
2010-07-01 17:44:33 +00:00
Dag-Erling Smørgrav
79ad329d0c Add HTTP digest authentication.
Submitted by:	Jean-Francois Dockes <jf@dockes.org>
Forgotten by:	des (repeatedly)
2010-01-19 10:19:55 +00:00
Murray Stokely
7f92799f67 Add support for HTTP 1.1 If-Modified-Since behavior.
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.

libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.

PR:		bin/87841
Submitted by:	Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by:	des, ru
MFC after:	3 weeks
2008-12-15 08:27:44 +00:00
Ruslan Ermilov
e374393a07 Don't fail mistakenly with -r when we already have the whole file.
Reviewed by:	des
2008-10-24 07:56:01 +00:00
Dag-Erling Smørgrav
340b079be0 Use memcpy(3) instead of the BSD-specific bcopy(3).
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	2 weeks
2008-02-08 09:48:48 +00:00
Dag-Erling Smørgrav
79257dd70a Add necessary cast for tolower() argument.
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	1 week
2008-02-06 11:39:55 +00:00
Dag-Erling Smørgrav
facd982794 As several people pointed out, I did all the ctype casts the wrong
way (not for the first time...)

Noticed by:	bde, ru ++
MFC after:	1 week
2007-12-19 00:26:36 +00:00
Dag-Erling Smørgrav
62a2681c93 Add support for the NO_PROXY / no_proxy environment variable as used by
lynx, curl etc.  Note that this patch differs significantly from that
in the PR, as the submitter refined it after submitting the PR.

PR:		110388
Submitted by:	Alexander Pohoyda <alexander.pohoyda@gmx.net>
MFC after:	3 weeks
2007-12-18 11:03:07 +00:00
Dag-Erling Smørgrav
836e34eeae Old patch I had lying around: correctly cast the argument to is*().
IWBNI gcc could warn about this the way it warns about printf() abuse.

MFC after:	1 week
2007-12-18 10:41:12 +00:00
Dag-Erling Smørgrav
a1b37df2d7 Clean up namespace violations.
MFC after:	1 week
2007-12-14 10:26:58 +00:00
Dag-Erling Smørgrav
cd84225739 Handle temporary redirects (HTTP status code 307)
PR:		kern/112515
Submitted by:	Ryan C. Gordon <icculus@icculus.org>
MFC after:	3 weeks
2007-05-08 19:28:03 +00:00
Dag-Erling Smørgrav
2738229eb8 Ignore HTTP_PROXY if it is defined but empty. This was already handled
correctly in the case of FTP_PROXY, because an empty FTP_PROXY has a
specific meaning ("don't use any proxy at all for ftp, even if HTTP_PROXY
is defined"), while an empty HTTP_PROXY has no meaning at all.

PR:		bin/85185
Submitted by:	Conall O'Brien <conallob=freebsd@maths.tcd.ie>
MFC after:	2 weeks
2005-08-24 12:28:05 +00:00
Kelly Yancey
448980e704 Set the TCP_NODELAY socket option and clear TCP_NOPUSH in order to flush
any pending HTTP request rather than calling shutdown(2) with SHUT_WR.
This makes libfetch (and thus fetch(1)) work again with Squid proxies
configured to not allow half-closed connections.

Reported by:	Pawel Worach (pawel.worach AT telia DOT com)
2005-03-02 19:09:28 +00:00
Kelly Yancey
20a2a51bc3 Set TCP_NOPUSH on HTTP requests, reducing the number of round-trips
necessary to establish each connection.

MFC after:	2 weeks
2005-02-16 00:22:20 +00:00
Dag-Erling Smørgrav
2cbbf9dac9 Update copyright years. 2004-09-21 18:35:21 +00:00
Dag-Erling Smørgrav
264a4c03fc Don't forget to allocate space for the terminating NUL when converting to
base 64.

PR:		misc/70022
Submitted by:	Herve Masson <herve-bsdbt@mindstep.com>
MFC after:	3 days
2004-08-07 20:23:50 +00:00
Dag-Erling Smørgrav
4d3d90ad0a Slight tweak to previous commit: don't forget to call _http_seterr() if
we get a genuine 416 reply.
2004-02-11 09:35:27 +00:00
Dag-Erling Smørgrav
b5e1214546 When restarting a transfer that has already completed, the server will
reply with a 416 error code (requested range not satisfiable) because
we ask it to start at the end of the file.  Handle this gracefully by
considering a 416 reply a success if the requested offset exactly
matches the length of the file and the requested length is zero.
2004-02-11 09:31:39 +00:00
Dag-Erling Smørgrav
2576e45915 Re-wrap some comments. 2004-02-11 09:23:35 +00:00
Dag-Erling Smørgrav
762892c047 Don't dereference flags if NULL (see http.c rev 1.87) 2003-03-29 15:15:38 +00:00
Dag-Erling Smørgrav
b91d6074d4 Don't parse the proxy URL unless we're actually going to use it. No real
functional difference, but debugging output will be less confusing.
2003-03-11 08:20:58 +00:00
Dag-Erling Smørgrav
930105c1e9 style(9): add parentheses to sizeof even when not strictly required.
MFC after:	3 days
2003-01-28 08:04:40 +00:00
Dag-Erling Smørgrav
591c019988 A negative offset means "get it all". 2003-01-22 17:52:27 +00:00
Dag-Erling Smørgrav
d5216a4fb7 Implement and document support for an HTTP_REFERER environment variable.
PR:		28171
Submitted by:	Andre Albsmeier <andre.albsmeier@mchp.siemens.de>
Approved by:	re (bmah)
MFC after:	1 week
2002-11-28 12:07:15 +00:00
Dag-Erling Smørgrav
a49b853d94 Use __func__ and break a long line. 2002-10-30 15:01:29 +00:00
Dag-Erling Smørgrav
cab8495092 Don't forget to set the error flag when _fetch_read() returns -1. 2002-10-30 04:42:01 +00:00
Dag-Erling Smørgrav
9f788e9c90 Eliminate two cases of undefined behaviour: total in _fetch_write() was
not initialized before use, and _http_growbuf() did not return a value
on success.

Reported by:	Peter Edwards <pmedwards@eircom.net>
MFC after:	2 weeks
2002-10-27 15:43:40 +00:00
Ruslan Ermilov
721d559c03 HTTP authentication got broken in rev. 1.49. 2002-10-03 10:42:19 +00:00
Dag-Erling Smørgrav
f8bd33a0ae Don't try to dereference conn when we know it's NULL. 2002-06-19 08:36:00 +00:00
Dag-Erling Smørgrav
3070f6cb06 Make SSL support conditional on NOCRYPT. 2002-06-05 21:35:35 +00:00
Dag-Erling Smørgrav
111e251009 Add SSL support + slight cleanup.
Submitted by:	Henry Whincup <henry@techiebod.com> (in principle)
2002-06-05 12:46:36 +00:00
Dag-Erling Smørgrav
9601e333a8 Wrap everything in struct connection, and enforce timeouts everywhere
(except for DNS operations).  Always use funopen() for HTTP, to support
both timeouts and SSL.
2002-06-05 12:19:08 +00:00
Dag-Erling Smørgrav
ccdd94bdd9 Rename struct cookie to struct httpio to avoid confusion (it's not an HTTP
cookie) and increase symmetry with equivalent FTP code.
2002-06-05 10:31:01 +00:00
Dag-Erling Smørgrav
d3b03a9006 Add comments to struct cookie. 2002-06-05 10:27:24 +00:00
Dag-Erling Smørgrav
3f4823c55d Fix a bug I introduced in the chunk decoder in the previous commit.. 2002-06-05 10:23:19 +00:00
Dag-Erling Smørgrav
dea29ca1d5 First step towards SSL support: wrap connections in a 'struct connection'
which contains the socket descriptor, the input buffer and (yet unused)
SSL state variables.  This has the neat side effect of greatly improving
reentrance (though we're not *quite* there yet) and opening the door to
HTTP connection caching.

This commit is inspired by email conversations with and patches from
Henry Whincup <henry@techiebod.com> last fall.
2002-06-05 10:05:03 +00:00
Mark Murray
4cd0119367 Do not use __progname directly (except in [gs]etprogname(3)).
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.
2002-03-29 22:43:43 +00:00