d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
157,183 Commits 4 Branches 49 Tags
Commit Graph

84 Commits

Author SHA1 Message Date
murray
72a890ccd7 Add support for HTTP 1.1 If-Modified-Since behavior. 
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.

libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.

PR:		bin/87841
Submitted by:	Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by:	des, ru
MFC after:	3 weeks
 2008-12-15 08:27:44 +00:00
ru
9a16e00e36 Don't fail mistakenly with -r when we already have the whole file. 
Reviewed by:	des
 2008-10-24 07:56:01 +00:00
des
ff0eb6dba4 Use memcpy(3) instead of the BSD-specific bcopy(3). 
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	2 weeks
 2008-02-08 09:48:48 +00:00
des
d129ae8c34 Add necessary cast for tolower() argument. 
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	1 week
 2008-02-06 11:39:55 +00:00
des
d198b4b8bb As several people pointed out, I did all the ctype casts the wrong 
way (not for the first time...)

Noticed by:	bde, ru ++
MFC after:	1 week
 2007-12-19 00:26:36 +00:00
des
f0e1aa6917 Add support for the NO_PROXY / no_proxy environment variable as used by 
lynx, curl etc.  Note that this patch differs significantly from that
in the PR, as the submitter refined it after submitting the PR.

PR:		110388
Submitted by:	Alexander Pohoyda <alexander.pohoyda@gmx.net>
MFC after:	3 weeks
 2007-12-18 11:03:07 +00:00
des
ddf13e4c29 Old patch I had lying around: correctly cast the argument to is*(). 
IWBNI gcc could warn about this the way it warns about printf() abuse.

MFC after:	1 week
 2007-12-18 10:41:12 +00:00
des
a1830118ba Clean up namespace violations. 
MFC after:	1 week
 2007-12-14 10:26:58 +00:00
des
396660deb6 Handle temporary redirects (HTTP status code 307) 
PR:		kern/112515
Submitted by:	Ryan C. Gordon <icculus@icculus.org>
MFC after:	3 weeks
 2007-05-08 19:28:03 +00:00
des
f202203775 Ignore HTTP_PROXY if it is defined but empty. This was already handled 
correctly in the case of FTP_PROXY, because an empty FTP_PROXY has a
specific meaning ("don't use any proxy at all for ftp, even if HTTP_PROXY
is defined"), while an empty HTTP_PROXY has no meaning at all.

PR:		bin/85185
Submitted by:	Conall O'Brien <conallob=freebsd@maths.tcd.ie>
MFC after:	2 weeks
 2005-08-24 12:28:05 +00:00
kbyanc
392eda7ce9 Set the TCP_NODELAY socket option and clear TCP_NOPUSH in order to flush 
any pending HTTP request rather than calling shutdown(2) with SHUT_WR.
This makes libfetch (and thus fetch(1)) work again with Squid proxies
configured to not allow half-closed connections.

Reported by:	Pawel Worach (pawel.worach AT telia DOT com)
 2005-03-02 19:09:28 +00:00
kbyanc
bdfff0682f Set TCP_NOPUSH on HTTP requests, reducing the number of round-trips 
necessary to establish each connection.

MFC after:	2 weeks
 2005-02-16 00:22:20 +00:00
des
d59f3ddc7e Update copyright years. 2004-09-21 18:35:21 +00:00
des
4fb65f044c Don't forget to allocate space for the terminating NUL when converting to 
base 64.

PR:		misc/70022
Submitted by:	Herve Masson <herve-bsdbt@mindstep.com>
MFC after:	3 days
 2004-08-07 20:23:50 +00:00
des
df87448759 Slight tweak to previous commit: don't forget to call _http_seterr() if 
we get a genuine 416 reply.
 2004-02-11 09:35:27 +00:00
des
33bdcbd9cf When restarting a transfer that has already completed, the server will 
reply with a 416 error code (requested range not satisfiable) because
we ask it to start at the end of the file.  Handle this gracefully by
considering a 416 reply a success if the requested offset exactly
matches the length of the file and the requested length is zero.
 2004-02-11 09:31:39 +00:00
des
ddda756172 Re-wrap some comments. 2004-02-11 09:23:35 +00:00
des
32c9a8b0bd Don't dereference flags if NULL (see http.c rev 1.87) 2003-03-29 15:15:38 +00:00
des
2d46bf6bbe Don't parse the proxy URL unless we're actually going to use it. No real 
functional difference, but debugging output will be less confusing.
 2003-03-11 08:20:58 +00:00
des
7a494517a4 style(9): add parentheses to sizeof even when not strictly required. 
MFC after:	3 days
 2003-01-28 08:04:40 +00:00
des
a5ab2dd958 A negative offset means "get it all". 2003-01-22 17:52:27 +00:00
des
b8dad5d242 Implement and document support for an HTTP_REFERER environment variable. 
PR:		28171
Submitted by:	Andre Albsmeier <andre.albsmeier@mchp.siemens.de>
Approved by:	re (bmah)
MFC after:	1 week
 2002-11-28 12:07:15 +00:00
des
799d66dac6 Use __func__ and break a long line. 2002-10-30 15:01:29 +00:00
des
007c9b98da Don't forget to set the error flag when _fetch_read() returns -1. 2002-10-30 04:42:01 +00:00
des
c118e55b51 Eliminate two cases of undefined behaviour: total in _fetch_write() was 
not initialized before use, and _http_growbuf() did not return a value
on success.

Reported by:	Peter Edwards <pmedwards@eircom.net>
MFC after:	2 weeks
 2002-10-27 15:43:40 +00:00
ru
48afd8f904 HTTP authentication got broken in rev. 1.49. 2002-10-03 10:42:19 +00:00
des
07d27b9d0a Don't try to dereference conn when we know it's NULL. 2002-06-19 08:36:00 +00:00
des
0642a9650f Make SSL support conditional on NOCRYPT. 2002-06-05 21:35:35 +00:00
des
ce6d4c4a6e Add SSL support + slight cleanup. 
Submitted by:	Henry Whincup <henry@techiebod.com> (in principle)
 2002-06-05 12:46:36 +00:00
des
0961c455f8 Wrap everything in struct connection, and enforce timeouts everywhere 
(except for DNS operations).  Always use funopen() for HTTP, to support
both timeouts and SSL.
 2002-06-05 12:19:08 +00:00
des
f69a71af46 Rename struct cookie to struct httpio to avoid confusion (it's not an HTTP 
cookie) and increase symmetry with equivalent FTP code.
 2002-06-05 10:31:01 +00:00
des
b85e1918d1 Add comments to struct cookie. 2002-06-05 10:27:24 +00:00
des
300df6a8f1 Fix a bug I introduced in the chunk decoder in the previous commit.. 2002-06-05 10:23:19 +00:00
des
e07ac3bab3 First step towards SSL support: wrap connections in a 'struct connection' 
which contains the socket descriptor, the input buffer and (yet unused)
SSL state variables.  This has the neat side effect of greatly improving
reentrance (though we're not *quite* there yet) and opening the door to
HTTP connection caching.

This commit is inspired by email conversations with and patches from
Henry Whincup <henry@techiebod.com> last fall.
 2002-06-05 10:05:03 +00:00
markm
76fe441a7b Do not use __progname directly (except in [gs]etprogname(3)). 
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.
 2002-03-29 22:43:43 +00:00
des
ec29926e4d Reindent, and add parentheses to return statements. Some functions in 
ftp.c and http.c now have exceedingly long lines due to deep nesting;
this will be corrected by reorganizing the code in a later revision.
 2002-02-05 22:13:51 +00:00
des
69f2f16cb4 Check the return value from read() when reading the CR/LF at the end of a 
chunk.

PR:		bin/33608
MFC after:	2 weeks
 2002-01-20 19:53:12 +00:00
des
99683cbbd4 In verbose mode, display the full error message from the server, stripping 
it of HTML tags and comments.

PR:		bin/32989
MFC after:	1 week
 2002-01-01 16:25:29 +00:00
des
e37508b70c Remove VT100 escapes from debugging messages now that they're enabled by 
default.

PR:		32988
MFC after:	3 days
 2002-01-01 14:48:09 +00:00
des
01882c47bb Conditionalize some debugging code that didn't use the DEBUG macro. 
MFC after:	1 week
 2001-12-09 15:05:58 +00:00
des
ff6a5a5c28 Properly handle being redirected to an FTP URL. 
Also fix a couple of not-so-minor bugs:
 - missing case for HTTP_SEE_OTHER
 - incorrect definition of HTTP_NEED_PROXY_AUTH

PR:		20259
MFC after:	2 weeks
 2001-12-04 01:12:51 +00:00
des
f8774d9490 Tons of type, style and warning fixes that have been rotting in my tree for 
ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were
willing to do something (-fno-builtin) about it.
 2001-10-18 08:29:26 +00:00
dillon
9e8adf09a9 Add __FBSDID()s to libfetch 2001-09-30 21:36:09 +00:00
des
1d4297a2b2 Implement a HTTP_USER_AGENT environment variable. 
PR:		27669
Submitted by:	Eoin Lawless <eoin@maths.tcd.ie>
 2001-05-27 11:00:36 +00:00
des
4f0b341263 Add rudimentary support for an authentication callback function. 2001-05-26 19:37:15 +00:00
archie
21cf7b313e Apply 'const' liberally. 
Fix some other minor glitches.
 2001-04-24 00:06:21 +00:00
des
1804e2a818 Support lower-case versions of the proxy environment variables. 
PR:		bin/25494
 2001-03-07 04:45:55 +00:00
des
985bff7699 Use the documented (and historical) defaults. Centralize the decision logic 
in order to avoid this bug in the future.

Submitted by:	se
 2000-11-10 08:43:40 +00:00
des
9c7c34c6a0 Use CHECK_FLAG 2000-10-29 15:56:10 +00:00
des
a0c41f1687 Rework proxy handling so that proxies can be specified as URLs. 
As a side effect, remove a lot of duplicate and now redundant code.
 2000-10-12 22:10:26 +00:00