d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
234,800 Commits 4 Branches 49 Tags 2 GiB
20b0fd729c
Commit Graph

386 Commits

Author SHA1 Message Date
trhodes
3514f4bda7 Add an EXAMPLES section. 
PR:		43899
Reviewed by:	des, ru
 2003-06-10 20:41:18 +00:00
ru
9bcac8799e Assorted mdoc(7) fixes. 
Approved by:	re (blanket)
 2003-05-22 13:02:28 +00:00
des
e5d2d778eb Retire the useless NOSECURE knob. 
Approved by:	re (scottl)
 2003-05-19 15:52:01 +00:00
des
9e8300e563 Mark libfetch as c99-clean. Always build at WARNS level 2 rather than 
juggling with levels 2 and 3, as this has lead to world breakage for
NOCRYPT users in the past.
 2003-05-01 14:39:43 +00:00
des
32c9a8b0bd Don't dereference flags if NULL (see http.c rev 1.87) 2003-03-29 15:15:38 +00:00
mtm
a65d822bcd The flags passed in to _ftp_get_proxy may be null 
Approved by:	des, markm (mentor)(implicit)
 2003-03-19 21:39:00 +00:00
des
2d46bf6bbe Don't parse the proxy URL unless we're actually going to use it. No real 
functional difference, but debugging output will be less confusing.
 2003-03-11 08:20:58 +00:00
des
66c619c7d8 Add and document support for a FETCH_BIND_ADDRESS environment variable 
specifying a local address to bind sockets to.  Caveat: lightly tested.

PR:		bin/37572
 2003-03-03 12:35:03 +00:00
markm
21f0f28e47 No need for the OPENSSL_NO_KRB5 switch anymore. 
Fixed by:	nectar
 2003-01-31 23:26:55 +00:00
markm
0d82b8beac Adjust for OpenSSL 0.9.7. 2003-01-28 23:03:15 +00:00
des
7a494517a4 style(9): add parentheses to sizeof even when not strictly required. 
MFC after:	3 days
 2003-01-28 08:04:40 +00:00
jwd
24a72fc271 Fix signed/unsigned comparison warning/error from 'make release' 2003-01-28 00:33:53 +00:00
des
b80b0e22d8 Document .netrc support. 2003-01-22 18:27:25 +00:00
des
48ceb170df Hook in the .netrc code + don't use pointers as if they were booleans. 2003-01-22 17:54:49 +00:00
des
ae10e138a9 Experimental support for .netrc. 2003-01-22 17:53:58 +00:00
des
cead81021c Initialization paranoia. 2003-01-22 17:53:15 +00:00
des
a5ab2dd958 A negative offset means "get it all". 2003-01-22 17:52:27 +00:00
des
a91193c322 Set auto-retry mode to avoid some spurious errors. 
Submitted by:	Andre Albsmeier <andre.albsmeier@siemens.com>
MFC after:	7 days
 2003-01-03 02:45:10 +00:00
schweikh
fec6546e12 english(4) police. 2002-12-27 12:15:40 +00:00
jake
4cbd3579e8 Reduce WARNS level to 3 in the non-crypto case to fix make release on sparc64. 
This may only be turning up now to changes in the cpp predefines for sparc64,
which may be turning on more strict alignment checking.

Approved by:	re
 2002-12-04 04:34:59 +00:00
des
b8dad5d242 Implement and document support for an HTTP_REFERER environment variable. 
PR:		28171
Submitted by:	Andre Albsmeier <andre.albsmeier@mchp.siemens.de>
Approved by:	re (bmah)
MFC after:	1 week
 2002-11-28 12:07:15 +00:00
fenner
1a1a58e225 Fix two typos. 2002-11-08 19:10:05 +00:00
des
799d66dac6 Use __func__ and break a long line. 2002-10-30 15:01:29 +00:00
des
216181b022 Fix a bug in fenner's _fetch_writev() patch (rev 1.36) 
Submitted by:	fenner
 2002-10-30 14:25:00 +00:00
des
a8da55afda Recognize the (incorrect) error code a MediaHawk server sends in reply to 
unrecognized commands such as MDTM.

Requested by:	Stephen Roome <stephen_roome@pepcross.com>
 2002-10-30 06:06:16 +00:00
des
deb00535da Clarify my feelings towards fetch / libfetch. 2002-10-30 04:53:58 +00:00
des
f8f3fd44d8 Recommit the non-broken parts of 1.34 and 1.37. 
Change the type and name of a variable introduced in 1.33.
 2002-10-30 04:43:00 +00:00
des
007c9b98da Don't forget to set the error flag when _fetch_read() returns -1. 2002-10-30 04:42:01 +00:00
imp
ef2e40e300 Reinstate revs 1.35-36 and 1.38. Revisions 1.34 and 1.37 were specifically 
the root cause of the bus errors I was experiencing.

Submitted by:	fenner
Tested by:	obrien
Prompted by:	peter
 2002-10-30 00:17:16 +00:00
obrien
ce6ecab91b Fix `pkg_add -r' by backing out revs 1.34-1.38. 
Revs 1.37-8 produce a bus error in some environments.
Revs 1.34-6 do not bus error, but write corrupted files.
 2002-10-29 12:17:43 +00:00
des
8f1da1ac7d libfetch is now WARNS5 clean in the non-SSL case. 2002-10-28 10:37:31 +00:00
des
d3836d6963 Fix an off-by-one error (> where >= should have been used) which caused 
_fetch_writev() to incorrectly report EPIPE in certain cases.

Also fix a number of const warnings by using __DECONST(), plus a signed /
unsigned comparison by casting the rhs to ssize_t.

Submitted by:	fenner, Craig Rodrigues <rodrigc@attbi.com>
 2002-10-28 10:19:03 +00:00
fenner
519bc6cb16 Up WARNS to 3 if not building with crypto. 
Approved by:	des
 2002-10-28 01:41:28 +00:00
des
37b3ac0423 Slight amendment to rev 1.34: instead of considering any short read an 
error, only report an error if no data was read at all (unless len was
0 to start with).  Otherwise, the final read of practically any transfer
will end in a fatal error.
 2002-10-27 17:20:49 +00:00
des
34e6c489c7 Introduce _fetch_writev(), which is the conn_t version of writev(2). In 
the SSL case, it is no different from the old _fetch_write(), but in the
non-SSL case it uses writev(2) to send the entire vector as a single
packet (provided it can fit in one packet).  Implement _fetch_write()
and _fetch_putln() in terms of _fetch_writev().

This should improve performance in the non-SSL case (by reducing protocol
overhead) and solve the problem where too-smart-for-their-own-good
firewalls reject FTP packets that do not end in CRLF.

PR:		bin/44123
Submitted by:	fenner
 2002-10-27 16:11:21 +00:00
des
c118e55b51 Eliminate two cases of undefined behaviour: total in _fetch_write() was 
not initialized before use, and _http_growbuf() did not return a value
on success.

Reported by:	Peter Edwards <pmedwards@eircom.net>
MFC after:	2 weeks
 2002-10-27 15:43:40 +00:00
des
cd8380e0b1 Back out the previous commit, and fix the bug rather than try to hide its 
symptoms: make timeouts and short transfers fatal, and set errno to an
appropriate value (ETIMEDOUT for a timeout, EPIPE for a short transfer).

MFC after:	2 weeks
 2002-10-27 15:08:21 +00:00
njl
06c0d543e0 The FTP connection caching needs a better interface -- connections are 
closed through _fetch_close() which is the only one who knows the connection
REALLY was closed (since ref -> 0).  However, FTP keeps its own local
cached_connection and checks if it is valid by comparing it to NULL.  This
is bogus since it may have been freed elsewhere by _fetch_close().

This change checks if we are closing the cached_connection and the ref is 1
(soon to be 0).  If so, set cached_connection to NULL so we don't
accidentally reuse it.  The REAL fix should be to move connection caching
to the common.c level (_fetch_* functions) and NULL the cache(s) in
_fetch_close().  Then all layers could benefit from caching.
 2002-10-25 01:17:32 +00:00
obrien
ddccb9e2b3 Use a warns setting we can catch regressions with. 2002-10-20 23:50:28 +00:00
tjr
8ed112060e When recycling a cached connection, increment the reference count so that 
the heap block does not get freed and reused. This should fix the
pkg_add -r crashes that have been happening for months.
 2002-10-20 10:36:27 +00:00
ru
48afd8f904 HTTP authentication got broken in rev. 1.49. 2002-10-03 10:42:19 +00:00
peter
0a7f0ba37e Zap now-unused SHLIB_MINOR 2002-09-28 00:25:32 +00:00
alfred
4f92df742a Fix an infinite loop when _fetch_read() can return 0 (if the 
connection is broken), take this into account and return at this
point.
 2002-09-20 21:50:57 +00:00
fenner
dd87f608a3 Make _fetch_connect() always set the error code. 
Tell ftp that _fetch_connect() always sets the error code (http already knew)
 2002-09-17 05:54:33 +00:00
des
4732944265 Surround prototypes with __BEGIN_DECLS / __END_DECLS. 
PR:		misc/40399
MFC after:	3 days
 2002-07-22 16:11:39 +00:00
ume
eb16909ba9 Cope with 2292bis-01 getaddrinfo (no NI_WITHSCOPEID, always attach 
scope identifier).

Approved by:	des
MFC after:	3 weeks
 2002-07-02 11:09:02 +00:00
des
0fa910cfde Reintroduce debugging code that somehow got lost in a previous revision. 2002-06-24 12:18:41 +00:00
des
10a50c435b Fix incorrect library ordering. I thought I'd committed this already... 2002-06-21 09:56:38 +00:00
des
07d27b9d0a Don't try to dereference conn when we know it's NULL. 2002-06-19 08:36:00 +00:00
des
5064d0b266 Add a reference count to struct fetchconn so we don't prematurely close and 
free a cached FTP connection.
 2002-06-11 11:27:28 +00:00
ru
876652376e libfetch now depends on libcrypto and libssl. 2002-06-06 13:45:46 +00:00
des
0642a9650f Make SSL support conditional on NOCRYPT. 2002-06-05 21:35:35 +00:00
des
694ae4fc45 During buildworld, "regular" libraries are built before crypto stuff, so 
libfetch can't depend on lib{crypto,ssl}.  Move the dependency to fetch
until we can figure out how to fix this.
 2002-06-05 21:25:33 +00:00
des
ce6d4c4a6e Add SSL support + slight cleanup. 
Submitted by:	Henry Whincup <henry@techiebod.com> (in principle)
 2002-06-05 12:46:36 +00:00
des
0961c455f8 Wrap everything in struct connection, and enforce timeouts everywhere 
(except for DNS operations).  Always use funopen() for HTTP, to support
both timeouts and SSL.
 2002-06-05 12:19:08 +00:00
des
71760f5130 Add the necessary dependencies for SSL. 2002-06-05 11:38:19 +00:00
des
f69a71af46 Rename struct cookie to struct httpio to avoid confusion (it's not an HTTP 
cookie) and increase symmetry with equivalent FTP code.
 2002-06-05 10:31:01 +00:00
des
b85e1918d1 Add comments to struct cookie. 2002-06-05 10:27:24 +00:00
des
300df6a8f1 Fix a bug I introduced in the chunk decoder in the previous commit.. 2002-06-05 10:23:19 +00:00
des
e07ac3bab3 First step towards SSL support: wrap connections in a 'struct connection' 
which contains the socket descriptor, the input buffer and (yet unused)
SSL state variables.  This has the neat side effect of greatly improving
reentrance (though we're not *quite* there yet) and opening the door to
HTTP connection caching.

This commit is inspired by email conversations with and patches from
Henry Whincup <henry@techiebod.com> last fall.
 2002-06-05 10:05:03 +00:00
markm
76fe441a7b Do not use __progname directly (except in [gs]etprogname(3)). 
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.
 2002-03-29 22:43:43 +00:00
phk
432db57cce Modernize my email address 2002-03-25 13:53:46 +00:00
des
36254aa73f Switch to a self-starting allocation scheme. 2002-02-05 22:15:16 +00:00
des
ec29926e4d Reindent, and add parentheses to return statements. Some functions in 
ftp.c and http.c now have exceedingly long lines due to deep nesting;
this will be corrected by reorganizing the code in a later revision.
 2002-02-05 22:13:51 +00:00
des
95a4ae2830 Eliminate a redundant loop. 2002-02-05 21:32:16 +00:00
des
69f2f16cb4 Check the return value from read() when reading the CR/LF at the end of a 
chunk.

PR:		bin/33608
MFC after:	2 weeks
 2002-01-20 19:53:12 +00:00
des
9f6167dc5a Mark uploads as O_WRONLY, not O_RDONLY. 
PR:		misc/34043
MFC after:	2 weeks
 2002-01-20 19:52:25 +00:00
des
cfe419eb21 Remove long-obsolete README. 2002-01-01 16:27:26 +00:00
des
99683cbbd4 In verbose mode, display the full error message from the server, stripping 
it of HTML tags and comments.

PR:		bin/32989
MFC after:	1 week
 2002-01-01 16:25:29 +00:00
des
e37508b70c Remove VT100 escapes from debugging messages now that they're enabled by 
default.

PR:		32988
MFC after:	3 days
 2002-01-01 14:48:09 +00:00
des
b073ce5f0e Don't build with NDEBUG. 
PR:		bin/32615
MFC after:	1 week
 2001-12-09 15:06:38 +00:00
des
01882c47bb Conditionalize some debugging code that didn't use the DEBUG macro. 
MFC after:	1 week
 2001-12-09 15:05:58 +00:00
des
a9520ca9d7 Introduce a fetchDebug global. Change the DEBUG macro so it only runs the 
debugging code if fetchDebug is set.

PR:		bin/32615
MFC after:	1 week
 2001-12-09 15:05:19 +00:00
des
ff6a5a5c28 Properly handle being redirected to an FTP URL. 
Also fix a couple of not-so-minor bugs:
 - missing case for HTTP_SEE_OTHER
 - incorrect definition of HTTP_NEED_PROXY_AUTH

PR:		20259
MFC after:	2 weeks
 2001-12-04 01:12:51 +00:00
des
8cabee0ca4 Add prototype for _ftp_request(). 2001-12-04 01:10:34 +00:00
des
5d89c5ae6e Reorganize to reduce code duplication. 2001-12-04 01:09:19 +00:00
des
38d04ef419 Add NO_WERROR so the build won't die because of discarded qualifiers 2001-10-19 10:08:38 +00:00
des
f59dae6318 Back out part of previous commit which was gcc-centric 2001-10-19 10:08:05 +00:00
des
f8774d9490 Tons of type, style and warning fixes that have been rotting in my tree for 
ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were
willing to do something (-fno-builtin) about it.
 2001-10-18 08:29:26 +00:00
ru
623da62a5a mdoc(7) police: Use the new .In macro for #include statements. 2001-10-01 16:09:29 +00:00
dillon
9e8adf09a9 Add __FBSDID()s to libfetch 2001-09-30 21:36:09 +00:00
des
a0569e9940 Use fseeko() instead of fseek() (u->offset is already an off_t), and mark 
some function arguments as unused.
 2001-09-05 12:22:28 +00:00
brian
903d158fae Handle snprintf() returning < 0 (not just -1) 
MFC after:	2 weeks
 2001-08-20 15:44:17 +00:00
brian
3b24d8f2af Handle snrintf overflows. 
Spotted by: bde
 2001-08-20 13:44:13 +00:00
brian
4e059b7f96 Handle snprintf() returning -1. 
MFC after:	2 weeks
 2001-08-20 12:50:21 +00:00
se
742d7aada7 The fix for schemeless and hostless URLs (rev. 1.27) broke the schemeless 
proxy specification, which seems to be valid according to the man page.

Change the logic to consider "hostname:port" a hostname and port instead
of a file URL.

Approved by:	des
 2001-07-28 21:28:14 +00:00
des
35acbfe0b3 Unbreak parsing URLs that have a host part but no document part. 
While we're here, fix a snprintf() usage warning.
 2001-07-17 20:22:33 +00:00
dd
a145482cf6 Remove whitespace at EOL. 2001-07-15 07:53:42 +00:00
dd
eaa6ee03b8 mdoc(7) police: remove extraneous .Pp before and/or after .Sh. 2001-07-09 09:54:33 +00:00
des
0ca9dbc9d1 Handle shemeless, hostless URLs correctly. 2001-07-08 15:59:15 +00:00
ru
9ca51e2245 mdoc(7) police: fixed markup and typo. 2001-07-05 11:24:26 +00:00
des
f47c06a197 Don't call _ftp_closefn() upon EOF in _ftp_readfn(); just return 0. This 
fixes a bug in pkg_add(1) (which nobody noticed because of another bug).
 2001-06-11 14:12:46 +00:00
des
1d4297a2b2 Implement a HTTP_USER_AGENT environment variable. 
PR:		27669
Submitted by:	Eoin Lawless <eoin@maths.tcd.ie>
 2001-05-27 11:00:36 +00:00
des
77734dca3c Document the authentication callback interface. 
Update RFC references (should have done that ages ago...)
 2001-05-26 19:38:34 +00:00
des
7be28011c0 Bump major number. 2001-05-26 19:37:26 +00:00
des
4f0b341263 Add rudimentary support for an authentication callback function. 2001-05-26 19:37:15 +00:00
des
4edf66ad45 MAXPATHLEN -> PATH_MAX 2001-05-26 19:36:49 +00:00
des
71613237c6 Plug memory leak. 
PR:		27506
 2001-05-26 17:23:38 +00:00
des
9af44bee15 Add some error codes that were added in RFC2616. 2001-05-23 18:52:02 +00:00
archie
21cf7b313e Apply 'const' liberally. 
Fix some other minor glitches.
 2001-04-24 00:06:21 +00:00
dd
02c02ffeef Grammar police: "its", not "it's", is the possessive form of "it". 2001-04-15 19:53:47 +00:00
des
a14b691e19 If the server's reply to the SIZE command is unparseable, reset us->size 
to -1 so the caller will know it's invalid.  This is an MFC candidate.
 2001-04-07 15:26:31 +00:00
ru
25ef23ac1c MAN[1-9] -> MAN. 2001-03-27 17:27:19 +00:00
des
bfaef3b3b9 Use high port range by default, and replace the 'h' option with an 'l' option 
that forces the ftp code to use the low (default) port range instead.
 2001-03-24 00:28:57 +00:00
des
79952149e8 Don't remember an EINTR, since the caller may want to restart the call. 2001-03-07 05:12:14 +00:00
des
1804e2a818 Support lower-case versions of the proxy environment variables. 
PR:		bin/25494
 2001-03-07 04:45:55 +00:00
ru
1476a16d46 mdoc(7) police: fixed broken references. 2001-01-16 11:52:00 +00:00
ru
3d8401c62e Prepare for mdoc(7)NG. 2001-01-16 09:08:22 +00:00
des
4a1dcddb01 The user name for anonymous ftp is now "anonymous". 
Remove the period after the last man page reference.
Add a reference to RFC1635, and sort the RFC references by number.
 2001-01-08 13:46:40 +00:00
des
0c87c94a31 Back out previous (accidental) commit. 2001-01-08 13:17:21 +00:00
des
e75c537ff5 Use "anonymous" rather than "ftp" as login name for anonymous ftp. 
Rather than have a separate (misnamed) FTP_ANONYMOUS_PASSWORD constant, use
FTP_ANONYMOUS_USER (i.e. "anonymous") to construct the anonymous ftp password
if getlogin() fails.
 2001-01-08 13:15:14 +00:00
des
68260023b0 Document FTP_LOGIN. 2000-12-22 18:03:21 +00:00
des
9d68cdd937 Check the FTP_LOGIN environment variable before falling back on 
FTP_ANONYMOUS_USER.
 2000-12-22 18:01:40 +00:00
des
ae0af0d011 Avoid a segfault (due to an unitialized pointer) when parsing URLs that have 
no scheme or host part.
 2000-12-13 11:21:09 +00:00
des
1f1139ecc0 Somewhere along the line, I misunderstood the whole FTP_PASSIVE_MODE debate 
and had libfetch selecting passive mode even when FTP_PASSIVE_MODE was not
set at all, which is really quite surprising unless you know about it. So
change it to the agreed default behaviour of selecting passive mode if
FTP_PASSIVE_MODE is set, but not "no".
 2000-12-06 09:23:27 +00:00
des
ac423cd763 Remove some obsolete comments. 2000-12-01 11:05:15 +00:00
des
1d928c8e6e Clean up the whitespace encoding code. 2000-12-01 11:04:57 +00:00
des
91475759ce Add SCHEME_HTTPS. 2000-12-01 11:04:43 +00:00
des
af385a33a0 Fix old-style proxy specs: default to FTP if FTP_PROXY was set; only default 
to HTTP if HTTP_PROXY was used instead.
 2000-11-27 13:42:56 +00:00
des
7affcbe9be Don't try to get the proxy port number from /etc/services. 2000-11-27 13:41:55 +00:00
des
24bc75d6d1 Don't go haywire if the server closes the connection in the middle of a 
multiline response (proper fix this time).
 2000-11-22 14:50:46 +00:00
des
c3ebcca666 Revert previous commit, it was somewhat hasty. 2000-11-22 14:44:48 +00:00
des
f8ecb7b0f8 Don't go haywire if the server closes the connection during a multiline 
response.
 2000-11-22 14:30:28 +00:00
ru
1a6c69e84a log 2000-11-22 09:23:54 +00:00
ru
3b7dc88538 Spell the des's name correctly. 2000-11-14 13:19:26 +00:00
ru
7d99729431 Use Fx macro wherever possible. 2000-11-14 11:20:58 +00:00
des
985bff7699 Use the documented (and historical) defaults. Centralize the decision logic 
in order to avoid this bug in the future.

Submitted by:	se
 2000-11-10 08:43:40 +00:00
des
9c7c34c6a0 Use CHECK_FLAG 2000-10-29 15:56:10 +00:00
des
0b058e3e71 Stricter error checking in the I/O functions. 2000-10-29 15:52:05 +00:00
des
f8eaa3d801 Add CHECK_FLAGS, a macro for (safely) checking if a particular flag is set. 2000-10-29 15:45:31 +00:00
des
0237382c52 Document FTP_PASSIVE_MODE and FTP_PASSWORD. 
PR:		docs/20626
Submitted by:	nik
 2000-10-29 12:58:08 +00:00
des
24f27bdb94 Fix this my way. David had absolutely no call overriding MAINTAINER without 
even giving me 24 hours to read his mail and find the bug.
 2000-10-28 20:53:02 +00:00
obrien
dd714aa7f1 Install per function manpages so one doesn't mistakenly think we don't 
have manpages for libfetch's functions.
 2000-10-28 20:32:39 +00:00
obrien
a90e794625 Repeat after me, "check to see that a pointer isn't NULL before 
dereferenceing it".  This fixes ``pkg_add -r''.
 2000-10-28 20:27:11 +00:00
des
ea5581599f MFS: don't expect result code until you close the data connection 2000-10-27 11:37:21 +00:00
des
d1e9930273 Use funopen() instead of fdopen(). This fixes three problems: 
- ftpTimeout was not honored when reading actual data, as opposed to
   talking protocol

 - connection caching was broken because _ftp_cached_connect() would see
   the result of the transfer instead of the result of the NOOP.

 - if the RETR succeeded, but an error occurred later (as can happen
   when talking to a proxy), the error would not be detected.

There still remains to register an atexit(3) callback to close the cached
connection gracefully instead of just dropping it on the floor.
 2000-10-22 12:07:28 +00:00
des
cbb8cbedb3 If the scheme is HTTP or HTTPS, percent-escape whitespace in the document 
part.

Submitted by:	green
 2000-10-21 14:58:18 +00:00
des
328bdfc322 Argh! Fix passive mode selection (again) 2000-10-17 19:01:18 +00:00
des
0292a311c7 Use the right user name 2000-10-13 09:36:46 +00:00
des
9a7ce3cd1a Document recent changes in URL parsing and proxy handling 2000-10-12 22:11:26 +00:00
des
62f1dcda19 ftp.c needs _http_request() 2000-10-12 22:10:43 +00:00
des
a0c41f1687 Rework proxy handling so that proxies can be specified as URLs. 
As a side effect, remove a lot of duplicate and now redundant code.
 2000-10-12 22:10:26 +00:00
des
238c26fcae Relax URL syntax so that schemeless URLs are supported. 2000-10-12 22:07:49 +00:00
des
4ce445deae Add macros for the names of the URL schemes we support. 2000-10-12 21:59:22 +00:00
des
88e20da596 Make the ENVIRONMENT section more legible, and mention that HTTP_AUTH is 
colon-separated.
 2000-10-12 16:55:52 +00:00
des
6fe9a920d1 Work around buggy servers such as NCSA httpd which send an incomplete 
HTTP-Version on the Status-Line (see RFC2616 sections 3.1 and 6.1).
 2000-09-24 12:22:12 +00:00
des
ebf8e6bdbf Document the FTP_PROXY and HTTP_PROXY variables better. 2000-08-31 15:13:22 +00:00
des
98f58e5a22 Grok 125 replies to passive RETR. This fixes an interoperability bug with 
Microsoft FTP Service.

Reported by:	asmodai, eivind
 2000-08-21 07:18:31 +00:00
des
5f91d2827b Fix signedness bogon. 2000-08-02 11:25:21 +00:00
des
0247d400b1 Don't fail if _ftp_stat() reports a protocol error, since that most likely 
means that the server doesn't understand SIZE or MDTM, which should not be a
hard failure.

Submitted by:	ume
 2000-07-27 08:48:48 +00:00
des
e534b00520 If the server reports the size as 0, treat it as unknown. This works around 
a bug in some ftp servers (most notably ftp.vmunix.com) which report the
size of a file correctly in ascii mode, but report it as 0 in binary mode.

Reported by:	asmodai

Also remove an unneeded initialization.
 2000-07-25 14:41:02 +00:00
des
2de7b8d72b Centralize the default port finding code. 
Work around YA Apache bug: don't send port in Host: header if it's the
default port.
 2000-07-25 11:45:38 +00:00
des
accb02b55a Ignore environment variables that are set but empty. 
Sort out the size / length confusion.  Always try to report the *real* file
size in the url_stat structure, no matter how much of it is actually being
sent, and try to detect inconsistencies between sizes.

Rearrange the request loop to avoid having to add meaningless code just to
silence compiler warnings.

Switch to a more sensible and consistent interface for the _http_parse*()
functions.
 2000-07-21 11:02:43 +00:00
des
ef0d473b84 Document the proxy support better. 2000-07-21 10:57:15 +00:00
des
c99122fbde The decision to use an HTTP proxy for FTP documents belongs in the FTP code 2000-07-21 10:46:22 +00:00
des
2fa9607541 Don't send port number to proxy unless it's not the standard FTP port 2000-07-21 10:25:33 +00:00
des
a260febdf9 Fix the case where the file name is preceded by a lone slash. 2000-07-20 02:03:56 +00:00
des
adc026dc20 Don't try to skip to the requested offset if the server returns more data 
than requested.  Instead, inform the caller of the real offset by modifying
the offset field in the original struct url, and let him decide how to handle
the situation.
 2000-07-19 23:43:49 +00:00
des
c5e4329667 Don't take a failure to stat a file to mean that it doesn't exist, as it may 
be a dynamically created file, e.g. a tarball on servers that support sending
tarballs of directories.
 2000-07-18 07:12:26 +00:00
des
3033318809 Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and 
fetchStat*().  In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetchGet*() is identical to calling
fetchXGet*() with the second argument set to NULL.
 2000-07-17 21:25:00 +00:00
des
9e42d615d8 Don't forget to set file descriptor to -1 after closing it, since the code 
outside the loop inspects it to determine whether or not we succeeded in
retrieving the requested document. This fixes a bug where fetchGetHTTP()
would return a FILE with an invalid file descriptor if it hit the redirect
limit without locating the requested document.
 2000-07-17 21:09:11 +00:00
des
147a437a42 Use a symbolic constant instead of hardcoding 999 2000-07-17 21:00:49 +00:00
des
a0fbdfaa34 Don't forget to declare fetchRestartCalls. 2000-07-17 20:52:06 +00:00
des
c1df3803c6 Introduce the (undocumented) variable fetchRestartCalls, which controls whether 
or not interrupted system calls will be restarted. This fixes a bug where
fetch(1) would hang (potentially forever) if a server stopped responding,
because the signal handler would absorb the user's efforts to interrupt the
transfer.
 2000-07-17 20:49:39 +00:00
des
94b7364484 Remove fetchContentType(), which disappeared in the HTTP rewrite. 2000-07-17 20:42:13 +00:00
des
5996cf5b9e #ifdef DEBUG -> #ifndef NDEBUG 
Pointed out by:	ache
 2000-07-16 23:18:44 +00:00
des
436b76a904 Reinsert the MIT copyright, which applies to the base64 code and was 
left out by accident during the rewrite.
 2000-07-16 01:04:10 +00:00
des
797b7f6ddd Fix a bug (misplaced continue) that caused redirects to fail. Lots of code 
moved around, but the acutal functional changes are small.
Add support for site-internal redirects (where the Location: header gives a
path instead of an absolute URI)

Pointed out by:	kuriyama
 2000-07-13 09:13:58 +00:00
des
a7682e2874 Fully document the HTTP authorization feature. 
Update the BUGS section to reflect the HTTP rewrite.
 2000-07-12 10:41:29 +00:00
des
59adaf8069 Extensive rewrite of the HTTP code. The new code is significantly cleaner and 
more robust, and somewhat more efficient. It also handles authorization and
redirects properly, and supports timeouts like the FTP code.

Many thanks to Umemoto-san for his assistance with IPv6 support, both here
and in other parts of libfetch.
 2000-07-12 10:39:56 +00:00
des
d908f153d0 Use fetch_putln() 2000-07-11 23:50:22 +00:00
des
848e1688b8 Add _fetch_putln() 2000-07-11 23:50:08 +00:00
des
ce80758a3f Document that basic authorization now kinda works. 2000-07-11 18:13:08 +00:00
des
c42f438ce8 Rework the authorization code. 
Only send absolute URI if connected to a proxy, since Apache doesn't always
understand absolute URIs.
Clean up some of the debugging output.
 2000-07-11 18:12:41 +00:00
des
686d930615 Clean up the debugging output 2000-07-11 18:07:09 +00:00
ume
ae7af5f4bd Free chunks obtained by getaddrinfo(3). 
Approved by:	des
 2000-07-10 16:28:28 +00:00
ume
4f24653720 Make EPSV work again. Separate parsing code of 229 replies from 
the code for 227 and 228.

Submitted by:	des
 2000-07-10 08:41:25 +00:00
des
866a6bff9a Don't assume the remote address in a 227 reply is enclosed in parentheses. 2000-07-08 09:27:47 +00:00
des
9f7754e0e5 Fix basic authentication, and add proxy authentication. 
Submitted by:	se
 2000-07-08 08:08:58 +00:00
des
2f7d5cb4f5 Make restart work in active mode, too. 
PR:		bin/18688
Submitted by:	Rudolf Cejka <cejkar@dcse.fee.vutbr.cz>
 2000-06-29 10:44:10 +00:00
des
b112ea1321 Handle multiline replies properly, instead of kinda-right. 2000-06-28 15:48:26 +00:00
des
d6712064a8 Don't incorrectly report a protocol error when we get a 302 and redirects 
are disabled.
 2000-06-21 09:49:51 +00:00
des
720d162772 Back out wes' commit with extreme prejudice. 2000-06-20 13:57:00 +00:00
wes
425601774e Add implementation of fetchListFTP. 
Reviewed by:	silence
 2000-06-20 05:32:41 +00:00
des
96fd818f66 Hackish support for 302 redirect. 2000-05-26 15:34:42 +00:00
des
2a39533c00 Honor FTP_PASSIVE_MODE so caller doesn't need to. 2000-05-26 12:56:53 +00:00
des
969e1c4e0d Output newline after debugging message 2000-05-25 16:50:08 +00:00
des
59892af9b1 Add (and document) fetchMakeURL() 2000-05-25 16:46:09 +00:00
des
e5d7688f99 Dump com_err, it's a stinking crock of shit. 2000-05-25 16:24:31 +00:00
ume
79e112b26f Use $FTP_PASSWORD for FTP password. If $FTP_PASSWORD is not 
found, `yourname@yourhost' is used.
 2000-05-22 13:01:13 +00:00
ume
1db238984d IPv6 support. 2000-05-20 18:23:51 +00:00
des
4721396b1d Better handling of some boundary conditions. 
Submitted by:	ume
 2000-05-19 09:45:42 +00:00
ume
3194168ffc Make HTTP_PROXY work for FTP. 
Reported by:	Ben Smithurst <ben@scientia.demon.co.uk>
Reviewed by:	des
 2000-05-15 09:05:36 +00:00
des
e10535fbc8 Document struct url. 
Document the default values for fetchStat*().
 2000-05-15 08:34:42 +00:00
des
5d01e7fd74 Initialize the struct url_stat at the beginning of _fetch_stat_file(). 2000-05-15 08:33:58 +00:00
des
ffd6fec6c0 Initialize the struct url_stat at the beginning of fetchStatFTP(). 2000-05-15 08:24:58 +00:00
des
23a4c9053b Remove unused 'verbose'. 
Initialize the struct url_stat at the beginning of fetchStatHTTP().
 2000-05-15 08:24:29 +00:00
des
741229b49a Add workaround for ftpds with the Y2K MDTM bug 2000-05-11 16:01:03 +00:00
des
2d5449b6f7 Bump major number, since struct url has changed. 
Discussed with:	jdp
 2000-05-11 15:29:26 +00:00
des
1e1a123aa3 Fix hard sentence break. 
Submitted by:	sheldonh
 2000-05-11 15:22:52 +00:00