Commit Graph

188 Commits

Author SHA1 Message Date
wollman
827e39354a Some fixes for HTTP:
1) Implement redirects (or try to, at least).
2) Implement automatic retry after 503 errors when Retry-After is given.
3) Implement a -a flag to enable both of these behaviors.
4) Recognize Transfer-Encoding headers and emit a warning that the file
  is likely to be damaged.
5) Bug fix: only write the amount of data we read.
6) Actually document some of these.
7) Fix the usage message to display flags in semi-alphabetical order.
1997-01-31 19:55:51 +00:00
wollman
444cbb04ee Here is my long-threatened revamping of fetch. Jean-Marc probably won't
recognize it any more.  This makes the following significant changes:

- The main body of the program doesn't know a thing about URIs,
  HTTP, or FTP.  This makes it possible to easily plug in other
  protocols.  (The next revision will probably be able to dynamically
  add new recognizers.)

- There are no longer arbitrary timeouts for the protocols.  If you want
  to set one for yourself, use the environment variables.

- FTP proxies are now supported (if I implemented it right).

- The HTTP implementation is much more complete, and can now do restarts,
  preserve modtimes, and mrun in mirror mode.  It's not yet up to 1.1,
  but it's getting there.

- Transaction TCP is now used for sending HTTP requests.  The HTTP/1.1 syntax
  for requesting that the connection be closed after one request is
  implemented.

In all of this, I have doubtless broken somebody.  Please test it and tell me
about the bugs.
1997-01-30 21:43:44 +00:00
jkh
b917141478 Handle timeouts in a slightly less baroque way. 1997-01-17 12:52:12 +00:00
wosch
4bcfb053ec Sort cross references. 1997-01-15 23:25:55 +00:00
jkh
808a36ef65 Make the long-awaited change from $Id$ to $FreeBSD$
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore.  This update would have been
insane otherwise.
1997-01-14 07:20:47 +00:00
ache
7a2c703c2b Report net connection error via hstrerror(h_errno) now instead of
simple fact that can't connect
1996-11-14 07:36:26 +00:00
ache
c9c621c36a Change to new ftpLogin interface
Now clearly say: Not logged in
instead of old: Broken pipe
(note it was original reason for all my libftpio changes)
1996-11-14 07:06:25 +00:00
peter
fb2d026372 Cosmetic bugfix. fetch was modifying it's argv[] strings in place, which
caused ps(1) to show strange things..
1996-11-10 14:46:50 +00:00
phk
7c5a2b76a6 bin/1933: don't print negative % for huge files (80Mb) 1996-10-31 14:24:35 +00:00
adam
f3ebc6792c old host:/filename syntax was broken
implement /filename syntax for completeness
1996-10-24 00:15:44 +00:00
jmz
6772418485 The -h flag implies ftp = 1. 1996-10-06 00:44:24 +00:00
wosch
89dea7c61d add missing comma(s) in .Xr macros 1996-09-23 22:24:39 +00:00
peter
43f6021163 Attempt to untangle the timeout code a bit, also make the default ftp
and http timeouts the same, since when using a http proxy to do ftp
transfers, the http timeout was being used for what is coming in via
ftp.
1996-09-19 18:07:24 +00:00
peter
ae3b6a683b Make fetch 64-bit safe, corresponding to libftpio changes.
Submitted by: Jason Thorpe <thorpej@nas.nasa.gov>, PR#1641
1996-09-19 17:31:34 +00:00
jkh
ccd8fb4fad Remove the code which has restart looking at the modtime. That's just
moronic!  We don't care about the time in restart mode, only the size.
It's *mirror* mode which cares about modtimes.
1996-09-10 19:49:41 +00:00
jkh
2ae01aec12 Allow proper ftp verbosity with a new -v flag. 1996-08-31 22:03:05 +00:00
jkh
7b0b861fef Don't make -l warn when it's not used, so we can potentially set it
all the time if we want to use "link" behavior for file URLs (and not
have fetch whine at us when they're not).
1996-08-23 06:21:17 +00:00
mpp
b4aab8b6f6 Use the .Fx macro where appropriate. 1996-08-23 00:57:08 +00:00
jkh
e9dba4b462 Print the correct thing when interrupted by a signal. 1996-08-22 23:25:24 +00:00
jkh
6e3ba357ba 1. Understand file:/path style URLs.
2. Add a -l flag for symlinking to rather than copying file:/path style URLs.
3. Add a -T flag for setting the timeout interval (overrides FTP_TIMEOUT if set)
1996-08-22 21:30:51 +00:00
jkh
3b73be8498 Fix a small potential bogon I noticed here with restarts. 1996-08-21 01:27:25 +00:00
jkh
4659e0fe44 Add support for HTTP proxies, fix some bugs with http transfers.
Submitted-By: Mikael Hybsch <micke@dynas.se>
1996-08-14 17:50:23 +00:00
jkh
633999a94b Fix broken modtime preservation code. *Now* it preserves the modtime
correctly!
Noticed-By: knarf@camelot.de (Frank Bartels)
1996-08-12 12:55:26 +00:00
jkh
3ed03544cf Defer to fashion. 1996-08-07 02:15:26 +00:00
jmz
1e79f06390 Suppress the messages from the server after login (ftpVerbose(ftp,0)). Errors
are already correctly reported.
1996-08-04 00:35:39 +00:00
jkh
5f947203c5 Adapt to new libftpio. [jkh]
Better error messages.
Submitted-By: "David E. O'Brien" <obrien@Nuxi.cs.ucdavis.edu>
1996-08-03 13:14:47 +00:00
jkh
df5a608d70 Whups - update the usage message too. 1996-07-18 00:08:58 +00:00
jkh
dc8c550225 Add -n flag to use the current modtime rather than preserving it. 1996-07-18 00:08:02 +00:00
jkh
3bfb590a22 New man page for fetch(1).
Submitted by: jmz (with some small tweaks)
1996-07-05 01:03:20 +00:00
jmz
557b0ffde9 Back out previous commit: do not use read(), but fread (). Change BUFFER_SIZE
to 1K to avoid waiting too long between 2 progress reports in the case of a
slow transfer.
Move the signal()/setitimer() code just before the read loop because the
ftpio library set and reset the timer internally.
1996-07-05 00:06:36 +00:00
jmz
64422261a0 Use read(fileno(fp), ...) instead of fread(..., fp) to avoid buffered input. 1996-07-02 01:49:47 +00:00
gpalmer
b99c167c70 Add -Wall to CFLAGS 1996-06-26 20:49:41 +00:00
ache
c9f677c96f Make GET request and response parsing HTTP/1.0 compatible 1996-06-25 21:33:18 +00:00
jmz
85adcb8a3f Do not exit if ftpGetSize() returns a value < 0. You may well try to
fetch a whole <directory>.tar.gz on a server running wu-ftp.
1996-06-24 01:03:05 +00:00
jkh
694ef53b8b Adjust the copyright to match reality, remove printing of gratuitous newline. 1996-06-22 23:24:13 +00:00
jkh
36706f9cf0 Clean up more bogons:
1. Add a copyright at the top.
	2. Make passive and binary modes work with new ftpio semantics.
	3. Add a `mirror' (-m) flag which only copies the remote file if it's
	   not the same size or is newer.
1996-06-22 21:42:14 +00:00
jkh
7b161df218 Address Andrey's complaints:
1. Always use file modtime, and if that's not usable then default
	   to current time.
	2. Allow HTTP_TIMEOUT to be set as an environment variable to override
	   default.
	3. Handle SIGTERM.
1996-06-22 20:16:57 +00:00
asami
b0bc5b5357 Jean-Marc's url fetch program, with Josh MacDonald's patches and
Jordan's ftpio library.

Submitted by:	jmz, jkh, jmacd (three-j!)
1996-06-19 09:32:11 +00:00