Commit Graph

190 Commits

Author SHA1 Message Date
Garrett Wollman
76dafb8954 Some bug-fixes, clean-ups, and one new feature:
- Fix the bug with URIs of the form ftp://host/filename.
- Fix some more string-termination bugs in util.c.
- Use safe_malloc() rather than testing the return value of
  regular malloc() in 15 places.
- Implement HTTP authentication, for both servers and proxies.
  Currently only ``basic'' authentication is supported; This Is A Bug
  (but less of one tjhan nmot supporting any authentication).

I think there is only one more feature which is required for full
HTTP/1.1 support, which is Transfer-Encoding: chunked; this should
not be toohard, but it isn't very important, either.
1997-02-05 19:59:18 +00:00
Bruce Evans
6525961b64 Null-terminate the string in percent_decode() so that we don't depend on
malloc() returning zeroed storage or dump core while starting up.

Found-by:	/etc/malloc.conf ->AJ
1997-02-02 09:16:37 +00:00
Garrett Wollman
2494e810b8 Some fixes for HTTP:
1) Implement redirects (or try to, at least).
2) Implement automatic retry after 503 errors when Retry-After is given.
3) Implement a -a flag to enable both of these behaviors.
4) Recognize Transfer-Encoding headers and emit a warning that the file
  is likely to be damaged.
5) Bug fix: only write the amount of data we read.
6) Actually document some of these.
7) Fix the usage message to display flags in semi-alphabetical order.
1997-01-31 19:55:51 +00:00
Garrett Wollman
78be319939 Here is my long-threatened revamping of fetch. Jean-Marc probably won't
recognize it any more.  This makes the following significant changes:

- The main body of the program doesn't know a thing about URIs,
  HTTP, or FTP.  This makes it possible to easily plug in other
  protocols.  (The next revision will probably be able to dynamically
  add new recognizers.)

- There are no longer arbitrary timeouts for the protocols.  If you want
  to set one for yourself, use the environment variables.

- FTP proxies are now supported (if I implemented it right).

- The HTTP implementation is much more complete, and can now do restarts,
  preserve modtimes, and mrun in mirror mode.  It's not yet up to 1.1,
  but it's getting there.

- Transaction TCP is now used for sending HTTP requests.  The HTTP/1.1 syntax
  for requesting that the connection be closed after one request is
  implemented.

In all of this, I have doubtless broken somebody.  Please test it and tell me
about the bugs.
1997-01-30 21:43:44 +00:00
Jordan K. Hubbard
ae0fde8e14 Handle timeouts in a slightly less baroque way. 1997-01-17 12:52:12 +00:00
Wolfram Schneider
da1ff3cb8a Sort cross references. 1997-01-15 23:25:55 +00:00
Jordan K. Hubbard
1130b656e5 Make the long-awaited change from $Id$ to $FreeBSD$
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore.  This update would have been
insane otherwise.
1997-01-14 07:20:47 +00:00
Andrey A. Chernov
528baa6dcc Report net connection error via hstrerror(h_errno) now instead of
simple fact that can't connect
1996-11-14 07:36:26 +00:00
Andrey A. Chernov
c7a626a567 Change to new ftpLogin interface
Now clearly say: Not logged in
instead of old: Broken pipe
(note it was original reason for all my libftpio changes)
1996-11-14 07:06:25 +00:00
Peter Wemm
e721a581cb Cosmetic bugfix. fetch was modifying it's argv[] strings in place, which
caused ps(1) to show strange things..
1996-11-10 14:46:50 +00:00
Poul-Henning Kamp
5745420bb0 bin/1933: don't print negative % for huge files (80Mb) 1996-10-31 14:24:35 +00:00
Adam David
bf2226a7db old host:/filename syntax was broken
implement /filename syntax for completeness
1996-10-24 00:15:44 +00:00
Jean-Marc Zucconi
b84aaff96e The -h flag implies ftp = 1. 1996-10-06 00:44:24 +00:00
Wolfram Schneider
e0e5145ce6 add missing comma(s) in .Xr macros 1996-09-23 22:24:39 +00:00
Peter Wemm
a43a248c32 Attempt to untangle the timeout code a bit, also make the default ftp
and http timeouts the same, since when using a http proxy to do ftp
transfers, the http timeout was being used for what is coming in via
ftp.
1996-09-19 18:07:24 +00:00
Peter Wemm
02fa2cc705 Make fetch 64-bit safe, corresponding to libftpio changes.
Submitted by: Jason Thorpe <thorpej@nas.nasa.gov>, PR#1641
1996-09-19 17:31:34 +00:00
Jordan K. Hubbard
f7c176c92b Remove the code which has restart looking at the modtime. That's just
moronic!  We don't care about the time in restart mode, only the size.
It's *mirror* mode which cares about modtimes.
1996-09-10 19:49:41 +00:00
Jordan K. Hubbard
7ae26f1450 Allow proper ftp verbosity with a new -v flag. 1996-08-31 22:03:05 +00:00
Jordan K. Hubbard
89fc51f330 Don't make -l warn when it's not used, so we can potentially set it
all the time if we want to use "link" behavior for file URLs (and not
have fetch whine at us when they're not).
1996-08-23 06:21:17 +00:00
Mike Pritchard
85cf659a76 Use the .Fx macro where appropriate. 1996-08-23 00:57:08 +00:00
Jordan K. Hubbard
f0abcb4154 Print the correct thing when interrupted by a signal. 1996-08-22 23:25:24 +00:00
Jordan K. Hubbard
fc6fcbf9fc 1. Understand file:/path style URLs.
2. Add a -l flag for symlinking to rather than copying file:/path style URLs.
3. Add a -T flag for setting the timeout interval (overrides FTP_TIMEOUT if set)
1996-08-22 21:30:51 +00:00
Jordan K. Hubbard
608051dc0d Fix a small potential bogon I noticed here with restarts. 1996-08-21 01:27:25 +00:00
Jordan K. Hubbard
477a3f5488 Add support for HTTP proxies, fix some bugs with http transfers.
Submitted-By: Mikael Hybsch <micke@dynas.se>
1996-08-14 17:50:23 +00:00
Jordan K. Hubbard
f2b2a91e75 Fix broken modtime preservation code. *Now* it preserves the modtime
correctly!
Noticed-By: knarf@camelot.de (Frank Bartels)
1996-08-12 12:55:26 +00:00
Jordan K. Hubbard
c668d40c24 Defer to fashion. 1996-08-07 02:15:26 +00:00
Jean-Marc Zucconi
d616694a11 Suppress the messages from the server after login (ftpVerbose(ftp,0)). Errors
are already correctly reported.
1996-08-04 00:35:39 +00:00
Jordan K. Hubbard
390a3f8faa Adapt to new libftpio. [jkh]
Better error messages.
Submitted-By: "David E. O'Brien" <obrien@Nuxi.cs.ucdavis.edu>
1996-08-03 13:14:47 +00:00
Jordan K. Hubbard
e0607117a5 Whups - update the usage message too. 1996-07-18 00:08:58 +00:00
Jordan K. Hubbard
6b5c2db22a Add -n flag to use the current modtime rather than preserving it. 1996-07-18 00:08:02 +00:00
Jordan K. Hubbard
7fefc698ab New man page for fetch(1).
Submitted by: jmz (with some small tweaks)
1996-07-05 01:03:20 +00:00
Jean-Marc Zucconi
4d705fe0f6 Back out previous commit: do not use read(), but fread (). Change BUFFER_SIZE
to 1K to avoid waiting too long between 2 progress reports in the case of a
slow transfer.
Move the signal()/setitimer() code just before the read loop because the
ftpio library set and reset the timer internally.
1996-07-05 00:06:36 +00:00
Jean-Marc Zucconi
144e75837e Use read(fileno(fp), ...) instead of fread(..., fp) to avoid buffered input. 1996-07-02 01:49:47 +00:00
Gary Palmer
84b54bb498 Add -Wall to CFLAGS 1996-06-26 20:49:41 +00:00
Andrey A. Chernov
1f1b7c85d3 Make GET request and response parsing HTTP/1.0 compatible 1996-06-25 21:33:18 +00:00
Jean-Marc Zucconi
ca3eb98721 Do not exit if ftpGetSize() returns a value < 0. You may well try to
fetch a whole <directory>.tar.gz on a server running wu-ftp.
1996-06-24 01:03:05 +00:00
Jordan K. Hubbard
35740f4fe3 Adjust the copyright to match reality, remove printing of gratuitous newline. 1996-06-22 23:24:13 +00:00
Jordan K. Hubbard
80951c9d5d Clean up more bogons:
1. Add a copyright at the top.
	2. Make passive and binary modes work with new ftpio semantics.
	3. Add a `mirror' (-m) flag which only copies the remote file if it's
	   not the same size or is newer.
1996-06-22 21:42:14 +00:00
Jordan K. Hubbard
bad91cabaf Address Andrey's complaints:
1. Always use file modtime, and if that's not usable then default
	   to current time.
	2. Allow HTTP_TIMEOUT to be set as an environment variable to override
	   default.
	3. Handle SIGTERM.
1996-06-22 20:16:57 +00:00
Satoshi Asami
5e5a2c7fb1 Jean-Marc's url fetch program, with Josh MacDonald's patches and
Jordan's ftpio library.

Submitted by:	jmz, jkh, jmacd (three-j!)
1996-06-19 09:32:11 +00:00