Dag-Erling Smørgrav
aa4b3574a8
Rework the stats code to avoid code duplication.
...
Fix a bug where the stats code would print the expected size instead of the
number of bytes received.
Fix the reading code so it'll support partial reads.
2000-07-12 08:29:52 +00:00
Dag-Erling Smørgrav
ec850e74c0
Don't segfault if given an invalid URL (doh!)
...
Submitted by: green
2000-07-09 10:14:22 +00:00
Dag-Erling Smørgrav
0b474bc608
Clarify the meaning of -m and -n (thanks to bde).
2000-07-08 09:34:53 +00:00
Dag-Erling Smørgrav
bb11a878ea
Straighten out the behvaiour of -m and -n (thanks to bde).
...
Check that stderr, not stdout, is a tty (thanks to green).
2000-07-08 09:34:33 +00:00
Dag-Erling Smørgrav
114c3c1ab8
Add MIT copyright
2000-07-05 12:04:31 +00:00
Dag-Erling Smørgrav
047843dd61
Note that the -h, -c and -f options are deprecated, and remove note that
...
incorrectly stated that they were not implemented.
Document the -d option.
2000-06-29 10:35:24 +00:00
Dag-Erling Smørgrav
a8369cd91b
Implement the -c option
2000-06-29 10:32:56 +00:00
Dag-Erling Smørgrav
df6f33d182
Don't forget to delete the output file if the request fails.
...
Don't delete the output file if -r was specified.
2000-06-29 08:39:29 +00:00
Dag-Erling Smørgrav
b39628e763
New libfetch-based fetch.
2000-06-28 16:55:15 +00:00
Jake Burkholder
e39756439c
Back out the previous change to the queue(3) interface.
...
It was not discussed and should probably not happen.
Requested by: msmith and others
2000-05-26 02:09:24 +00:00
Jake Burkholder
740a1973a6
Change the way that the queue(3) structures are declared; don't assume that
...
the type argument to *_HEAD and *_ENTRY is a struct.
Suggested by: phk
Reviewed by: phk
Approved by: mdodd
2000-05-23 20:41:01 +00:00
Martin Cracauer
301cba219f
Print a warning and exit with != 0 when at least one downloaded file
...
is shorter than previously announced by the server.
Tested by asami.
Approved by: jkh
2000-03-08 13:02:11 +00:00
Sheldon Hearn
87faa07bec
Remove single-space hard sentence breaks. These degrade the quality
...
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.
2000-03-01 12:20:22 +00:00
Brian Feldman
d6a8296aed
Add the -F option. This is for forcing restarts with -r by inhibiting
...
transmission of the If-Range HTTP header field.
1999-10-23 15:22:53 +00:00
Peter Wemm
c3aac50f28
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
Brian Feldman
90331af0d2
"transfered" -> "transferred"
...
Reviewed by: dict(1)
1999-07-02 08:57:06 +00:00
Brian Feldman
378949dbca
This fixes the problem of cancelling an FTP transfer resulting in
...
a bogus error ("parsing URI".) I also fixed an inconsistency in
another fs_status.
1999-06-21 02:38:16 +00:00
John-Mark Gurney
85020e14f0
make md5 checksums actually work instead of simply calculating the length
...
of an empty buffer... the output file wasn't readable... also warn that
we can't checksum on stdout and print out the base64 encoded version of the
md5 checksum...
Site to actually return md5 digest: web.golux.com
Verified that fetch was broken: Ken Coar <Ken.Coar@Golux.Com>
1999-05-18 19:37:37 +00:00
Garrett Wollman
c7ede78e51
Restore \n at end of usage message.
...
Complained-about-by: charnier
1999-03-08 19:16:14 +00:00
Alexander Langer
c5e72606a7
Improve error message wording when attempting to link to a non-existent
...
file on the local host.
PR: 10042
Submitted by: Chris Costello <phoenix@calldei.com>
1999-03-02 04:14:33 +00:00
Garrett Wollman
189da04438
Deal with broken Web sites which return 302 responses rather than 404
...
and an error document when the requested resource does not exist. Grrr.
Requested by: asami
1999-02-23 18:51:13 +00:00
Bill Fenner
d17bf266c4
Remove the FTP_PASSIVE_MODE "fix"; libftpio handles this.
1999-02-05 01:01:17 +00:00
Bill Fenner
054672c795
Warn about collapsing multiple slashes into 1 in ftp URL's.
...
Look at the FTP_PASSIVE_MODE environment variable like the man page says.
PR: bin/9464
Submitted by: John A. Shue <John.Shue@symmetron.com>
Add references to RFC's 1790, 959, 850.
PR: doc/6564
1999-02-03 20:43:29 +00:00
Bill Fenner
93eb50393d
Don't try to parse a colon in a URL as a port
...
(e.g. http://www.host.name/foo:bar )
PR: bin/5072
Submitted by: Takeshi WATANABE <watanabe@komadori.planet.kobe-u.ac.jp>
1999-02-03 20:24:53 +00:00
Garrett Wollman
9caa4ddf6b
Fix logic error in RFC 850 kluge.
1999-01-15 17:10:31 +00:00
Garrett Wollman
c146843096
For RFC 850 dates received in HTTP responses, implement the century pivot
...
described in RFC 2068. Include a reference to same in the manual page.
1999-01-15 16:56:22 +00:00
Martin Cracauer
b451f4524e
Add -s option, just report size of file that would be fetched.
...
Reviewed by: -current list
1998-12-08 13:00:49 +00:00
Dag-Erling Smørgrav
9f63f2aa1c
main.c:
...
Fix usage string. Fix getopt() string. Fix ordering of compatibility options.
fetch.1:
Fix synopsis. Fix ordering of T and t options. Fix minor grammar nit.
1998-11-08 23:18:48 +00:00
Bill Fenner
60a53b7d32
If we know the content-length, only read that number of bytes from
...
the server. There exists a broken server which sends a few extra
garbage bytes in response to HTTP/1.1 requests.
1998-10-26 02:39:21 +00:00
Jordan K. Hubbard
0637c2cccd
Add -S flag; this is benign since fetch's behavior is unchanged if you
...
don't use it. That's why I'm bringing it in during our "code slush"
Submitted by: Stefan Esser <se@mi.uni-koeln.de>
1998-09-20 00:01:26 +00:00
Dag-Erling Smørgrav
82e3dde754
DTRT for file: URLs when output file is "-".
...
PR: bin/7203
1998-09-17 00:15:15 +00:00
Stefan Eßer
cc522880b4
Fix proxy authorization code:
...
- failed to use authorization parameters passed in the environment, if
/dev/tty could not be opened (i.e. if running from cron)
- mixed use of /dev/tty and stdin for prompt and reading of the result
1998-07-12 09:07:36 +00:00
Garrett Wollman
cffe96c903
Fix my last change: the .Fx macro doesn't do anything useful without an
...
argument, so use `.Tn FreeBSD' instead.
1998-05-09 21:39:49 +00:00
Garrett Wollman
eeed58d824
Documentation fix: delete references to T/TCP as they are not relevant
...
to the case in question. (Said case being quite clearly defined by
the basic TCP specification, RFC 792.)
1998-05-09 20:50:37 +00:00
Dag-Erling Smørgrav
49778f39ac
Arrggghhhh... forgot to strip the / after outputting %2f
1998-05-09 08:56:07 +00:00
Dag-Erling Smørgrav
757c6582c1
Map a leading / in -c argument to %2f instead of dropping it silently.
...
PR: bin/6558
1998-05-09 08:41:23 +00:00
Bill Fenner
5e92200712
Describe absolute paths in FTP URL's. This patch has been sitting
...
around uncommitted for 2 months =(
1998-05-09 07:33:09 +00:00
Philippe Charnier
306005e78c
.Sh AUTHOR -> .Sh AUTHORS. Use .An/.Aq.
1998-03-23 07:48:45 +00:00
John Birrell
5362bb64c5
gcc wants to see long long variables passed to %q printf formats, so
...
cast to that instead of quad_t.
1998-02-20 05:11:42 +00:00
John Birrell
2ffc763234
time() needs a pointer to a time_t, but tv_sec in a timeval is a
...
long (yuk). So give time() what it wants and let the compiler
promote the variable when it is assigned to tv_sec.
1998-02-20 05:08:53 +00:00
Bill Fenner
18d1eb6776
Suggest using "-b" or "-t" if the connection is reset (typical symptom
...
of broken TCP stack).
1997-11-22 01:00:42 +00:00
John Polstra
a5bd62c599
Add missing arguments detected by "-Wformat".
1997-11-18 03:27:34 +00:00
David E. O'Brien
e3f8c97942
style-police got me. :-)
...
Also fixed some "didn't make the most sense" problems.
Submitted by: bde
1997-11-12 04:39:33 +00:00
David E. O'Brien
e4b172e407
If "-l" is given, make sure the target of the link exist.
1997-11-08 22:15:55 +00:00
Andrey A. Chernov
9e8930055e
Change diags to see what part of url actually parsed, see
...
previous change about #? parms
1997-11-01 05:47:41 +00:00
Andrey A. Chernov
4ad97ec183
Don't search for ':' port in ?# params part
...
Require to correctly parse http://...cgi?http:// ...
1997-10-31 23:33:44 +00:00
Bill Fenner
21d466b21a
Interpret multiple slashes as a single slash in ftp: URL's. This
...
is contrary to RFC1738, which specifies that empty elements between
slashes really mean to send a CWD command with an empty argument.
Most FTP servers consider this an error, so strict RFC compliance
is advisable in this case.
1997-10-08 18:43:53 +00:00
Bill Fenner
a5cd8a3572
Use the RFC1738 interpretation of ftp: URL's, i.e. CWD to each
...
slash-seperated element of the URL and then RETR the last element.
PR: bin/4670
Reviewed by: wollman
1997-10-06 01:09:56 +00:00
Martin Cracauer
3e27c094e4
fetch(3) doesn't get asctime(3) format, wrong length assumed
...
PR: bin/4625
Submitted by: "Timo J. Rinne" <tri@pooh.tky.hut.fi>
Obtained from:bin/4625
1997-09-28 11:25:59 +00:00
Satoshi Asami
e5ed1f5994
Be consistent with other utilities in spelling "Kbytes". (Actually,
...
there even was one of them about ten lines above in the same file.)
Reminded by: joerg's commit to usr.bin/ftp/ftp.c rev 1.9
1997-08-19 01:57:53 +00:00
Andrey A. Chernov
f4e4504f06
Add -t option which turns T/TCP off as workaround for some broken servers
...
Submitted by: Marc Slemko <marcs@znep.com>
1997-08-05 20:18:39 +00:00
Garrett Wollman
12b7829739
Work around a brokenness in the HTTP spec by generating the Host header
...
for ftp: URLs as well. This can't possibly be done in the general case,
but since we only claim to support http and ftp, we'll adhere to the
bogus requirement (RFC 2068, s. 14.25) anyway... >sigh<
Submitted by: =?iso-8859-1?Q?=C5ge_R=F8bekk?= <aagero@aage.priv.no>
1997-07-26 20:18:43 +00:00
Garrett Wollman
efd4ad4cd2
Fix the error message for when mirroring and restarting are rejected
...
due to the output not being a regular file. Also split the error message
in any error case here to be on two lines to lessen the likelihood of it
being too long to fit on just one.
1997-07-26 20:00:05 +00:00
Garrett Wollman
2ec04c7076
Implement HTTP 1.1's ``chunked'' Transfer-Encoding (ick). This hasn't
...
been extensively tested, but I now can successfully retrieve
<http://www.apache.org/index.html >, so I guess that's a victory of some
sort.
Also move the initialization of ``autherror'' to hopefully eliminate
the reported loop involving authentication. Still need to implement
MD5 digest authentication.
1997-07-26 19:25:56 +00:00
Garrett Wollman
0c8cc99e5b
Provide a new `-b' flag to work around some broken HTTP/TCP implementations
...
that can't deal with a half-closed connection.
1997-07-25 19:35:44 +00:00
Philippe Charnier
b28977d169
One newline should be enough.
1997-07-02 06:28:32 +00:00
Philippe Charnier
0bd47384ea
Do not use argv[0] in usage().
1997-07-01 06:37:34 +00:00
Andrey A. Chernov
5e04857ec1
Now understand password in ftp:// URLs
1997-05-31 14:45:41 +00:00
John-Mark Gurney
27e0fa20aa
fix a bug in processing of FTP_TIMEOUT environment variable.
...
Closes PR#2947
Submitted-by: Peter Childs <pjchilds@imforei.apana.org.au>
1997-03-11 15:13:28 +00:00
John-Mark Gurney
817eca811d
fix restarting http transfers and understanding response from server.
...
Slight modification of patch by wollman.
Closes PR#2870
Reviewed by: wollman
Submitted by: Marc Slemko <marcs@znep.com>
1997-03-06 20:01:32 +00:00
John-Mark Gurney
c1599df8b3
fix a couple problems with fetch:
...
. don't try to interpet a colon in the pathname as a port number
. don't report an errno message when one don't exist
1997-03-06 10:01:54 +00:00
Bill Fenner
8a2f8e395a
Fix FTP_PROXY to use user@host[@port] for FTP proxy and eliminate
...
undocumented FTP_PROXY_USER
Make FTP file errors contian hostname and path.
Pass the FTP port to libftp.
Partially un-HTMLify error messages returned from server
Handle "HTTP NNN" instead of "HTTP/V.vv NNN" response sent by
pre-HTTP/1.0 servers
Reviewed by: wollman
1997-03-05 18:57:16 +00:00
Wolfram Schneider
bc5d5816e7
These man page will appear in 2.2, not 3.0.
1997-02-22 23:43:32 +00:00
Peter Wemm
c115df18cd
Revert $FreeBSD$ to $Id$
1997-02-22 19:58:13 +00:00
John-Mark Gurney
429100430f
Fix fetch so that all status information is sent to stderr, prevents
...
corruption of file being recieved to stdout.
Submitted-by: Kent Vander Velden <graphix@iastate.edu>
Closes PR#2739
1997-02-17 04:11:47 +00:00
John-Mark Gurney
88e8266d80
Modified the display() function to recognize when the output it going to a
...
file. When it is don't do the percentage update, but still display the file
size and rate.
Closes PR#2725
Reviewed by: wollman
1997-02-14 19:08:18 +00:00
Garrett Wollman
857292949c
Fix remote request for HTTP proxies. Should close PR#2670.
1997-02-11 20:46:06 +00:00
Garrett Wollman
cb0cd746b5
Fix PR#2700: report correct transmission speed, and restart transfers
...
from the restart point, not the end of the file (blush!).
Submitted by: John-Mark Gurney <jmg@nike.efn.org>
1997-02-10 18:49:42 +00:00
Garrett Wollman
df6c4acc3c
Fix off-by-one error in proxy host:port parser.
...
Submitted by: fenner
1997-02-07 17:55:01 +00:00
Garrett Wollman
76dafb8954
Some bug-fixes, clean-ups, and one new feature:
...
- Fix the bug with URIs of the form ftp://host/filename.
- Fix some more string-termination bugs in util.c.
- Use safe_malloc() rather than testing the return value of
regular malloc() in 15 places.
- Implement HTTP authentication, for both servers and proxies.
Currently only ``basic'' authentication is supported; This Is A Bug
(but less of one tjhan nmot supporting any authentication).
I think there is only one more feature which is required for full
HTTP/1.1 support, which is Transfer-Encoding: chunked; this should
not be toohard, but it isn't very important, either.
1997-02-05 19:59:18 +00:00
Bruce Evans
6525961b64
Null-terminate the string in percent_decode() so that we don't depend on
...
malloc() returning zeroed storage or dump core while starting up.
Found-by: /etc/malloc.conf ->AJ
1997-02-02 09:16:37 +00:00
Garrett Wollman
2494e810b8
Some fixes for HTTP:
...
1) Implement redirects (or try to, at least).
2) Implement automatic retry after 503 errors when Retry-After is given.
3) Implement a -a flag to enable both of these behaviors.
4) Recognize Transfer-Encoding headers and emit a warning that the file
is likely to be damaged.
5) Bug fix: only write the amount of data we read.
6) Actually document some of these.
7) Fix the usage message to display flags in semi-alphabetical order.
1997-01-31 19:55:51 +00:00
Garrett Wollman
78be319939
Here is my long-threatened revamping of fetch. Jean-Marc probably won't
...
recognize it any more. This makes the following significant changes:
- The main body of the program doesn't know a thing about URIs,
HTTP, or FTP. This makes it possible to easily plug in other
protocols. (The next revision will probably be able to dynamically
add new recognizers.)
- There are no longer arbitrary timeouts for the protocols. If you want
to set one for yourself, use the environment variables.
- FTP proxies are now supported (if I implemented it right).
- The HTTP implementation is much more complete, and can now do restarts,
preserve modtimes, and mrun in mirror mode. It's not yet up to 1.1,
but it's getting there.
- Transaction TCP is now used for sending HTTP requests. The HTTP/1.1 syntax
for requesting that the connection be closed after one request is
implemented.
In all of this, I have doubtless broken somebody. Please test it and tell me
about the bugs.
1997-01-30 21:43:44 +00:00
Jordan K. Hubbard
ae0fde8e14
Handle timeouts in a slightly less baroque way.
1997-01-17 12:52:12 +00:00
Wolfram Schneider
da1ff3cb8a
Sort cross references.
1997-01-15 23:25:55 +00:00
Jordan K. Hubbard
1130b656e5
Make the long-awaited change from $Id$ to $FreeBSD$
...
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.
Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.
1997-01-14 07:20:47 +00:00
Andrey A. Chernov
528baa6dcc
Report net connection error via hstrerror(h_errno) now instead of
...
simple fact that can't connect
1996-11-14 07:36:26 +00:00
Andrey A. Chernov
c7a626a567
Change to new ftpLogin interface
...
Now clearly say: Not logged in
instead of old: Broken pipe
(note it was original reason for all my libftpio changes)
1996-11-14 07:06:25 +00:00
Peter Wemm
e721a581cb
Cosmetic bugfix. fetch was modifying it's argv[] strings in place, which
...
caused ps(1) to show strange things..
1996-11-10 14:46:50 +00:00
Poul-Henning Kamp
5745420bb0
bin/1933: don't print negative % for huge files (80Mb)
1996-10-31 14:24:35 +00:00
Adam David
bf2226a7db
old host:/filename syntax was broken
...
implement /filename syntax for completeness
1996-10-24 00:15:44 +00:00
Jean-Marc Zucconi
b84aaff96e
The -h flag implies ftp = 1.
1996-10-06 00:44:24 +00:00
Wolfram Schneider
e0e5145ce6
add missing comma(s) in .Xr macros
1996-09-23 22:24:39 +00:00
Peter Wemm
a43a248c32
Attempt to untangle the timeout code a bit, also make the default ftp
...
and http timeouts the same, since when using a http proxy to do ftp
transfers, the http timeout was being used for what is coming in via
ftp.
1996-09-19 18:07:24 +00:00
Peter Wemm
02fa2cc705
Make fetch 64-bit safe, corresponding to libftpio changes.
...
Submitted by: Jason Thorpe <thorpej@nas.nasa.gov>, PR#1641
1996-09-19 17:31:34 +00:00
Jordan K. Hubbard
f7c176c92b
Remove the code which has restart looking at the modtime. That's just
...
moronic! We don't care about the time in restart mode, only the size.
It's *mirror* mode which cares about modtimes.
1996-09-10 19:49:41 +00:00
Jordan K. Hubbard
7ae26f1450
Allow proper ftp verbosity with a new -v flag.
1996-08-31 22:03:05 +00:00
Jordan K. Hubbard
89fc51f330
Don't make -l warn when it's not used, so we can potentially set it
...
all the time if we want to use "link" behavior for file URLs (and not
have fetch whine at us when they're not).
1996-08-23 06:21:17 +00:00
Mike Pritchard
85cf659a76
Use the .Fx macro where appropriate.
1996-08-23 00:57:08 +00:00
Jordan K. Hubbard
f0abcb4154
Print the correct thing when interrupted by a signal.
1996-08-22 23:25:24 +00:00
Jordan K. Hubbard
fc6fcbf9fc
1. Understand file:/path style URLs.
...
2. Add a -l flag for symlinking to rather than copying file:/path style URLs.
3. Add a -T flag for setting the timeout interval (overrides FTP_TIMEOUT if set)
1996-08-22 21:30:51 +00:00
Jordan K. Hubbard
608051dc0d
Fix a small potential bogon I noticed here with restarts.
1996-08-21 01:27:25 +00:00
Jordan K. Hubbard
477a3f5488
Add support for HTTP proxies, fix some bugs with http transfers.
...
Submitted-By: Mikael Hybsch <micke@dynas.se>
1996-08-14 17:50:23 +00:00
Jordan K. Hubbard
f2b2a91e75
Fix broken modtime preservation code. *Now* it preserves the modtime
...
correctly!
Noticed-By: knarf@camelot.de (Frank Bartels)
1996-08-12 12:55:26 +00:00
Jordan K. Hubbard
c668d40c24
Defer to fashion.
1996-08-07 02:15:26 +00:00
Jean-Marc Zucconi
d616694a11
Suppress the messages from the server after login (ftpVerbose(ftp,0)). Errors
...
are already correctly reported.
1996-08-04 00:35:39 +00:00
Jordan K. Hubbard
390a3f8faa
Adapt to new libftpio. [jkh]
...
Better error messages.
Submitted-By: "David E. O'Brien" <obrien@Nuxi.cs.ucdavis.edu>
1996-08-03 13:14:47 +00:00
Jordan K. Hubbard
e0607117a5
Whups - update the usage message too.
1996-07-18 00:08:58 +00:00
Jordan K. Hubbard
6b5c2db22a
Add -n flag to use the current modtime rather than preserving it.
1996-07-18 00:08:02 +00:00
Jordan K. Hubbard
7fefc698ab
New man page for fetch(1).
...
Submitted by: jmz (with some small tweaks)
1996-07-05 01:03:20 +00:00