d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
155,195 Commits 4 Branches 49 Tags 2 GiB
dbe6a58879
Commit Graph

228 Commits

Author SHA1 Message Date
des
4a3e9f26cb Add -N option to specify a netrc file. 
Fix handling of -v option.
Don't treat negative offsets as valid positive ones.
Clean up the ETA and transfer rate code.  Show transfer rate along with
ETA if the verbose level is higher than 1.
 2003-01-22 18:33:39 +00:00
des
3eb14891e1 Use the URL instead of the output file name when reporting errors relating 
to the remote document.

PR:		33856
MFC after:	1 week
Approved by:	re
 2002-11-27 20:52:07 +00:00
des
57b4b0e512 Don't set atime / mtime if we don't have an output file, which can happen 
in certain failure scenarii (mostly DNS trouble).
 2002-11-13 16:04:20 +00:00
fenner
73fa7c5c52 Don't divide by zero to calculate the ETA, even if we haven't gotten 
any data in 30 seconds.
 2002-11-12 07:09:43 +00:00
fenner
abf784c214 Don't exit with success after timeout during authentication. 2002-11-07 22:16:45 +00:00
des
deb00535da Clarify my feelings towards fetch / libfetch. 2002-10-30 04:53:58 +00:00
des
911ff37bf7 Don't complain about not knowing the remote file size when working in 
quiet mode.

PR:		bin/37079
Submitted by:	Nicolas Rachinsky <list@rachinsky.de>
 2002-10-27 17:33:08 +00:00
des
877e52987a Whitespace and indentation cleanup. 2002-10-27 15:32:51 +00:00
des
f8dcf07de6 Forgot to disable alarms after fetchXGet() in previous commit. 2002-10-27 15:32:06 +00:00
des
c1a4236482 Add an ETA timer that kicks in after 30 seconds. 
Re-add alarm(2) calls around the calls to fetchStat(3) and fetchXGet(3),
since these calls can still time out on DNS lookups or TCP connect(2).

Remove the alarm(2) calls in the main loop, since all methods properly
handle transfer timeouts (as opposed to connection timeouts).

Set the sigalrm flag if a timeout occurs in the main loop.

Move the signal: label up a little so we still set the atime and mtime
when the transfer times out or is interrupted, so that restarted transfers
will work as expected (as long as the file still exists).

MFC after:	2 weeks
 2002-10-27 15:15:13 +00:00
des
19b896d323 s/optarg/s/ (cut'n'paste braino) 
Spotted by:	kris
 2002-08-27 10:19:24 +00:00
des
403946d7ed fetch(1) is WARNS-5 clean (tested on i386 and Alpha) 2002-07-28 21:09:57 +00:00
des
9413d584e7 Use strtol(3) instead of a home-grown version, and fix a "possibly 
unitialized variable" warning.
 2002-07-28 21:09:25 +00:00
des
5a55cbe1eb Reorder LDADD to fix static linking. 2002-06-12 12:46:50 +00:00
ru
876652376e libfetch now depends on libcrypto and libssl. 2002-06-06 13:45:46 +00:00
des
88a7e46829 Figures they had to call it NOCRYPT instead of NOCRYPTO. 2002-06-05 21:28:47 +00:00
des
694ae4fc45 During buildworld, "regular" libraries are built before crypto stuff, so 
libfetch can't depend on lib{crypto,ssl}.  Move the dependency to fetch
until we can figure out how to fix this.
 2002-06-05 21:25:33 +00:00
des
4d6b787d2d Usage style sweep: spell "usage" with a small 'u'. 
Also change one case of blatant __progname abuse (several more remain)
This commit does not touch anything in src/{contrib,crypto,gnu}/.
 2002-04-22 13:44:47 +00:00
charnier
a07fb1cc07 Use `The .Nm utility' 2002-04-19 23:44:58 +00:00
bde
c2a1e52896 Restored the part of 1.45 that was clobbered in rev.1.46: 
#include <sys/time.h> instead of depending on namespace pollution in
<sys/stat.h> for the declaration of struct timeval.
 2002-03-27 02:23:18 +00:00
charnier
896929d2ba Add FBSDID. Remove unused include. Getopt returns -1 not EOF. 2002-03-26 12:09:49 +00:00
bde
b3ad16b2d7 #include <sys/time.h> instead of depending on namespace pollution in 
<sys/stat.h> for the declaration of struct timeval.

Removed unused includes (<time.h> doesn't declare anything of interest;
only <sys/time.h> does).

Sorted includes a bit.
 2002-02-25 05:00:39 +00:00
des
4c0d10cd1b If downloading to a temporary file, set the timestamp on the temp file, not 
on the file we're replacing.

PR:		bin/34992
Submitted by:	Shunichiro Ariura <syun1rou@blackshell.org>
MFC after:	1 week
 2002-02-16 14:02:57 +00:00
markm
ae2558974f Remove leaf node WARNS?=2 (that mainly I added). This should 
help the GCC3 transition and CURRENT in general.
 2002-02-08 22:31:43 +00:00
ru
66616c46ce Document fetch(1) reaction to SIGINFO. 
PR:		docs/33640
Submitted by:	Maxim Konovalov <maxim@macomnet.ru>
Approved by:	des
 2002-01-16 16:16:08 +00:00
des
a1922f6bbf Set fetchDebug if v_level is 3 or more. 
PR:		bin/32615
MFC after:	1 week
 2001-12-09 15:07:26 +00:00
iedowse
526c677d03 Remove an extraneous argument to fprintf that -Wformat noticed. 
Reviewed by:	des
 2001-11-10 18:39:54 +00:00
des
572d6d9be0 Remove a debugging warnx() that got committed by mistake. 2001-11-03 01:31:46 +00:00
des
3274d63b58 Don't print running stats unless we're the foreground process. 
PR:		30764
 2001-09-23 22:03:39 +00:00
mike
fe5200b07e Add a cast to silence a warning (error because of WARNS=2) on Alpha. 
Discussed with:	des
 2001-09-10 17:23:57 +00:00
des
80b788da32 If the local file does not exist, or is a regular file, and we're not trying 
to resume a transfer, download the requested document into a temporary file
which we later rename.  This avoids leaving half-completed files around in
case of a crash (it'll still leave a half-completed file, but with a  hope-
fully non-conflicting name), and should reduce the need for human inter-
vention on ports-building machines.

The temporary file name for "foo/bar" is constructed by invoking mkstemps()
with the pattern "foo/.fetch.XXXXXX.bar"

Requested by:	obrien
 2001-09-08 15:17:15 +00:00
des
4debb1b901 Remove a blank line that snuck in with the previous commit. 
Set the default timeout to 120 seconds instead of 0 (no timeout).
 2001-08-18 10:48:55 +00:00
des
f1dd0e9302 Add some comments. 2001-08-17 22:22:45 +00:00
se
815cafd7d0 Include value of command line argument that causes an error message or 
warning in said message, since fetch may be run from a makefile or script
which does not print the command line.

Approved by:	des
 2001-07-28 21:34:56 +00:00
des
45c6fea352 If we're writing to stdout, we don't care about the size. 
Submitted by:	wollman
 2001-07-17 21:23:43 +00:00
des
1d3699a988 WARNS cleanup + remove an unused macro and an unused function argument. 2001-07-17 21:22:32 +00:00
dd
911ca14c87 Remove whitespace at EOL. 2001-07-15 08:06:20 +00:00
des
edbf7599a6 Fix an embarassing if-test-reversal bug that broke scheme guessing. 2001-07-08 15:17:26 +00:00
des
e576be9a24 Don't try to print a completion percentage for zero-size files. 
PR:		27780
Submitted by:	Craig Leres <leres@ee.lbl.gov>
 2001-06-01 10:24:58 +00:00
des
c652271f23 Damn. Fix typo in previous commit. 2001-05-27 10:46:19 +00:00
des
b7626578b0 When running on a tty, install an authentication callback. 2001-05-26 19:51:05 +00:00
des
30512a1dcf Bring the usage message in synch with reality. 
PR:		bin/26160
 2001-03-28 21:01:07 +00:00
des
deabc3a379 Now that libfetch uses the high port range by default, add a -U option to 
make it use the low (default) port range instead.
 2001-03-24 00:32:11 +00:00
des
0a97af408f If SIGINFO is received during the transfer, print a status line similar 
to the summary shown at the end of the transfer.

Some days, I really hate Unix...

Suggested by:	cracauer
 2001-03-07 05:33:50 +00:00
des
41113a8cdf Fix a formatting bug (MFC candidate) 2001-03-07 03:22:11 +00:00
ru
8a6f8b5fe4 mdoc(7) police: split punctuation characters + misc fixes. 2001-02-01 16:38:02 +00:00
des
d6547b8df5 Warn if the size of the remote file isn't known. 
If the -R option was specified, don't truncate the local file even if its
mtime is incorrect.

PR:		bin/23719
 2000-12-22 18:23:19 +00:00
des
5612c73391 If the URL did not specify a scheme, try to guess it from the host name. 2000-12-13 11:26:27 +00:00
des
517ed0bad5 If the transfer timed out, but we don't know how large the file is supposed 
to be, assume it was truncated.
 2000-11-30 10:08:22 +00:00
kris
303a4f96b9 Format string auditing 2000-11-27 07:35:53 +00:00
ru
835a46c9c2 mdoc(7) police: do not split author names in the AUTHORS section. 2000-11-22 09:35:58 +00:00
ru
3b7dc88538 Spell the des's name correctly. 2000-11-14 13:19:26 +00:00
des
1763356b83 Understand the difference between an empty file and a non-existent file. 
This has been sitting in my tree for ages...
 2000-10-19 21:05:59 +00:00
des
10398016b8 Don't try to set the mtime of the output file if it's not a regular file. 
Pointed out by:	cwt
 2000-09-02 09:48:34 +00:00
des
23f9ab64bf Don't unlink the target file if it's not a regular file. 2000-08-31 11:24:15 +00:00
des
cef359aa2c Restore the old semantics of deleting the output file when interrupted. 
Some people liked this and some didn't, so POLA won.
 2000-07-26 07:28:33 +00:00
des
d88213abd9 Don't print summary in quiet mode. 2000-07-21 14:21:29 +00:00
des
d81bffbebb Don't display a running count in quiet mode. 
Don't add the offset to the size; it's libfetch's job to report the correct
size of the requested file.
 2000-07-21 11:08:03 +00:00
des
3ce22a8bc5 Fix two bugs related to resumed transfers: 
- if the dates didn't match, fetch would append the received file to the
    existing file instead of replacing it.

  - if the local file was complete and up-to-date, fetch would miscalculate
    the expected size and report a failure instead of a success, because it
    had no way of knowing that the server was actually resending the entire
    file since the requested offset was invalid.
 2000-07-19 23:56:45 +00:00
des
e8e313d17b Braino: don't use passive mode unless the user requests it. 
PR:		bin/20025
Submitted by:	HIYAMA Takeshi <th@cis.ibaraki.ac.jp>
 2000-07-19 09:12:36 +00:00
ben
3932cf62e5 * Remove description of obsolete -t option. 
* Insert newline before "and" in ".Fl c and".

Approved by:	des
 2000-07-18 21:05:11 +00:00
des
29b66ff09c The description of the semantics of -v and -q has long since been OBE. 2000-07-17 22:44:36 +00:00
des
0a49b612e7 While I'm at it, break a line that was too long, remove a pointless diagnostic 
and adjust the verbosity level of another.
 2000-07-17 22:44:00 +00:00
des
6bedb18bde Document the fact that -r and -m are mutually exclusive. 2000-07-17 21:51:48 +00:00
des
403f5cc58e Rearrange / rewrite large portions of fetch() to take advantage of new 
libfetch features (fetchRestartCalls, fetchXGet()).

Since it doesn't make much sense to have m_flag and r_flag set at the same
time, and it can actually cause trouble in some cases, die if they're both
set.

Set the SA_RESETHAND flag for SIGINT so that when we've caught one, we can
kill ourselves with a second SIGINT (thus notifying our parent of our tragic
fate) instead of just exiting.

These changes fix several problems that would show up when fetching ports,
as well as speeding up HTTP transfers quite a bit (at least for relatively
small files).

Most of these changes were prompted by an interaction problem with an HTTP
server called SWS-1.0, which exhibited two bugs, the first of which prevented
fetch from working around the second (the first was not sending content-type
in reply to HEAD requests, the second was sending garbage after the end of
the requested file).
 2000-07-17 21:49:01 +00:00
des
d4605ce5a4 Better signal handling. 
Submitted by:   green
 2000-07-16 00:47:00 +00:00
des
9ced946355 Fix a bug in the stats display that showed up when resuming aborted transfers. 
Submitted by:	green
 2000-07-13 08:37:39 +00:00
des
6146eecaff Rework the stats code for the nth time. Much cleaner now. 
Always display the completion percentage if stderr is a tty.
Drop the char-by-char transfer mode, it was based on an incorrect assumption
regarding the semantics of fread().
Finally (I hope) straighten out the business of setting the mtime, as well as
when to remove the output file and when not to.
Thanks are owed to the many who have provided nearly instantaneous and
highly constructive feedback and suggestions about these matters.
 2000-07-12 20:53:07 +00:00
des
106d1927ea Now that the HTTP code supports timeouts, we don't need to use alarm() 
around the call to fetchStat().
Catch SIGINT, and rework the signal handling so it doesn't skimp on the
cleanup after a timeout or interrupt. Also, don't just bail out after a
timeout; there may be more files to fetch.
 2000-07-12 11:02:01 +00:00
des
16b092fba0 Rework the stats code to avoid code duplication. 
Fix a bug where the stats code would print the expected size instead of the
number of bytes received.
Fix the reading code so it'll support partial reads.
 2000-07-12 08:29:52 +00:00
des
7613572c34 Don't segfault if given an invalid URL (doh!) 
Submitted by:		green
 2000-07-09 10:14:22 +00:00
des
1fddde897c Clarify the meaning of -m and -n (thanks to bde). 2000-07-08 09:34:53 +00:00
des
9ee4aebc40 Straighten out the behvaiour of -m and -n (thanks to bde). 
Check that stderr, not stdout, is a tty (thanks to green).
 2000-07-08 09:34:33 +00:00
des
40127f9018 Add MIT copyright 2000-07-05 12:04:31 +00:00
des
cd4c3a8be3 Note that the -h, -c and -f options are deprecated, and remove note that 
incorrectly stated that they were not implemented.
Document the -d option.
 2000-06-29 10:35:24 +00:00
des
c8fb726f76 Implement the -c option 2000-06-29 10:32:56 +00:00
des
9a9fe3065a Don't forget to delete the output file if the request fails. 
Don't delete the output file if -r was specified.
 2000-06-29 08:39:29 +00:00
des
9b9369d90e New libfetch-based fetch. 2000-06-28 16:55:15 +00:00
jake
961b97d434 Back out the previous change to the queue(3) interface. 
It was not discussed and should probably not happen.

Requested by:		msmith and others
 2000-05-26 02:09:24 +00:00
jake
d93fbc9916 Change the way that the queue(3) structures are declared; don't assume that 
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by:	phk
Reviewed by:	phk
Approved by:	mdodd
 2000-05-23 20:41:01 +00:00
cracauer
1a6acdd95f Print a warning and exit with != 0 when at least one downloaded file 
is shorter than previously announced by the server.

Tested by asami.

Approved by:	jkh
 2000-03-08 13:02:11 +00:00
sheldonh
49c4458c80 Remove single-space hard sentence breaks. These degrade the quality 
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.
 2000-03-01 12:20:22 +00:00
green
b6ebb73004 Add the -F option. This is for forcing restarts with -r by inhibiting 
transmission of the If-Range HTTP header field.
 1999-10-23 15:22:53 +00:00
peter
3b842d34e8 $Id$ -> $FreeBSD$ 1999-08-28 01:08:13 +00:00
green
090af66e8b "transfered" -> "transferred" 
Reviewed by:	dict(1)
 1999-07-02 08:57:06 +00:00
green
0aac238e5b This fixes the problem of cancelling an FTP transfer resulting in 
a bogus error ("parsing URI".) I also fixed an inconsistency in
another fs_status.
 1999-06-21 02:38:16 +00:00
jmg
09c8cd859c make md5 checksums actually work instead of simply calculating the length 
of an empty buffer...  the output file wasn't readable...  also warn that
we can't checksum on stdout and print out the base64 encoded version of the
md5 checksum...

Site to actually return md5 digest: web.golux.com
Verified that fetch was broken: Ken Coar <Ken.Coar@Golux.Com>
 1999-05-18 19:37:37 +00:00
wollman
bd6b0d4dcd Restore \n at end of usage message. 
Complained-about-by: charnier
 1999-03-08 19:16:14 +00:00
alex
0af144ae9f Improve error message wording when attempting to link to a non-existent 
file on the local host.

PR:		10042
Submitted by:	Chris Costello <phoenix@calldei.com>
 1999-03-02 04:14:33 +00:00
wollman
9c3b9cf062 Deal with broken Web sites which return 302 responses rather than 404 
and an error document when the requested resource does not exist.  Grrr.

Requested by:	asami
 1999-02-23 18:51:13 +00:00
fenner
44f9119dc9 Remove the FTP_PASSIVE_MODE "fix"; libftpio handles this. 1999-02-05 01:01:17 +00:00
fenner
adb8ba8144 Warn about collapsing multiple slashes into 1 in ftp URL's. 
Look at the FTP_PASSIVE_MODE environment variable like the man page says.

PR:		bin/9464
Submitted by:	John A. Shue <John.Shue@symmetron.com>

Add references to RFC's 1790, 959, 850.

PR:		doc/6564
 1999-02-03 20:43:29 +00:00
fenner
af6685ec33 Don't try to parse a colon in a URL as a port 
(e.g. http://www.host.name/foo:bar)

PR:		bin/5072
Submitted by:	Takeshi WATANABE <watanabe@komadori.planet.kobe-u.ac.jp>
 1999-02-03 20:24:53 +00:00
wollman
acfce95642 Fix logic error in RFC 850 kluge. 1999-01-15 17:10:31 +00:00
wollman
78818f4318 For RFC 850 dates received in HTTP responses, implement the century pivot 
described in RFC 2068.  Include a reference to same in the manual page.
 1999-01-15 16:56:22 +00:00
cracauer
c4f0cd91df Add -s option, just report size of file that would be fetched. 
Reviewed by:	-current list
 1998-12-08 13:00:49 +00:00
des
34ddce5767 main.c: 
Fix usage string. Fix getopt() string. Fix ordering of compatibility options.
fetch.1:
 Fix synopsis. Fix ordering of T and t options. Fix minor grammar nit.
 1998-11-08 23:18:48 +00:00
fenner
1f7615e10a If we know the content-length, only read that number of bytes from 
the server.  There exists a broken server which sends a few extra
garbage bytes in response to HTTP/1.1 requests.
 1998-10-26 02:39:21 +00:00
jkh
79ff6f224d Add -S flag; this is benign since fetch's behavior is unchanged if you 
don't use it.  That's why I'm bringing it in during our "code slush"
Submitted by:	Stefan Esser <se@mi.uni-koeln.de>
 1998-09-20 00:01:26 +00:00
des
8b07d06aca DTRT for file: URLs when output file is "-". 
PR:		bin/7203
 1998-09-17 00:15:15 +00:00