Commit Graph

100 Commits

Author SHA1 Message Date
Jung-uk Kim
01ed3ca3de Remove defunct SSLv2 support from fetch(1) and fetch(3). 2015-03-25 18:56:36 +00:00
Dag-Erling Smørgrav
c2ea176dac Increase the default (and minimum) buffer size from 4 kB to 16 kB. Also,
propagate the buffer size to libc, which uses a 1 kB buffer by default,
negating any hypothetical benefit of increasing fetch(1)'s buffer size.

MFC after:	3 days
2014-01-28 14:32:04 +00:00
Dag-Erling Smørgrav
2043423cd1 whitespace and bump copyright 2014-01-28 14:29:24 +00:00
Dag-Erling Smørgrav
dcd47379ff Implement certificate verification, and many other SSL-related
imrovements; complete details in the PR.

PR:		kern/175514
Submitted by:	Michael Gmelin <freebsd@grem.de>
MFC after:	1 week
2013-07-26 15:53:43 +00:00
Eitan Adler
a96469b791 Correct check to see if file changes.
Reported by:	swildner@DragonFlyBSD.org
Reviewed by:	des
2013-06-02 18:10:07 +00:00
Andre Oppermann
162361c198 Fix bandwidth reporting when doing a restarted download with "-r".
The offset is already accounted for in xs->lastrcvd and doesn't
have to be subtracted again.

Reported by:	Florian Smeets <flo@smeets.im>
Submitted by:	Mateusz Guzik <mjguzik@gmail.com>
Tested by:	Florian Smeets <flo@smeets.im>
MFC after:	1 week
2012-12-09 22:54:03 +00:00
Eitan Adler
2214137ab1 Add check for failure of mkstemp and setenv.
Reviewed by:	des
Approved by:	cperciva (implicit)
Obtained from:	DragonFlyBSD
MFC after:	1 week
2012-12-08 18:41:16 +00:00
Andre Oppermann
ae7dc19cd1 Change fetch(1) to:
o Report the instantaneous bandwidth instead of an average since the
  beginning of the download.

o At the finish of the download report the average bandwidth and also
  the total time it took instead of 00m00s.

Reviewed by:	des
MFC after:	1 week
2012-11-16 12:05:10 +00:00
Ed Schouten
bf70becee6 More -Wmissing-variable-declarations fixes.
In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.
2012-10-19 14:49:42 +00:00
Dag-Erling Smørgrav
2a7daafe67 Fix two issues related to the use of SIGINFO in fetch(1) to display
progress information.  The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal.  The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.

PR:		bin/153240
Submitted by:	Mark <markjdb@gmail.com>
MFC after:	3 weeks
2012-01-18 15:13:21 +00:00
Dag-Erling Smørgrav
bb6e332901 Document the fact that passive mode is now the default.
Update copyright dates and strip my middle name.
2011-09-27 19:02:44 +00:00
Dag-Erling Smørgrav
5c5052cca4 Use fseeko() instead of fseek(). The rest of the code is off_t-aware,
but the use of fseek() means fetch(1) can't correctly resume a transfer
that was interrupted past the 2 GB mark.

Pointed out by:	ache@
MFC after:	3 weeks
2011-09-27 17:11:31 +00:00
Dag-Erling Smørgrav
27274e81b2 Followup to r225599: the fseek() was a no-op since the file was opened
in append mode.  Open it in read-write mode instead.  Also move the
fseek up one level to cover the (unlikely but not impossible) case where
the server accepts ranges but does not send a Content-Size header.

PR:		bin/117277
MFC after:	3 weeks
2011-09-27 15:57:13 +00:00
Dag-Erling Smørgrav
a17e51c108 When resuming an HTTP download, we failed to verify that the range
returned by the server matched what we requested, and blindly appended
what we received to what we already had.  This could go two ways: if the
delivered offset was higher than expected, the local file would contain
duplicate data, while if it was lower than expected, there would be data
missing from the middle of the file.  Furthermore, if the transfer was
interrupted again, each subsequent attempt would compound the error.
Fix the first problem by restarting the transfer from scratch if there
is a gap, and the second by explicitly seeking to the correct location
in the local file so as to overwrite any duplicated data.

PR:		bin/117277
Approved by:	re (kib)
MFC after:	3 weeks
2011-09-15 22:50:31 +00:00
Ruslan Ermilov
8181961a73 Treat an empty argument as an error, instead of fetching the
contents of the root directory.

MFC after:	1 week
2009-12-30 23:01:49 +00:00
Xin LI
821df508e8 Revert most part of 200420 as requested, as more review and polish is
needed.
2009-12-13 03:14:06 +00:00
Xin LI
6f2d322192 Remove unneeded header includes from usr.bin/ except contributed code.
Tested with:	make universe
2009-12-11 23:35:38 +00:00
Dag-Erling Smørgrav
3187195884 The FTP_TIMEOUT and HTTP_TIMEOUT environment variables were ignored because
T_secs already had a non-zero default.  Unbreak by moving the default to
ftp_timeout / http_timeout.

Submitted by:	Christoph Mallon <christoph.mallon@gmx.de>
MFC after:	2 weeks
2009-01-17 13:34:56 +00:00
Murray Stokely
e8f612b687 1. Update fetch to consistently return 1 on error, as the man page states,
rather than usually returning 1 but in a few instances using a sysexits(3)
   return value.

2. Remove a few unused variables from libfetch.

PR:		docs/122470  (1, only)
Reviewed by:	des
2008-12-17 18:00:18 +00:00
Murray Stokely
7f92799f67 Add support for HTTP 1.1 If-Modified-Since behavior.
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.

libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.

PR:		bin/87841
Submitted by:	Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by:	des, ru
MFC after:	3 weeks
2008-12-15 08:27:44 +00:00
Ruslan Ermilov
e801aa4856 Improve usage and sync SYNOPSIS with usage. 2008-12-13 17:48:06 +00:00
Dag-Erling Smørgrav
2b8907e7e2 Apply the same defaults to https as to http.
Submitted by:	Mike Tancsa <mike@sentex.net>
MFC after:	1 week
2008-12-11 06:54:36 +00:00
David E. O'Brien
8acee1c0ca Use the SCHEME_ knobs rather than knowing what they expand to.
Reviewed by:	des
2008-08-21 14:12:34 +00:00
Dag-Erling Smørgrav
1bac3abc07 Try to preserve ownership and permissions when replacing an existing file.
PR:		bin/104702
Submitted by:	Kevin Day <toasty@dragondata.com>
MFC after:	1 week
2006-11-10 22:05:41 +00:00
Dag-Erling Smørgrav
15e2d64e15 Only clear sb.st_size if it is clearly wrong or meaningless. This fixes
mirror mode.

PR:		bin/86940
MFC after:	2 weeks
2005-12-30 23:36:26 +00:00
Dag-Erling Smørgrav
0c87874496 Reflect progress in process title.
Submitted by:	flz
MFC after:	2 weeks
2005-12-30 18:17:11 +00:00
Colin Percival
8c355b0889 Fix buffer overflow. This is FreeBSD-SA-04:16.fetch.
Approved by:	des
2004-11-18 12:01:30 +00:00
Dag-Erling Smørgrav
2cbbf9dac9 Update copyright years. 2004-09-21 18:35:21 +00:00
Dag-Erling Smørgrav
44336f2f9c The check for r_flag was accidentally removed in the previous commit.
Submitted by:	SANETO Takanori <sanewo@ba2.so-net.ne.jp>
MFC after:	3 days
2004-08-26 15:51:10 +00:00
Dag-Erling Smørgrav
13901dd25a Fix a couple of edge cases in which sb.st_size may be incorrect or
meaningless.  In particular, don't assume that it is left untouched if
stat(2) fails; that assumption happens to fail at high optimization
levels on some platforms.

MFC after:	1 week
2004-08-15 22:22:35 +00:00
Dag-Erling Smørgrav
e9aa05f18c Don't strip trailing linear whitespace from passwords.
MFC after:	2 weeks
2004-07-27 11:34:25 +00:00
Dag-Erling Smørgrav
ebeb3bab5b Style nits. 2004-07-27 11:30:35 +00:00
Dag-Erling Smørgrav
c8488e24e7 Truncate long file names in stat_display(), as was originally intended.
MFC after:	1 week
2004-07-05 11:21:50 +00:00
Dag-Erling Smørgrav
adce61f9b4 The description of the -S option in the man page says we won't fail if the
remote size is unknown, but we do.  Resolve this in the man page's favor.

Requested by:   Andre Albsmeier <andre.albsmeier@siemens.com>
MFC after:	1 week
2004-06-24 13:43:57 +00:00
Lukas Ertl
be28a6af38 Fix integer overflow in the file size output when dealing with
large files (i.e. DVD images).

Reviewed by:  des@
2004-05-19 11:07:30 +00:00
Dag-Erling Smørgrav
3d49384208 Workaround for servers that ignore byte ranges when using chunked
encoding.

MFC after:	3 days
2004-04-06 12:24:26 +00:00
Dag-Erling Smørgrav
5476ba5545 Use %zu for size_t, like God intended. 2004-02-18 21:51:02 +00:00
Dag-Erling Smørgrav
eae1afd9a2 Fix printf() format bug in previous commit (size_t != int).
Use %ju and an (uintmax_t) cast to print size_t values.
Use %jd and an (intmax_t) cast to print off_t values.
Use off_t variables to hold the difference between two off_t values.
Don't bother with unsigned char where char will do nicely.
2004-02-18 21:37:36 +00:00
Dag-Erling Smørgrav
3f59deafb3 Revamp the statistics code, and switch to a much more compact display
format.  The old code tried to produce the exact same output as the
pre-libfetch implementation, but I no longer see any value in this.
2004-02-18 15:12:00 +00:00
Dag-Erling Smørgrav
b12d1620fb Use floating point arithmetic to compute the ETA to avoid integer overflow
during slow transfers of large files.
2003-03-11 21:33:43 +00:00
Dag-Erling Smørgrav
29568c0191 Clean up the ETA logic a bit and make sure it works for restarted transfers. 2003-03-11 08:21:51 +00:00
Dag-Erling Smørgrav
32c8cd2915 Slight cleanup of the ETA / BPS code. Avoid a division by zero. 2003-01-23 13:06:13 +00:00
Dag-Erling Smørgrav
13da7d9934 Add -N option to specify a netrc file.
Fix handling of -v option.
Don't treat negative offsets as valid positive ones.
Clean up the ETA and transfer rate code.  Show transfer rate along with
ETA if the verbose level is higher than 1.
2003-01-22 18:33:39 +00:00
Dag-Erling Smørgrav
3d7c408ce4 Use the URL instead of the output file name when reporting errors relating
to the remote document.

PR:		33856
MFC after:	1 week
Approved by:	re
2002-11-27 20:52:07 +00:00
Dag-Erling Smørgrav
a6266f245c Don't set atime / mtime if we don't have an output file, which can happen
in certain failure scenarii (mostly DNS trouble).
2002-11-13 16:04:20 +00:00
Bill Fenner
f10d3c5113 Don't divide by zero to calculate the ETA, even if we haven't gotten
any data in 30 seconds.
2002-11-12 07:09:43 +00:00
Bill Fenner
05f45e0c2c Don't exit with success after timeout during authentication. 2002-11-07 22:16:45 +00:00
Dag-Erling Smørgrav
eb87c5a73a Don't complain about not knowing the remote file size when working in
quiet mode.

PR:		bin/37079
Submitted by:	Nicolas Rachinsky <list@rachinsky.de>
2002-10-27 17:33:08 +00:00
Dag-Erling Smørgrav
97727a7dc1 Whitespace and indentation cleanup. 2002-10-27 15:32:51 +00:00
Dag-Erling Smørgrav
0c6d34af10 Forgot to disable alarms after fetchXGet() in previous commit. 2002-10-27 15:32:06 +00:00