Commit Graph

231 Commits

Author SHA1 Message Date
des
2fa9607541 Don't send port number to proxy unless it's not the standard FTP port 2000-07-21 10:25:33 +00:00
des
a260febdf9 Fix the case where the file name is preceded by a lone slash. 2000-07-20 02:03:56 +00:00
des
adc026dc20 Don't try to skip to the requested offset if the server returns more data
than requested.  Instead, inform the caller of the real offset by modifying
the offset field in the original struct url, and let him decide how to handle
the situation.
2000-07-19 23:43:49 +00:00
des
c5e4329667 Don't take a failure to stat a file to mean that it doesn't exist, as it may
be a dynamically created file, e.g. a tarball on servers that support sending
tarballs of directories.
2000-07-18 07:12:26 +00:00
des
3033318809 Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and
fetchStat*().  In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetchGet*() is identical to calling
fetchXGet*() with the second argument set to NULL.
2000-07-17 21:25:00 +00:00
des
9e42d615d8 Don't forget to set file descriptor to -1 after closing it, since the code
outside the loop inspects it to determine whether or not we succeeded in
retrieving the requested document. This fixes a bug where fetchGetHTTP()
would return a FILE with an invalid file descriptor if it hit the redirect
limit without locating the requested document.
2000-07-17 21:09:11 +00:00
des
147a437a42 Use a symbolic constant instead of hardcoding 999 2000-07-17 21:00:49 +00:00
des
a0fbdfaa34 Don't forget to declare fetchRestartCalls. 2000-07-17 20:52:06 +00:00
des
c1df3803c6 Introduce the (undocumented) variable fetchRestartCalls, which controls whether
or not interrupted system calls will be restarted. This fixes a bug where
fetch(1) would hang (potentially forever) if a server stopped responding,
because the signal handler would absorb the user's efforts to interrupt the
transfer.
2000-07-17 20:49:39 +00:00
des
94b7364484 Remove fetchContentType(), which disappeared in the HTTP rewrite. 2000-07-17 20:42:13 +00:00
des
5996cf5b9e #ifdef DEBUG -> #ifndef NDEBUG
Pointed out by:	ache
2000-07-16 23:18:44 +00:00
des
436b76a904 Reinsert the MIT copyright, which applies to the base64 code and was
left out by accident during the rewrite.
2000-07-16 01:04:10 +00:00
des
797b7f6ddd Fix a bug (misplaced continue) that caused redirects to fail. Lots of code
moved around, but the acutal functional changes are small.
Add support for site-internal redirects (where the Location: header gives a
path instead of an absolute URI)

Pointed out by:	kuriyama
2000-07-13 09:13:58 +00:00
des
a7682e2874 Fully document the HTTP authorization feature.
Update the BUGS section to reflect the HTTP rewrite.
2000-07-12 10:41:29 +00:00
des
59adaf8069 Extensive rewrite of the HTTP code. The new code is significantly cleaner and
more robust, and somewhat more efficient. It also handles authorization and
redirects properly, and supports timeouts like the FTP code.

Many thanks to Umemoto-san for his assistance with IPv6 support, both here
and in other parts of libfetch.
2000-07-12 10:39:56 +00:00
des
d908f153d0 Use fetch_putln() 2000-07-11 23:50:22 +00:00
des
848e1688b8 Add _fetch_putln() 2000-07-11 23:50:08 +00:00
des
ce80758a3f Document that basic authorization now kinda works. 2000-07-11 18:13:08 +00:00
des
c42f438ce8 Rework the authorization code.
Only send absolute URI if connected to a proxy, since Apache doesn't always
understand absolute URIs.
Clean up some of the debugging output.
2000-07-11 18:12:41 +00:00
des
686d930615 Clean up the debugging output 2000-07-11 18:07:09 +00:00
ume
ae7af5f4bd Free chunks obtained by getaddrinfo(3).
Approved by:	des
2000-07-10 16:28:28 +00:00
ume
4f24653720 Make EPSV work again. Separate parsing code of 229 replies from
the code for 227 and 228.

Submitted by:	des
2000-07-10 08:41:25 +00:00
des
866a6bff9a Don't assume the remote address in a 227 reply is enclosed in parentheses. 2000-07-08 09:27:47 +00:00
des
9f7754e0e5 Fix basic authentication, and add proxy authentication.
Submitted by:	se
2000-07-08 08:08:58 +00:00
des
2f7d5cb4f5 Make restart work in active mode, too.
PR:		bin/18688
Submitted by:	Rudolf Cejka <cejkar@dcse.fee.vutbr.cz>
2000-06-29 10:44:10 +00:00
des
b112ea1321 Handle multiline replies properly, instead of kinda-right. 2000-06-28 15:48:26 +00:00
des
d6712064a8 Don't incorrectly report a protocol error when we get a 302 and redirects
are disabled.
2000-06-21 09:49:51 +00:00
des
720d162772 Back out wes' commit with extreme prejudice. 2000-06-20 13:57:00 +00:00
wes
425601774e Add implementation of fetchListFTP.
Reviewed by:	silence
2000-06-20 05:32:41 +00:00
des
96fd818f66 Hackish support for 302 redirect. 2000-05-26 15:34:42 +00:00
des
2a39533c00 Honor FTP_PASSIVE_MODE so caller doesn't need to. 2000-05-26 12:56:53 +00:00
des
969e1c4e0d Output newline after debugging message 2000-05-25 16:50:08 +00:00
des
59892af9b1 Add (and document) fetchMakeURL() 2000-05-25 16:46:09 +00:00
des
e5d7688f99 Dump com_err, it's a stinking crock of shit. 2000-05-25 16:24:31 +00:00
ume
79e112b26f Use $FTP_PASSWORD for FTP password. If $FTP_PASSWORD is not
found, `yourname@yourhost' is used.
2000-05-22 13:01:13 +00:00
ume
1db238984d IPv6 support. 2000-05-20 18:23:51 +00:00
des
4721396b1d Better handling of some boundary conditions.
Submitted by:	ume
2000-05-19 09:45:42 +00:00
ume
3194168ffc Make HTTP_PROXY work for FTP.
Reported by:	Ben Smithurst <ben@scientia.demon.co.uk>
Reviewed by:	des
2000-05-15 09:05:36 +00:00
des
e10535fbc8 Document struct url.
Document the default values for fetchStat*().
2000-05-15 08:34:42 +00:00
des
5d01e7fd74 Initialize the struct url_stat at the beginning of _fetch_stat_file(). 2000-05-15 08:33:58 +00:00
des
ffd6fec6c0 Initialize the struct url_stat at the beginning of fetchStatFTP(). 2000-05-15 08:24:58 +00:00
des
23a4c9053b Remove unused 'verbose'.
Initialize the struct url_stat at the beginning of fetchStatHTTP().
2000-05-15 08:24:29 +00:00
des
741229b49a Add workaround for ftpds with the Y2K MDTM bug 2000-05-11 16:01:03 +00:00
des
2d5449b6f7 Bump major number, since struct url has changed.
Discussed with:	jdp
2000-05-11 15:29:26 +00:00
des
1e1a123aa3 Fix hard sentence break.
Submitted by:	sheldonh
2000-05-11 15:22:52 +00:00
des
a9ed660f51 Reorganize some of the http code and split it into more functions.
Implement fetchStatHTTP().
Unbungle struct url, and add fetchFreeURL().
Document it.
2000-05-11 13:31:02 +00:00
sheldonh
5443174128 Supply only one author name per instance of %A, as per mdoc.samples(7).
PR:		18465
Submitted by:	Kazu TAKAMUNE <takamune@avrl.mei.co.jp>
2000-05-10 09:49:04 +00:00
des
df9519feab While we're at it, add a length field too in case we want to fully implement
partial fetches later.
2000-05-07 20:52:40 +00:00
des
5f7a182a3e Implement restart 2000-05-07 20:51:31 +00:00
des
0ee0a9878b Use sizeof more consistently. 2000-05-07 20:02:52 +00:00
des
58fa435eca Use sizeof more consistently.
Outline sanity checking of server port spec.
2000-05-07 20:01:55 +00:00
des
3db481e95d Implement restart.
Use sizeof more consistently.
Outline sanity checking of server port spec.
2000-05-07 20:00:12 +00:00
des
c23ed76a90 Implement restart 2000-05-07 19:56:07 +00:00
des
2695a66902 Add offset field to struct url 2000-05-07 19:46:33 +00:00
phantom
0ab35641b3 Introduce .Lb macro to libfetch manpage
Sort .Nm values
Change first column width in errors list (table look much better now)
Remove redundant comma
2000-04-22 16:08:41 +00:00
sheldonh
329223e6f2 Remove single-space hard sentence breaks. These degrade the quality
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.
2000-03-02 09:14:21 +00:00
des
e81fbe2ad9 Fix timestamp handling.
Submitted by:	bde
2000-01-26 12:50:46 +00:00
rgrimes
c127d0e61e Replace beforeinstall target with new variables used by .mk system.
Reviewed by:	marcel, and make world
2000-01-14 07:57:47 +00:00
des
c701cfdd7c More old uncommitted patches: implement timeouts at the protocol level.
Currently only supported for ftp connections.
2000-01-07 12:58:40 +00:00
des
3acbd8ffda Add error codes for protocol errors. 2000-01-07 12:56:27 +00:00
des
1e930b38f8 Patches I've had lying around for several months:
* Add the 'h' ftp flag (allocate local port in high range)
 * Add the 'd' flag (use direct connection even if proxy is defined)
 * Make sure flags != NULL before calling strchr().
 * Minor changes to some comments.
2000-01-07 10:59:12 +00:00
peter
76f0c923fe $Id$ -> $FreeBSD$ 1999-08-28 00:22:10 +00:00
bde
3af5374f3c Fixed prototype for fetchParseURL(). 1999-03-05 18:45:32 +00:00
des
5c7a3846e8 Add MAINTAINER. 1999-01-29 12:15:34 +00:00
des
9eb6071298 Implement and document file list retrieval. 1998-12-21 19:41:50 +00:00
des
b8af0c7d8d Don't specify a language to compile_et.
Rename fetchConnect to _fetch_connect since it's internal.
1998-12-18 14:32:48 +00:00
des
f1482d2faf Implement and document fetchStatFTP.
Update description of struct url_stat in the man page.
Clean up error handling in ftp.c.
1998-12-16 15:29:03 +00:00
des
f13cf4998e Remove unused header files. Fix a few shadowing bugs. Improve error reporting. 1998-12-16 11:44:31 +00:00
des
37f6ae6efa Add verbose flag, and support functions.
Brucify the Makefile.
Differentiate atime and mtime in fetch*Stat().
Fix a few pointer bugs.
Tweak some error messages.
Don't #include sys/param.h and stdio.h in fetch.h.
Document that sys/param.h and stdio.h must be #included before fetch.h.
1998-12-16 10:24:55 +00:00
des
86d45a4ff7 Fix the "Unknown HTTP error" message.
Make compile_et generate prototypes.
1998-12-15 12:24:26 +00:00
des
32bb859dc0 fetch_err.h is a generated file, so don't look for it in ${.CURDIR}. 1998-11-07 08:59:38 +00:00
des
93ce759c73 Second of a series of cleanups to libfetch.
This commit introduces the following features:

 a) the fetchStat*() functions, which return meta-information for a
    document, such as size, modification time, etc.

 b) the use of the com_err(3) facilities to report errors.

It also fixes a bunch of style bugs and a few logic bugs and somewhat
improves the man page.

Changed files, in alphabetical order:

 Makefile:
  Don't generate macros in {ftp,http}err.c.

  Generate category fields for the error message lists.

  Compile the error table.

  Install fetch_err.h along with fetch.h.

 common.c:
  Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the
  error code in the _netdb_seterr() macro.

  Add categories to the _netdb_errlist table.

  Report errors through the Common Error library.

 common.h:
  Add the DEBUG macros.

  Add prototype for fetchConnect().

  Remove the prototype for _fetch_errstring(), which is local to common.c

  Add a categroy field to struct fetcherr, and define constants for
  error categories.

  Define macros for _{url,netdb,ftp,http}_seterr().

 errors.et: (new file)
  List error categories.

 fetch.3:
  Document the fetchStat*() functions.

  Move the "unimplemented functionality" comments from NOTES to BUGS.

  Document that applications which use libfetch must also use
  libcom_err, and list existing error codes.

  Undocument fetchLastErr{Code,String}.

  Remove the (empty) DIAGNOSTICS section.

  Mention Eugene Skepner in the AUTHORS section.

 fetch.c:
  Move the DEBUG macros to common.c

  Add fetchStat() and fetchStatURL().

  Generate error messages for URL parser errors, and fix a minor bug
  in the parser.

  Use 'struct url' instead of 'url_t'.

  Remove fetchLastErr{Code,String}.

 fetch.h:
  Use 'struct url' instead of 'url_t', and remove the typedef.

  Define struct url_stat (used by fetchStat()).

  Add prototypes for fetchStat*().

  Remove the declarations for fetchLastErr{Code,String}.

  Include fetch_err.h.

 fetch_err.et: (new file)
  Error table for libfetch.

 file.c:
  Add fetchStatFile().

  Use 'struct url' instead of 'url_t'.

 ftp.c:
  Add fetchStatFTP().

  Use 'struct url' instead of 'url_t'.

  Don't use fetchLastErrCode.

 ftp.errors:
  Add categories to all error messages.

 http.c:
  Add fetchStatHTTP().

  Use 'struct url' instead of 'url_t'.

  Don't use fetchLastErr{Code,Text}.

 http.errors:
  Add categories to all error messages.

Prompted by: jkh and Eugene Skepner
Numerous sugestions from: Garett Wollman and Eugene Skepner
1998-11-06 22:14:08 +00:00
des
e958241739 First of a series of cleanups to libfetch. Changed files, in
alphabetical order:

 Makefile:
  Add common.c to SRCS.

  Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG

  Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr
  instead.

 README:
  Remove the todo list, which is out of date anyway.

 common.c: (new file)
  Gather utility functions in this file.

  Merge the error reporting functions intp _fetch_errstring(),
  _fetch_seterr() and _fetch_syserr().

  Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect
  fails.

 common.h: (new file)
  Gather internal prototypes and structures in this files.

 fetch.3:
  Undocument fetchFreeURL().

  Document a few more known bugs.

  Document fetchLastErrCode and fetchLastErrText.

 fetch.c:
  Add descriptive comments to all functions that lacked them.

  Move fetchConnect() to common.c.

  Obviate the need for fetchFreeURL(), and remove it.

 fetch.h:
  Modify struct url_t so the document part is at the end.

 ftp.c:
  Remove code that is duplicated elsewhere.

 http.c:
  Remove code that is duplicated elsewhere.

Prompted by: jkh
1998-11-05 19:48:17 +00:00
des
ef532c67ad Bring the man page more or less up to date with the code. 1998-09-26 20:42:44 +00:00
bde
7268de7a17 Made this actually work when there's an object directory:
- replaced bogus dependencies of distribution sources on generated
  sources by the same ones that bsd.lib.mk would generate if it knew
  all the sources.  We shoot ourself in the foot by not naming the
  generated (included) sources *.h, so we can just put the generated
  sources in SRCS.
- replaced -I${.CURDIR} by `-I.'.  Here `.' is an alias for ${.OBJDIR}.
  -I${.CURDIR} didn't do anything, since ${.CURDIR} is the default in
  all cases here (it would be necessary for ""-style includes made from
  sources in ${.OBJDIR}.

Don't use `+=' for setting CLEANFILES for the first time.

Added $Id$.
1998-08-17 20:39:09 +00:00
des
b315880890 Commit a bunch of patches that have been accumulating:
- Fix the README to reflect the new status of the ftp code.
 - Change tons of 'if (xxx < 0)' to 'if (xxx == -1)'
 - Add two new interface functions
 - Fix the Makefile so it actually works (yay!)

Now the manpage is lagging even further behind... :( Next on the todo
list is to clean up the http code.
1998-08-17 09:30:19 +00:00
des
42566affcd Forgot to remove base64.c from SRCS in the Makefile. 1998-07-12 22:37:45 +00:00
des
54143a63f8 Base64 code (and the MIT copyright) moved to http.c
FTP STORe and APPEnd added.
FTP proxy support added (untested).
1998-07-12 22:34:40 +00:00
des
7592a9749c FTP retrieve works. 1998-07-11 21:29:08 +00:00
des
5b74d7e9c7 Too many changes to list. Basically, FTP is nearly there and error
reporting is kinda sorted out. Now HTTP needs to catch up...
1998-07-11 18:56:01 +00:00
des
9c2d60ed01 Imported libfetch into the tree. It compiles, but there's still some
work to do. I especially need help with the man page.
1998-07-09 16:52:44 +00:00