closed through _fetch_close() which is the only one who knows the connection
REALLY was closed (since ref -> 0). However, FTP keeps its own local
cached_connection and checks if it is valid by comparing it to NULL. This
is bogus since it may have been freed elsewhere by _fetch_close().
This change checks if we are closing the cached_connection and the ref is 1
(soon to be 0). If so, set cached_connection to NULL so we don't
accidentally reuse it. The REAL fix should be to move connection caching
to the common.c level (_fetch_* functions) and NULL the cache(s) in
_fetch_close(). Then all layers could benefit from caching.
which contains the socket descriptor, the input buffer and (yet unused)
SSL state variables. This has the neat side effect of greatly improving
reentrance (though we're not *quite* there yet) and opening the door to
HTTP connection caching.
This commit is inspired by email conversations with and patches from
Henry Whincup <henry@techiebod.com> last fall.
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.
proxy specification, which seems to be valid according to the man page.
Change the logic to consider "hostname:port" a hostname and port instead
of a file URL.
Approved by: des
Rather than have a separate (misnamed) FTP_ANONYMOUS_PASSWORD constant, use
FTP_ANONYMOUS_USER (i.e. "anonymous") to construct the anonymous ftp password
if getlogin() fails.
and had libfetch selecting passive mode even when FTP_PASSIVE_MODE was not
set at all, which is really quite surprising unless you know about it. So
change it to the agreed default behaviour of selecting passive mode if
FTP_PASSIVE_MODE is set, but not "no".
- ftpTimeout was not honored when reading actual data, as opposed to
talking protocol
- connection caching was broken because _ftp_cached_connect() would see
the result of the transfer instead of the result of the NOOP.
- if the RETR succeeded, but an error occurred later (as can happen
when talking to a proxy), the error would not be detected.
There still remains to register an atexit(3) callback to close the cached
connection gracefully instead of just dropping it on the floor.
a bug in some ftp servers (most notably ftp.vmunix.com) which report the
size of a file correctly in ascii mode, but report it as 0 in binary mode.
Reported by: asmodai
Also remove an unneeded initialization.
Sort out the size / length confusion. Always try to report the *real* file
size in the url_stat structure, no matter how much of it is actually being
sent, and try to detect inconsistencies between sizes.
Rearrange the request loop to avoid having to add meaningless code just to
silence compiler warnings.
Switch to a more sensible and consistent interface for the _http_parse*()
functions.
than requested. Instead, inform the caller of the real offset by modifying
the offset field in the original struct url, and let him decide how to handle
the situation.
fetchStat*(). In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetchGet*() is identical to calling
fetchXGet*() with the second argument set to NULL.
outside the loop inspects it to determine whether or not we succeeded in
retrieving the requested document. This fixes a bug where fetchGetHTTP()
would return a FILE with an invalid file descriptor if it hit the redirect
limit without locating the requested document.
or not interrupted system calls will be restarted. This fixes a bug where
fetch(1) would hang (potentially forever) if a server stopped responding,
because the signal handler would absorb the user's efforts to interrupt the
transfer.
moved around, but the acutal functional changes are small.
Add support for site-internal redirects (where the Location: header gives a
path instead of an absolute URI)
Pointed out by: kuriyama
more robust, and somewhat more efficient. It also handles authorization and
redirects properly, and supports timeouts like the FTP code.
Many thanks to Umemoto-san for his assistance with IPv6 support, both here
and in other parts of libfetch.
* Add the 'h' ftp flag (allocate local port in high range)
* Add the 'd' flag (use direct connection even if proxy is defined)
* Make sure flags != NULL before calling strchr().
* Minor changes to some comments.
Brucify the Makefile.
Differentiate atime and mtime in fetch*Stat().
Fix a few pointer bugs.
Tweak some error messages.
Don't #include sys/param.h and stdio.h in fetch.h.
Document that sys/param.h and stdio.h must be #included before fetch.h.
This commit introduces the following features:
a) the fetchStat*() functions, which return meta-information for a
document, such as size, modification time, etc.
b) the use of the com_err(3) facilities to report errors.
It also fixes a bunch of style bugs and a few logic bugs and somewhat
improves the man page.
Changed files, in alphabetical order:
Makefile:
Don't generate macros in {ftp,http}err.c.
Generate category fields for the error message lists.
Compile the error table.
Install fetch_err.h along with fetch.h.
common.c:
Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the
error code in the _netdb_seterr() macro.
Add categories to the _netdb_errlist table.
Report errors through the Common Error library.
common.h:
Add the DEBUG macros.
Add prototype for fetchConnect().
Remove the prototype for _fetch_errstring(), which is local to common.c
Add a categroy field to struct fetcherr, and define constants for
error categories.
Define macros for _{url,netdb,ftp,http}_seterr().
errors.et: (new file)
List error categories.
fetch.3:
Document the fetchStat*() functions.
Move the "unimplemented functionality" comments from NOTES to BUGS.
Document that applications which use libfetch must also use
libcom_err, and list existing error codes.
Undocument fetchLastErr{Code,String}.
Remove the (empty) DIAGNOSTICS section.
Mention Eugene Skepner in the AUTHORS section.
fetch.c:
Move the DEBUG macros to common.c
Add fetchStat() and fetchStatURL().
Generate error messages for URL parser errors, and fix a minor bug
in the parser.
Use 'struct url' instead of 'url_t'.
Remove fetchLastErr{Code,String}.
fetch.h:
Use 'struct url' instead of 'url_t', and remove the typedef.
Define struct url_stat (used by fetchStat()).
Add prototypes for fetchStat*().
Remove the declarations for fetchLastErr{Code,String}.
Include fetch_err.h.
fetch_err.et: (new file)
Error table for libfetch.
file.c:
Add fetchStatFile().
Use 'struct url' instead of 'url_t'.
ftp.c:
Add fetchStatFTP().
Use 'struct url' instead of 'url_t'.
Don't use fetchLastErrCode.
ftp.errors:
Add categories to all error messages.
http.c:
Add fetchStatHTTP().
Use 'struct url' instead of 'url_t'.
Don't use fetchLastErr{Code,Text}.
http.errors:
Add categories to all error messages.
Prompted by: jkh and Eugene Skepner
Numerous sugestions from: Garett Wollman and Eugene Skepner
alphabetical order:
Makefile:
Add common.c to SRCS.
Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG
Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr
instead.
README:
Remove the todo list, which is out of date anyway.
common.c: (new file)
Gather utility functions in this file.
Merge the error reporting functions intp _fetch_errstring(),
_fetch_seterr() and _fetch_syserr().
Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect
fails.
common.h: (new file)
Gather internal prototypes and structures in this files.
fetch.3:
Undocument fetchFreeURL().
Document a few more known bugs.
Document fetchLastErrCode and fetchLastErrText.
fetch.c:
Add descriptive comments to all functions that lacked them.
Move fetchConnect() to common.c.
Obviate the need for fetchFreeURL(), and remove it.
fetch.h:
Modify struct url_t so the document part is at the end.
ftp.c:
Remove code that is duplicated elsewhere.
http.c:
Remove code that is duplicated elsewhere.
Prompted by: jkh
- replaced bogus dependencies of distribution sources on generated
sources by the same ones that bsd.lib.mk would generate if it knew
all the sources. We shoot ourself in the foot by not naming the
generated (included) sources *.h, so we can just put the generated
sources in SRCS.
- replaced -I${.CURDIR} by `-I.'. Here `.' is an alias for ${.OBJDIR}.
-I${.CURDIR} didn't do anything, since ${.CURDIR} is the default in
all cases here (it would be necessary for ""-style includes made from
sources in ${.OBJDIR}.
Don't use `+=' for setting CLEANFILES for the first time.
Added $Id$.
- Fix the README to reflect the new status of the ftp code.
- Change tons of 'if (xxx < 0)' to 'if (xxx == -1)'
- Add two new interface functions
- Fix the Makefile so it actually works (yay!)
Now the manpage is lagging even further behind... :( Next on the todo
list is to clean up the http code.
