Commit Graph

78 Commits

Author SHA1 Message Date
Dag-Erling Smørgrav
767207650c As pointed out by several people, r273114 was incorrect: it unconditionally
disabled everything except TLS 1.0.  Replace it with a more carefully
wrought patch:

 - Switch the default for SSLv3 from on to off
 - Add environment variables to control TLS 1.1 and 1.2
 - In verbose mode, report which version is used
 - Update the man page to reflect these changes.

MFC after:	1 week
2014-10-15 07:35:50 +00:00
Dag-Erling Smørgrav
d75ca0c1fd Drop support for SSLv3.
MFC after:	1 week
2014-10-15 02:34:26 +00:00
Baptiste Daroussin
3674911c29 Add a comment to explain the EAGAIN is only there for POSIX compliance
Resquested by:	kib
Reviewed by:	des
2014-07-15 15:29:43 +00:00
Baptiste Daroussin
4472d6e1df Support EAGAIN in fetch_writev
Reviewed by:	des
Approved by:	des
2014-07-10 13:04:52 +00:00
Dag-Erling Smørgrav
e9de92ad83 Look for root certificates in /usr/local/etc/ssl before /etc/ssl.
MFH:	1 week
2014-05-17 03:39:56 +00:00
Dag-Erling Smørgrav
4524013cd3 Bump copyright dates 2014-01-30 08:37:23 +00:00
Dag-Erling Smørgrav
9c1ca3a1dd r261230 broke the cases where the amount of data to be read is not
known in advance, or where the caller doesn't care and just keeps
reading until it hits EOF.

In fetch_read(): the socket is non-blocking, so read() will return 0
on EOF, and -1 (errno == EAGAIN) when the connection is still open but
there is no data waiting.  In the first case, we should immediately
return 0.  The EINTR case was also broken, although not in a way that
matters.

In fetch_writev(): use timersub() and timercmp() as in fetch_read().

In http_fillbuf(): set errno to a sensible value when an invalid chunk
header is encountered.

In http_readfn(): as in fetch_read(), a zero return from down the
stack indicates EOF, not an error.  Furthermore, when io->error is
EINTR, clear it (but no errno) before returning so the caller can
retry after dealing with the interrupt.

MFC after:	3 days
2014-01-29 12:48:19 +00:00
Dag-Erling Smørgrav
215a27f1a4 Solve http buffering issues and hangs once and for all (hopefully!) by
simply not trying to return exactly what the caller asked for - just
return whatever we got and let the caller be the judge of whether it
was enough.  If an error occurs or the connection times out after we
already received some data, return a short read, under the assumption
that the next call will fail or time out before we read anything.

As it turns out, none of the code that calls fetch_read() assumes an
all-or-nothing result anyway, except for a couple of lines where we
read the CR LF at the end of a hunk in HTTP hunked encoding, so the
changes outside of fetch_read() and http_readfn() are minimal.

While there, replace select(2) with poll(2).

MFC after:	3 days
2014-01-28 12:48:17 +00:00
Dag-Erling Smørgrav
3443b2bc31 Fix format string.
Submitted by:	Jörg Sonnenberger <joerg@NetBSD.org>
MFC after:	1 week
2014-01-20 11:13:05 +00:00
Bryan Drewery
5313966ea6 Fix build with GCC
SSL_set_tlsext_host_name(3) internally does not modify the host buffer
pased to it. So it is safe to DECONST the struct url* here.

Reported by:	gjb
Approved by:	bapt (implicit)
MFC after:	1 week
X-MFC-With:	r258347
2013-11-19 16:11:03 +00:00
Bryan Drewery
4cb272a079 Support SNI in libfetch
SNI is Server Name Indentification which is a protocol for TLS that
indicates the host that is being connected to at the start of the
handshake. It allows to use Virtual Hosts on HTTPS.

Submitted by:	sbz
Submitted by:	Michael Gmelin <freebsd@grem.de> [1]
PR:		kern/183583 [1]
Reviewed by:	des
Approved by:	bapt
MFC after:	1 week
2013-11-19 15:35:26 +00:00
Dag-Erling Smørgrav
dcd47379ff Implement certificate verification, and many other SSL-related
imrovements; complete details in the PR.

PR:		kern/175514
Submitted by:	Michael Gmelin <freebsd@grem.de>
MFC after:	1 week
2013-07-26 15:53:43 +00:00
Jilles Tjoelker
28fd93073a libfetch: Avoid SIGPIPE on network connections.
To avoid unexpected process termination from SIGPIPE when writing to a
closed network connection, enable SO_NOSIGPIPE on all network connections.

The POSIX standard MSG_NOSIGNAL is not used since it requires modifying all
send calls to add this flag. This is particularly nasty for SSL connections.

Reviewed by:	des
Tested by:	bapt
MFC after:	5 days
2012-05-28 19:22:23 +00:00
Dag-Erling Smørgrav
30787285b5 Since the socket is non-blocking, it is necessary to use select(2) even
when there is no timeout, because read(2) will return immediately if there
is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on
slow connections.

MFC after:	3 weeks
Noticed by:	Yanhui Shen <shen.elf@gmail.com>
2012-04-30 12:11:45 +00:00
Dag-Erling Smørgrav
e6e3bcd376 Fix two nits in previous commit pointed out by pjd@.
MFC after:	3 weeks
2012-01-23 09:23:07 +00:00
Dag-Erling Smørgrav
2a7daafe67 Fix two issues related to the use of SIGINFO in fetch(1) to display
progress information.  The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal.  The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.

PR:		bin/153240
Submitted by:	Mark <markjdb@gmail.com>
MFC after:	3 weeks
2012-01-18 15:13:21 +00:00
Dag-Erling Smørgrav
578153f1ba latin1 -> utf8 2011-10-19 11:43:51 +00:00
Dag-Erling Smørgrav
6337341d81 Update copyright dates and strip my middle name. 2011-09-27 18:57:26 +00:00
Dag-Erling Smørgrav
15b68c63bb Mark all socket and file descriptors close-on-exec.
PR:		bin/151866
MFC after:	3 weeks
2011-05-13 07:21:41 +00:00
Ed Maste
a9d0c84909 Move variable declarations into the conditional block where they are
used, to fix warning if WITH_SSL is not set.

Submitted by:	Sean Bruno
MFC after:	1 week
2010-10-24 01:05:10 +00:00
Dag-Erling Smørgrav
caaffed8f0 Redo fetch_read() using non-blocking sockets. This is necessary to
avoid a hang in the SSL case if the server sends a close notification
before we are done reading.  In the non-SSL case, it can provide a
minor (but probably not noticeable) performance improvement for small
transfers.

MFC after:	3 weeks
2010-07-28 16:11:22 +00:00
Fabien Thomas
c0d2581bcb PR: 139751
Approved by: des
Obtained from: Xavier Heiny <xavier.heiny@netasq.com>
MFC after: 3 weeks
2009-10-21 18:29:26 +00:00
Colin Percival
fc2841a92f Fix one-byte buffer overflow: NUL gets written to the buffer, but isn't
counted in the width specification in scanf.

This is not a security problem, since this function is only used to
parse a user's configuration file.

Submitted by:	Joerg Sonnenberger
Obtained from:	dragonflybsd
MFC after:	1 week
2008-04-15 23:29:51 +00:00
Dag-Erling Smørgrav
5092cf0569 s/wait/delta/ to avoid namespace collision.
MFC after:	2 weeks
2008-03-20 09:55:27 +00:00
Dag-Erling Smørgrav
340b079be0 Use memcpy(3) instead of the BSD-specific bcopy(3).
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	2 weeks
2008-02-08 09:48:48 +00:00
Dag-Erling Smørgrav
facd982794 As several people pointed out, I did all the ctype casts the wrong
way (not for the first time...)

Noticed by:	bde, ru ++
MFC after:	1 week
2007-12-19 00:26:36 +00:00
Dag-Erling Smørgrav
62a2681c93 Add support for the NO_PROXY / no_proxy environment variable as used by
lynx, curl etc.  Note that this patch differs significantly from that
in the PR, as the submitter refined it after submitting the PR.

PR:		110388
Submitted by:	Alexander Pohoyda <alexander.pohoyda@gmx.net>
MFC after:	3 weeks
2007-12-18 11:03:07 +00:00
Dag-Erling Smørgrav
a1b37df2d7 Clean up namespace violations.
MFC after:	1 week
2007-12-14 10:26:58 +00:00
Dag-Erling Smørgrav
55cf7be1ab Fix a memory leak: when freeing the connection structure, don't forget to
free the connection buffer as well.

PR:		bin/76153
MFC after:	1 week
2005-02-16 12:46:46 +00:00
Dag-Erling Smørgrav
2cbbf9dac9 Update copyright years. 2004-09-21 18:35:21 +00:00
Hajimu UMEMOTO
3d82ba4313 preparation for RFC3493. EAI_NODATA was deprecated. 2003-10-23 13:50:01 +00:00
Dag-Erling Smørgrav
c42cb9d906 Add and document support for a FETCH_BIND_ADDRESS environment variable
specifying a local address to bind sockets to.  Caveat: lightly tested.

PR:		bin/37572
2003-03-03 12:35:03 +00:00
Dag-Erling Smørgrav
930105c1e9 style(9): add parentheses to sizeof even when not strictly required.
MFC after:	3 days
2003-01-28 08:04:40 +00:00
John W. De Boskey
9015b953d6 Fix signed/unsigned comparison warning/error from 'make release' 2003-01-28 00:33:53 +00:00
Dag-Erling Smørgrav
07350d12cf Experimental support for .netrc. 2003-01-22 17:53:58 +00:00
Dag-Erling Smørgrav
f8020ddefe Set auto-retry mode to avoid some spurious errors.
Submitted by:	Andre Albsmeier <andre.albsmeier@siemens.com>
MFC after:	7 days
2003-01-03 02:45:10 +00:00
Dag-Erling Smørgrav
7504527ed2 Fix a bug in fenner's _fetch_writev() patch (rev 1.36)
Submitted by:	fenner
2002-10-30 14:25:00 +00:00
Dag-Erling Smørgrav
bb13d0af67 Recommit the non-broken parts of 1.34 and 1.37.
Change the type and name of a variable introduced in 1.33.
2002-10-30 04:43:00 +00:00
Warner Losh
a4a37038bb Reinstate revs 1.35-36 and 1.38. Revisions 1.34 and 1.37 were specifically
the root cause of the bus errors I was experiencing.

Submitted by:	fenner
Tested by:	obrien
Prompted by:	peter
2002-10-30 00:17:16 +00:00
David E. O'Brien
b68fbebd5a Fix `pkg_add -r' by backing out revs 1.34-1.38.
Revs 1.37-8 produce a bus error in some environments.
Revs 1.34-6 do not bus error, but write corrupted files.
2002-10-29 12:17:43 +00:00
Dag-Erling Smørgrav
32a4a82829 Fix an off-by-one error (> where >= should have been used) which caused
_fetch_writev() to incorrectly report EPIPE in certain cases.

Also fix a number of const warnings by using __DECONST(), plus a signed /
unsigned comparison by casting the rhs to ssize_t.

Submitted by:	fenner, Craig Rodrigues <rodrigc@attbi.com>
2002-10-28 10:19:03 +00:00
Dag-Erling Smørgrav
1a5424b137 Slight amendment to rev 1.34: instead of considering any short read an
error, only report an error if no data was read at all (unless len was
0 to start with).  Otherwise, the final read of practically any transfer
will end in a fatal error.
2002-10-27 17:20:49 +00:00
Dag-Erling Smørgrav
2761348f78 Introduce _fetch_writev(), which is the conn_t version of writev(2). In
the SSL case, it is no different from the old _fetch_write(), but in the
non-SSL case it uses writev(2) to send the entire vector as a single
packet (provided it can fit in one packet).  Implement _fetch_write()
and _fetch_putln() in terms of _fetch_writev().

This should improve performance in the non-SSL case (by reducing protocol
overhead) and solve the problem where too-smart-for-their-own-good
firewalls reject FTP packets that do not end in CRLF.

PR:		bin/44123
Submitted by:	fenner
2002-10-27 16:11:21 +00:00
Dag-Erling Smørgrav
9f788e9c90 Eliminate two cases of undefined behaviour: total in _fetch_write() was
not initialized before use, and _http_growbuf() did not return a value
on success.

Reported by:	Peter Edwards <pmedwards@eircom.net>
MFC after:	2 weeks
2002-10-27 15:43:40 +00:00
Dag-Erling Smørgrav
e24f60e74f Back out the previous commit, and fix the bug rather than try to hide its
symptoms: make timeouts and short transfers fatal, and set errno to an
appropriate value (ETIMEDOUT for a timeout, EPIPE for a short transfer).

MFC after:	2 weeks
2002-10-27 15:08:21 +00:00
Alfred Perlstein
a6756ecc22 Fix an infinite loop when _fetch_read() can return 0 (if the
connection is broken), take this into account and return at this
point.
2002-09-20 21:50:57 +00:00
Bill Fenner
40cfbfd508 Make _fetch_connect() always set the error code.
Tell ftp that _fetch_connect() always sets the error code (http already knew)
2002-09-17 05:54:33 +00:00
Dag-Erling Smørgrav
66ffb8a371 Reintroduce debugging code that somehow got lost in a previous revision. 2002-06-24 12:18:41 +00:00
Dag-Erling Smørgrav
f606d589b9 Add a reference count to struct fetchconn so we don't prematurely close and
free a cached FTP connection.
2002-06-11 11:27:28 +00:00
Dag-Erling Smørgrav
3070f6cb06 Make SSL support conditional on NOCRYPT. 2002-06-05 21:35:35 +00:00