Commit Graph

325 Commits

Author SHA1 Message Date
bdrewery
b8eaeebf21 MFC r258347,r258349:
Support SNI in libfetch

  SNI is Server Name Indentification which is a protocol for TLS that
  indicates the host that is being connected to at the start of the
  handshake. It allows to use Virtual Hosts on HTTPS.

PR:		kern/183583
Approved by:	bapt (implicit)
Approved by:	re (gjb)
2013-11-29 21:13:30 +00:00
des
0b46e38a09 Even though it doesn't really make sense in the context of a CONNECT
request, RFC 2616 14.23 mandates the presence of the Host: header in
all HTTP 1.1 requests.

PR:		kern/181445
Submitted by:	Kimo <kimor79@yahoo.com>
MFC after:	3 days
2013-08-22 07:43:36 +00:00
des
0b1a1b2338 Include an Accept header in requests.
PR:		kern/180917
MFC after:	1 week
2013-07-30 13:07:55 +00:00
des
29d3efe0a8 Implement certificate verification, and many other SSL-related
imrovements; complete details in the PR.

PR:		kern/175514
Submitted by:	Michael Gmelin <freebsd@grem.de>
MFC after:	1 week
2013-07-26 15:53:43 +00:00
des
50344d2c6f Use the correct request syntax for proxied (tunneled) HTTPS requests.
PR:		bin/180666
MFC after:	3 days
2013-07-21 06:59:56 +00:00
kientzle
7a1b8ef999 Fix -Wunsequenced warning.
Submitted by:	dt71@gmx.com
2013-06-29 15:51:27 +00:00
des
b2e53cc2ea Use the CONNECT method to proxy HTTPS connections through HTTP proxies.
PR:		bin/80176
Submitted by:	Yuichiro NAITO <naito.yuichiro@gmail.com>
2013-04-12 22:05:15 +00:00
des
f81cc66625 Fix weird indentation. 2012-11-16 12:31:43 +00:00
eadler
106fb1fefe Implement HTTP 305 redirect handling.
PR:		172452
Submitted by:	gcooper
Reviewed by:	des
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:15 +00:00
eadler
f647c87f9c Don't deny non-temporary redirects if the -A option is set (per
the man page) [0]

While here add support for draft-reschke-http-status-308-07

PR:		172451 [0]
Submitted by:	gcooper [0]
Reviewed by:	des
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:10 +00:00
eadler
e82cfe8e30 Be a bit more lenient in the maximum number of redirects allowed.
Chrome and Firefox have a limit of 20. IE has a limit of 8.

Reviewed by:	des
Approved by:	cperciva
MFC after:	3 days
2012-10-22 03:00:04 +00:00
des
d123c9234a Use libmd if and only if OpenSSL is not available.
PR:		bin/171402
MFC after:	3 days
2012-09-14 13:00:43 +00:00
eadler
f1f9181f8f Adding missing return statements during error conditions.
PR:		kern/171187
Submitted by:	Mark Johnston <markjdb@gmail.com>
Reviewed by:	des
Approved by:	cperciva
MFC after:	2 weeks
2012-09-14 12:15:13 +00:00
jkim
e393e47b1e Merge OpenSSL 1.0.1c.
Approved by:	benl (maintainer)
2012-07-12 19:30:53 +00:00
jilles
9118921236 libfetch: Avoid SIGPIPE on network connections.
To avoid unexpected process termination from SIGPIPE when writing to a
closed network connection, enable SO_NOSIGPIPE on all network connections.

The POSIX standard MSG_NOSIGNAL is not used since it requires modifying all
send calls to add this flag. This is particularly nasty for SSL connections.

Reviewed by:	des
Tested by:	bapt
MFC after:	5 days
2012-05-28 19:22:23 +00:00
des
ad64077869 Don't reuse credentials if redirected to a different host.
Submitted by:	Niels Heinen <heinenn@google.com>
MFC after:	3 weeks
2012-04-30 12:12:48 +00:00
des
34d79578ae Since the socket is non-blocking, it is necessary to use select(2) even
when there is no timeout, because read(2) will return immediately if there
is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on
slow connections.

MFC after:	3 weeks
Noticed by:	Yanhui Shen <shen.elf@gmail.com>
2012-04-30 12:11:45 +00:00
emaste
b6c463eabc Support percent-encoded user and password
RFC 1738 specifies that any ":", "@", or "/" within a user name or
password in a URL is percent-encoded, to avoid ambiguity with the use
of those characters as URL component separators.

Reviewed by:    rstone@
MFC after:	1 month
2012-04-11 15:42:02 +00:00
eadler
1ef5fe44d3 Remove trailing whitespace per mdoc lint warning
Disussed with:	gavin
No objection from:	doc
Approved by:	joel
MFC after:	3 days
2012-03-29 05:02:12 +00:00
des
4b95b05db4 Fix two nits in previous commit pointed out by pjd@.
MFC after:	3 weeks
2012-01-23 09:23:07 +00:00
des
939a66af62 Fix two issues related to the use of SIGINFO in fetch(1) to display
progress information.  The first is that fetch_read() (used in the HTTP
code but not the FTP code) can enter an infinite loop if it has previously
been interrupted by a signal.  The second is that when it is interrupted,
fetch_read() will discard any data it may have read up to that point.
Luckily, both bugs are extremely timing-sensitive and therefore difficult
to trigger.

PR:		bin/153240
Submitted by:	Mark <markjdb@gmail.com>
MFC after:	3 weeks
2012-01-18 15:13:21 +00:00
des
5d19d6ad80 latin1 -> utf8 2011-10-19 11:43:51 +00:00
des
b99d90c0bb Update copyright dates and strip my middle name. 2011-09-27 18:57:26 +00:00
des
5d14e77ed4 Bump date. 2011-09-27 18:53:35 +00:00
des
36218c0055 Think first, commit second.
1. Allow the caller to select active mode.
2. Fix the envar logic so it *always* overrides the caller's flags.
3. Document the change from active to passive.
2011-09-27 18:42:09 +00:00
des
23b1a74459 Long overdue: make passive mode the default for ftp. 2011-09-27 18:23:58 +00:00
des
410ff8d8e2 Mark all socket and file descriptors close-on-exec.
PR:		bin/151866
MFC after:	3 weeks
2011-05-13 07:21:41 +00:00
des
18a608d91f Builds cleanly at the default WARNS level (WARNS=6). 2011-05-12 21:30:46 +00:00
des
9740a3f77d Increase WARNS to 4. 2011-05-12 21:26:42 +00:00
des
fb4144ec68 Mechanical whitespace cleanup. 2011-05-12 21:18:55 +00:00
des
db3b3a9ab5 Increase WARNS to 3. 2011-05-12 21:12:24 +00:00
des
ae98b060b9 Fix a bug related to connection caching which could cause a crash after
a STAT command.

PR:		kern/153748 (different patch)
Submitted by:	Mark Johnston <markjdb@gmail.com>
MFC after:	2 weeks
2011-01-17 17:25:46 +00:00
emaste
c50e7d3467 Move variable declarations into the conditional block where they are
used, to fix warning if WITH_SSL is not set.

Submitted by:	Sean Bruno
MFC after:	1 week
2010-10-24 01:05:10 +00:00
des
5df649f424 Redo fetch_read() using non-blocking sockets. This is necessary to
avoid a hang in the SSL case if the server sends a close notification
before we are done reading.  In the non-SSL case, it can provide a
minor (but probably not noticeable) performance improvement for small
transfers.

MFC after:	3 weeks
2010-07-28 16:11:22 +00:00
des
6aaf496254 Fix a couple of embarrassing mistakes in the previous commit.
Submitted by:	Dimitry Andric <dimitry@andric.com>
2010-07-28 15:29:18 +00:00
des
9c3639722d If the A flag is supplied, http_request() will attempt the request only
once, even if authentication is required, instead of retrying with the
proper credentials.  Fix this by bumping the countdown if the origin or
proxy server requests authentication so that the initial unauthenticated
request does not count as an attempt.

PR:		148087
Submitted by:	Tom Evans <tevans.uk@googlemail.com>
MFC after:	2 weeks
2010-07-01 17:44:33 +00:00
des
d2d2cf7b2e Document HTTP digest authentication support.
Mention some prominent past contributors: Hajimu Umemoto (ipv6), Henry
Whincup (https), Jukka Ukkonen (if-modified-since) and Jean-François
Dockes (digest auth)
2010-01-26 15:06:22 +00:00
des
0e83531890 Add missing DPADD and fix whitespace.
Submitted by:	ru
2010-01-19 13:31:57 +00:00
des
709f5f5dad Add HTTP digest authentication.
Submitted by:	Jean-Francois Dockes <jf@dockes.org>
Forgotten by:	des (repeatedly)
2010-01-19 10:19:55 +00:00
attilio
6b1883b1f0 Fix a socket leak in ftp_request() after that a connection is established.
Submitted by:	Sandvine Incorporated
Reviewed by:	des, emaste
Sponsored by:	Sandvine Incorporated
MFC:		1 week
2009-11-25 14:57:07 +00:00
fabient
5ac7c6cc4e PR: 139751
Approved by: des
Obtained from: Xavier Heiny <xavier.heiny@netasq.com>
MFC after: 3 weeks
2009-10-21 18:29:26 +00:00
kensmith
9c2c634ee9 Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE.  Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by:    kib
Approved by:    re (rwatson)
2009-07-19 17:25:24 +00:00
murray
a12379d1b7 1. Update fetch to consistently return 1 on error, as the man page states,
rather than usually returning 1 but in a few instances using a sysexits(3)
   return value.

2. Remove a few unused variables from libfetch.

PR:		docs/122470  (1, only)
Reviewed by:	des
> Description of fields to fill in above:                     76 columns --|
> PR:            If a GNATS PR is affected by the change.
> Submitted by:  If someone else sent in the change.
> Reviewed by:   If someone else reviewed your modification.
> Approved by:   If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after:     N [day[s]|week[s]|month[s]].  Request a reminder email.
> Security:      Vulnerability reference (one per line) or description.
> Empty fields above will be automatically removed.

M    usr.bin/fetch/fetch.c
M    lib/libfetch/fetch.c
2008-12-17 18:00:18 +00:00
murray
72a890ccd7 Add support for HTTP 1.1 If-Modified-Since behavior.
fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.

libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.

PR:		bin/87841
Submitted by:	Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by:	des, ru
MFC after:	3 weeks
2008-12-15 08:27:44 +00:00
ru
9a16e00e36 Don't fail mistakenly with -r when we already have the whole file.
Reviewed by:	des
2008-10-24 07:56:01 +00:00
cperciva
4ee0999e20 Fix one-byte buffer overflow: NUL gets written to the buffer, but isn't
counted in the width specification in scanf.

This is not a security problem, since this function is only used to
parse a user's configuration file.

Submitted by:	Joerg Sonnenberger
Obtained from:	dragonflybsd
MFC after:	1 week
2008-04-15 23:29:51 +00:00
des
0b6cf6e8e4 s/wait/delta/ to avoid namespace collision.
MFC after:	2 weeks
2008-03-20 09:55:27 +00:00
des
ff0eb6dba4 Use memcpy(3) instead of the BSD-specific bcopy(3).
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	2 weeks
2008-02-08 09:48:48 +00:00
des
1de1bb1bc6 s/MAXPATHLEN/PATH_MAX/ to reflect five-year old change to the code :)
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	2 weeks
2008-02-08 09:44:34 +00:00
des
d129ae8c34 Add necessary cast for tolower() argument.
Submitted by:	Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after:	1 week
2008-02-06 11:39:55 +00:00