Commit Graph

371 Commits

Author SHA1 Message Date
schweikh
19b98edeaa Correct some grammar. 2013-06-30 17:59:40 +00:00
ed
3764c06cce Don't let hastd use C11 atomics.
Due to possible concerns about the stability of C11 atomics, use our
existing atomics API instead.

Requested by:	pjd
2013-06-29 20:13:39 +00:00
ed
4ea862e26e Let hastd use C11 atomics.
C11 atomics now work on all the architectures. Have at least a single
piece of software in our base system that uses C11 atomics. This
somewhat makes it less likely that we break it because of LLVM imports,
etc.
2013-06-15 22:17:59 +00:00
jkim
d752b745df Improve compatibility with old flex and fix build with GCC. 2013-05-22 17:47:45 +00:00
trociny
f39c963dc2 Get rid of libl dependency. We needed it only to provide yywrap. But
yywrap is not necessary when parsing a single hast.conf file.

Suggested by:	kib
Reviewed by:	pjd
2013-05-11 09:51:44 +00:00
ed
a09c9f5550 Partially revert my last change.
I forgot that I still had a locally applied patch to my copy of Clang
that needs to be pushed in before we should use C11 atomics.
2013-04-27 05:06:25 +00:00
ed
e802849740 Use C11 <stdatomic.h> instead of our non-standard <machine/atomic.h>.
Reviewed by:	pjd
2013-04-27 05:01:29 +00:00
ed
120125784a Add the Clang specific -Wmissing-variable-declarations to WARNS=6.
This compiler flag enforces that that people either mark variables
static or use an external declarations for the variable, similar to how
-Wmissing-prototypes works for functions.

Due to the fact that Yacc/Lex generate code that cannot trivially be
changed to not warn because of this (lots of yy* variables), add a
NO_WMISSING_VARIABLE_DECLARATIONS that can be used to turn off this
specific compiler warning.

Announced on:	toolchain@
2013-04-19 19:45:00 +00:00
sjg
97d8b94956 sync from head 2013-04-12 20:48:55 +00:00
pjd
3d6eb97a85 Now that ioctl(2) is allowed in capability mode and we can limit ioctls for the
given descriptors, use Capsicum sandboxing for hastd in primary and secondary
modes. Allow for DIOCGDELETE and DIOCGFLUSH ioctls on provider descriptor and
for G_GATE_CMD_MODIFY, G_GATE_CMD_START, G_GATE_CMD_DONE and G_GATE_CMD_DESTROY
on GEOM Gate descriptor.

Sponsored by:	The FreeBSD Foundation
2013-03-14 23:14:47 +00:00
pjd
aa00097868 Minor corrections. 2013-03-14 23:11:52 +00:00
pjd
5f4ba049f3 Delete requests can be larger than MAXPHYS. 2013-03-14 23:03:48 +00:00
sjg
6d37b86f2b Updated dependencies 2013-03-11 17:21:52 +00:00
trociny
8690e69f6a Add i/o error counters to hastd(8) and make hastctl(8) display
them.  This may be useful for detecting problems with HAST disks.

Discussed with and reviewed by:	pjd
MFC after:	1 week
2013-02-25 20:09:07 +00:00
pjd
0bf19fd812 - Add support for 'memsync' mode. This is the fastest replication mode that's
why it will now be the default.
- Bump protocol version to 2 and add backward compatibility for version 1.
- Allow to specify hosts by kern.hostid as well (in addition to hostname and
  kern.hostuuid) in configuration file.

Sponsored by:	Panzura
Tested by:	trociny
2013-02-17 21:12:34 +00:00
sjg
0ee5295509 Updated dependencies 2013-02-16 01:23:54 +00:00
obrien
3028e3f8ab Sync with HEAD. 2013-02-08 16:10:16 +00:00
kevlo
804c67a486 Fix socket calls on error post-r243965.
Submitted by:	Garrett Cooper
2012-12-21 15:54:13 +00:00
pjd
712fa475ac Revert r228695. We use __func__ here as a format to distinguish between
abort and assert. It would be cleaner to use NULL or "" here, but gcc
complains in both cases.
2012-11-05 00:38:14 +00:00
sjg
778e93c51a Sync from head 2012-11-04 02:52:03 +00:00
marcel
9dd41e3647 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
trociny
48789932a6 Metaflush on/off values don't need quotes.
Reviewed by:	pjd
MFC after:	3 days
2012-07-16 20:43:28 +00:00
pjd
1fb1092d75 Make use of GEOM Gate direct reads feature. This allows HAST to serve
reads with native speed of the underlying provider.
There are three situations when direct reads are not used:
1. Data is being synchronized and synchronization source is the secondary
   node, which means secondary node has more recent data and we should read
   from it.
2. Local read failed and we have to try to read from the secondary node.
3. Local component is unavailable and all I/O requests are served from the
   secondary node.

Sponsored by:	Panzura, http://www.panzura.com
MFC after:	1 month
2012-07-04 20:20:48 +00:00
pjd
c34c233e83 Check if there is cmsg at all.
MFC after:	3 days
2012-07-01 16:26:07 +00:00
hselasky
6a1a47c355 Revert: r236909
Pointyhat: me
2012-06-11 20:27:52 +00:00
hselasky
6b0d4730f1 Use the correct clock source when computing timeouts.
MFC after:	1 week
2012-06-11 19:20:59 +00:00
pjd
2143a387af Simplify the code by using snprlcat().
MFC after:	3 days
2012-06-03 10:50:46 +00:00
wblock
9fa9a2acad Fixes to man8 groff mandoc style, usage mistakes, or typos.
PR:		168016
Submitted by:	Nobuyuki Koganemaru
Approved by:	gjb
MFC after:	3 days
2012-05-24 02:24:03 +00:00
bapt
310ab6d7ff Fix world after byacc import:
- old yacc(1) use to magicially append stdlib.h, while new one don't
- new yacc(1) do declare yyparse by itself, fix redundant declaration of
  'yyparse'

Approved by:	des (mentor)
2012-05-22 16:33:10 +00:00
gjb
67d88d49d4 General mdoc(7) and typo fixes.
PR:		167804
Submitted by:	Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after:	3 days
2012-05-12 15:08:22 +00:00
trociny
5780d99776 If hastd is invoked with "-P pidfile" option always create pidfile
regardless of whether -F (foreground) option is set or not.

Also, if -P option is specified, ignore pidfile setting from configuration
not only on start but on reload too. This fixes the issue when for hastd
run with -P option reload caused the pidfile change.

Reviewed by:	pjd
MFC after:	1 week
2012-03-29 20:11:16 +00:00
trociny
4eba690e91 Fix typo.
MFC after:	3 days
2012-03-23 20:18:48 +00:00
pjd
3e86e21237 Nice range comparison.
MFC after:	3 days
2012-02-11 16:41:52 +00:00
trociny
1d510ea553 If a local write request is from the synchronization thread, when it
is synchronizing data that is out of date on the local component, we
should not send G_GATE_CMD_DONE acknowledge to the kernel.

This fixes the issue, observed in async mode, when on synchronization
from the remote component the worker terminated with "G_GATE_CMD_DONE
failed" error.

Reported by:	Artem Kajalainen <artem kayalaynen ru>
Reviewed by:	pjd
MFC after:	1 week
2012-02-05 15:23:32 +00:00
trociny
d2786f0a2c Fix the regression introduced in r226859: if the local component is
out of date BIO_READ requests got lost instead of being sent to the
remote component.

Reviewed by:	pjd
MFC after:	1 week
2012-02-05 15:21:08 +00:00
pjd
306b4958cf Fix typo in comment.
MFC after:	3 days
2012-02-04 07:59:12 +00:00
pjd
538ff2e745 - Fix documentation to note that /etc/hast.conf is the default configuration
file for hastd(8) and hastctl(8) and not hast.conf.
- In copyright statement correct that this file is documentation, not software.
- Bump date.

MFC after:	3 days
2012-01-24 23:43:13 +00:00
pjd
983cddf8a6 Free memory that won't be used in child.
MFC after:	1 week
2012-01-22 11:20:42 +00:00
pjd
bf37868a59 Fix minor memory leak.
MFC after:	3 days
2012-01-21 20:13:37 +00:00
pjd
62f2fd64bb Remove another unused token.
MFC after:	3 days
2012-01-20 21:49:56 +00:00
pjd
8e6c8deb43 Remove unused token 'port'.
MFC after:	3 days
2012-01-20 21:45:24 +00:00
pjd
b07aec8b25 Style cleanups.
MFC after:	3 days
2012-01-13 23:25:35 +00:00
pjd
e437b28043 - Fix a bug where pidfile was removed in SIGHUP when it hasn't changed in
configuration file.
- Log the fact that pidfile has changed.

MFC after:	3 days
2012-01-10 22:41:09 +00:00
pjd
c5fe5a76f2 For functions that return -1 on failure check exactly for -1 and not for
any negative number.

MFC after:	3 days
2012-01-10 22:39:07 +00:00
pjd
29f76d890e Don't touch pidfiles when running in foreground. Before that change we
would create an empty pidfile on start and check if it changed on SIGHUP.

MFC after:	3 days
2012-01-10 22:24:57 +00:00
uqs
5f1ca9b982 Spelling fixes for sbin/ 2012-01-07 16:09:33 +00:00
pjd
cbcf1832ad fork(2) returns -1 on failure, not some random negative number.
MFC after:	3 days
2012-01-06 23:44:26 +00:00
pjd
a272071bf0 Constify argument.
MFC after:	3 days
2012-01-06 12:27:17 +00:00
dim
9d35683411 Use NO_WCAST_ALIGN for usr.bin/hastctl and usr.bin/hastd; the alignment
warnings in sbin/hastd/lzf.c are only emitted for i386 and amd64, and
there they can be safely ignored.

MFC after:	1 week
2011-12-19 15:46:15 +00:00
pjd
0283ff3e72 Use lex's standard way of not generating unused function.
Inspired by:	r228555
MFC after:	1 week
2011-12-18 20:41:58 +00:00
pjd
509e6459e0 Don't use function name as format string.
Detected by:	clang
MFC after:	1 week
2011-12-18 20:40:19 +00:00
pjd
8d73b322fa Remove redundant assignment.
Found by:	Clang Static Analyzer
MFC after:	1 week
2011-12-15 22:05:23 +00:00
pjd
91f3438889 Simplify code by changing functions types from int to avoid, as the functions
always return 0.

Found by:	Clang Static Analyzer
MFC after:	1 week
2011-12-15 22:03:17 +00:00
pjd
a852ec0ef9 Remove redundant setting of the error variable.
Found by:	Clang Static Analyzer
MFC after:	1 week
2011-12-15 22:01:34 +00:00
pjd
f13dc0df13 Remove redundant space.
MFC after:	3 days
2011-10-27 20:36:35 +00:00
pjd
14cf798458 Implement 'async' mode for HAST.
MFC after:	3 days
2011-10-27 20:32:57 +00:00
pjd
c017e98c55 Minor cleanups.
MFC after:	3 days
2011-10-27 20:15:37 +00:00
pjd
739f931164 Reduce indentation.
MFC after:	3 days
2011-10-27 20:13:39 +00:00
pjd
986d3757ae Improve comment so it doesn't suggest race is possible, but that we handle
the race.

MFC after:	3 days
2011-10-27 20:10:21 +00:00
pjd
72922c3d6b - Eliminate the need for hio_nv.
- Introduce hio_clear() function for clearing hio before returning it
  onto free queue.

MFC after:	3 days
2011-10-27 20:01:23 +00:00
pjd
fcce680c47 Monor cleanups.
MFC after:	3 days
2011-10-27 18:49:16 +00:00
pjd
c2e715af70 Delay resuid generation until first connection to secondary, not until first
write. This way on first connection we will synchronize only the extents that
were modified during the lifetime of primary node, not entire GEOM provider.

MFC after:	3 days
2011-10-27 18:45:01 +00:00
pjd
315841e3ac Correct comments.
MFC after:	3 days
2011-10-27 16:22:17 +00:00
pjd
ed683cf8c0 Allow to specify pidfile in HAST configuration file.
MFC after:	1 week
2011-10-17 12:22:09 +00:00
pjd
06a2c15f45 Remove redundant space.
MFC after:	1 week
2011-10-17 09:59:04 +00:00
pjd
cc73b6a041 When path to the configuration file is relative, obtain full path,
so we can always find the file, even after daemonizing and changing
working directory to /.

MFC after:	1 week
2011-10-17 09:54:07 +00:00
pjd
ae2bc06327 Correct typo.
MFC after:	3 days
2011-09-28 13:25:27 +00:00
pjd
25b3d91c28 If the underlying provider doesn't support BIO_FLUSH, log it only once
and don't bother trying in the future.

MFC after:	3 days
2011-09-28 13:19:47 +00:00
pjd
3fbe0df4ff Break a bit earlier.
MFC after:	3 days
2011-09-28 13:13:43 +00:00
pjd
374501b495 After every activemap change flush disk's write cache, so that write
reordering won't make the actual write to be committed before marking
the coresponding extent as dirty.

It can be disabled in configuration file.

If BIO_FLUSH is not supported by the underlying file system we log a warning
and never send BIO_FLUSH again to that GEOM provider.

MFC after:	3 days
2011-09-28 13:08:51 +00:00
pjd
2c59e5d543 Use PJDLOG_ASSERT() and PJDLOG_ABORT() everywhere instead of assert().
MFC after:	3 days
2011-09-27 08:50:37 +00:00
pjd
1f93bdc27f No need to wrap pjdlog functions around with KEEP_ERRNO() macro.
MFC after:	3 days
2011-09-27 08:26:09 +00:00
pjd
ac89e4145a - Convert some impossible conditions into assertions.
- Add missing 'if' in comment.

MFC after:	3 days
2011-09-27 08:04:01 +00:00
pjd
b294ac14be Correct two mistakes when converting asserts to PJDLOG_ASSERT()/PJDLOG_ABORT().
MFC after:	3 days
2011-09-27 07:59:10 +00:00
pjd
26b0e1eedd Prefer PJDLOG_ASSERT() and PJDLOG_ABORT() over assert() and abort().
pjdlog versions will log problem to syslog when application is running in
background.

MFC after:	3 days
2011-09-27 07:57:15 +00:00
pjd
ff11c3069e No need to use KEEP_ERRNO() macro around pjdlog functions, as they don't
modify errno.

MFC after:	3 days
2011-09-27 07:52:39 +00:00
pjd
e0053a0ea2 Ensure that pjdlog functions don't modify errno.
MFC after:	3 days
2011-09-27 06:43:51 +00:00
trociny
ec6755c2ee Fix indentation.
Approved by:	pjd (mentor)
2011-07-13 05:32:55 +00:00
trociny
3628e4b620 Remove useless initialization.
Approved by:	pjd (mentor)
MFC after:	3 days
2011-07-05 06:12:28 +00:00
trociny
82faa3e641 Check the returned value of activemap_write_complete() and update matadata on
disk if needed. This should fix a potential case when extents are cleared in
activemap but metadata is not updated on disk.

Suggested by:	pjd
Approved by:	pjd (mentor)
2011-06-28 21:01:32 +00:00
trociny
55657cdc99 Make activemap_write_start/complete check the keepdirty list, when
stating if we need to update activemap on disk. This makes keepdirty
serve its purpose -- to reduce number of metadata updates.

Discussed with:	pjd
Approved by:	pjd (mentor)
2011-06-28 20:57:54 +00:00
pjd
bfdf57abb6 Compile hastd and hastctl with capsicum support.
X-MFC after:	capsicum merge
2011-06-27 09:15:41 +00:00
pjd
faa028ceab Compile capsicum support only if HAVE_CAPSICUM is defined.
MFC after:	3 days
2011-06-27 09:14:25 +00:00
pjd
6a3c287a89 Log a warning if we cannot sandbox using capsicum, but only under debug level 1.
It would be too noisy to log it as a proper warning as CAPABILITIES are not
compiled into GENERIC by default.

MFC after:	3 days
2011-06-27 09:10:48 +00:00
trociny
a262e24ff4 In HAST we use two sockets - one for only sending the data and one for
only receiving the data. In r220271 the unused directions were
disabled using shutdown(2).

Unfortunately, this broke automatic receive buffer sizing, which
currently works only for connections in ETASBLISHED state. It was a
root cause of the issue reported by users, when connection between
primary and secondary could get stuck.

Disable the code introduced in r220271 until the issue with automatic
buffer sizing is not resolved.

Reported by:	Daniel Kalchev <daniel@digsys.bg>, danger, sobomax
Tested by:	Daniel Kalchev <daniel@digsys.bg>, danger
Approved by:	pjd (mentor)
MFC after:	1 week
2011-06-17 07:07:26 +00:00
sobomax
47a79e7f33 Revert r222688.
Requested by:	Mikolaj Golub
2011-06-16 08:31:06 +00:00
sobomax
0198596495 Read from the socket using the same max buffer size as we use while
sending. What happens otherwise is that the sender splits all the
traffic into 32k chunks, while the receiver is waiting for the whole
packet. Then for a certain packet sizes, particularly 66607 bytes in
my case, the communication stucks to secondary is expecting to
read one chunk of 66607 bytes, while primary is sending two chunks
of 32768 bytes and third chunk of 1071. Probably due to TCP windowing
and buffering the final chunk gets stuck somewhere, so neither server
not client can make any progress.

This patch also protect from short reads, as according to the manual
page there are some cases when MSG_WAITALL can give less data than
expected.

MFC after:	3 days
2011-06-04 16:01:30 +00:00
trociny
e74ba5594f If READ from the local node failed we send the request to the remote
node. There is no use in doing this for synchronization requests.

Approved by:	pjd (mentor)
MFC after:	1 week
2011-05-29 21:20:47 +00:00
pjd
42a14e17b5 Keep statistics on number of BIO_READ, BIO_WRITE, BIO_DELETE and BIO_FLUSH
requests as well as number of activemap updates.

Number of BIO_WRITEs and activemap updates are especially interesting, because
if those two are too close to each other, it means that your workload needs
bigger number of dirty extents. Activemap should be updated as rarely as
possible.

MFC after:	1 week
2011-05-23 21:15:19 +00:00
pjd
93ce9f3fcb To handle BIO_FLUSH and BIO_DELETE requests in secondary worker we need
to use ioctl(2). This is why we can't use capsicum for now to sandbox
secondary. Capsicum is still used to sandbox hastctl.

MFC after:	1 week
2011-05-23 20:59:50 +00:00
pjd
0ebed84dd0 Recognize HIO_FLUSH requests.
MFC after:	1 week
2011-05-21 20:21:20 +00:00
pjd
a2cf5c4542 Document IPv6 support.
MFC after:	3 weeks
2011-05-20 11:21:39 +00:00
pjd
87ff33b2c9 If no listen address is specified, bind by default to:
tcp4://0.0.0.0:8457
	tcp6://[::]:8457

MFC after:	3 weeks
2011-05-20 11:16:25 +00:00
pjd
65d1a6f181 Rename ipv4/ipv6 to tcp4/tcp6.
MFC after:	3 weeks
2011-05-20 11:15:27 +00:00
pjd
fadabb377a Now that hell is fully frozen it is good time to add IPv6 support to HAST.
MFC after:	3 weeks
2011-05-20 11:14:05 +00:00
pjd
5e404c67de Allow [ ] characters in strings. They might be used in IPv6 addresses.
MFC after:	3 weeks
2011-05-20 11:10:39 +00:00
pjd
a432a45a88 Rename tcp4 to tcp in preparation for IPv6 support.
MFC after:	3 weeks
2011-05-20 11:09:02 +00:00
pjd
23cbf888e0 Rename proto_tcp4.c to proto_tcp.c in preparation for IPv6 support.
MFC after:	2 weeks
2011-05-20 11:06:17 +00:00
pjd
5cf7b46f54 In preparation for IPv6 support allow to specify multiple addresses to
listen on.

MFC after:	3 weeks
2011-05-19 23:18:42 +00:00
pjd
a47ef66623 - Add support for AF_INET6 sockets for %S format character.
- Use inet_ntop(3) instead of reimplementing it.
- Use %hhu for unsigned char instead of casting it to unsigned int and
  using %u.

MFC after:	1 week
2011-05-18 22:43:56 +00:00