Commit Graph

1026 Commits

Author SHA1 Message Date
Enji Cooper
c53d56999c Fix the build after r317942 by adding usr.bin/csplit to BSD.tests.dist
Pointyhat to:	cem
MFC with:	r317942
Sponsored by:	Dell EMC Isilon
2017-05-08 17:13:00 +00:00
Brooks Davis
a7dc31283a Remove the NATM framework including the en(4), fatm(4), hatm(4), and
patm(4) devices.

Maintaining an address family and framework has real costs when we make
infrastructure improvements.  In the case of NATM we support no devices
manufactured in the last 20 years and some will not even work in modern
motherboards (some newer devices that patm(4) could be updated to
support apparently exist, but we do not currently have support).

With this change, support remains for some netgraph modules that don't
require NATM support code. It is unclear if all these should remain,
though ng_atmllc certainly stands alone.

Note well: FreeBSD 11 supports NATM and will continue to do so until at
least September 30, 2021.  Improvements to the code in FreeBSD 11 are
certainly welcome.

Reviewed by:	philip
Approved by:	harti
2017-04-24 21:21:49 +00:00
Baptiste Daroussin
50502545ce Readd Big5: some large databases setup are still requiring it.
Reported by:	"張君天(Chun-Tien Chang)" <tcs@kitty.2y.idv.tw>
2017-04-20 18:21:50 +00:00
Baptiste Daroussin
e229090553 Import zstandard 1.1.4 in base
zstandard is a new compression library/tool which is very fast at
compression/decompression

For now import as a private library
2017-04-15 20:05:22 +00:00
Enji Cooper
b824378b14 sbuf(3): add some basic functional tests for the library
Areas not covered still [positive functionality wise] are:
- sbuf_{clear,get,set}_flags
- sbuf_new (in particular, with fixed buffers, etc).

Some basic negative testing has been added, but more will be added in the
future.

This work was in part to validate work done by cem in r288223, and ian
before that.

MFC after:	2 months
Sponsored by:	Dell EMC Isilon
2017-04-06 05:29:28 +00:00
Alan Somers
6dc025ea3a Fix file descriptor and memory leaks in pr(1)
Also, hook NetBSD's pr test into the build, and add three more test cases.

Reported by:	Coverity, Valgrind
CID:		271650 271651 271652 271653 271654 271655 271656 271656
CID:		271657 271658 271659 1006939 1006940 1006941 1006942 1009098
Reviewed by:	ngie
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D9137
2017-04-04 20:03:57 +00:00
Enji Cooper
10f81a9b3b lib/libkvm: start adding basic tests for kvm(3)
- kvm_close: add a testcase to verify support for errno = EINVAL / -1
  (see D10065) when kd == NULL is provided to the libcall.
- kvm_geterr:
-- Add a negative testcase for kd == NULL returning "" (see D10022).
-- Add two positive testcases:
--- test the error case using kvm_write on a O_RDONLY descriptor.
--- test the "no error" case using kvm_read(3) and kvm_nlist(3) as
    helper routines and by injecting a bogus error message via
    _kvm_err (an internal API) _kvm_err was used as there isn't a
    formalized way to clear the error output, and because
    kvm_nlist always returns ENOENT with the NULL terminator today.
- kvm_open, kvm_open2:
-- Add some basic negative tests for kvm_open(3) and kvm_open2(3).
   Testing positive cases with a specific
   `corefile`/`execfile`/`resolver` requires more work and would require
   user intervention today in order to reliably test this out.

Reviewed by:	markj
MFC after:	2 months
Sponsored by:	Dell EMC Isilon
Differential Revision:	D10024
2017-03-28 17:37:49 +00:00
Adrian Chadd
f906f2025b [private] add libevent1 and sqlite3 include files for our private libraries.
This, like including ucl private headers, is useful for writing new base
system tools.  Yes, anyone using these libraries shouldn't assume ABI
compatibility.

Reviewed by:	bdrewery, bapt
Differential Revision:	https://reviews.freebsd.org/D10123
2017-03-27 22:34:43 +00:00
Baptiste Daroussin
b285596f4e Remove empty Big5 directory from share/nls 2017-03-19 18:03:52 +00:00
Baptiste Daroussin
ababdab0ce Remove zh_TW.Big5 locale
After discussion with many Taiwanese, in IT or not. Big5 is not used anymore.
It is not able to represent lots of the characters used in the language.
2017-03-19 17:52:46 +00:00
Marius Strobl
72dec0792a - Add support for eMMC "partitions". Besides the user data area, i. e.
the default partition, eMMC v4.41 and later devices can additionally
  provide up to:
  1 enhanced user data area partition
  2 boot partitions
  1 RPMB (Replay Protected Memory Block) partition
  4 general purpose partitions (optionally with a enhanced or extended
    attribute)

  Of these "partitions", only the enhanced user data area one actually
  slices the user data area partition and, thus, gets handled with the
  help of geom_flashmap(4). The other types of partitions have address
  space independent from the default partition and need to be switched
  to via CMD6 (SWITCH), i. e. constitute a set of additional "disks".

  The second kind of these "partitions" doesn't fit that well into the
  design of mmc(4) and mmcsd(4). I've decided to let mmcsd(4) hook all
  of these "partitions" up as disk(9)'s (except for the RPMB partition
  as it didn't seem to make much sense to be able to put a file-system
  there and may require authentication; therefore, RPMB partitions are
  solely accessible via the newly added IOCTL interface currently; see
  also below). This approach for one resulted in cleaner code. Second,
  it retains the notion of mmcsd(4) children corresponding to a single
  physical device each. With the addition of some layering violations,
  it also would have been possible for mmc(4) to add separate mmcsd(4)
  instances with one disk each for all of these "partitions", however.
  Still, both mmc(4) and mmcsd(4) share some common code now e. g. for
  issuing CMD6, which has been factored out into mmc_subr.c.

  Besides simply subdividing eMMC devices, some Intel NUCs having UEFI
  code in the boot partitions etc., another use case for the partition
  support is the activation of pseudo-SLC mode, which manufacturers of
  eMMC chips typically associate with the enhanced user data area and/
  or the enhanced attribute of general purpose partitions.

  CAVEAT EMPTOR: Partitioning eMMC devices is a one-time operation.

- Now that properly issuing CMD6 is crucial (so data isn't written to
  the wrong partition for example), make a step into the direction of
  correctly handling the timeout for these commands in the MMC layer.
  Also, do a SEND_STATUS when CMD6 is invoked with an R1B response as
  recommended by relevant specifications. However, quite some work is
  left to be done in this regard; all other R1B-type commands done by
  the MMC layer also should be followed by a SEND_STATUS (CMD13), the
  erase timeout calculations/handling as documented in specifications
  are entirely ignored so far, the MMC layer doesn't provide timeouts
  applicable up to the bridge drivers and at least sdhci(4) currently
  is hardcoding 1 s as timeout for all command types unconditionally.
  Let alone already available return codes often not being checked in
  the MMC layer ...

- Add an IOCTL interface to mmcsd(4); this is sufficiently compatible
  with Linux so that the GNU mmc-utils can be ported to and used with
  FreeBSD (note that due to the remaining deficiencies outlined above
  SANITIZE operations issued by/with `mmc` currently most likely will
  fail). These latter will be added to ports as sysutils/mmc-utils in
  a bit. Among others, the `mmc` tool of the GNU mmc-utils allows for
  partitioning eMMC devices (tested working).

- For devices following the eMMC specification v4.41 or later, year 0
  is 2013 rather than 1997; so correct this for assembling the device
  ID string properly.

- Let mmcsd.ko depend on mmc.ko. Additionally, bump MMC_VERSION as at
  least for some of the above a matching pair is required.

- In the ACPI front-end of sdhci(4) describe the Intel eMMC and SDXC
  controllers as such in order to match the PCI one.
  Additionally, in the entry for the 80860F14 SDXC controller remove
  the eMMC-only SDHCI_QUIRK_INTEL_POWER_UP_RESET.

OKed by:	imp
Submitted by:	ian (mmc_switch_status() implementation)
2017-03-16 22:23:04 +00:00
Enji Cooper
b0b1dbdd49 Start adding basic tests for cam(3)
This change contains several negative and positive tests for:
- cam_open_device
- cam_close_device
- cam_getccb
- cam_freeccb

This also contains a test for the failure case noted in bug 217649,
i.e., O_RDWR must be specified because pass(4) requires it.

This test unfortunately cannot assume that cam-capable devices are
present, so the user must explicitly provide a device via
`test_suites.FreeBSD.cam_test_device`. In the future, a test kernel
module might be shipped, or ctl(4) might be used, as a test device
when testing out libcam, which will allow the tests to do away with
having to specify an explicit test device.

Reviewed by:	asomers, ken (earlier diff)
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision: D9928
2017-03-15 18:00:54 +00:00
Enji Cooper
e22ad7bca1 Move .../sys/geom/eli/pbkdf2... to .../sys/geom/class/eli/...
This change moves the tests added in r313962 to an existing directory
structure used by the geli TAP tests. It also, renames the test from
pbkdf2 to pbkdf2_test .

The changes to ObsoleteFiles.inc are being committed separately as they
aren't needed for the MFC to ^/stable/11, etc, if the MFC for the tests
is done all in one commit.

MFC after:	2 weeks
X-MFC with:	r313962, r313972-r313973
Reviewed by:	allanjude
Sponsored by:	Dell EMC Isilon
Differential Revision:	D9985
2017-03-14 07:00:22 +00:00
Baptiste Daroussin
84e1ba258b Add the diff to the tests mtree
Reported by:	lwhsu
2017-03-11 06:27:06 +00:00
Baptiste Daroussin
d2baa3fdee texinfo is gone in r276551 remove the related directories
Reported by:	jbeich
2017-03-08 08:52:15 +00:00
Cy Schubert
05de3f339a Fix install due to incorrect placement of pwait dir in r314886.
Reported by:	Shawn Webb <shawn.webb@hardenedbsd.org>
MFC after:	2 weeks
X-MFC with:	r314886
2017-03-08 05:27:04 +00:00
Bryan Drewery
b06b52baac pwait: Add a -t flag to specify a timeout before exiting, and tests.
The exit status will be 124, as the timeout(1) utility uses.

Reviewed by:	jilles
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D9697
2017-03-07 22:16:55 +00:00
Enji Cooper
7d9ade5da1 Integrate indent tests added in r313544 into ATF/Kyua and the FreeBSD
test suite

This change does the following:

- Introduces symmetry in the test inputs/outputs by adding the exit
  code to the files. This simplified the test driver notably by
  requiring less filename/test name manipulation.
- Adds a test driver for the testcases added in r313544, patterned
  after bin/sh/tests/functional_test.sh . The driver calls indent as
  noted in r313544, with an exception: The $FreeBSD$ RCS keyword's
  expansion is reindented with indent, which means that the output
  differs from the expected output. Thus, all lines with $FreeBSD$
  in them are deleted on the fly, both in the input file and the
  output file.

  The test inputs/outputs are copied to the kyua sandbox before the
  test is run as the pathing in some of the files relies on pathing
  normalized to the current directory (copying the files is the
  easiest way to resolve the issue).

Approved by:	pstef (maintainer)
Reviewed by:	pstef
X-MFC with:	r313544
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9682
2017-03-03 20:15:22 +00:00
Dimitry Andric
6ae9acde63 Merge ^/head r313896 through r314128. 2017-02-23 07:45:58 +00:00
Enji Cooper
81e8601f58 Remove lib/libpam tests after they were removed from the source tree in r313975
X-MFC with:	r313975
Sponsored by:	Dell EMC Isilon
2017-02-20 01:45:12 +00:00
Allan Jude
85c15ab853 improve PBKDF2 performance
The PBKDF2 in sys/geom/eli/pkcs5v2.c is around half the speed it could be

GELI's PBKDF2 uses a simple benchmark to determine a number of iterations
that will takes approximately 2 seconds. The security provided is actually
half what is expected, because an attacker could use the optimized
algorithm to brute force the key in half the expected time.

With this change, all newly generated GELI keys will be approximately 2x
as strong. Previously generated keys will talk half as long to calculate,
resulting in faster mounting of encrypted volumes. Users may choose to
rekey, to generate a new key with the larger default number of iterations
using the geli(8) setkey command.

Security of existing data is not compromised, as ~1 second per brute force
attempt is still a very high threshold.

PR:		202365
Original Research:	https://jbp.io/2015/08/11/pbkdf2-performance-matters/
Submitted by:	Joe Pixton <jpixton@gmail.com> (Original Version), jmg (Later Version)
Reviewed by:	ed, pjd, delphij
Approved by:	secteam, pjd (maintainer)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D8236
2017-02-19 19:30:31 +00:00
Dimitry Andric
1a36faad54 Merge ^/head r313301 through r313643. 2017-02-11 14:04:18 +00:00
Enji Cooper
9b3ece1c2e MFhead@r313243 2017-02-04 18:06:09 +00:00
Dimitry Andric
a4aa656aa5 Merge ^/head r312309 through r312623. 2017-01-22 16:05:13 +00:00
Ed Maste
6b02cd2c8f Remove obsolete /usr/lib/debug/usr/lib/private dir
Missed in r282420

Reported by:	dim
2017-01-20 03:14:18 +00:00
Enji Cooper
71164a14d0 Integrate .../contrib/netbsd-tests/usr.bin/uniq into the FreeBSD test
suite as .../usr.bin/uniq/tests

Sponsored by:	Dell EMC Isilon
2017-01-14 06:51:31 +00:00
Dimitry Andric
8a6fe8ce60 Merge ^/head r311812 through r311939. 2017-01-11 21:05:13 +00:00
Alan Somers
cdb7a6fc42 Fix memory leaks during "tail -r" of an irregular file
* Rewrite r_buf to use standard tail queues instead of a hand-rolled
  circular linked list. Free dynamic allocations when done.
* Remove an optimization for the case where the file is a multiple of 128KB
  in size and there is a scarcity of memory.
* Add ATF tests for "tail -r" and its variants.

Reported by:	Valgrind
Reviewed by:	ngie
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D9067
2017-01-10 20:43:32 +00:00
Dimitry Andric
2b532af829 Merge ^/head r311314 through r311459. 2017-01-05 20:50:44 +00:00
Dimitry Andric
53fe1d28fa Adjust version numbers for the clang library directory. 2017-01-05 18:32:18 +00:00
Alan Somers
0dbb4093ef Fix typo from r311349
Reported by:	lwhsu
Pointy-hat-to:	asomers
MFC after:	4 weeks
X-MFC-with:	311349
2017-01-05 15:07:04 +00:00
Alan Somers
371f86d244 tabs -> spaces in etc/mtree
MFC after:	4 weeks
2017-01-05 02:47:56 +00:00
Dimitry Andric
5b41a5b675 Update build glue for llvm/clang 3.9.1. 2016-11-26 01:13:53 +00:00
Dimitry Andric
67bc8c8b9e Merge ^/head r308491 through r308841. 2016-11-19 16:05:55 +00:00
Sepherosa Ziehau
168fce73b5 hyperv/vss: Add driver and tools for VSS
VSS stands for "Volume Shadow Copy Service".  Unlike virtual machine
snapshot, it only takes snapshot for the virtual disks, so both
filesystem and applications have to aware of it, and cooperate the
whole VSS process.

This driver exposes two device files to the userland:

    /dev/hv_fsvss_dev

    Normally userland programs should _not_ mess with this device file.
    It is currently used by the hv_vss_daemon(8), which freezes and
    thaws the filesystem.  NOTE: currently only UFS is supported, if
    the system mounts _any_ other filesystems, the hv_vss_daemon(8)
    will veto the VSS process.

    If hv_vss_daemon(8) was disabled, then this device file must be
    opened, and proper ioctls must be issued to keep the VSS working.

    /dev/hv_appvss_dev

    Userland application can opened this device file to receive the
    VSS freeze notification, hold the VSS for a while (mainly to flush
    application data to filesystem), release the VSS process, and
    receive the VSS thaw notification i.e. applications can run again.

    The VSS will still work, even if this device file is not opened.
    However, only filesystem consistency is promised, if this device
    file is not opened or is not operated properly.

hv_vss_daemon(8) is started by devd(8) by default.  It can be disabled
by editting /etc/devd/hyperv.conf.

Submitted by:	Hongjiang Zhang <honzhan microsoft com>
Reviewed by:	kib, mckusick
MFC after:	3 weeks
Sponsored by:	Microsoft
Differential Revision:	https://reviews.freebsd.org/D8224
2016-11-15 02:36:12 +00:00
Dimitry Andric
a2b802ce70 Merge ^/head r303250 through r308226. 2016-11-02 19:18:24 +00:00
Baptiste Daroussin
fdec22c37d syslogd(8): add an 'include' keyword
All the '.conf' files not beginning with a '.' contained int he directory
following the keyword will be included.

This keyword can only be used in the first level configuration files.

Modify the default syslogd.conf to 'include' /etc/syslog.d and
/usr/local/etc/syslog.d

It simplify a lot handling of syslog from automation tools.

Reviewed by:	markj, kib (via irc)
Approved by:	markj
MFC after:	2 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D8402
2016-11-01 01:41:24 +00:00
Dimitry Andric
02ebdc7823 Merge ^/head r307736 through r308146. 2016-10-31 19:02:42 +00:00
Baptiste Daroussin
b2fd8384ff cron(8): add support for /etc/cron.d and /usr/local/etc/cron.d
For automation tools it is way easier to maintain files in directories rather
than modifying /etc/crontab.

The files in those directories are in the same format as /etc/crontab

Reviewed by:	adrian
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	Gandi.net
Differential Revision:	https://reviews.freebsd.org/D8400
2016-10-31 18:20:12 +00:00
Dimitry Andric
5763f79695 Merge ^/head r307383 through r307735. 2016-10-21 16:29:40 +00:00
Enji Cooper
669c253531 Integrate contrib/netbsd-tests/fs/tmpfs into the FreeBSD test suite
as tests/sys/fs

These testcases exercise tmpfs support

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2016-10-21 05:24:08 +00:00
Marcel Moolenaar
50875ed2c1 Re-apply change 306811 or alternatively, revert change 307385. 2016-10-16 02:43:51 +00:00
Marcel Moolenaar
9ffbf09f2f Revert change 306811 so that the change can be re-done using
svn copy instead of svn move.  This to preserve history on
the originals headers as well.
2016-10-16 02:05:22 +00:00
Dimitry Andric
a0e610c439 Merge ^/head r306906 through r307382. 2016-10-15 22:49:04 +00:00
Baptiste Daroussin
a0135a1e35 Remove GNU rcs from base.
GNU rcs is still available as a package:
- rcs: Latest GPLv3 GNU rcs version.
- rcs57: Copy of the latest version of GNU rcs (GPLv2) from base.

Relnotes:	yes
2016-10-15 12:07:37 +00:00
Dimitry Andric
242b248284 Merge ^/head r306412 through r306905. 2016-10-09 13:30:57 +00:00
Marcel Moolenaar
0974f66d06 In order to allow mkimg(1) (and other tools) to become a build tool
that can be compiled on various OSes (including on older versions
of FreeBSD), make it possible to have it include the partitioning
scheme definitions without pulling in FreeBSD specifics.
In particular this means:
 o  move the scheme definitions iand related defines to header files
    under sys/disk,
 o  make them (more) portable by using uint#_t (where applicable)
    and renaming defines so that they at least have a good prefix,
 o  make the new headers stand-alone so that they don't need FreeBSD
    definitions, like struct uuid(*)
 o  keep the original headers for compatibility, but rewrite them to
    get the scheme definitions from <sys/disk/$scheme.h>.

(*) since UUID/GUID type definitions are non-portable and the GPT
scheme uses them, make it possible to have the scheme definitions
use an external type by allowing consumers of the header to set
GPT_UUID_TYPE. When GPT_UUID_TYPE has not been defined, the header
will use it's own type definition, which is the same as struct uuid.
The gpt_uuid_t typedef is created to abstract the details and allows
consumers to refer to a single type.

There is not conflict between the partitioning scheme headers and
what is defined in them. All headers can be included in the same
source files.

Note: consumers of the old headers have not been changed yet. Such
will be done if and when needed/beneficial.

Reviewed by:	imp, jhb
MFC after:	1 month
Sponsored by:	Bracket Computing
2016-10-07 15:42:20 +00:00
Dimitry Andric
93badfa1f2 Merge ^/head r305687 through r305890. 2016-09-16 20:49:12 +00:00
Oleksandr Tymoshenko
2b3f6d6650 Add evdev protocol implementation
evdev is a generic input event interface compatible with Linux
evdev API at ioctl level. It allows using unmodified (apart from
header name) input evdev drivers in Xorg, Wayland, Qt.

This commit has only generic kernel API. evdev support for individual
hardware drivers like ukbd, ums, atkbd, etc. will be committed later.

Project was started by Jakub Klama as part of GSoC 2014. Jakub's
evdev implementation was later used as a base, updated and finished
by Vladimir Kondratiev.

Submitted by:	Vladimir Kondratiev <wulf@cicgroup.ru>
Reviewed by:	adrian, hans
Differential Revision:	https://reviews.freebsd.org/D6998
2016-09-11 18:56:38 +00:00
Dimitry Andric
a75e9a0239 Merge ^/head r305623 through r305686. 2016-09-10 17:00:08 +00:00