Much of this was originally applied by Darek in
6f5840d22 but then had to be reverted due to NVMe-oF
MR issues in a817ccf5. Now that the NVMe-oF MR issues
have been resolved, we can reapply these changes.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Change-Id: I58d603523901c78dbc81279078a09b2a24c091e6
Reviewed-on: https://review.gerrithub.io/436547
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Application Makefiles can now just add xx_MODULES_LIB_LIST
to SPDK_LIB_LIST. This is possible now since all
SPDK libraries are linked with --whole-archive, so there
is no need to differentiate between "modules" libraries
and other SPDK libraries.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iabf81a388b72d3b2a2f48287a8491ddc977722ac
Reviewed-on: https://review.gerrithub.io/434277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This allows a lot of simplification to SPDK application
makefiles.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5fa463f6369834b84a8d92e79fa7768082209d7a
Reviewed-on: https://review.gerrithub.io/434274
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There were several applications that were missing
either SOCK_MODULES_LINKER_ARGS or COPY_MODULES_LINKER_ARGS.
For the ones missing SOCK_MODULES_LINKER_ARGS (fuse and rocksdb),
the nvme inititator with TCP transport would not have worked
at all.
Adding COPY_MODULES_LINKER_ARGS to the bdev fio plugin enables
ioat which isn't critical, but adding it makes it consistent
with other apps and will allow its Makefile to be simplified in
some future patches.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0098350d75d27ad2b2d408221b727698f5e902e4
Reviewed-on: https://review.gerrithub.io/434260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:
Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).
Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
blob and blob_bdev are already included via the lvol bdev module
dependencies in mk/spdk.modules.mk. This patch adds blobfs there
too.
These changes are needed for some upcoming patches which will
apply --whole-archive to all libraries linked into the application.
That patch will require that we never include the same library
twice.
Shared library builds *require* that blobfs be listed before
blob on the link line, hence adding blobfs here even though it's
not technically associated with a bdev module.
We can also simplify spdk.modules.mk while we're here - just
filter out bdev_lvol to disable lvol for applications like
blob_cli that need to disable it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I01e5e6b91d947463749f942f54c12f48229bbe5a
Reviewed-on: https://review.gerrithub.io/432916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Apparently this patch revealed an issue somewhere in SGL
handling in NVMf initiator which is causing our CI to fail.
Let's revert it while we work on a proper fix.
This reverts commit 6f5840d225.
This is not a full revert, some app.c changes to the usage text
were kept unchanged.
Change-Id: Iddea5c2b9df50bd12ef8f6226165883f6622ab33
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/432576
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: Ben Walker <benjamin.walker@intel.com>
Now that we utilize DPDK dynamic memory allocation, we
no longer need to set the default pre-reserved memory size.
It'll now be 0, which implies all memory will be allocated
at runtime.
The option to pre-reserve all hugepages on the system is
now only available on BSD, so also clarify that bit in the
app usage text.
Change-Id: I5a8a1d9bf14ad6d938532d7e6254a45e4a81bb92
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/432204
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I066085a0606d64a0d95ab2d28340aa35d83efdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I64221416aae7cd28dd7cab5badb8fb3a534b9d3f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
This makes more sense as a first class library.
Change-Id: Ibd5c578f8708bd8c7d83fe1629e97c9a3316b56b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414698
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
We need -lrt as well as -luuid. I'm not exactly sure how
this worked previously - but updating my dev box to
Ubuntu 18.04 LTS hit this problem. Any linking done
in the SPDK repository always uses SYS_LIBS which
includes uuid and rt so we should use it for linking
RocksDB db_bench as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I243e7745ebe528d3b6c1e705cdd20716a041a098
Reviewed-on: https://review.gerrithub.io/414478
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Our implementation of WritableFile::Allocate() was wrapped in #ifdef
ROCKSDB_FALLOCATE_PRESENT, which is set based on whether the target
system supports fallocate(); SPDK doesn't use the kernel filesystem
interface, so using that check makes no sense.
The #ifdef was also incorrectly wrapping WritableFile::RangeSync() and
WritableFile::GetUniqueId(), which are totally unrelated to fallocate.
Change-Id: I9655074353f79cf7a63e3e3d9c455cd39a683062
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405332
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
All of the uses of this member variable treat it as uint64_t.
Change-Id: Iba9cf2eae1f2bd79617e2785ad498526f1115c10
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405331
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Make sure spdk_fs_open_file() succeeds in all of the RocksDB environment
file constructors. NewSequentialFile() already checked this, but
SpdkRandomAccessFile and SpdkWritableFile did not.
Change-Id: Ie298e0cd29366eb698a0358740310fec62af7f9c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405326
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Modifies behavior of spdk_app_start() and spdk_app_parse_args()
such that they return on failure instead of terminating with
exit().
Change-Id: I82566417f04e1ae2e3ca60a00c72e664db26c9e4
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/401243
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Require braces around all conditional statements, e.g.:
if (cond)
statement();
becomes:
if (cond) {
statement();
}
This is the style used through most of the SPDK code, but several
exceptions crept in over time. Add the astyle option to make sure we
are consistent.
Change-Id: I5a71980147fe8dfb471ff42e8bc06db2124a1a7f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/390914
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Make this generic and not directly dependent on
the event framework. That way our libraries can
register pollers without adding a dependency.
Change-Id: I7ad7a7ddc131596ca1791a7b0f43dabfda050f5f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/387690
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
RPC is core functionality for SPDK applications and should always
be initialized last (after all subsystems have been initialized).
So make RPC a first class citizen and integrate it with the
app framework directly instead of making it an "optional" subsystem.
Then we initializing it after all subsystems have completed
initialization, and tear it down before tearing down subsystems.
We can also do some other cleanup while here - for example, reactors
are already started when spdk_rpc_initialize() is called, so remove
the extra event that was added during initialization since this is no
longer needed.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4cc63586a6d55be68786629a2176c61a88979267
Reviewed-on: https://review.gerrithub.io/385914
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is in preparation for enabling hot remove of logical volumes when
their underlying blobstore device is hot-removed.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I310a3f64f0de5d628609c20a1a3b4d38df0755aa
Reviewed-on: https://review.gerrithub.io/377041
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia07efcdec742d6ae5339e9f1c860ab58107d9f85
Signed-off-by: Jin Tsai <caijin11331007@gmail.com>
Reviewed-on: https://review.gerrithub.io/379928
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia96ae78ff9530d953181ac5f7255a38f3c8ec430
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Reviewed-on: https://review.gerrithub.io/375392
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For multiple db test, there will be some files not close at the end of test,
this patch make sure to close all the files before shutdown the app.
Change-Id: Ie87d9e44b55a9192a19c35c439fbf33268f8ca4c
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/375168
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
This moves the thread name setting code into the generic SPDK thread
setup code, so now all spdk_threads can be named, not just ones created
by the event framework.
Change-Id: I6c824cf4bcf12fe64a8e2fc7cdc2d6c949021e40
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375220
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7f9b24e5ee625ba6336a69a6c2a40498ccd6eabe
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/374399
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
No longer directly include env_posix.cc. Instead,
use the EnvWrapper class from the public header
env.h.
Change-Id: Iab97f943d26ca9f07ad2b80f4f3706e1593f3caa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/370585
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This will avoid possible complitation issue.
Change-Id: I423d5e200c53cae4f8c5317d8b04e08a09c3b59a
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/371435
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Myrocks engine uses errno(ENOENT) as one of the condition
to create new database, this is okay for POSIX environment,
to support MySQL, spdk environment layer set errno to right
value as well.
Change-Id: Ib633f915ffafde0eacc0a8a0c451e510396d9176
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/369320
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Retire the old claim/unclaim semantics in favor of
open/close. Clients must now open a bdev to get
an spdk_bdev_desc, then pass this desc to get an
I/O channel.
This allows multiple clients to open a bdev,
although only one may open a bdev with write
access.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d319f1278170124169a8a75fd791e926b3f7171
Reviewed-on: https://review.gerrithub.io/367611
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Recent versions of RocksDB have moved the env plugins
to the env/ directory from util/
Change-Id: Ie9478e871e9e656721aa963bc92183636e567f00
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/366699
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Remove dpdk_ prefix in spdk_app_opts and spdk_env_opts
Change-Id: I6f231f67072b808e84945d41b1fe31a180beb350
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/365787
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This separates the RPC library from the event framework.
Change-Id: I26a9cb318b56d44ec9337f8a4db98967eb87ad95
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/365283
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This removes the strict dependency on the SPDK
event framework.
Change-Id: Ie6b6f9585d09ab1df0f25bf33d53e7cdecd83faa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/365281
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Much like bdev modules inside the bdev directory,
add a subsystems directory inside of event. The subsystem
specific code for the bdev library is placed in to
a separate library in that directory, breaking the
strict dependency of the bdev library on the event subsystem
code.
Change-Id: I255941b823a9ec3e2d62f22a586414949d8ff5ad
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/365055
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In this patch, spdk_app_init will be merged into
spdk_app_start.
Due to this change, we need to change event_perf
to use spdk_app_start, since we cannot use spdk_app_init
anymore. So the related changes for event_perf is to
make it work with reactor framework.
Change-Id: Id67edf209fd628cca361a499068c93aeedfe6167
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/364153
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Move the necessary RocksDB Makefile changes into a Makefile
fragment, and put that fragment into SPDK. This allows SPDK
changes to update the RocksDB build.
Change-Id: I4fee0ba4e1ec7fe35b51187c8700109e9cd29206
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/365057
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This code was previously in our fork of RocksDB. Move it here
so that API breaking changes can update it.
Change-Id: Icae3e22380b9bd3de8c1ec5b6f82909f812d204b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/364531
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
