Dumping every object file for every symbol is too heavy.
Use a temporary storage.
Before:
$ rm -rf master && make defconfig O=master
$ time make EXTRA_CFLAGS=-g O=master
[...]
real 2m24.063s
user 1m16.985s
sys 1m46.372s
After:
$ rm -rf master && make defconfig O=master
$ time make EXTRA_CFLAGS=-g O=master
[...]
real 1m37.110s
user 0m49.417s
sys 0m51.803s
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
There should not be blank lines at end of files.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
For some reason, ICC creates additional global symbols with a . which
triggers an error in the check that validates that symbols in the
experimental section are properly published in the map file of the
library. Filter them out.
Fixes: 3290ac14eb ("buildtools: detect discrepancies for experimental symbols")
Reported-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
When promoting those symbols as stable, there is no check to ensure that
the final result is consistent.
Add a little script to get the symbols per section from the library map
files.
Validate that all experimental symbols in object files are referenced by
library map files.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Rather than having a separate version.map file for linux/BSD and an
exports definition file for windows for each library, generate the
latter from the former automatically at build time.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The post-install script to symlink the PMDs from their own PMD directory to
the regular lib directory (so they would be found by ld at runtime) was
using the "-r" flag to ln to create relative symlinks. This flag is
unsupported by ln on FreeBSD causing the ninja install step to fail.
Reworking the script to take the relative driver path as parameter removes
the need for ln to calculate the relative path ensuring compatibility with
FreeBSD.
As part of the fix, we move the registration of the install script to the
config/meson.build file, from the top level one. This improves readability
as the script takes as parameters the variables set in that file.
Fixes: ed4d43d73e ("build: symlink drivers to library directory")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Define variables for "is_linux", "is_freebsd" and "is_windows"
to make the code shorter for comparisons and more readable.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Added meson workarounds to build helloworld on Windows.
Windows currently only supports kvargs and eal libraries.
This change restricts the build flow to supported libraries
only.
Signed-off-by: Anand Rawat <anand.rawat@intel.com>
Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Harini Ramakrishnan <harini.ramakrishnan@microsoft.com>
The libraries provided by rdma-core may be statically linked
if enabling CONFIG_RTE_IBVERBS_LINK_STATIC in the make-based build.
If CONFIG_RTE_BUILD_SHARED_LIB is disabled, the applications
will embed the mlx PMDs with ibverbs and the mlx libraries.
If CONFIG_RTE_BUILD_SHARED_LIB is enabled,
the mlx PMDs will embed ibverbs and the mlx libraries.
Support with meson may be added later.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
When building using make, the Makefile in the build directory contained
the name of the build directory to be passed as an "O=" parameter to
the DPDK SDK makefiles. Unfortunately, this meant that the compilation
would always fail if the build directory was renamed. To remove this
limitation, we can use $(CURDIR) instead of the directory name.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
With "make -C test/" command getting following warnings:
awk: cmd. line:1: fatal: cannot open file `/cmdline_test/cmdline_test/'
for reading (No such file or directory)
awk: cmd. line:1: fatal: cannot open file
`/test-pipeline/test-pipeline/' for reading (No such file or
directory)
awk: cmd. line:1: fatal: cannot open file `/test-acl/test-acl/'
for reading (No such file or directory)
This is because unexpected/invalid MAPFILE param passed to
check-experimental-syms.sh
There is no easy way to unify MAPFILE for different build options,
instead add an input verification to script, and silently ignore wrong
values.
Fixes: a6ec31597a ("mk: add experimental tag check")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
build error:
== Build drivers/net/tap
mktemp: cannot create temp file /tmp/dpdk.auto-config-h.sh.XXX.c:
Invalid argument
.../buildtools/auto-config-h.sh: line 86: : No such file or directory
.../drivers/net/tap/Makefile:55: recipe for target
'tap_autoconf.h.new' failed
Above error observed on Wind River Linux 8.0
`mktemp` command in that system has a restrictions to have X in
the template at the end and at least six of them.
Complied to mktemp requirements and add -xc flag to compiler to say
`temp` file is a C file
Fixes: ff37ca5d37 ("devtools: use a common prefix for temporary files")
Reported-by: Shuai Zhu <shuaix.zhu@intel.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
We use _GNU_SOURCE all over the place, but often times we miss
defining it, resulting in broken builds on musl. Rather than
fixing every library's and driver's and application's makefile,
fix it by simply defining _GNU_SOURCE by default for all
builds.
Remove all usages of _GNU_SOURCE in source files and makefiles,
and also fixup a couple of instances of using __USE_GNU instead
of _GNU_SOURCE.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
The experimental symbol check script matched on the regexes
"\.text.*$SYM" and "\.text\.experimental.*$SYM" which allows for
substring matches.
If a symbol is leading substring of another one (e.g. symbol foo
is a substring of symbol foobar), it would match on symbols
when it shouldn't.
It is fixed by matching additionally on the end of line
so that symbols are an exact match.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Some temporary files were generated in /tmp, others in the current
directory, and none was "dpdk prefixed".
All these files have a common path prefix now: $TMPDIR/dpdk.
TMPDIR is /tmp by default.
Note: the previous use of mktemp, with a template but without -t,
was generating a file in the current directory.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The "-Wimplicit-fallthrough=2" option was introduced into gcc 7.0, it was
enabled when the cross compiler gcc is greater than 7.0, but for the host
side buildtools/pmdinfogen, if the native gcc is older than 7.0, or the
host cc compiler is clang, it should not be enabled.
The fix is to differentiate the host gcc Werror options from the cross gcc.
gcc -Wp,-MD,./.pmdinfogen.o.d.tmp -W -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations -Wold-style-definition
-Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual
-Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated
-Werror -Wimplicit-fallthrough=2 -Dbbb -Wno-format-truncation -g
-I/dpdk/build/include -o pmdinfogen.o -c
~/dpdk/buildtools/pmdinfogen/pmdinfogen.c gcc: error:
unrecognized command line option ‘-Wimplicit-fallthrough=2’
Fixes: ced3e6f8 ("mk: adjust gcc flags for new gcc 7 warnings")
Cc: stable@dpdk.org
Signed-off-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Steve Capper <steve.capper@arm.com>
Tested-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
If build fails because of failed experimental check and stdout is
redirected to /dev/null, it is absolutely unclear why build fails.
Fixes: a4bcd61de8 ("buildtools: add script to check experimental API exports")
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Coverity flags an issue where the resources used by the FILE object for
the temporary input file are leaked. This is a very minor issue, but is
easily fixed, while also avoiding later problems where we try to close
an invalid file descriptor in the failure case.
The fix is to use "dup()" to get a new file descriptor number rather than
using the value directly from fileno. This allows us to close the file
opened with tmpfile() within in scope block, while allowing the duplicate
to pass to the outer block and be closed when the function terminates.
As a side-effect I/O in the function is therefore changed from using stdio
fread/fwrite to read/write system calls.
Coverity issue: 260399
Fixes: 0d68533617 ("pmdinfogen: allow using stdin and stdout")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Since meson 0.44, changing any file inside a PMD directory (quite
correctly) triggers a full re-run of meson on build, rather than an
incremental build as with earlier versions. This rerun is needed because
we use "grep" in meson to search for files on which to run pmdinfogen, and
changing any of those files means that grep and, therefore meson, needs to
be rerun. [Previous versions of meson did not track this dependency on the
grep command, and so did incremental builds only.]
If, however, we take advantage of pmdinfogen's ability to use stdin and
stdout instead of files, we can instead use a shell script to process an
entire static archive and generate a single .c file from it. This
eliminates the need for grep, and means that changes to a PMD file only
need an incremental build - a significant time saving.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Any flags added to the project args are automatically added to all builds,
both native and cross-compiled. This is not what we want for the -march
flag as a valid -march for the cross-compile is not valid for pmdinfogen
which is a native-build tool.
Instead we store the march flag as a variable, and add it to the default
cflags for all libs, drivers, examples, etc. This will allow pmdinfogen to
compile successfully in a cross-compilation environment.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
With the introduction of bus drivers, we now have a situation where
driver libraries will start to depend upon each other. Because of this,
the driver libs need to be discoverable by the dynamic loader.
There are three options to fix this:
1. Force the user to put the $libdir/dpdk/drivers folder into their
library path.
2. Move all libraries from drivers sub-directory to $libdir.
3. Symlink all libraries from the subfolder to the main library dir.
Option 1 is not great for usability or distro packaging, and option 2
means that we can't have EAL load all drivers from a known path
automatically (as it would error out on non-PMD libs), so option 3 was
chosen as the best fix. The only downside is that on a "ninja uninstall"
the symlinks are not removed, as they are unknown to meson/ninja.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Add the buildtools folder, and more specifically the pmdinfogen binary to
the meson and ninja build. This will be needed for building the PMDs in the
driver folder later, as the pmd info output from the tool needs to be
included in those libs.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Luca Boccassi <luca.boccassi@gmail.com>
This tools reads the given version map for a directory, and checks to
ensure that, for each symbol listed in the export list, the corresponding
definition is tagged as __rte_experimental, erroring out if its not. In this
way, we can ensure that the EXPERIMENTAL api is kept in sync with the tags
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Rather than having to work off files all the time, allow stdin and stdout
to be used as the source and destination for pmdinfogen. This will allow
other possible usages from scripts, e.g. taking files from ar archive and
building a single .pmd.c file from all the .o files in it.
for f in `ar t librte_pmd_xyz.a` ; do
ar p librte_pmd_xyz.a $f | pmdinfogen - - >> xyz_info.c
done
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Cross compiling DPDK for BE mode on ARM results into errors
"PMDINFO portal/dpaa2_hw_dpio.o.pmd.c No drivers registered"
The original code assumes the sh_size to be 32 bit, while it can
be Elf32_Word or Elf64_Xword based on 32bit or 64 bit systems.
This patches replaces the sh_size conversion routines to use ADDR_SIZE
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Cc: stable@dpdk.org
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Replace the BSD license header with the SPDX tag for
scripting files with only an Intel copyright on them.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
In func locate_pmd_entries(), pointer 'new' returned from call to func
'calloc' may be NULL. It is dereferenced without null point check.
Signed-off-by: Yong Wang <wang.yong19@zte.com.cn>
There are random build errors in test reports [1]. Build error
is not directly related to DPDK but observed during DPDK build.
When I get similar unexpected build errors in my system, found
out that /dev/null is invalid.
It seems ICC overwrites the /dev/null with "icc -o /dev/null" instead
of sending output to /dev/null. This is not always reproducible, so
hard to say what exactly is triggering the error.
I suspect test-report build errors can be because of the same reason,
and it is good to add a protection for this case.
Instead of sending output to /dev/null save it to the tmp folder and
remove it back when done.
[1]
http://dpdk.org/ml/archives/test-report/2017-November/034053.html
Failure #3
/usr/src/linux-headers-4.4.0-97-generic/include/linux/sysfs.h:517:37:
error: pointer targets in passing argument 2 of ‘kernfs_find_and_get’
differ in signedness [-Werror=pointer-sign]
return kernfs_find_and_get(parent, name);
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
This commit zeros out the elf_info struct at startup of the
pmdinfogen code. If it is not zeroed, later in the code gcc
produces "may be unused" prints. Clang does not report any
issue.
This issue is only observed when compiling pmdinfogen as an
optimized build, hence this warning is not disabled in the
existing Makefile.
This commit enables a simplification in the meson build
system, removing the requirement for "-Wno-maybe-uninitialized".
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Before this patch, the management of dependencies between directories
had several issues:
- the generation of .depdirs, done at configuration is slow: it can take
more than one minute on some slow targets (usually ~10s on a standard
PC without -j).
- for instance, it is possible to express a dependency like:
- app/foo depends on lib/librte_foo
- and lib/librte_foo depends on app/bar
But this won't work because the directories are traversed with a
depth-first algorithm, so we have to choose between doing 'app' before
or after 'lib'.
- the script depdirs-rule.sh is too complex.
- we cannot use "make -d" for debug, because the output of make is used for
the generation of .depdirs.
This patch moves the DEPDIRS-* variables in the upper Makefile, making
the dependencies much easier to calculate. A DEPDIRS variable is still
used to process library dependencies in LDLIBS.
After this commit, "make config" is almost immediate.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Tested-by: Robin Jarry <robin.jarry@6wind.com>
Tested-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
When creating the symlinks for header files to the include folder, the
relpath script dereferenced all symlinks. This made it impossible to
have file A.h renamed to B.h and then symlinked back to its original
name. This is useful to be able to do when refactoring or reworking
a library. Change this so that we just use the dirname of the path from
readlink, we can use the basename as it was originally, even if it was a
symlink.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Coverity reports a forward null dereference from a for loop
that works with a variable previously tested for null that had no error
handling or condition to prevent it. Pretty obvious fix below.
Coverity issue: 139593
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
There is already a directory buildtools for pmdinfogen used by
the build system. The scripts used in makefiles are moved here.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Add a new macro RTE_PMD_REGISTER_KMOD_DEP() that allows a driver to
declare the list of kernel modules required to run properly.
Today, most PCI drivers require uio/vfio.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
pmdinfogen has a bug in which, during build, it pulls in rte_byteorder.h to
obtain the rte macros for byteswapping between the cpu byte order and big or
little endian. Unfortunately, pmdinfogen is a tool that is only meant to be run
during the build of dpdk components, and so, it runs on the host. In cross
compile environments however, the rte_byteorder.h is configured using a target
cpu, who's endianness may differ from that of the host, leading to improper
swapping.
The fix is to use host system defined byte swapping routines rather than the
dpdk provided routines. Note that we are using non posix compliant routines, as
the posix compliant api only addresses 16 and 32 bit swaps, and we also need 64
bit swaps. Those macros exist (via endian.h), but BSD and Linux put that header
in different locations so some ifdeffery is required.
Tested successfully by myself on Linux and BSD systems.
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Compile error:
CC mlx5.o.pmd.o
mlx5.o.pmd.c:1:227:
error: no newline at end of file [-Werror,-Wnewline-eof]
...__attribute__((used)) = "PMD_INFO_STRING= {...}";
^
Produced with clang 3.8.0 and MLX5_PMD and MLX5_DEBUG
config options enabled.
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
The following tools may be installed system-wide.
It may be cleaner and more convenient to find them with the same
dpdk- prefix (especially for autocompletion).
Moreover, the script dpdk_nic_bind.py deserves a new name because it is
not restricted to NICs and can be used for e.g. crypto.
These files are renamed:
pmdinfogen -> dpdk-pmdinfogen
pmdinfo.py -> dpdk-pmdinfo.py
dpdk_pdump -> dpdk-pdump
dpdk_proc_info -> dpdk-procinfo
dpdk_nic_bind.py -> dpdk-devbind.py
setup.sh -> dpdk-setup.sh
The tools pmdinfogen, pmdinfo.py and dpdk_pdump are new in 16.07.
The scripts dpdk_nic_bind.py and setup.sh may have been used with
previous releases by end users. That's why a symbolic link still
provide the old name in the installed tools directory.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
error on compilation caused by missing include for libgen.h.
buildtools/pmdinfogen/pmdinfogen.c:402:4: error:
implicit declaration of function 'basename' is invalid in C99
basename(argv[0]));
Fixes: 840e5dfea3 ("pmdinfogen: fix usage message")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The name of the tool is pmdinfogen.
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
When compiled with a standard clang, pmdinfogen can raise a warning:
buildtools/pmdinfogen/pmdinfogen.c:365:1: warning:
control reaches end of non-void function
Actually there can be more warnings with stricter compilers.
In order to catch them early and fix most of them, the DPDK standard flags
WERROR_FLAGS are used.
The warnings fixed are:
no previous prototype for ...
no return statement in function returning non-void
variable ‘secstrings’ set but not used
‘sec_name’ defined but not used
‘get_symbol_index’ defined but not used
pointer of type ‘void *’ used in arithmetic
Fixes: 98b0fdb0ff ("pmdinfogen: add buildtools and pmdinfogen utility")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
pmdinfogen is a tool used to parse object files and build json strings for
use in later determining hardware support in a dso or application binary.
pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
this_pmd_tbl<n> (where n is a integer counter). It records the name of
each of these tuples, using the later to find the symbolic name of the
pci_table for physical devices that the object supports. With this
information, it outputs a C file with a single line of the form:
static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
PMD_DRIVER_INFO=<json string>";
Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
json encoded string that hold relevant pmd information, including the pmd
name, type and optional array of pci device/vendor ids that the driver
supports.
This c file is suitable for compiling to object code, then relocatably
linking into the parent file from which the C was generated. This creates
an entry in the string table of the object that can inform a later tool
about hardware support.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
