d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

eal/x86: use lock-prefixed instructions for SMP barrier

Browse Source 
On x86 it is possible to use lock-prefixed instructions to get
the similar effect as mfence.
As pointed by Java guys, on most modern HW that gives a better
performance than using mfence:
https://shipilev.net/blog/2014/on-the-fence-with-dependencies/
That patch adopts that technique for rte_smp_mb() implementation.
On BDW 2.2 mb_autotest on single lcore reports 2X cycle reduction,
i.e. from ~110 to ~55 cycles per operation.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
This commit is contained in:
Konstantin Ananyev 2018-01-15 15:09:31 +00:00 committed by Thomas Monjalon
parent 93da5b59af
commit 096ffd811f
1 changed files with 42 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
lib/librte_eal/common/include/arch/x86/rte_atomic.h
View File

@ -27,12 +27,52 @@ extern "C" {
#define rte_rmb() _mm_lfence()
#define rte_smp_mb() rte_mb()
#define rte_smp_wmb() rte_compiler_barrier()
#define rte_smp_rmb() rte_compiler_barrier()
/*
* From Intel Software Development Manual; Vol 3;
* 8.2.2 Memory Ordering in P6 and More Recent Processor Families:
* ...
* . Reads are not reordered with other reads.
* . Writes are not reordered with older reads.
* . Writes to memory are not reordered with other writes,
* with the following exceptions:
* . streaming stores (writes) executed with the non-temporal move
* instructions (MOVNTI, MOVNTQ, MOVNTDQ, MOVNTPS, and MOVNTPD); and
* . string operations (see Section 8.2.4.1).
* ...
* . Reads may be reordered with older writes to different locations but not
* with older writes to the same location.
* . Reads or writes cannot be reordered with I/O instructions,
* locked instructions, or serializing instructions.
* . Reads cannot pass earlier LFENCE and MFENCE instructions.
* . Writes ... cannot pass earlier LFENCE, SFENCE, and MFENCE instructions.
* . LFENCE instructions cannot pass earlier reads.
* . SFENCE instructions cannot pass earlier writes ...
* . MFENCE instructions cannot pass earlier reads, writes ...
*
* As pointed by Java guys, that makes possible to use lock-prefixed
* instructions to get the same effect as mfence and on most modern HW
* that gives a better perfomance then using mfence:
* https://shipilev.net/blog/2014/on-the-fence-with-dependencies/
* Basic idea is to use lock prefixed add with some dummy memory location
* as the destination. From their experiments 128B(2 cache lines) below
* current stack pointer looks like a good candidate.
* So below we use that techinque for rte_smp_mb() implementation.
*/
static __rte_always_inline void
rte_smp_mb(void)
{
#ifdef RTE_ARCH_I686
asm volatile("lock addl $0, -128(%%esp); " ::: "memory");
#else
asm volatile("lock addl $0, -128(%%rsp); " ::: "memory");
#endif
}
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_compiler_barrier()