d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
numam-dpdk/doc/api/doxy-api-index.md
Jerin Jacob bab9497ef7 regexdev: introduce API 
As RegEx usage become more used by DPDK applications, for example:
* Next Generation Firewalls (NGFW)
* Deep Packet and Flow Inspection (DPI)
* Intrusion Prevention Systems (IPS)
* DDoS Mitigation
* Network Monitoring
* Data Loss Prevention (DLP)
* Smart NICs
* Grammar based content processing
* URL, spam and adware filtering
* Advanced auditing and policing of user/application security policies
* Financial data mining - parsing of streamed financial feeds
* Application recognition.
* Dmemory introspection.
* Natural Language Processing (NLP)
* Sentiment Analysis.
* Big data database acceleration.
* Computational storage.

Number of PMD providers started to work on HW implementation,
along side with SW implementations.

This lib adds the support for those kind of devices.

The RegEx Device API is composed of two parts:
- The application-oriented RegEx API that includes functions to setup
  a RegEx device (configure it, setup its queue pairs and start it),
  update the rule database and so on.

- The driver-oriented RegEx API that exports a function allowing
  a RegEx poll Mode Driver (PMD) to simultaneously register itself as
  a RegEx device driver.

RegEx device components and definitions:

    +-----------------+
    |                 |
    |                 o---------+    rte_regexdev_[en|de]queue_burst()
    |   PCRE based    o------+  |               |
    |  RegEx pattern  |      |  |  +--------+   |
    | matching engine o------+--+--o        |   |    +------+
    |                 |      |  |  | queue  |<==o===>|Core 0|
    |                 o----+ |  |  | pair 0 |        |      |
    |                 |    | |  |  +--------+        +------+
    +-----------------+    | |  |
           ^               | |  |  +--------+
           |               | |  |  |        |        +------+
           |               | +--+--o queue  |<======>|Core 1|
       Rule|Database       |    |  | pair 1 |        |      |
    +------+----------+    |    |  +--------+        +------+
    |     Group 0     |    |    |
    | +-------------+ |    |    |  +--------+        +------+
    | | Rules 0..n  | |    |    |  |        |        |Core 2|
    | +-------------+ |    |    +--o queue  |<======>|      |
    |     Group 1     |    |       | pair 2 |        +------+
    | +-------------+ |    |       +--------+
    | | Rules 0..n  | |    |
    | +-------------+ |    |       +--------+
    |     Group 2     |    |       |        |        +------+
    | +-------------+ |    |       | queue  |<======>|Core n|
    | | Rules 0..n  | |    +-------o pair n |        |      |
    | +-------------+ |            +--------+        +------+
    |     Group n     |
    | +-------------+ |<-------rte_regexdev_rule_db_update()
    | |             | |<-------rte_regexdev_rule_db_compile_activate()
    | | Rules 0..n  | |<-------rte_regexdev_rule_db_import()
    | +-------------+ |------->rte_regexdev_rule_db_export()
    +-----------------+

RegEx: A regular expression is a concise and flexible means for matching
strings of text, such as particular characters, words, or patterns of
characters. A common abbreviation for this is â~@~\RegExâ~@~].

RegEx device: A hardware or software-based implementation of RegEx
device API for PCRE based pattern matching syntax and semantics.

PCRE RegEx syntax and semantics specification:
http://regexkit.sourceforge.net/Documentation/pcre/pcrepattern.html

RegEx queue pair: Each RegEx device should have one or more queue pair to
transmit a burst of pattern matching request and receive a burst of
receive the pattern matching response. The pattern matching
request/response embedded in *rte_regex_ops* structure.

Rule: A pattern matching rule expressed in PCRE RegEx syntax along with
Match ID and Group ID to identify the rule upon the match.

Rule database: The RegEx device accepts regular expressions and converts
them into a compiled rule database that can then be used to scan data.
Compilation allows the device to analyze the given pattern(s) and
pre-determine how to scan for these patterns in an optimized fashion that
would be far too expensive to compute at run-time. A rule database
contains a set of rules that compiled in device specific binary form.

Match ID or Rule ID: A unique identifier provided at the time of rule
creation for the application to identify the rule upon match.

Group ID: Group of rules can be grouped under one group ID to enable
rule isolation and effective pattern matching. A unique group identifier
provided at the time of rule creation for the application to identify
the rule upon match.

Scan: A pattern matching request through *enqueue* API.

It may possible that a given RegEx device may not support all the
features
of PCRE. The application may probe unsupported features through
struct rte_regexdev_info::pcre_unsup_flags

By default, all the functions of the RegEx Device API exported by a PMD
are lock-free functions which assume to not be invoked in parallel on
different logical cores to work on the same target object. For instance,
the dequeue function of a PMD cannot be invoked in parallel on two logical
cores to operates on same RegEx queue pair. Of course, this function
can be invoked in parallel by different logical core on different queue
pair. It is the responsibility of the upper level application to
enforce this rule.

In all functions of the RegEx API, the RegEx device is
designated by an integer >= 0 named the device identifier *dev_id*

At the RegEx driver level, RegEx devices are represented by a generic
data structure of type *rte_regexdev*.
RegEx devices are dynamically registered during the PCI/SoC device
probing phase performed at EAL initialization time.
When a RegEx device is being probed, a *rte_regexdev* structure and
a new device identifier are allocated for that device. Then, the
regexdev_init() function supplied by the RegEx driver matching the
probed device is invoked to properly initialize the device.

The role of the device init function consists of resetting the hardware
or software RegEx driver implementations.

If the device init operation is successful, the correspondence between
the device identifier assigned to the new device and its associated
*rte_regexdev* structure is effectively registered.
Otherwise, both the *rte_regexdev* structure and the device identifier
are freed.

The functions exported by the application RegEx API to setup a device
designated by its device identifier must be invoked in the following
order:
    - rte_regexdev_configure()
    - rte_regexdev_queue_pair_setup()
    - rte_regexdev_start()

Then, the application can invoke, in any order, the functions
exported by the RegEx API to enqueue pattern matching job, dequeue
pattern matching response, get the stats, update the rule database,
get/set device attributes and so on

If the application wants to change the configuration (i.e. call
rte_regexdev_configure() or rte_regexdev_queue_pair_setup()), it must
call rte_regexdev_stop() first to stop the device and then do the
reconfiguration before calling rte_regexdev_start() again. The enqueue and
dequeue functions should not be invoked when the device is stopped.

Finally, an application can close a RegEx device by invoking the
rte_regexdev_close() function.

Each function of the application RegEx API invokes a specific function
of the PMD that controls the target device designated by its device
identifier.

For this purpose, all device-specific functions of a RegEx driver are
supplied through a set of pointers contained in a generic structure of
type *regexdev_ops*.
The address of the *regexdev_ops* structure is stored in the
*rte_regexdev* structure by the device init function of the RegEx driver,
which is invoked during the PCI/SoC device probing phase, as explained
earlier.

In other words, each function of the RegEx API simply retrieves the
*rte_regexdev* structure associated with the device identifier and
performs an indirect invocation of the corresponding driver function
supplied in the *regexdev_ops* structure of the *rte_regexdev*
structure.

For performance reasons, the address of the fast-path functions of the
RegEx driver is not contained in the *regexdev_ops* structure.
Instead, they are directly stored at the beginning of the *rte_regexdev*
structure to avoid an extra indirect memory access during their
invocation.

RTE RegEx device drivers do not use interrupts for enqueue or dequeue
operation. Instead, RegEx drivers export Poll-Mode enqueue and dequeue
functions to applications.

The *enqueue* operation submits a burst of RegEx pattern matching
request to the RegEx device and the *dequeue* operation gets a burst of
pattern matching response for the ones submitted through *enqueue*
operation.

Typical application utilisation of the RegEx device API will follow the
following programming flow.

- rte_regexdev_configure()
- rte_regexdev_queue_pair_setup()
- rte_regexdev_rule_db_update() Needs to invoke if precompiled rule
  database not
  provided in rte_regexdev_config::rule_db for rte_regexdev_configure()
  and/or application needs to update rule database.
- rte_regexdev_rule_db_compile_activate() Needs to invoke if
  rte_regexdev_rule_db_update function was used.
- Create or reuse exiting mempool for *rte_regex_ops* objects.
- rte_regexdev_start()
- rte_regexdev_enqueue_burst()
- rte_regexdev_dequeue_burst()

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Ori Kam <orika@mellanox.com>
2020-07-07 00:24:38 +02:00

7.1 KiB
Raw Blame History

API

The public API headers are grouped by topics:

  • device: [dev] (@ref rte_dev.h), [ethdev] (@ref rte_ethdev.h), [ethctrl] (@ref rte_eth_ctrl.h), [rte_flow] (@ref rte_flow.h), [rte_tm] (@ref rte_tm.h), [rte_mtr] (@ref rte_mtr.h), [bbdev] (@ref rte_bbdev.h), [cryptodev] (@ref rte_cryptodev.h), [security] (@ref rte_security.h), [compressdev] (@ref rte_compressdev.h), [compress] (@ref rte_comp.h), [regexdev] (@ref rte_regexdev.h), [eventdev] (@ref rte_eventdev.h), [event_eth_rx_adapter] (@ref rte_event_eth_rx_adapter.h), [event_eth_tx_adapter] (@ref rte_event_eth_tx_adapter.h), [event_timer_adapter] (@ref rte_event_timer_adapter.h), [event_crypto_adapter] (@ref rte_event_crypto_adapter.h), [rawdev] (@ref rte_rawdev.h), [metrics] (@ref rte_metrics.h), [bitrate] (@ref rte_bitrate.h), [latency] (@ref rte_latencystats.h), [devargs] (@ref rte_devargs.h), [PCI] (@ref rte_pci.h), [vdev] (@ref rte_bus_vdev.h), [vfio] (@ref rte_vfio.h)

  • device specific: [softnic] (@ref rte_eth_softnic.h), [bond] (@ref rte_eth_bond.h), [vhost] (@ref rte_vhost.h), [vdpa] (@ref rte_vdpa.h), [KNI] (@ref rte_kni.h), [ixgbe] (@ref rte_pmd_ixgbe.h), [i40e] (@ref rte_pmd_i40e.h), [ice] (@ref rte_pmd_ice.h), [bnxt] (@ref rte_pmd_bnxt.h), [dpaa] (@ref rte_pmd_dpaa.h), [dpaa2] (@ref rte_pmd_dpaa2.h), [dpaa2_mempool] (@ref rte_dpaa2_mempool.h), [dpaa2_cmdif] (@ref rte_pmd_dpaa2_cmdif.h), [dpaa2_qdma] (@ref rte_pmd_dpaa2_qdma.h), [crypto_scheduler] (@ref rte_cryptodev_scheduler.h)

  • memory: [memseg] (@ref rte_memory.h), [memzone] (@ref rte_memzone.h), [mempool] (@ref rte_mempool.h), [malloc] (@ref rte_malloc.h), [memcpy] (@ref rte_memcpy.h)

  • timers: [cycles] (@ref rte_cycles.h), [timer] (@ref rte_timer.h), [alarm] (@ref rte_alarm.h)

  • locks: [atomic] (@ref rte_atomic.h), [mcslock] (@ref rte_mcslock.h), [rwlock] (@ref rte_rwlock.h), [spinlock] (@ref rte_spinlock.h), [ticketlock] (@ref rte_ticketlock.h), [RCU] (@ref rte_rcu_qsbr.h)

  • CPU arch: [branch prediction] (@ref rte_branch_prediction.h), [cache prefetch] (@ref rte_prefetch.h), [SIMD] (@ref rte_vect.h), [byte order] (@ref rte_byteorder.h), [CPU flags] (@ref rte_cpuflags.h), [CPU pause] (@ref rte_pause.h), [I/O access] (@ref rte_io.h)

  • CPU multicore: [interrupts] (@ref rte_interrupts.h), [launch] (@ref rte_launch.h), [lcore] (@ref rte_lcore.h), [per-lcore] (@ref rte_per_lcore.h), [service cores] (@ref rte_service.h), [keepalive] (@ref rte_keepalive.h), [power/freq] (@ref rte_power.h)

  • layers: [ethernet] (@ref rte_ether.h), [ARP] (@ref rte_arp.h), [HIGIG] (@ref rte_higig.h), [ICMP] (@ref rte_icmp.h), [ESP] (@ref rte_esp.h), [IPsec] (@ref rte_ipsec.h), [IPsec group] (@ref rte_ipsec_group.h), [IPsec SA] (@ref rte_ipsec_sa.h), [IPsec SAD] (@ref rte_ipsec_sad.h), [IP] (@ref rte_ip.h), [SCTP] (@ref rte_sctp.h), [TCP] (@ref rte_tcp.h), [UDP] (@ref rte_udp.h), [GTP] (@ref rte_gtp.h), [GRO] (@ref rte_gro.h), [GSO] (@ref rte_gso.h), [frag/reass] (@ref rte_ip_frag.h), [LPM IPv4 route] (@ref rte_lpm.h), [LPM IPv6 route] (@ref rte_lpm6.h), [VXLAN] (@ref rte_vxlan.h)

  • QoS: [metering] (@ref rte_meter.h), [scheduler] (@ref rte_sched.h), [RED congestion] (@ref rte_red.h)

  • hashes: [hash] (@ref rte_hash.h), [jhash] (@ref rte_jhash.h), [thash] (@ref rte_thash.h), [FBK hash] (@ref rte_fbk_hash.h), [CRC hash] (@ref rte_hash_crc.h)

  • classification [reorder] (@ref rte_reorder.h), [distributor] (@ref rte_distributor.h), [EFD] (@ref rte_efd.h), [ACL] (@ref rte_acl.h), [member] (@ref rte_member.h), [flow classify] (@ref rte_flow_classify.h), [BPF] (@ref rte_bpf.h)

  • containers: [mbuf] (@ref rte_mbuf.h), [mbuf pool ops] (@ref rte_mbuf_pool_ops.h), [ring] (@ref rte_ring.h), [stack] (@ref rte_stack.h), [tailq] (@ref rte_tailq.h), [bitmap] (@ref rte_bitmap.h)

  • packet framework:

    • [port] (@ref rte_port.h): [ethdev] (@ref rte_port_ethdev.h), [ring] (@ref rte_port_ring.h), [frag] (@ref rte_port_frag.h), [reass] (@ref rte_port_ras.h), [sched] (@ref rte_port_sched.h), [kni] (@ref rte_port_kni.h), [src/sink] (@ref rte_port_source_sink.h)
    • [table] (@ref rte_table.h): [lpm IPv4] (@ref rte_table_lpm.h), [lpm IPv6] (@ref rte_table_lpm_ipv6.h), [ACL] (@ref rte_table_acl.h), [hash] (@ref rte_table_hash.h), [array] (@ref rte_table_array.h), [stub] (@ref rte_table_stub.h)
    • [pipeline] (@ref rte_pipeline.h) [port_in_action] (@ref rte_port_in_action.h) [table_action] (@ref rte_table_action.h)
    • [graph] (@ref rte_graph.h): [graph_worker] (@ref rte_graph_worker.h)
    • graph_nodes: [eth_node] (@ref rte_node_eth_api.h), [ip4_node] (@ref rte_node_ip4_api.h)

  • basic: [bitops] (@ref rte_bitops.h), [approx fraction] (@ref rte_approx.h), [random] (@ref rte_random.h), [config file] (@ref rte_cfgfile.h), [key/value args] (@ref rte_kvargs.h), [string] (@ref rte_string_fns.h)

  • debug: [jobstats] (@ref rte_jobstats.h), [telemetry] (@ref rte_telemetry.h), [pdump] (@ref rte_pdump.h), [hexdump] (@ref rte_hexdump.h), [debug] (@ref rte_debug.h), [log] (@ref rte_log.h), [errno] (@ref rte_errno.h), [trace] (@ref rte_trace.h), [trace_point] (@ref rte_trace_point.h)

  • misc: [EAL config] (@ref rte_eal.h), [common] (@ref rte_common.h), [experimental APIs] (@ref rte_compat.h), [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h)