net/mlx5: document E-Switch limitations with HWS

This patch adds the following limitations to the mlx5 PMD guide:

- With HW Steering and E-Switch enabled, transfer proxy port must
  be started before any port representor.
- With HW Steering and E-Switch enabled, all representors
  must be stopped before transfer proxy port is stopped.

Documentation of mlx5 PMD's implementations of
rte_eth_dev_start() and rte_eth_dev_stop() is updated accordingly:

- rte_eth_dev_start() returns (-EAGAIN) when transfer proxy port
  cannot be started.
- rte_eth_dev_stop() returns (-EBUSY) when port representor
  cannot be stopped.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

doc/guides/nics/mlx5.rst

@@ -161,6 +161,19 @@ Limitations
   - NIC ConnectX-5 and before are not supported.
   - Partial match with item template is not supported.
   - IPv6 5-tuple matching is not supported.
+  - With E-Switch enabled, ports which share the E-Switch domain
+    should be started and stopped in a specific order:
+
+    - When starting ports, the transfer proxy port should be started first
+      and port representors should follow.
+    - When stopping ports, all of the port representors
+      should be stopped before stopping the transfer proxy port.
+
+    If ports are started/stopped in an incorrect order,
+    ``rte_eth_dev_start()``/``rte_eth_dev_stop()`` will return an appropriate error code:
+
+    - ``-EAGAIN`` for ``rte_eth_dev_start()``.
+    - ``-EBUSY`` for ``rte_eth_dev_stop()``.
 
 - When using Verbs flow engine (``dv_flow_en`` = 0), flow pattern without any
   specific VLAN will match for VLAN packets as well:

drivers/net/mlx5/mlx5_trigger.c

@@ -1138,6 +1138,10 @@ mlx5_hw_representor_port_allowed_start(struct rte_eth_dev *dev)
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
+ *   The following error values are defined:
+ *
+ *   - -EAGAIN: If port representor cannot be started,
+ *     because transfer proxy port is not started.
  */
 int
 mlx5_dev_start(struct rte_eth_dev *dev)
@@ -1394,6 +1398,13 @@ mlx5_hw_proxy_port_allowed_stop(struct rte_eth_dev *dev)
  *
  * @param dev
  *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ *   The following error values are defined:
+ *
+ *   - -EBUSY: If transfer proxy port cannot be stopped,
+ *     because other port representors are still running.
  */
 int
 mlx5_dev_stop(struct rte_eth_dev *dev)