133 lines
4.8 KiB
Groff
133 lines
4.8 KiB
Groff
|
.TH "RDMA_CM" 7 "2008-01-02" "librdmacm" "Librdmacm Programmer's Manual" librdmacm
|
||
|
|
.SH NAME
|
||
|
|
rdma_cm \- RDMA communication manager.
|
||
|
|
.SH SYNOPSIS
|
||
|
|
.B "#include <rdma/rdma_cma.h>"
|
||
|
|
.SH "DESCRIPTION"
|
||
|
|
Used to establish communication over RDMA transports.
|
||
|
|
.SH "NOTES"
|
||
|
|
The RDMA CM is a communication manager used to setup reliable, connected
|
||
|
|
and unreliable datagram data transfers. It provides an RDMA transport
|
||
|
|
neutral interface for establishing connections. The API is based on sockets,
|
||
|
|
but adapted for queue pair (QP) based semantics: communication must be
|
||
|
|
over a specific RDMA device, and data transfers are message based.
|
||
|
|
.P
|
||
|
|
The RDMA CM only provides the communication management (connection setup /
|
||
|
|
teardown) portion of an RDMA API. It works in conjunction with the verbs
|
||
|
|
API defined by the libibverbs library. The libibverbs library provides the
|
||
|
|
interfaces needed to send and receive data.
|
||
|
|
.SH "CLIENT OPERATION"
|
||
|
|
This section provides a general overview of the basic operation for the active,
|
||
|
|
or client, side of communication. A general connection flow would be:
|
||
|
|
.IP rdma_create_event_channel
|
||
|
|
create channel to receive events
|
||
|
|
.IP rdma_create_id
|
||
|
|
allocate an rdma_cm_id, this is conceptually similar to a socket
|
||
|
|
.IP rdma_resolve_addr
|
||
|
|
obtain a local RDMA device to reach the remote address
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_ADDR_RESOLVED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.IP rdma_create_qp
|
||
|
|
allocate a QP for the communication
|
||
|
|
.IP rdma_resolve_route
|
||
|
|
determine the route to the remote address
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_ROUTE_RESOLVED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.IP rdma_connect
|
||
|
|
connect to the remote server
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_ESTABLISHED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.P
|
||
|
|
Perform data transfers over connection
|
||
|
|
.IP rdma_disconnect
|
||
|
|
tear-down connection
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_DISCONNECTED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.IP rdma_destroy_qp
|
||
|
|
destroy the QP
|
||
|
|
.IP rdma_destroy_id
|
||
|
|
release the rdma_cm_id
|
||
|
|
.IP rdma_destroy_event_channel
|
||
|
|
release the event channel
|
||
|
|
.P
|
||
|
|
An almost identical process is used to setup unreliable datagram (UD)
|
||
|
|
communication between nodes. No actual connection is formed between QPs
|
||
|
|
however, so disconnection is not needed.
|
||
|
|
.P
|
||
|
|
Although this example shows the client initiating the disconnect, either side
|
||
|
|
of a connection may initiate the disconnect.
|
||
|
|
.SH "SERVER OPERATION"
|
||
|
|
This section provides a general overview of the basic operation for the passive,
|
||
|
|
or server, side of communication. A general connection flow would be:
|
||
|
|
.IP rdma_create_event_channel
|
||
|
|
create channel to receive events
|
||
|
|
.IP rdma_create_id
|
||
|
|
allocate an rdma_cm_id, this is conceptually similar to a socket
|
||
|
|
.IP rdma_bind_addr
|
||
|
|
set the local port number to listen on
|
||
|
|
.IP rdma_listen
|
||
|
|
begin listening for connection requests
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_CONNECT_REQUEST event with a new rdma_cm_id
|
||
|
|
.IP rdma_create_qp
|
||
|
|
allocate a QP for the communication on the new rdma_cm_id
|
||
|
|
.IP rdma_accept
|
||
|
|
accept the connection request
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_ESTABLISHED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.P
|
||
|
|
Perform data transfers over connection
|
||
|
|
.IP rdma_get_cm_event
|
||
|
|
wait for RDMA_CM_EVENT_DISCONNECTED event
|
||
|
|
.IP rdma_ack_cm_event
|
||
|
|
ack event
|
||
|
|
.IP rdma_disconnect
|
||
|
|
tear-down connection
|
||
|
|
.IP rdma_destroy_qp
|
||
|
|
destroy the QP
|
||
|
|
.IP rdma_destroy_id
|
||
|
|
release the connected rdma_cm_id
|
||
|
|
.IP rdma_destroy_id
|
||
|
|
release the listening rdma_cm_id
|
||
|
|
.IP rdma_destroy_event_channel
|
||
|
|
release the event channel
|
||
|
|
.SH "RETURN CODES"
|
||
|
|
.IP "= 0"
|
||
|
|
success
|
||
|
|
.IP "= -1"
|
||
|
|
error - see errno for more details
|
||
|
|
.P
|
||
|
|
Librdmacm functions return 0 to indicate success, and a -1 return value
|
||
|
|
to indicate failure. If a function operates asynchronously, a return value of 0
|
||
|
|
means that the operation was successfully started. The operation could still
|
||
|
|
complete in error; users should check the status of the related event. If the
|
||
|
|
return value is -1, then errno will contain additional information
|
||
|
|
regarding the reason for the failure.
|
||
|
|
.P
|
||
|
|
Prior versions of the library would return -errno and not set errno for some cases
|
||
|
|
related to ENOMEM, ENODEV, ENODATA, EINVAL, and EADDRNOTAVAIL codes. Applications
|
||
|
|
that want to check these codes and have compatability with prior library versions
|
||
|
|
must manually set errno to the negative of the return code if it is < -1.
|
||
|
|
.SH "SEE ALSO"
|
||
|
|
rdma_create_event_channel(3), rdma_get_cm_event(3), rdma_create_id(3),
|
||
|
|
rdma_resolve_addr(3), rdma_bind_addr(3), rdma_create_qp(3),
|
||
|
|
rdma_resolve_route(3), rdma_connect(3), rdma_listen(3), rdma_accept(3),
|
||
|
|
rdma_reject(3), rdma_join_multicast(3), rdma_leave_multicast(3), rdma_notify(3),
|
||
|
|
rdma_ack_cm_event(3), rdma_disconnect(3), rdma_destroy_qp(3), rdma_destroy_id(3),
|
||
|
|
rdma_destroy_event_channel(3), rdma_get_devices(3), rdma_free_devices(3),
|
||
|
|
rdma_get_peer_addr(3), rdma_get_local_addr(3),
|
||
|
|
rdma_get_dst_port(3), rdma_get_src_port(3), rdma_set_option(3)
|
||
|
|
ucmatose(1), udaddy(1), mckey(1), rping(1)
|