kernel APIs. List of sources used: 1) rdma-core was cloned from "https://github.com/linux-rdma/rdma-core.git" Top commit d65138ef93af30b3ea249f3a84aa6a24ba7f8a75 2) OpenSM was cloned from git://git.openfabrics.org/~halr/opensm.git Top commit 85f841cf209f791c89a075048a907020e924528d 3) libibmad was cloned from "git://git.openfabrics.org/~iraweiny/libibmad.git" Tag 1.3.13 with some additional patches from Mellanox. 4) infiniband-diags was cloned from "git://git.openfabrics.org/~iraweiny/infiniband-diags.git" Tag 1.6.7 with some additional patches from Mellanox. Added the required Makefiles for building and installing. Sponsored by: Mellanox Technologies
400 lines
12 KiB
Groff
400 lines
12 KiB
Groff
.\" Man page generated from reStructuredText.
|
|
.
|
|
.TH IBNETDISCOVER 8 "" "" "Open IB Diagnostics"
|
|
.SH NAME
|
|
IBNETDISCOVER \- discover InfiniBand topology
|
|
.
|
|
.nr rst2man-indent-level 0
|
|
.
|
|
.de1 rstReportMargin
|
|
\\$1 \\n[an-margin]
|
|
level \\n[rst2man-indent-level]
|
|
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
-
|
|
\\n[rst2man-indent0]
|
|
\\n[rst2man-indent1]
|
|
\\n[rst2man-indent2]
|
|
..
|
|
.de1 INDENT
|
|
.\" .rstReportMargin pre:
|
|
. RS \\$1
|
|
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
|
|
. nr rst2man-indent-level +1
|
|
.\" .rstReportMargin post:
|
|
..
|
|
.de UNINDENT
|
|
. RE
|
|
.\" indent \\n[an-margin]
|
|
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
.nr rst2man-indent-level -1
|
|
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
|
|
..
|
|
.SH SYNOPSIS
|
|
.sp
|
|
ibnetdiscover [options] [<topology\-file>]
|
|
.SH DESCRIPTION
|
|
.sp
|
|
ibnetdiscover performs IB subnet discovery and outputs a human readable
|
|
topology file. GUIDs, node types, and port numbers are displayed
|
|
as well as port LIDs and NodeDescriptions. All nodes (and links) are displayed
|
|
(full topology). Optionally, this utility can be used to list the current
|
|
connected nodes by nodetype. The output is printed to standard output
|
|
unless a topology file is specified.
|
|
.SH OPTIONS
|
|
.sp
|
|
\fB\-l, \-\-list\fP
|
|
List of connected nodes
|
|
.sp
|
|
\fB\-g, \-\-grouping\fP
|
|
Show grouping. Grouping correlates IB nodes by different vendor specific
|
|
schemes. It may also show the switch external ports correspondence.
|
|
.sp
|
|
\fB\-H, \-\-Hca_list\fP
|
|
List of connected CAs
|
|
.sp
|
|
\fB\-S, \-\-Switch_list\fP
|
|
List of connected switches
|
|
.sp
|
|
\fB\-R, \-\-Router_list\fP
|
|
List of connected routers
|
|
.sp
|
|
\fB\-s, \-\-show\fP
|
|
Show progress information during discovery.
|
|
.sp
|
|
\fB\-f, \-\-full\fP
|
|
Show full information (ports\(aq speed and width, vlcap)
|
|
.sp
|
|
\fB\-p, \-\-ports\fP
|
|
Obtain a ports report which is a
|
|
list of connected ports with relevant information (like LID, portnum,
|
|
GUID, width, speed, and NodeDescription).
|
|
.sp
|
|
\fB\-m, \-\-max_hops\fP
|
|
Report max hops discovered.
|
|
.\" Define the common option -z
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \fB\-\-outstanding_smps, \-o <val>\fP
|
|
Specify the number of outstanding SMP\(aqs which should be issued during the scan
|
|
.sp
|
|
Default: 2
|
|
.UNINDENT
|
|
.SS Cache File flags
|
|
.\" Define the common option cache
|
|
.
|
|
.sp
|
|
\fB\-\-cache <filename>\fP
|
|
Cache the ibnetdiscover network data in the specified filename. This
|
|
cache may be used by other tools for later analysis.
|
|
.\" Define the common option load-cache
|
|
.
|
|
.sp
|
|
\fB\-\-load\-cache <filename>\fP
|
|
Load and use the cached ibnetdiscover data stored in the specified
|
|
filename. May be useful for outputting and learning about other
|
|
fabrics or a previous state of a fabric.
|
|
.\" Define the common option diff
|
|
.
|
|
.sp
|
|
\fB\-\-diff <filename>\fP
|
|
Load cached ibnetdiscover data and do a diff comparison to the current
|
|
network or another cache. A special diff output for ibnetdiscover
|
|
output will be displayed showing differences between the old and current
|
|
fabric. By default, the following are compared for differences: switches,
|
|
channel adapters, routers, and port connections.
|
|
.\" Define the common option diffcheck
|
|
.
|
|
.sp
|
|
\fB\-\-diffcheck <key(s)>\fP
|
|
Specify what diff checks should be done in the \fB\-\-diff\fP option above.
|
|
Comma separate multiple diff check key(s). The available diff checks
|
|
are: \fBsw = switches\fP, \fBca = channel adapters\fP, \fBrouter\fP = routers,
|
|
\fBport\fP = port connections, \fBlid\fP = lids, \fBnodedesc\fP = node
|
|
descriptions. Note that \fBport\fP, \fBlid\fP, and \fBnodedesc\fP are
|
|
checked only for the node types that are specified (e.g. \fBsw\fP,
|
|
\fBca\fP, \fBrouter\fP). If \fBport\fP is specified alongside \fBlid\fP
|
|
or \fBnodedesc\fP, remote port lids and node descriptions will also be compared.
|
|
.SS Port Selection flags
|
|
.\" Define the common option -C
|
|
.
|
|
.sp
|
|
\fB\-C, \-\-Ca <ca_name>\fP use the specified ca_name.
|
|
.\" Define the common option -P
|
|
.
|
|
.sp
|
|
\fB\-P, \-\-Port <ca_port>\fP use the specified ca_port.
|
|
.\" Explanation of local port selection
|
|
.
|
|
.SS Local port Selection
|
|
.sp
|
|
Multiple port/Multiple CA support: when no IB device or port is specified
|
|
(see the "local umad parameters" below), the libibumad library
|
|
selects the port to use by the following criteria:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.INDENT 0.0
|
|
.IP 1. 3
|
|
the first port that is ACTIVE.
|
|
.IP 2. 3
|
|
if not found, the first port that is UP (physical link up).
|
|
.UNINDENT
|
|
.sp
|
|
If a port and/or CA name is specified, the libibumad library attempts
|
|
to fulfill the user request, and will fail if it is not possible.
|
|
.sp
|
|
For example:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
ibaddr # use the first port (criteria #1 above)
|
|
ibaddr \-C mthca1 # pick the best port from "mthca1" only.
|
|
ibaddr \-P 2 # use the second (active/up) port from the first available IB device.
|
|
ibaddr \-C mthca0 \-P 2 # use the specified port only.
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.SS Configuration flags
|
|
.\" Define the common option -z
|
|
.
|
|
.sp
|
|
\fB\-\-config, \-z <config_file>\fP Specify alternate config file.
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
Default: /etc/infiniband-diags/ibdiag.conf
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.\" Define the common option -z
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \fB\-\-outstanding_smps, \-o <val>\fP
|
|
Specify the number of outstanding SMP\(aqs which should be issued during the scan
|
|
.sp
|
|
Default: 2
|
|
.UNINDENT
|
|
.\" Define the common option --node-name-map
|
|
.
|
|
.sp
|
|
\fB\-\-node\-name\-map <node\-name\-map>\fP Specify a node name map.
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
This file maps GUIDs to more user friendly names. See FILES section.
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.\" Define the common option -t
|
|
.
|
|
.sp
|
|
\fB\-t, \-\-timeout <timeout_ms>\fP override the default timeout for the solicited mads.
|
|
.\" Define the common option -y
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \fB\-y, \-\-m_key <key>\fP
|
|
use the specified M_key for requests. If non\-numeric value (like \(aqx\(aq)
|
|
is specified then a value will be prompted for.
|
|
.UNINDENT
|
|
.SS Debugging flags
|
|
.\" Define the common option -d
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-d
|
|
raise the IB debugging level.
|
|
May be used several times (\-ddd or \-d \-d \-d).
|
|
.UNINDENT
|
|
.\" Define the common option -e
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-e
|
|
show send and receive errors (timeouts and others)
|
|
.UNINDENT
|
|
.\" Define the common option -h
|
|
.
|
|
.sp
|
|
\fB\-h, \-\-help\fP show the usage message
|
|
.\" Define the common option -v
|
|
.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \fB\-v, \-\-verbose\fP
|
|
increase the application verbosity level.
|
|
May be used several times (\-vv or \-v \-v \-v)
|
|
.UNINDENT
|
|
.\" Define the common option -V
|
|
.
|
|
.sp
|
|
\fB\-V, \-\-version\fP show the version info.
|
|
.SH FILES
|
|
.\" Common text for the config file
|
|
.
|
|
.SS CONFIG FILE
|
|
.sp
|
|
/etc/infiniband-diags/ibdiag.conf
|
|
.sp
|
|
A global config file is provided to set some of the common options for all
|
|
tools. See supplied config file for details.
|
|
.\" Common text to describe the node name map file.
|
|
.
|
|
.SS NODE NAME MAP FILE FORMAT
|
|
.sp
|
|
The node name map is used to specify user friendly names for nodes in the
|
|
output. GUIDs are used to perform the lookup.
|
|
.sp
|
|
This functionality is provided by the opensm\-libs package. See \fBopensm(8)\fP
|
|
for the file location for your installation.
|
|
.sp
|
|
\fBGenerically:\fP
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
# comment
|
|
<guid> "<name>"
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.sp
|
|
\fBExample:\fP
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
# IB1
|
|
# Line cards
|
|
0x0008f104003f125c "IB1 (Rack 11 slot 1 ) ISR9288/ISR9096 Voltaire sLB\-24D"
|
|
0x0008f104003f125d "IB1 (Rack 11 slot 1 ) ISR9288/ISR9096 Voltaire sLB\-24D"
|
|
0x0008f104003f10d2 "IB1 (Rack 11 slot 2 ) ISR9288/ISR9096 Voltaire sLB\-24D"
|
|
0x0008f104003f10d3 "IB1 (Rack 11 slot 2 ) ISR9288/ISR9096 Voltaire sLB\-24D"
|
|
0x0008f104003f10bf "IB1 (Rack 11 slot 12 ) ISR9288/ISR9096 Voltaire sLB\-24D"
|
|
|
|
# Spines
|
|
0x0008f10400400e2d "IB1 (Rack 11 spine 1 ) ISR9288 Voltaire sFB\-12D"
|
|
0x0008f10400400e2e "IB1 (Rack 11 spine 1 ) ISR9288 Voltaire sFB\-12D"
|
|
0x0008f10400400e2f "IB1 (Rack 11 spine 1 ) ISR9288 Voltaire sFB\-12D"
|
|
0x0008f10400400e31 "IB1 (Rack 11 spine 2 ) ISR9288 Voltaire sFB\-12D"
|
|
0x0008f10400400e32 "IB1 (Rack 11 spine 2 ) ISR9288 Voltaire sFB\-12D"
|
|
|
|
# GUID Node Name
|
|
0x0008f10400411a08 "SW1 (Rack 3) ISR9024 Voltaire 9024D"
|
|
0x0008f10400411a28 "SW2 (Rack 3) ISR9024 Voltaire 9024D"
|
|
0x0008f10400411a34 "SW3 (Rack 3) ISR9024 Voltaire 9024D"
|
|
0x0008f104004119d0 "SW4 (Rack 3) ISR9024 Voltaire 9024D"
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.\" Common text to describe the Topology file.
|
|
.
|
|
.SS TOPOLOGY FILE FORMAT
|
|
.sp
|
|
The topology file format is human readable and largely intuitive.
|
|
Most identifiers are given textual names like vendor ID (vendid), device ID
|
|
(device ID), GUIDs of various types (sysimgguid, caguid, switchguid, etc.).
|
|
PortGUIDs are shown in parentheses (). For switches, this is shown on the
|
|
switchguid line. For CA and router ports, it is shown on the connectivity
|
|
lines. The IB node is identified followed by the number of ports and a quoted
|
|
the node GUID. On the right of this line is a comment (#) followed by the
|
|
NodeDescription in quotes. If the node is a switch, this line also contains
|
|
whether switch port 0 is base or enhanced, and the LID and LMC of port 0.
|
|
Subsequent lines pertaining to this node show the connectivity. On the
|
|
left is the port number of the current node. On the right is the peer node
|
|
(node at other end of link). It is identified in quotes with nodetype
|
|
followed by \- followed by NodeGUID with the port number in square brackets.
|
|
Further on the right is a comment (#). What follows the comment is
|
|
dependent on the node type. If it it a switch node, it is followed by
|
|
the NodeDescription in quotes and the LID of the peer node. If it is a
|
|
CA or router node, it is followed by the local LID and LMC and then
|
|
followed by the NodeDescription in quotes and the LID of the peer node.
|
|
The active link width and speed are then appended to the end of this
|
|
output line.
|
|
.sp
|
|
An example of this is:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
#
|
|
# Topology file: generated on Tue Jun 5 14:15:10 2007
|
|
#
|
|
# Max of 3 hops discovered
|
|
# Initiated from node 0008f10403960558 port 0008f10403960559
|
|
|
|
Non\-Chassis Nodes
|
|
|
|
vendid=0x8f1
|
|
devid=0x5a06
|
|
sysimgguid=0x5442ba00003000
|
|
switchguid=0x5442ba00003080(5442ba00003080)
|
|
Switch 24 "S\-005442ba00003080" # "ISR9024 Voltaire" base port 0 lid 6 lmc 0
|
|
[22] "H\-0008f10403961354"[1](8f10403961355) # "MT23108 InfiniHost Mellanox Technologies" lid 4 4xSDR
|
|
[10] "S\-0008f10400410015"[1] # "SW\-6IB4 Voltaire" lid 3 4xSDR
|
|
[8] "H\-0008f10403960558"[2](8f1040396055a) # "MT23108 InfiniHost Mellanox Technologies" lid 14 4xSDR
|
|
[6] "S\-0008f10400410015"[3] # "SW\-6IB4 Voltaire" lid 3 4xSDR
|
|
[12] "H\-0008f10403960558"[1](8f10403960559) # "MT23108 InfiniHost Mellanox Technologies" lid 10 4xSDR
|
|
|
|
vendid=0x8f1
|
|
devid=0x5a05
|
|
switchguid=0x8f10400410015(8f10400410015)
|
|
Switch 8 "S\-0008f10400410015" # "SW\-6IB4 Voltaire" base port 0 lid 3 lmc 0
|
|
[6] "H\-0008f10403960984"[1](8f10403960985) # "MT23108 InfiniHost Mellanox Technologies" lid 16 4xSDR
|
|
[4] "H\-005442b100004900"[1](5442b100004901) # "MT23108 InfiniHost Mellanox Technologies" lid 12 4xSDR
|
|
[1] "S\-005442ba00003080"[10] # "ISR9024 Voltaire" lid 6 1xSDR
|
|
[3] "S\-005442ba00003080"[6] # "ISR9024 Voltaire" lid 6 4xSDR
|
|
|
|
vendid=0x2c9
|
|
devid=0x5a44
|
|
caguid=0x8f10403960984
|
|
Ca 2 "H\-0008f10403960984" # "MT23108 InfiniHost Mellanox Technologies"
|
|
[1](8f10403960985) "S\-0008f10400410015"[6] # lid 16 lmc 1 "SW\-6IB4 Voltaire" lid 3 4xSDR
|
|
|
|
vendid=0x2c9
|
|
devid=0x5a44
|
|
caguid=0x5442b100004900
|
|
Ca 2 "H\-005442b100004900" # "MT23108 InfiniHost Mellanox Technologies"
|
|
[1](5442b100004901) "S\-0008f10400410015"[4] # lid 12 lmc 1 "SW\-6IB4 Voltaire" lid 3 4xSDR
|
|
|
|
vendid=0x2c9
|
|
devid=0x5a44
|
|
caguid=0x8f10403961354
|
|
Ca 2 "H\-0008f10403961354" # "MT23108 InfiniHost Mellanox Technologies"
|
|
[1](8f10403961355) "S\-005442ba00003080"[22] # lid 4 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR
|
|
|
|
vendid=0x2c9
|
|
devid=0x5a44
|
|
caguid=0x8f10403960558
|
|
Ca 2 "H\-0008f10403960558" # "MT23108 InfiniHost Mellanox Technologies"
|
|
[2](8f1040396055a) "S\-005442ba00003080"[8] # lid 14 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR
|
|
[1](8f10403960559) "S\-005442ba00003080"[12] # lid 10 lmc 1 "ISR9024 Voltaire" lid 6 1xSDR
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.sp
|
|
When grouping is used, IB nodes are organized into chassis which are
|
|
numbered. Nodes which cannot be determined to be in a chassis are
|
|
displayed as "Non\-Chassis Nodes". External ports are also shown on the
|
|
connectivity lines.
|
|
.SH AUTHORS
|
|
.INDENT 0.0
|
|
.TP
|
|
.B Hal Rosenstock
|
|
< \fI\%halr@voltaire.com\fP >
|
|
.TP
|
|
.B Ira Weiny
|
|
< \fI\%ira.weiny@intel.com\fP >
|
|
.UNINDENT
|
|
.\" Generated by docutils manpage writer.
|
|
.
|