pax_global_header00006660000000000000000000000064147305216470014523gustar00rootroot0000000000000052 comment=bb1ac326bd5d3283d52fbbdb7aea3272681763b3 smc-tools-1.8.4/000077500000000000000000000000001473052164700134555ustar00rootroot00000000000000smc-tools-1.8.4/.gitignore000066400000000000000000000000341473052164700154420ustar00rootroot00000000000000*.o *.so smc smc_pnet smcss smc-tools-1.8.4/CONTRIBUTING.md000066400000000000000000000043521473052164700157120ustar00rootroot00000000000000Contributing to smc-tools ========================= License ------- All contributions have to be submitted under the Eclipse Public License v1.0. See also the [LICENSE](LICENSE) file. Developer's Certificate of Origin and Signed-off-by --------------------------------------------------- The sign-off is a simple line at the end of the explanation for the patch, which certifies that you wrote it or otherwise have the right to pass it on as an open-source patch. With the Signed-off-by line you certify the below: ``` Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ``` If you can certify the above, just add a line stating the following at the bottom of each of your commit messages: ``` Signed-off-by: Random Developer ``` Please use your real name and a valid e-mail address (no pseudonyms or anonymous contributions). Submitting code --------------- The preferred way is to create GitHub pull requests for your code contributions. Please create separate pull requests for each logical enhancement, new feature, or fix. smc-tools-1.8.4/LICENSE000066400000000000000000000273471473052164700144770ustar00rootroot00000000000000 Eclipse Public License - v 1.0 THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. 1. DEFINITIONS "Contribution" means: a) in the case of the initial Contributor, the initial code and documentation distributed under this Agreement, and b) in the case of each subsequent Contributor: i) changes to the Program, and ii) additions to the Program; where such changes and/or additions to the Program originate from and are distributed by that particular Contributor. A Contribution 'originates' from a Contributor if it was added to the Program by such Contributor itself or anyone acting on such Contributor's behalf. Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program. "Contributor" means any person or entity that distributes the Program. "Licensed Patents" mean patent claims licensable by a Contributor which are necessarily infringed by the use or sale of its Contribution alone or when combined with the Program. "Program" means the Contributions distributed in accordance with this Agreement. "Recipient" means anyone who receives the Program under this Agreement, including all Contributors. 2. GRANT OF RIGHTS a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form. b) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed Patents to make, use, sell, offer to sell, import and otherwise transfer the Contribution of such Contributor, if any, in source code and object code form. This patent license shall apply to the combination of the Contribution and the Program if, at the time the Contribution is added by the Contributor, such addition of the Contribution causes such combination to be covered by the Licensed Patents. The patent license shall not apply to any other combinations which include the Contribution. No hardware per se is licensed hereunder. c) Recipient understands that although each Contributor grants the licenses to its Contributions set forth herein, no assurances are provided by any Contributor that the Program does not infringe the patent or other intellectual property rights of any other entity. Each Contributor disclaims any liability to Recipient for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, each Recipient hereby assumes sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow Recipient to distribute the Program, it is Recipient's responsibility to acquire that license before distributing the Program. d) Each Contributor represents that to its knowledge it has sufficient copyright rights in its Contribution, if any, to grant the copyright license set forth in this Agreement. 3. REQUIREMENTS A Contributor may choose to distribute the Program in object code form under its own license agreement, provided that: a) it complies with the terms and conditions of this Agreement; and b) its license agreement: i) effectively disclaims on behalf of all Contributors all warranties and conditions, express and implied, including warranties or conditions of title and non-infringement, and implied warranties or conditions of merchantability and fitness for a particular purpose; ii) effectively excludes on behalf of all Contributors all liability for damages, including direct, indirect, special, incidental and consequential damages, such as lost profits; iii) states that any provisions which differ from this Agreement are offered by that Contributor alone and not by any other party; and iv) states that source code for the Program is available from such Contributor, and informs licensees how to obtain it in a reasonable manner on or through a medium customarily used for software exchange. When the Program is made available in source code form: a) it must be made available under this Agreement; and b) a copy of this Agreement must be included with each copy of the Program. Contributors may not remove or alter any copyright notices contained within the Program. Each Contributor must identify itself as the originator of its Contribution, if any, in a manner that reasonably allows subsequent Recipients to identify the originator of the Contribution. 4. COMMERCIAL DISTRIBUTION Commercial distributors of software may accept certain responsibilities with respect to end users, business partners and the like. While this license is intended to facilitate the commercial use of the Program, the Contributor who includes the Program in a commercial product offering should do so in a manner which does not create potential liability for other Contributors. Therefore, if a Contributor includes the Program in a commercial product offering, such Contributor ("Commercial Contributor") hereby agrees to defend and indemnify every other Contributor ("Indemnified Contributor") against any losses, damages and costs (collectively "Losses") arising from claims, lawsuits and other legal actions brought by a third party against the Indemnified Contributor to the extent caused by the acts or omissions of such Commercial Contributor in connection with its distribution of the Program in a commercial product offering. The obligations in this section do not apply to any claims or Losses relating to any actual or alleged intellectual property infringement. In order to qualify, an Indemnified Contributor must: a) promptly notify the Commercial Contributor in writing of such claim, and b) allow the Commercial Contributor to control, and cooperate with the Commercial Contributor in, the defense and any related settlement negotiations. The Indemnified Contributor may participate in any such claim at its own expense. For example, a Contributor might include the Program in a commercial product offering, Product X. That Contributor is then a Commercial Contributor. If that Commercial Contributor then makes performance claims, or offers warranties related to Product X, those performance claims and warranties are such Commercial Contributor's responsibility alone. Under this section, the Commercial Contributor would have to defend claims against the other Contributors related to those performance claims and warranties, and if a court requires any other Contributor to pay any damages as a result, the Commercial Contributor must pay those damages. 5. NO WARRANTY EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the appropriateness of using and distributing the Program and assumes all risks associated with its exercise of rights under this Agreement , including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and unavailability or interruption of operations. 6. DISCLAIMER OF LIABILITY EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 7. GENERAL If any provision of this Agreement is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this Agreement, and without further action by the parties hereto, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable. If Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed. All Recipient's rights under this Agreement shall terminate if it fails to comply with any of the material terms or conditions of this Agreement and does not cure such failure in a reasonable period of time after becoming aware of such noncompliance. If all Recipient's rights under this Agreement terminate, Recipient agrees to cease use and distribution of the Program as soon as reasonably practicable. However, Recipient's obligations under this Agreement and any licenses granted by Recipient relating to the Program shall continue and survive. Everyone is permitted to copy and distribute copies of this Agreement, but in order to avoid inconsistency the Agreement is copyrighted and may only be modified in the following manner. The Agreement Steward reserves the right to publish new versions (including revisions) of this Agreement from time to time. No one other than the Agreement Steward has the right to modify this Agreement. The Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation may assign the responsibility to serve as the Agreement Steward to a suitable separate entity. Each new version of the Agreement will be given a distinguishing version number. The Program (including Contributions) may always be distributed subject to the version of the Agreement under which it was received. In addition, after a new version of the Agreement is published, Contributor may elect to distribute the Program (including its Contributions) under the new version. Except as expressly stated in Sections 2(a) and 2(b) above, Recipient receives no rights or licenses to the intellectual property of any Contributor under this Agreement, whether expressly, by implication, estoppel or otherwise. All rights in the Program not expressly granted under this Agreement are reserved. This Agreement is governed by the laws of the State of New York and the intellectual property laws of the United States of America. No party to this Agreement will bring a legal action under this Agreement more than one year after the cause of action arose. Each party waives its rights to a jury trial in any resulting litigation. smc-tools-1.8.4/Makefile000066400000000000000000000177121473052164700151250ustar00rootroot00000000000000# # SMC Tools - Shared Memory Communication Tools # # Copyright IBM Corp. 2016, 2018 # # All rights reserved. This program and the accompanying materials # are made available under the terms of the Eclipse Public License v1.0 # which accompanies this distribution, and is available at # http://www.eclipse.org/legal/epl-v10.html # SMC_TOOLS_RELEASE = 1.8.4 VER_MAJOR = $(shell echo $(SMC_TOOLS_RELEASE) | cut -d '.' -f 1) ARCHTYPE = $(shell uname -m) ARCH := $(shell getconf LONG_BIT) DISTRO := $(shell lsb_release -si 2>/dev/null) ifneq ("${V}","1") MAKEFLAGS += --quiet cmd = echo $1$2; else cmd = endif CCC = $(call cmd," CC ",$@)${CC} LINK = $(call cmd," LINK ",$@)${CC} GEN = $(call cmd," GEN ",$@)sed DESTDIR ?= PREFIX = /usr BINDIR = ${PREFIX}/bin MANDIR = ${PREFIX}/share/man BASH_AUTODIR = $(shell pkg-config --variable=completionsdir bash-completion 2>/dev/null) OWNER = $(shell id -un) GROUP = $(shell id -gn) INSTALL_FLAGS_BIN = -g $(GROUP) -o $(OWNER) -m755 INSTALL_FLAGS_MAN = -g $(GROUP) -o $(OWNER) -m644 INSTALL_FLAGS_LIB = -g $(GROUP) -o $(OWNER) -m4755 STUFF_32BIT = 0 # # Check that 31/32-bit build tools are available. # ifeq ($(ARCH),64) ifeq ($(DISTRO),Ubuntu) LIBDIR = ${PREFIX}/usr/lib else LIBDIR = ${PREFIX}/lib64 endif ifneq ("$(wildcard ${PREFIX}/include/gnu/stubs-32.h)","") STUFF_32BIT = 1 LIBDIR32 = ${PREFIX}/lib endif else ifeq ($(DISTRO),Ubuntu) LIBDIR = ${PREFIX}/lib/s390-linux-gnu else LIBDIR = ${PREFIX}/lib endif endif all: libsmc-preload.so libsmc-preload32.so smcd smcr smcss smc_pnet CFLAGS ?= -Wall -O3 -g ifneq ($(shell sh -c 'command -v pkg-config'),) LIBNL_CFLAGS = $(shell pkg-config --silence-errors --cflags libnl-genl-3.0) LIBNL_LFLAGS = $(shell pkg-config --silence-errors --libs libnl-genl-3.0) else LIBNL_CFLAGS = -I /usr/include/libnl3 LIBNL_LFLAGS = -lnl-genl-3 -lnl-3 endif ALL_CFLAGS += ${CFLAGS} -DSMC_TOOLS_RELEASE=$(SMC_TOOLS_RELEASE) \ ${LIBNL_CFLAGS} ${OPTFLAGS} ALL_LDFLAGS += ${LDFLAGS} ${LIBNL_LFLAGS} -lm ifeq ($(ARCHTYPE),s390x) MACHINE_OPT32="-m31" else MACHINE_OPT32="-m32" endif util.o: util.c util.h ${CCC} ${ALL_CFLAGS} -c util.c libnetlink.o: libnetlink.c libnetlink.h ${CCC} ${ALL_CFLAGS} ${ALL_LDFLAGS} -c libnetlink.c smc-preload.o: smc-preload.c ${CCC} ${ALL_CFLAGS} -fPIC -c smc-preload.c libsmc-preload.so: smc-preload.o ${LINK} ${ALL_LDFLAGS} -shared smc-preload.o -ldl -Wl,-z,defs,-soname,$@.$(VER_MAJOR) -o $@ chmod u+s $@ libsmc-preload32.so: smc-preload.c ifeq ($(ARCH),64) ifeq ($(STUFF_32BIT),1) ${CCC} ${ALL_CFLAGS} -fPIC -c ${MACHINE_OPT32} $< -o smc-preload32.o ${LINK} ${ALL_LDFLAGS} -shared smc-preload32.o ${MACHINE_OPT32} -ldl -Wl,-soname,$@.$(VER_MAJOR) -o $@ chmod u+s $@ else $(warning "Warning: Skipping 31/32-bit library build because 31/32-bit build tools") $(warning " are unavailable. SMC will not support 31/32 bit applications") $(warning " unless the glibc devel package for the appropriate addressing") $(warning " mode is installed and the preload libraries are rebuilt.") endif endif %d.o: %.c smctools_common.h ${CCC} ${ALL_CFLAGS} -DSMCD -c $< -o $@ %r.o: %.c smctools_common.h ${CCC} ${ALL_CFLAGS} -DSMCR -c $< -o $@ %.o: %.c smctools_common.h ${CCC} ${ALL_CFLAGS} -c $< -o $@ smc: smc.o info.o ueid.o seid.o dev.o linkgroup.o libnetlink.o util.o ${CCC} ${ALL_CFLAGS} ${ALL_LDFLAGS} $^ -o $@ smcd: smcd.o infod.o ueidd.o seidd.o devd.o linkgroupd.o statsd.o libnetlink.o util.o ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ smcr: smcr.o infor.o ueidr.o seidr.o devr.o linkgroupr.o statsr.o libnetlink.o util.o ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ smc_pnet: smc_pnet.c smctools_common.h @if [ ! -e /usr/include/libnl3/netlink/netlink.h ]; then \ printf "**************************************************************\n" >&2; \ printf "* Missing build requirement for: %-45s\n" $@ >&2; \ printf "* Install package..............: %-45s\n" "devel package for libnl3" >&2; \ printf "* Install package..............: %-45s\n" "devel package for libnl3-genl" >&2; \ printf "* NOTE: Package names might differ by platform\n" >&2; \ printf "* On Ubuntu try libnl-3-dev and libnl-genl-3-dev\n" >&2; \ printf "**************************************************************\n" >&2; \ exit 1; \ fi ${CCC} ${ALL_CFLAGS} $< ${ALL_LDFLAGS} -o $@ smcss: smcss.o libnetlink.o ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ install: all echo " INSTALL" install -d -m755 $(DESTDIR)$(LIBDIR) $(DESTDIR)$(BINDIR) $(DESTDIR)$(MANDIR)/man7 \ $(DESTDIR)$(BASH_AUTODIR) $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_LIB) libsmc-preload.so $(DESTDIR)$(LIBDIR) #ifeq ($(STUFF_32BIT),1) # install -d -m755 $(DESTDIR)$(LIBDIR32) # install $(INSTALL_FLAGS_LIB) libsmc-preload32.so $(DESTDIR)$(LIBDIR32)/libsmc-preload.so #endif install $(INSTALL_FLAGS_BIN) smc_run $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_BIN) smcd $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_BIN) smcr $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_BIN) smcss $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_BIN) smc_pnet $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_BIN) smc_dbg $(DESTDIR)$(BINDIR) ifeq ($(shell uname -m | cut -c1-4),s390) install $(INSTALL_FLAGS_BIN) smc_rnics $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_MAN) smc_rnics.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_BIN) smc_chk $(DESTDIR)$(BINDIR) install $(INSTALL_FLAGS_MAN) smc_chk.8 $(DESTDIR)$(MANDIR)/man8 endif install $(INSTALL_FLAGS_MAN) af_smc.7 $(DESTDIR)$(MANDIR)/man7 install $(INSTALL_FLAGS_MAN) smc_run.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smc_pnet.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcss.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcr.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-linkgroup.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-device.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-info.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-stats.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-ueid.8 $(DESTDIR)$(MANDIR)/man8 install $(INSTALL_FLAGS_MAN) smcd-seid.8 $(DESTDIR)$(MANDIR)/man8 ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-linkgroup.8 $(DESTDIR)$(MANDIR)/man8/smcr-linkgroup.8 ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-device.8 $(DESTDIR)$(MANDIR)/man8/smcr-device.8 ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-info.8 $(DESTDIR)$(MANDIR)/man8/smcr-info.8 ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-stats.8 $(DESTDIR)$(MANDIR)/man8/smcr-stats.8 ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-ueid.8 $(DESTDIR)$(MANDIR)/man8/smcr-ueid.8 ifneq ($(BASH_AUTODIR),) install $(INSTALL_FLAGS_MAN) smc-tools.autocomplete $(DESTDIR)$(BASH_AUTODIR)/smc-tools ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_rnics ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_chk ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_dbg ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smcss ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_pnet ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc endif check: if ( command -v cppcheck >/dev/null ); then \ echo "Running cppcheck"; \ cppcheck . 2>&1; \ else \ echo "cppcheck not available"; \ fi @echo; #if type -p valgrind >/dev/null; then \ if ( command -v valgrind >/dev/null ); then \ echo "Running valgrind"; \ valgrind --leak-check=full --show-leak-kinds=all ./smcss 2>&1; \ valgrind --leak-check=full --show-leak-kinds=all ./smc_pnet 2>&1; \ valgrind --leak-check=full --show-leak-kinds=all ./smcd info 2>&1; \ valgrind --leak-check=full --show-leak-kinds=all ./smcd stats 2>&1; \ else \ echo "valgrind not available"; \ fi @echo; clean: echo " CLEAN" rm -f *.o *.so *.a smc smcd smcr smcss smc_pnet smc-tools-1.8.4/README.md000066400000000000000000000153111473052164700147350ustar00rootroot00000000000000SMC Tools ========= Utilities for use with `AF_SMC` sockets. This package consists of the following tools: - `libsmc-preload.so` : preload library. - `smc` : List linkgroups, links, devices, and more - `smc_chk` : SMC support diagnostics - `smc_pnet` : C program for PNET Table handling - `smc_rnics` : List available RDMA NICs - `smc_run` : preload library environment setup script. - `smcss` : C program for displaying the information about active SMC sockets. The preload shared library `libsmc-preload.so` provides mapping of TCP socket operations to SMC sockets. The environment setup script `smc_run` sets up the preload environment for the `libsmc-preload.so` shared library before starting application. The `smcss` program is used to gather and display information about the SMC sockets. The `smc_pnet` program is used to create, destroy, and change the SMC-R PNET table. In addition the package contains the `AF_SMC` manpage (`man af_smc`). License ------- See [LICENSE](LICENSE). Code Contributions ------------------ See [CONTRIBUTING.md](CONTRIBUTING.md). Release History: ================ * __v1.8.4 (2024-12-17)__ Changes: - `smcd`/`smcr` statistics: add statistics on sndbufs/RMBs usage Bug fixes: - `smc_rnics`: Fix smc_rnics showing the wrong physical port - `util.h`: Fix build with CFLAGS="-g" - `smc_run`: Fix segfault issue during creating concurrent sockets - `Makefile`: Make sure to show the right release number * __v1.8.3 (2023-07-28)__ Bug fixes: - `Makefile`: Adjust default library path for Ubuntu - `smc_stats`: Fix man page name * __v1.8.2 (2022-09-26)__ Bug fixes: - `smc_run`: Fix for single quotes in parameters - `Makefile`: Fix target `check` - `smcss`/`smc_pnet`: Fix option `--version` - `smcr`/`smc_dbg`: Fix for showing the correct hardware capabailities for RoCE Express3 cards * __v1.8.1 (2022-04-14)__ Changes: - `smc_rnics`: Recognize RoCE Express3 cards * __v1.8.0 (2022-04-11)__ Changes: - `smc_dbg`: Add stats and `smc info` output - `smc_rnics`: - List unknown devices with option `-a` - Include software-set PNET IDs - `smc_chk`: Indicate PNET IDs set by `smc_pnet`. Bug fixes: - `smc_rnics`: Display correct PNET ID for unknown Mellanox cards - `smc_run`: Fix output of version info * __v1.7.0 (2021-10-29)__ Changes: - Add support for SMC-Rv2 - `smcd`/`smcr`: Add support for new commands `seid` and `ueid` to manage system and user EIDs * __v1.6.1 (2021-10-01)__ Bug fixes: - `smcd`/`smcr` statistics: - Fix memory overread in is_data_consistent() - Fix memory and file handle leaks - Use correct fallback counter values after reset * __v1.6.0 (2021-07-01)__ Changes: - `smcd`/`smcr`: Add new command `stats` - `smc_rnics`: Recognize unknown Mellanox cards - `smc_run`: Add various command-line switches Bug fixes: - `smc_chk`: Remove 'EXPERIMENTAL' flag - `smc_chk`: Improve cleanup - `smc_chk`: Start server with intended port - `Makefile`: Install `smc_chk.8` on s390 only - `Makefile`: Fix extra compile flags handling - `smc_rnics`: Handle malformed FID arguments * __v1.5.0 (2021-01-29)__ Changes: - `smcd`/`smcr`: Add new command `info` - `smc_rnics`: Use `n/a` to indicate missing PNET ID - `smc_chk`: New tool to perform SMC eligilibilty checks, requires `man` and `python3` to be installed - `man` pages: Consistency improvements Bug fixes: - `smc_pnet.8`: Use correct spelling for 'PNET ID' - `smc_rnics`: Suppress output of port attribute for offline devices * __v1.4.0 (2020-11-03)__ Changes: - Add SMC-Dv2 support - `smc`: Add new tools `smcd` and `smcr` to list linkgroups, links and devices. Requires Linux kernel 5.11 or higher. - `smc_rnics`: Display enabled devices per default, add new option `--all` - `smc_rnics`: Sort output by FID Bug fixes: - `smc_rnics`/`smc_dbg`: Fix PNETID for multiport devices - `smcss`/`smc_pnet`: Consistent use of option `-v` * __v1.3.1 (2020-09-14)__ Changes: - `smcss`: Add further error codes to man page Bug fixes: - `smcss`: Display more than 321 connections - `smc_rnics`: Suppress any unknown non-networking device unless option `-r` is specified * __v1.3.0 (2020-06-16)__ Changes: - `smcss`: Add description of Linux error codes to man page - `smc_rnics`: * Sort output by PCHID * Replace spaces in output by underscores for easier parsing * Add new option `--IB-dev` to display IB-specific attributes Bug fixes: - smc_rnics: * FIDs can have up to 4 digits and are planned to be extended to a total of 8 digits - adjusting output format accordingly * Do not display port attribute for RoCE Express2 devices unless we have an accurate value * __v1.2.2 (2019-10-24)__ Changes: - Add bash autocompletion support - `Makefile`: Drop 31 Bit install due to rpmbuild conflict Bug fixes: - `smcss`: Do not show connection mode for already closed sockets - `smc_rnics`: Set interface to "n/a" for ISM devices * __v1.2.1 (2019-04-15)__ Bug fixes: - `smc_rnics`: Install man page on s390 only - `libsmc..`: Handle behavior flags in type argument to `socket()` call - `Makefile`: Fixed install target on Ubuntu for platforms other than s390 - `smc_pnet`: Changes in support of kernel 5.1 * __v1.2.0 (2019-02-08)__ Changes: - `smc_rnics`: Initial version added - `smc_dbg`: Initial version added Bug fixes: - `smcss`: Parse address family of ip address * __v1.1.0 (2018-06-29)__ Changes: - `smcss`: Add IPv6 support - `libsmc..`: Add IPv6 support - `smcss`: Output format changed - `libsmc..`: Rename preload library to `libsmc-preload.so` - `Makefile`: Improve distro compatibility - `Makefile`: Add `SONAME` to shared libraries - `Makefile`: Do not strip binaries on install - `Makefile`: Use `LDFLAGS` to allow addition of externally set link flags - `libsmc..`: Remove hardcoded reference to libc - Manpages: Formatting changes Bug fixes: - `Makefile`: Fix target `install` dependencies - `smcss`: Fix `--version` output - `smc_pnet`: Fix `--version` output - `smc_run`: Append preload library to `LD_PRELOAD` instead of potentially overwriting pre-set values - `libsmc..`: Set suid flag to work with suid executables * __v1.0.0 (2017-02-13)__ The initial version Copyright IBM Corp. 2016, 2020 smc-tools-1.8.4/af_smc.7000066400000000000000000000100011473052164700147650ustar00rootroot00000000000000.\" .\" Copyright IBM Corp. 2016, 2018 .\" Author(s): Ursula Braun .\" Thomas Richter .\" ---------------------------------------------------------------------- .TH AF_SMC 7 "January 2017" "smc-tools" "Linux Programmer's Manual" .SH NAME AF_SMC - Sockets for SMC communication .SH SYNOPSIS .B #include .sp .B "#define AF_SMC 43" .sp .B "#define SMCPROTO_SMC 0" .sp .B "#define SMCPROTO_SMC6 1" .PP .IB tcp_sockfd " = socket(" AF_SMC ", " SOCK_STREAM ", " SMCPROTO_SMC ); .sp .IB tcp_sockfd " = socket(" AF_SMC ", " SOCK_STREAM ", " SMCPROTO_SMC6 ); .SH DESCRIPTION .I Shared Memory Communication via RDMA (SMC) is a socket over the RDMA communication protocol that allows existing TCP socket applications to transparently benefit from RDMA when exchanging data over an .I RDMA over Converged Ethernet (RoCE) network. Those networks are not routable. SMC provides host-to-host direct memory access without traditional TCP/IP processing overhead. SMC offers preservation of existing IP topology and IP security, and introduces minimal administrative and operational changes. The exploitation of SMC is transparent to TCP socket applications. .PP The new address family .B AF_SMC supports the SMC protocol on Linux. It keeps the address format of .B AF_INET and .B AF_INET6 sockets and supports streaming socket types only. .SS Usage modes Two usage modes are possible: .IP "AF_SMC native usage" uses the socket domain .B AF_SMC instead of .B AF_INET and .BR AF_INET6 . Specify .B SMCPROTO_SMC for .B AF_INET compatible socket semantics, and .B SMC_PROTO_SMC6 for .B AF_INET6 respectively. .IP "Usage of AF_INET socket applications with SMC preload library" converts .B AF_INET and .B AF_INET6 sockets to .B AF_SMC sockets. The SMC preload library is part of the SMC tools package. .PP SMC socket capabilities are negotiated at connection setup. If one peer is not SMC capable, further socket processing falls back to TCP usage automatically. .SS Implementation details: Links and Link Groups To run RDMA traffic to a peer, a so-called .I link is established between a local RoCE card and a remote RoCE card. To enhance availability, you can configure alternate links with automatic fail over. Primary and backup links to a certain peer are combined in a so-called .I link .IR group . .SS RoCE adapter mapping: Creation of a pnet table The SMC protocol requires grouping of multiple physical networks - standard Ethernet and RoCE networks. Such groups are called .I Physical Networks (PNets). For SMC, RoCE adapter mapping is configured within a table called .BR "pnet table" . Any available Ethernet interface can be combined with available .I RDMA-capable network interface cards (RNICs), if they belong to the same Converged Ethernet fabric. To configure RoCE Adapter mapping, you must create a pnet table. Modify the table with the smc-tools command .IR smc_pnet . .PP For details see .BR smc_pnet (8). .SS Displaying SMC socket state information SMC socket state information can be obtained with the smc-tools command .IR smcss . For details see .BR smcss (8). .SS Starting a TCP application to work with SMC To use an existing TCP application to work with SMC, use the SMC preload library. The SMC Tools package provides the command .I smc_run to convert .B AF_INET and .B AF_INET6 socket calls to .B AF_SMC socket calls by means of the preload technique. For more information about the preload mechanism, see also .BR ld.so (8). .PP Example: .IP .B smc_run ftp .PP This command-line example starts an FTP client over SMC. .PP .SS MTU and Infiniband data transfer Infiniband traffic may use MTU values 256, 512, 1024, 2048, or 4096. SMC determines the configured MTU size of the RoCE Ethernet port, announces this MTU size to the peer during connection start, and chooses the minimum MTU size of both peers. .SH "SEE ALSO" .BR socket (2), .BR ip (7), .BR tcp (7), .BR socket (7), .BR smc_chk (8) .BR smc_run (8), .BR smcss (8), .BR smc_pnet (8) .SH "HISTORY" .TP .B AF_SMC, version 1.0.0 .RS 4 .IP "\bu" 2 Initial version. .RE smc-tools-1.8.4/dev.c000066400000000000000000000371231473052164700144050ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "dev.h" #define MASK_ROCE_V1_HEX 0x1004 #define MASK_ROCE_V2_HEX 0x1016 #define MASK_ROCE_V3_HEX 0x101e #define MASK_ISM_V1_HEX 0x04ed static int netdev_entered = 0; static int ibdev_entered = 0; static int type_entered = 0; static int all_entered = 0; #if defined(SMCD) static int dev_smcr = 0; static int dev_smcd = 1; #else static int dev_smcr = 1; static int dev_smcd = 0; #endif static int d_level = 0; static char target_ibdev[IB_DEVICE_NAME_MAX] = {0}; static char target_type[SMC_TYPE_STR_MAX] = {0}; static char target_ndev[IFNAMSIZ] = {0}; static struct nla_policy smc_gen_dev_smcd_sock_policy[SMC_NLA_DEV_MAX + 1] = { [SMC_NLA_DEV_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_DEV_USE_CNT] = { .type = NLA_U32 }, [SMC_NLA_DEV_IS_CRIT] = { .type = NLA_U8 }, [SMC_NLA_DEV_PCI_FID] = { .type = NLA_U32 }, [SMC_NLA_DEV_PCI_CHID] = { .type = NLA_U16 }, [SMC_NLA_DEV_PCI_VENDOR] = { .type = NLA_U16 }, [SMC_NLA_DEV_PCI_DEVICE] = { .type = NLA_U16 }, [SMC_NLA_DEV_PCI_ID] = { .type = NLA_NUL_STRING }, [SMC_NLA_DEV_PORT] = { .type = NLA_NESTED }, [SMC_NLA_DEV_PORT2] = { .type = NLA_NESTED }, [SMC_NLA_DEV_IB_NAME] = { .type = NLA_NUL_STRING }, }; static struct nla_policy smc_gen_dev_port_smcd_sock_policy[SMC_NLA_DEV_PORT_MAX + 1] = { [SMC_NLA_DEV_PORT_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_DEV_PORT_PNET_USR] = { .type = NLA_U8 }, [SMC_NLA_DEV_PORT_PNETID] = { .type = NLA_NUL_STRING }, [SMC_NLA_DEV_PORT_NETDEV] = { .type = NLA_U32 }, [SMC_NLA_DEV_PORT_STATE] = { .type = NLA_U8 }, [SMC_NLA_DEV_PORT_VALID] = { .type = NLA_U8 }, [SMC_NLA_DEV_PORT_LNK_CNT] = { .type = NLA_U32 }, }; static void usage(void) { fprintf(stderr, #if defined(SMCD) "Usage: smcd device [show] [all]\n" #elif defined(SMCR) "Usage: smcr device [show] [all]\n" " [ibdev ]\n" " [netdev ]\n" #else "Usage: smc device [show] [all] [type {smcd | smcr}]\n" " [ibdev ]\n" " [netdev ]\n" #endif ); exit(-1); } static const char *smc_ib_port_state(unsigned int x) { static char buf[16]; switch (x) { case 0: return "INACTIVE"; case 1: return "ACTIVE"; default: sprintf(buf, "%#x?", x); return buf; } } static const char *smc_ib_dev_type(unsigned int x) { static char buf[16]; switch (x) { case MASK_ROCE_V1_HEX: return "RoCE_Express"; case MASK_ROCE_V2_HEX: return "RoCE_Express2"; case MASK_ROCE_V3_HEX: return "RoCE_Express3"; case MASK_ISM_V1_HEX: return "ISM"; default: sprintf(buf, "%#x", x); return buf; } } static void print_devs_smcd_header(void) { printf("FID "); printf("Type "); printf("PCI-ID "); printf("PCHID "); printf("InUse "); printf("#LGs "); printf("PNET-ID "); printf("\n"); } static void print_devs_smcr_header(void) { printf("Net-Dev "); printf("IB-Dev "); printf("IB-P "); printf("IB-State "); printf("Type "); printf("Crit "); if (d_level >= SMC_DETAIL_LEVEL_V) { printf(" FID "); printf("PCI-ID "); printf("PCHID "); } printf("#Links "); printf("PNET-ID "); printf("\n"); } static int filter_item(struct smc_diag_dev_info *dev, int idx) { int ignore = 0; if (is_str_empty(target_ibdev) && is_str_empty(target_ndev)) { return ignore; /* No filter set */ } else if (!is_str_empty(target_ndev)) { if (strncmp(target_ndev, (char *)dev->netdev[idx], sizeof(target_ndev)) == 0) ignore = 0; else ignore = 1; } else if (!is_str_empty(target_ibdev)) { if (strncmp(target_ibdev, (char *)dev->dev_name, sizeof(target_ibdev)) == 0) ignore = 0; else ignore = 1; } return ignore; } static void show_devs_smcr_details(struct smc_diag_dev_info *dev, int idx) { char buf[SMC_MAX_PNETID_LEN+1] = {0}; if (dev->port_valid[idx] && !filter_item(dev, idx)) { if (strnlen((char*)dev->netdev[idx], sizeof(dev->netdev[idx])) > (IFNAMSIZ - 1)) printf("%-.15s ", (char *)&dev->netdev[idx]); else printf("%-15s ", (char *)&dev->netdev[idx]); if (strnlen((char*)dev->dev_name, sizeof(dev->dev_name)) > SMC_MAX_IBNAME) printf("%-.8s ", dev->dev_name); else printf("%-8s ", dev->dev_name); printf("%4d ", idx+1); printf("%8s ", smc_ib_port_state(dev->port_state[idx])); printf("%-13s ", smc_ib_dev_type(dev->pci_device)); printf("%3s ", dev->is_critical?"Yes":"No"); if (d_level >= SMC_DETAIL_LEVEL_V) { printf("%04x ", dev->pci_fid); printf("%-12s ", dev->pci_id); printf("%04x ", dev->pci_pchid); } printf("%5d ", dev->lnk_cnt_by_port[idx]); if (dev->pnetid_by_user[idx]) snprintf(buf, sizeof(buf),"*%s", dev->pnet_id[idx]); else snprintf(buf, sizeof(buf),"%s", dev->pnet_id[idx]); printf(" %-16s ", trim_space(buf)); printf("\n"); } } static int fill_dev_port_smcr_struct(struct smc_diag_dev_info *dev, struct nlattr **attrs, int idx) { struct nlattr *port_attrs[SMC_NLA_DEV_PORT_MAX + 1]; if (!attrs[SMC_NLA_DEV_PORT + idx]) { dev->port_valid[idx] = 0; return NL_OK; } if (nla_parse_nested(port_attrs, SMC_NLA_DEV_PORT_MAX, attrs[SMC_NLA_DEV_PORT + idx], smc_gen_dev_port_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_dev_port_smcd_sock_policy\n"); return NL_STOP; } if (port_attrs[SMC_NLA_DEV_PORT_PNETID]) snprintf((char*)&dev->pnet_id[idx], sizeof(dev->pnet_id[idx]), "%s", nla_get_string(port_attrs[SMC_NLA_DEV_PORT_PNETID])); if (port_attrs[SMC_NLA_DEV_PORT_PNET_USR]) dev->pnetid_by_user[idx] = nla_get_u8(port_attrs[SMC_NLA_DEV_PORT_PNET_USR]); if (port_attrs[SMC_NLA_DEV_PORT_NETDEV] && nla_get_u32(port_attrs[SMC_NLA_DEV_PORT_NETDEV])) if_indextoname(nla_get_u32(port_attrs[SMC_NLA_DEV_PORT_NETDEV]), (char*)dev->netdev[idx]); if (port_attrs[SMC_NLA_DEV_PORT_STATE]) dev->port_state[idx] = nla_get_u8(port_attrs[SMC_NLA_DEV_PORT_STATE]); if (port_attrs[SMC_NLA_DEV_PORT_VALID]) dev->port_valid[idx] = nla_get_u8(port_attrs[SMC_NLA_DEV_PORT_VALID]); if (port_attrs[SMC_NLA_DEV_PORT_LNK_CNT]) dev->lnk_cnt_by_port[idx] = nla_get_u32(port_attrs[SMC_NLA_DEV_PORT_LNK_CNT]); return NL_OK; } static int fill_dev_smcr_struct(struct smc_diag_dev_info *dev, struct nlattr **attrs) { struct nlattr *dev_attrs[SMC_NLA_DEV_MAX + 1]; int i; if (nla_parse_nested(dev_attrs, SMC_NLA_DEV_MAX, attrs[SMC_GEN_DEV_SMCR], smc_gen_dev_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_dev_smcd_sock_policy\n"); return NL_STOP; } if (dev_attrs[SMC_NLA_DEV_IS_CRIT]) dev->is_critical = nla_get_u8(dev_attrs[SMC_NLA_DEV_IS_CRIT]); if (dev_attrs[SMC_NLA_DEV_PCI_FID]) dev->pci_fid = nla_get_u32(dev_attrs[SMC_NLA_DEV_PCI_FID]); if (dev_attrs[SMC_NLA_DEV_PCI_CHID]) dev->pci_pchid = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_CHID]); if (dev_attrs[SMC_NLA_DEV_PCI_VENDOR]) dev->pci_vendor = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_VENDOR]); if (dev_attrs[SMC_NLA_DEV_PCI_DEVICE]) dev->pci_device = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_DEVICE]); if (dev_attrs[SMC_NLA_DEV_PCI_ID]) snprintf((char*)dev->pci_id, sizeof(dev->pci_id), "%s", nla_get_string(dev_attrs[SMC_NLA_DEV_PCI_ID])); if (dev_attrs[SMC_NLA_DEV_IB_NAME]) snprintf((char*)dev->dev_name, sizeof(dev->dev_name), "%s", nla_get_string(dev_attrs[SMC_NLA_DEV_IB_NAME])); for (i = 0; i < SMC_MAX_PORTS; i++) { if (fill_dev_port_smcr_struct(dev, &dev_attrs[0], i) != NL_OK) return NL_STOP; } return NL_OK; } static int show_dev_smcr_info(struct nlattr **attr) { struct smc_diag_dev_info dev; int i; if (attr[SMC_GEN_DEV_SMCR]) { if (fill_dev_smcr_struct(&dev, attr) != NL_OK) return NL_STOP; for (i = 0; i < SMC_MAX_PORTS; i++) { show_devs_smcr_details(&dev, i); } } return NL_OK; } static int fill_dev_port_smcd_struct(struct smc_diag_dev_info *dev, struct nlattr **attrs, int idx) { struct nlattr *port_attrs[SMC_NLA_DEV_PORT_MAX + 1]; if (nla_parse_nested(port_attrs, SMC_NLA_DEV_PORT_MAX, attrs[SMC_NLA_DEV_PORT], smc_gen_dev_port_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_dev_port_smcd_sock_policy\n"); return NL_STOP; } if (port_attrs[SMC_NLA_DEV_PORT_PNETID]) snprintf((char*)&dev->pnet_id[idx], sizeof(dev->pnet_id[idx]), "%s", nla_get_string(port_attrs[SMC_NLA_DEV_PORT_PNETID])); if (port_attrs[SMC_NLA_DEV_PORT_PNET_USR]) dev->pnetid_by_user[idx] = nla_get_u8(port_attrs[SMC_NLA_DEV_PORT_PNET_USR]); return NL_OK; } static int fill_dev_smcd_struct(struct smc_diag_dev_info *dev, struct nlattr **attrs) { struct nlattr *dev_attrs[SMC_NLA_DEV_MAX + 1]; if (nla_parse_nested(dev_attrs, SMC_NLA_DEV_MAX, attrs[SMC_GEN_DEV_SMCD], smc_gen_dev_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_dev_smcd_sock_policy\n"); return NL_STOP; } if (dev_attrs[SMC_NLA_DEV_USE_CNT]) dev->use_cnt = nla_get_u32(dev_attrs[SMC_NLA_DEV_USE_CNT]); if (dev_attrs[SMC_NLA_DEV_IS_CRIT]) dev->is_critical = nla_get_u8(dev_attrs[SMC_NLA_DEV_IS_CRIT]); if (dev_attrs[SMC_NLA_DEV_PCI_FID]) dev->pci_fid = nla_get_u32(dev_attrs[SMC_NLA_DEV_PCI_FID]); if (dev_attrs[SMC_NLA_DEV_PCI_CHID]) dev->pci_pchid = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_CHID]); if (dev_attrs[SMC_NLA_DEV_PCI_VENDOR]) dev->pci_vendor = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_VENDOR]); if (dev_attrs[SMC_NLA_DEV_PCI_DEVICE]) dev->pci_device = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_DEVICE]); if (dev_attrs[SMC_NLA_DEV_PCI_ID]) snprintf((char*)dev->pci_id, sizeof(dev->pci_id), "%s", nla_get_string(dev_attrs[SMC_NLA_DEV_PCI_ID])); if (fill_dev_port_smcd_struct(dev, &dev_attrs[0], 0) != NL_OK) return NL_STOP; return NL_OK; } static int show_dev_smcd_info(struct nlattr **attr) { char buf[SMC_MAX_PNETID_LEN+1] = {0}; struct smc_diag_dev_info dev; if (attr[SMC_GEN_DEV_SMCD]) { if (fill_dev_smcd_struct(&dev, attr) != NL_OK) return NL_STOP; printf("%04x ", dev.pci_fid); printf("%-4s ", smc_ib_dev_type(dev.pci_device)); printf("%-12s ", dev.pci_id); printf("%04x ", dev.pci_pchid); printf("%-4s ", dev.is_critical?"Yes":"No"); printf("%5d ", dev.use_cnt); if (dev.pnetid_by_user[0]) snprintf(buf, sizeof(buf),"*%s", dev.pnet_id[0]); else snprintf(buf, sizeof(buf),"%s", dev.pnet_id[0]); printf(" %-16s ", trim_space(buf)); printf("\n"); } return NL_OK; } static int handle_gen_dev_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); static int header_printed = 0; int rc = NL_OK; if (!header_printed) { if (dev_smcr) print_devs_smcr_header(); else print_devs_smcd_header(); header_printed = 1; } if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_DEV_SMCD] && !attrs[SMC_GEN_DEV_SMCR]) return NL_STOP; if (dev_smcd && attrs[SMC_GEN_DEV_SMCD]) rc = show_dev_smcd_info(&attrs[0]); if (dev_smcr && attrs[SMC_GEN_DEV_SMCR]) rc = show_dev_smcr_info(&attrs[0]); return rc; } static void handle_cmd_params(int argc, char **argv) { if (((argc == 1) && (contains(argv[0], "help") == 0)) || (argc > 4)) usage(); if ((argc > 0) && (contains(argv[0], "show") != 0)) PREV_ARG(); /* no object given, so use the default "show" */ while (NEXT_ARG_OK()) { NEXT_ARG(); if (ibdev_entered) { snprintf(target_ibdev, sizeof(target_ibdev), "%s", argv[0]); ibdev_entered = 0; break; } else if (netdev_entered) { snprintf(target_ndev, sizeof(target_ndev), "%s", argv[0]); netdev_entered = 0; break; } else if (type_entered) { snprintf(target_type, sizeof(target_type), "%s", argv[0]); if (strncmp(target_type, "smcd", SMC_TYPE_STR_MAX) == 0) { dev_smcd = 1; dev_smcr = 0; } else if ((strnlen(target_type, sizeof(target_type)) < 4) || (strncmp(target_type, "smcr", SMC_TYPE_STR_MAX) != 0)) { print_type_error(); } type_entered = 0; break; } else if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "all") == 0) { all_entered=1; #if !defined(SMCD) && !defined(SMCR) } else if (contains(argv[0], "type") == 0) { type_entered=1; #endif #if !defined(SMCD) } else if (contains(argv[0], "ibdev") == 0) { ibdev_entered =1; } else if (contains(argv[0], "netdev") == 0) { netdev_entered =1; #endif } else { usage(); } } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); } int invoke_devs(int argc, char **argv, int detail_level) { int rc = EXIT_SUCCESS; d_level = detail_level; handle_cmd_params(argc, argv); if (dev_smcd) rc = gen_nl_handle_dump(SMC_NETLINK_GET_DEV_SMCD, handle_gen_dev_reply, NULL); else rc = gen_nl_handle_dump(SMC_NETLINK_GET_DEV_SMCR, handle_gen_dev_reply, NULL); return rc; } /* arg is an (int *) */ static int count_ism_devices_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); int *ism_count = (int *)arg; if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_DEV_SMCD]) return NL_STOP; (*ism_count)++; return NL_OK; } int dev_count_ism_devices(int *ism_count) { *ism_count = 0; return gen_nl_handle_dump(SMC_NETLINK_GET_DEV_SMCD, count_ism_devices_reply, ism_count); } struct count_roce_args { int *rocev1_count; int *rocev2_count; int *rocev3_count; }; /* arg is an (struct count_roce_args *) */ static int count_roce_devices_reply(struct nl_msg *msg, void *arg) { struct count_roce_args *args = (struct count_roce_args *)arg; struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_DEV_SMCR]) return NL_STOP; /* Determine PCI device type */ { struct nlattr *dev_attrs[SMC_NLA_DEV_MAX + 1]; short i = 0; if (nla_parse_nested(dev_attrs, SMC_NLA_DEV_MAX, attrs[SMC_GEN_DEV_SMCR], smc_gen_dev_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_dev_smcd_sock_policy\n"); return NL_STOP; } if (dev_attrs[SMC_NLA_DEV_PCI_DEVICE]) i = nla_get_u16(dev_attrs[SMC_NLA_DEV_PCI_DEVICE]); if (i == MASK_ROCE_V1_HEX) (*args->rocev1_count)++; if (i == MASK_ROCE_V2_HEX) (*args->rocev2_count)++; if (i == MASK_ROCE_V3_HEX) (*args->rocev3_count)++; } return NL_OK; } int dev_count_roce_devices(int *rocev1_count, int *rocev2_count, int *rocev3_count) { struct count_roce_args args = { .rocev1_count = rocev1_count, .rocev2_count = rocev2_count, .rocev3_count = rocev3_count, }; *rocev1_count = 0; *rocev2_count = 0; *rocev3_count = 0; return gen_nl_handle_dump(SMC_NETLINK_GET_DEV_SMCR, count_roce_devices_reply, &args); } smc-tools-1.8.4/dev.h000066400000000000000000000013101473052164700143770ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef DEV_H_ #define DEV_H_ extern struct rtnl_handle rth; int invoke_devs(int argc, char **argv, int detail_level); int dev_count_ism_devices(int *ism_count); int dev_count_roce_devices(int *rocev1_count, int *rocev2_count, int *rocev3_count); #endif /* DEV_H_ */ smc-tools-1.8.4/info.c000066400000000000000000000117271473052164700145640ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2021 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "info.h" #include "dev.h" static int show_cmd = 0; static int ism_count, rocev1_count, rocev2_count, rocev3_count; static struct nla_policy smc_gen_info_policy[SMC_NLA_SYS_MAX + 1] = { [SMC_NLA_SYS_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_SYS_VER] = { .type = NLA_U8 }, [SMC_NLA_SYS_REL] = { .type = NLA_U8 }, [SMC_NLA_SYS_IS_ISM_V2] = { .type = NLA_U8 }, [SMC_NLA_SYS_LOCAL_HOST]= { .type = NLA_NUL_STRING }, [SMC_NLA_SYS_SEID] = { .type = NLA_NUL_STRING }, [SMC_NLA_SYS_IS_SMCR_V2]= { .type = NLA_U8 }, }; static void usage(void) { fprintf(stderr, #if defined(SMCD) "Usage: smcd info [show]\n" #elif defined(SMCR) "Usage: smcr info [show]\n" #else "Usage: smc info [show]\n" #endif ); exit(-1); } static int handle_gen_info_reply(struct nl_msg *msg, void *arg) { struct nlattr *info_attrs[SMC_NLA_SYS_MAX + 1]; struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); int rc = NL_OK; char tmp[80]; int smc_version = 1; if (!show_cmd) return rc; if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_SYS_INFO]) return rc; if (nla_parse_nested(info_attrs, SMC_NLA_DEV_MAX, attrs[SMC_GEN_SYS_INFO], smc_gen_info_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_info_policy\n"); return NL_STOP; } printf("Kernel Capabilities\n"); /* Version */ tmp[0] = '\0'; if (info_attrs[SMC_NLA_SYS_VER] && info_attrs[SMC_NLA_SYS_REL]) { sprintf(tmp, "%d.%d", nla_get_u8(info_attrs[SMC_NLA_SYS_VER]), nla_get_u8(info_attrs[SMC_NLA_SYS_REL])); smc_version = nla_get_u8(info_attrs[SMC_NLA_SYS_VER]); } printf("SMC Version: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); /* Hostname */ tmp[0] = '\0'; if (info_attrs[SMC_NLA_SYS_LOCAL_HOST]) { sprintf(tmp, "%s", nla_get_string(info_attrs[SMC_NLA_SYS_LOCAL_HOST])); } printf("SMC Hostname: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); /* SMC-D */ sprintf(tmp, "%s", "v1"); if (smc_version >= 2) { strcat(tmp, " v2"); } printf("SMC-D Features: %s\n", tmp); /* SMC-R */ sprintf(tmp, "%s", "v1"); if (info_attrs[SMC_NLA_SYS_IS_SMCR_V2] && nla_get_u8(info_attrs[SMC_NLA_SYS_IS_SMCR_V2])) { strcat(tmp, " v2"); } printf("SMC-R Features: %s\n", tmp); printf("\n"); printf("Hardware Capabilities\n"); /* SEID */ tmp[0] = '\0'; if (info_attrs[SMC_NLA_SYS_SEID]) { sprintf(tmp, "%s", nla_get_string(info_attrs[SMC_NLA_SYS_SEID])); } printf("SEID: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); /* ISM hardware */ tmp[0] = '\0'; if (ism_count) { /* Kernel found any ISM device */ sprintf(tmp, "%s", "v1"); /* dev found, v1 is possible */ if (info_attrs[SMC_NLA_SYS_IS_ISM_V2]) { if (nla_get_u8(info_attrs[SMC_NLA_SYS_IS_ISM_V2])) strcat(tmp, " v2"); } } printf("ISM: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); /* RoCE hardware */ tmp[0] = '\0'; if (rocev1_count || rocev2_count || rocev3_count) { /* Kernel found any RoCE device */ strcpy(tmp, ""); if (rocev1_count || rocev2_count || rocev3_count) strcat(tmp, "v1 "); if (rocev2_count || rocev3_count) strcat(tmp, "v2"); } printf("RoCE: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); return rc; } static void handle_cmd_params(int argc, char **argv) { if (argc == 0) { show_cmd = 1; /* no object given, so use the default "show" */ return; } while (1) { if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "show") == 0) { show_cmd = 1; break; } else { usage(); } if (!NEXT_ARG_OK()) break; NEXT_ARG(); } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); } int invoke_info(int argc, char **argv, int detail_level) { int rc = EXIT_SUCCESS; handle_cmd_params(argc, argv); if (show_cmd) { if (dev_count_ism_devices(&ism_count)) { fprintf(stderr, "Error: Failed to retrieve ISM device count\n"); return EXIT_FAILURE; } if (dev_count_roce_devices(&rocev1_count, &rocev2_count, &rocev3_count)) { fprintf(stderr, "Error: Failed to retrieve RoCE device count\n"); return EXIT_FAILURE; } rc = gen_nl_handle_dump(SMC_NETLINK_GET_SYS_INFO, handle_gen_info_reply, NULL); } else { printf("Error: Unknown command\n"); /* we should never come here ... */ return EXIT_FAILURE; } return rc; } smc-tools-1.8.4/info.h000066400000000000000000000010261473052164700145600ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2021 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef INFO_H_ #define INFO_H_ extern struct rtnl_handle rth; int invoke_info(int argc, char **argv, int detail_level); #endif /* INFO_H_ */ smc-tools-1.8.4/libnetlink.c000066400000000000000000000153151473052164700157610ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Ursula Braun * Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "smctools_common.h" #include "libnetlink.h" #define MAGIC_SEQ 123456 int smc_id = 0; struct nl_sock *sk; /* Operations on sock_diag netlink socket */ int rtnl_open(struct rtnl_handle *rth) { socklen_t addr_len; int rcvbuf = 1024 * 1024; int sndbuf = 32768; rth->fd = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_SOCK_DIAG); if (rth->fd < 0) { perror("Error: Cannot open netlink socket"); return EXIT_FAILURE; } if (setsockopt(rth->fd, SOL_SOCKET, SO_SNDBUF, &sndbuf, sizeof(sndbuf)) < 0) { perror("Error: SO_SNDBUF"); return EXIT_FAILURE; } if (setsockopt(rth->fd, SOL_SOCKET, SO_RCVBUF, &rcvbuf, sizeof(rcvbuf)) < 0) { perror("Error: SO_RCVBUF"); return EXIT_FAILURE; } memset(&rth->local, 0, sizeof(rth->local)); rth->local.nl_family = AF_NETLINK; rth->local.nl_groups = 0; if (bind(rth->fd, (struct sockaddr*)&rth->local, sizeof(rth->local)) < 0) { perror("Error: Cannot bind netlink socket"); return EXIT_FAILURE; } addr_len = sizeof(rth->local); if (getsockname(rth->fd, (struct sockaddr*)&rth->local, &addr_len) < 0) { perror("Error: getsockname"); return EXIT_FAILURE; } if (addr_len != sizeof(rth->local)) { fprintf(stderr, "Error: Wrong address length %d\n", addr_len); return EXIT_FAILURE; } if (rth->local.nl_family != AF_NETLINK) { fprintf(stderr, "Error: Wrong address family %d\n", rth->local.nl_family); return EXIT_FAILURE; } rth->seq = time(NULL); return 0; } void rtnl_close(struct rtnl_handle *rth) { if (rth->fd >= 0) { close(rth->fd); rth->fd = -1; } } int rtnl_dump(struct rtnl_handle *rth, void (*handler)(struct nlmsghdr *nlh)) { int msglen, found_done = 0; struct sockaddr_nl nladdr; struct iovec iov; struct msghdr msg = { .msg_name = &nladdr, .msg_namelen = sizeof(nladdr), .msg_iov = &iov, .msg_iovlen = 1, }; char buf[32768]; struct nlmsghdr *h = (struct nlmsghdr *)buf; memset(buf, 0, sizeof(buf)); iov.iov_base = buf; iov.iov_len = sizeof(buf); again: msglen = recvmsg(rth->fd, &msg, 0); if (msglen < 0) { if (errno == EINTR || errno == EAGAIN) goto again; fprintf(stderr, "Error: Netlink receive error %s (%d)\n", strerror(errno), errno); return EXIT_FAILURE; } if (msglen == 0) { fprintf(stderr, "Error: Unexpected EOF on netlink\n"); return EXIT_FAILURE; } while(NLMSG_OK(h, msglen)) { if (h->nlmsg_flags & NLM_F_DUMP_INTR) fprintf(stderr, "Error: Dump interrupted\n"); if (h->nlmsg_type == NLMSG_DONE) { found_done = 1; break; } if (h->nlmsg_type == NLMSG_ERROR) { if (h->nlmsg_len < NLMSG_LENGTH(sizeof(struct nlmsgerr))) { fprintf(stderr, "Error: Incomplete message\n"); } else { perror("RTNETLINK answers"); } return EXIT_FAILURE; } (*handler)(h); h = NLMSG_NEXT(h, msglen); } if (msg.msg_flags & MSG_TRUNC) { fprintf(stderr, "Error: Message truncated\n"); goto again; } if (!found_done) { h = (struct nlmsghdr *)buf; goto again; } return EXIT_SUCCESS; } void parse_rtattr(struct rtattr *tb[], int max, struct rtattr *rta, int len) { unsigned short type; memset(tb, 0, sizeof(struct rtattr *) * (max + 1)); while (RTA_OK(rta, len)) { type = rta->rta_type; if ((type <= max) && (!tb[type])) tb[type] = rta; rta = RTA_NEXT(rta,len); } if (len) fprintf(stderr, "Error: Deficit %d, rta_len=%d\n", len, rta->rta_len); } int sockdiag_send(int fd, unsigned char cmd) { struct sockaddr_nl nladdr = { .nl_family = AF_NETLINK }; DIAG_REQUEST(req, struct smc_diag_req r, MAGIC_SEQ); struct msghdr msg; struct iovec iov[1]; int iovlen = 1; memset(&req.r, 0, sizeof(req.r)); req.r.diag_family = PF_SMC; iov[0] = (struct iovec) { .iov_base = &req, .iov_len = sizeof(req) }; msg = (struct msghdr) { .msg_name = (void *)&nladdr, .msg_namelen = sizeof(nladdr), .msg_iov = iov, .msg_iovlen = iovlen, }; req.r.diag_ext = cmd; if (sendmsg(fd, &msg, 0) < 0) { close(fd); return EXIT_FAILURE; } return 0; } /* Operations on generic netlink sockets */ int gen_nl_open(char *pname) { int rc = EXIT_FAILURE; /* Allocate a netlink socket and connect to it */ sk = nl_socket_alloc(); if (!sk) { nl_perror(NLE_NOMEM, "Error"); return rc; } rc = genl_connect(sk); if (rc) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err1; } smc_id = genl_ctrl_resolve(sk, SMC_GENL_FAMILY_NAME); if (smc_id < 0) { rc = EXIT_FAILURE; if (smc_id == -NLE_OBJ_NOTFOUND) fprintf(stderr, "Error: SMC module not loaded\n"); else nl_perror(smc_id, "Error"); goto err2; } return EXIT_SUCCESS; err2: nl_close(sk); err1: nl_socket_free(sk); return rc; } int gen_nl_handle(int cmd, int nlmsg_flags, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg) { struct nl_msg *msg; int rc; nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, arg); /* Allocate a netlink message and set header information. */ msg = nlmsg_alloc(); if (!msg) { nl_perror(NLE_NOMEM, "Error"); goto errout; } if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, cmd, SMC_GENL_FAMILY_VERSION)) { nl_perror(NLE_NOMEM, "Error"); goto errout; } /* Send message */ rc = nl_send_auto(sk, msg); if (rc < 0) { nl_perror(rc, "Error"); goto errout; } /* Receive reply message, returns number of cb invocations. */ rc = nl_recvmsgs_default(sk); if (rc < 0) { if (rc == -NLE_OPNOTSUPP) { fprintf(stderr, "Error: Operation not supported by kernel\n"); } else { nl_perror(rc, "Error"); } goto errout; } nlmsg_free(msg); return EXIT_SUCCESS; errout: nlmsg_free(msg); return EXIT_FAILURE; } int gen_nl_handle_dump(int cmd, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg) { return gen_nl_handle(cmd, NLM_F_DUMP, cb_handler, arg); } void gen_nl_close() { if (sk) { nl_close(sk); nl_socket_free(sk); sk = NULL; } } uint64_t nl_attr_get_uint(const struct nlattr *nla) { if (nla && nla_len(nla) == sizeof(uint32_t)) return nla_get_u32(nla); return nla_get_u64(nla); } smc-tools-1.8.4/libnetlink.h000066400000000000000000000041621473052164700157640ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Ursula Braun * Guvenc Gulce * * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef SMC_LIBNETLINK_H_ #define SMC_LIBNETLINK_H_ #include #include #include #include #include #include #include static const struct nla_policy smc_gen_net_policy[SMC_GEN_MAX + 1] = { [SMC_GEN_UNSPEC] = { .type = NLA_UNSPEC, }, [SMC_GEN_STATS] = { .type = NLA_NESTED, }, [SMC_GEN_FBACK_STATS] = { .type = NLA_NESTED, }, }; struct rtnl_handle { int fd; struct sockaddr_nl local; struct sockaddr_nl peer; __u32 seq; __u32 dump; int proto; FILE *dump_fp; int flags; }; #define DIAG_REQUEST(_req, _r, _seq) \ struct { \ struct nlmsghdr nlh; \ _r; \ } _req = { \ .nlh = { \ .nlmsg_type = SOCK_DIAG_BY_FAMILY, \ .nlmsg_flags = NLM_F_ROOT|NLM_F_REQUEST, \ .nlmsg_seq = _seq, \ .nlmsg_len = sizeof(_req), \ }, \ } int rtnl_open(struct rtnl_handle *rth); void rtnl_close(struct rtnl_handle *rth); int rtnl_dump(struct rtnl_handle *rth, void (*handler)(struct nlmsghdr *nlh)); void parse_rtattr(struct rtattr *tb[], int max, struct rtattr *rta, int len); int sockdiag_send(int fd, unsigned char cmd); void set_extension(int ext); int gen_nl_open(); void gen_nl_close(); int gen_nl_handle(int cmd, int nlmsg_flags, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg); int gen_nl_handle_dump(int cmd, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg); uint64_t nl_attr_get_uint(const struct nlattr *nla); #endif /* SMC_LIBNETLINK_H_ */ smc-tools-1.8.4/linkgroup.c000066400000000000000000000476341473052164700156510ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * User space program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "linkgroup.h" #define SMC_MASK_LINK_ID 0xFFFFFF00 #define SMC_INVALID_LINK_ID 0xFFFFFFFF static __u32 unmasked_trgt_lgid = 0; static int netdev_entered = 0; static __u32 target_lgid = 0; static int ibdev_entered = 0; static int type_entered = 0; static int all_entered = 0; static int show_links = 0; #if defined(SMCD) static int lgr_smcr = 0; static int lgr_smcd = 1; #else static int lgr_smcr = 1; static int lgr_smcd = 0; #endif static int d_level = 0; static char target_ibdev[IB_DEVICE_NAME_MAX] = {0}; static char target_type[SMC_TYPE_STR_MAX] = {0}; static char target_ndev[IFNAMSIZ] = {0}; static struct nla_policy smc_gen_lgr_smcr_sock_policy[SMC_NLA_LGR_R_MAX + 1] = { [SMC_NLA_LGR_R_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_LGR_R_ID] = { .type = NLA_U32 }, [SMC_NLA_LGR_R_ROLE] = { .type = NLA_U8 }, [SMC_NLA_LGR_R_TYPE] = { .type = NLA_U8 }, [SMC_NLA_LGR_R_PNETID] = { .type = NLA_NUL_STRING, .maxlen = SMC_MAX_PNETID_LEN + 1 }, [SMC_NLA_LGR_R_VLAN_ID] = { .type = NLA_U8 }, [SMC_NLA_LGR_R_CONNS_NUM] = { .type = NLA_U32 }, }; static struct nla_policy smc_gen_lgr_smcd_sock_policy[SMC_NLA_LGR_D_MAX + 1] = { [SMC_NLA_LGR_D_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_LGR_D_ID] = { .type = NLA_U32 }, [SMC_NLA_LGR_D_PNETID] = { .type = NLA_NUL_STRING, .maxlen = SMC_MAX_PNETID_LEN + 1 }, [SMC_NLA_LGR_D_VLAN_ID] = { .type = NLA_U8 }, [SMC_NLA_LGR_D_CONNS_NUM] = { .type = NLA_U32 }, }; static struct nla_policy smc_gen_link_smcr_sock_policy[SMC_NLA_LINK_MAX + 1] = { [SMC_NLA_LINK_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_LINK_ID] = { .type = NLA_U8 }, [SMC_NLA_LINK_IB_DEV] = { .type = NLA_NUL_STRING, .maxlen = IB_DEVICE_NAME_MAX + 1 }, [SMC_NLA_LINK_IB_PORT] = { .type = NLA_U8 }, [SMC_NLA_LINK_GID] = { .type = NLA_NUL_STRING, .maxlen = 40 + 1 }, [SMC_NLA_LINK_PEER_GID] = { .type = NLA_NUL_STRING, .maxlen = 40 + 1 }, [SMC_NLA_LINK_CONN_CNT] = { .type = NLA_U32 }, [SMC_NLA_LINK_NET_DEV] = { .type = NLA_U32}, [SMC_NLA_LINK_UID] = { .type = NLA_U32 }, [SMC_NLA_LINK_PEER_UID] = { .type = NLA_U32 }, [SMC_NLA_LINK_STATE] = { .type = NLA_U32 }, }; static struct nla_policy smc_gen_lgr_v2_sock_policy[SMC_NLA_LGR_V2_MAX + 1] = { [SMC_NLA_LGR_V2_VER] = { .type = NLA_U8 }, [SMC_NLA_LGR_V2_REL] = { .type = NLA_U8 }, [SMC_NLA_LGR_V2_OS] = { .type = NLA_U8 }, [SMC_NLA_LGR_V2_NEG_EID] = { .type = NLA_NUL_STRING, .maxlen = SMC_MAX_EID_LEN + 1 }, [SMC_NLA_LGR_V2_PEER_HOST] = { .type = NLA_NUL_STRING, .maxlen = SMC_MAX_HOSTNAME_LEN + 1 }, }; static struct nla_policy smc_gen_lgr_r_v2_sock_policy[SMC_NLA_LGR_R_V2_MAX + 1] = { [SMC_NLA_LGR_R_V2_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_LGR_R_V2_DIRECT] = { .type = NLA_U8 }, }; static void usage(void) { fprintf(stderr, #if defined(SMCD) "Usage: smcd linkgroup [show] [all | LG-ID]\n" #elif defined(SMCR) "Usage: smcr linkgroup [show | link-show] [all | LG-ID]\n" " [ibdev ]\n" " [netdev ]\n" #else "Usage: smc linkgroup [show | link-show] [all | LG-ID] [type {smcd | smcr}]\n" " [ibdev ]\n" " [netdev ]\n" #endif ); exit(-1); } static void print_lgr_smcr_header(void) { if (!show_links && d_level >= SMC_DETAIL_LEVEL_V) return; printf("LG-ID "); printf("LG-Role "); printf("LG-Type "); if (show_links) { printf("Net-Dev "); printf("Link-State "); printf("#Conns "); if (d_level >= SMC_DETAIL_LEVEL_V) { printf("Link-UID "); printf("Peer-UID "); printf("IB-Dev "); printf("IB-P "); if (d_level >= SMC_DETAIL_LEVEL_VV) { printf("Local-GID "); printf("Peer-GID "); } } } else { printf("VLAN "); printf("#Conns "); printf("PNET-ID "); } printf("\n"); } static void print_lgr_smcd_header(void) { if (d_level >= SMC_DETAIL_LEVEL_V) return; printf("LG-ID "); printf("VLAN "); printf("#Conns "); printf("PNET-ID " ); printf("\n"); } static const char *smc_peer_os(unsigned int x) { static char buf[32]; switch (x) { case 1: return "ZOS"; case 2: return "LINUX"; case 3: return "AIX"; default: sprintf(buf, "Unknown (%#x)", x); return buf; } } static const char *smc_link_state(unsigned int x) { static char buf[16]; switch (x) { case 0: return "LINK_UNUSED"; case 1: return "LINK_INACTIVE"; case 2: return "LINK_ACTIVATING"; case 3: return "LINK_ACTIVE"; default: sprintf(buf, "%#x?", x); return buf; } } static const char *smc_lgr_type(unsigned int x) { static char buf[16]; switch (x) { case 0: return "NONE"; case 1: return "SINGLE"; case 2: return "SYM"; case 3: return "ASYMP"; case 4: return "ASYML"; default: sprintf(buf, "%#x?", x); return buf; } } static int filter_smcd_item(struct smcd_diag_dmbinfo_v2 *lgr) { int ignore = 0; if (unmasked_trgt_lgid == 0 ) return ignore; /* No filter set */ else if (unmasked_trgt_lgid != lgr->v1.linkid) ignore = 1; return ignore; } static int filter_smcr_item(struct smc_diag_linkinfo_v2 *link, struct smc_diag_lgr *lgr) { int ignore = 0; if (is_str_empty(target_ibdev) && is_str_empty(target_ndev) && (unmasked_trgt_lgid == 0 )) { return ignore; /* No filter set */ }else if (!is_str_empty(target_ndev) && show_links) { if (strncmp(target_ndev, (char*)link->netdev, sizeof(target_ndev)) == 0) ignore = 0; else ignore = 1; } else if (!is_str_empty(target_ibdev) && show_links) { if (strncmp(target_ibdev, (char*)link->v1.ibname, sizeof(target_ibdev)) == 0) ignore = 0; else ignore = 1; } else if (unmasked_trgt_lgid != 0 ) { if (target_lgid == *(__u32*)lgr->lgr_id) ignore = 0; else ignore = 1; } return ignore; } static int fill_link_struct(struct smc_diag_linkinfo_v2 *link, struct nlattr **attrs) { struct nlattr *link_attrs[SMC_NLA_LINK_MAX + 1]; __u32 temp_link_uid; if (nla_parse_nested(link_attrs, SMC_NLA_LINK_MAX, attrs[SMC_GEN_LINK_SMCR], smc_gen_link_smcr_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_link_smcr_sock_policy\n"); return NL_STOP; } if (link_attrs[SMC_NLA_LINK_STATE]) link->link_state = nla_get_u32(link_attrs[SMC_NLA_LINK_STATE]); if (link_attrs[SMC_NLA_LINK_CONN_CNT]) link->conn_cnt = nla_get_u32(link_attrs[SMC_NLA_LINK_CONN_CNT]); if (link_attrs[SMC_NLA_LINK_UID]) { temp_link_uid = nla_get_u32(link_attrs[SMC_NLA_LINK_UID]); memcpy(&link->link_uid[0], &temp_link_uid, sizeof(temp_link_uid)); } if (link_attrs[SMC_NLA_LINK_IB_PORT]) link->v1.ibport = nla_get_u8(link_attrs[SMC_NLA_LINK_IB_PORT]); if (link_attrs[SMC_NLA_LINK_PEER_UID]) { temp_link_uid = nla_get_u32(link_attrs[SMC_NLA_LINK_PEER_UID]); memcpy(&link->peer_link_uid[0], &temp_link_uid, sizeof(temp_link_uid)); } if (link_attrs[SMC_NLA_LINK_NET_DEV] && nla_get_u32(link_attrs[SMC_NLA_LINK_NET_DEV])) if_indextoname(nla_get_u32(link_attrs[SMC_NLA_LINK_NET_DEV]), (char*)link->netdev); if (link_attrs[SMC_NLA_LINK_IB_DEV]) snprintf((char*)link->v1.ibname, sizeof(link->v1.ibname), "%s", nla_get_string(link_attrs[SMC_NLA_LINK_IB_DEV])); if (link_attrs[SMC_NLA_LINK_GID]) snprintf((char*)link->v1.gid, sizeof(link->v1.gid), "%s", nla_get_string(link_attrs[SMC_NLA_LINK_GID])); if (link_attrs[SMC_NLA_LINK_PEER_GID]) snprintf((char*)link->v1.peer_gid, sizeof(link->v1.peer_gid), "%s", nla_get_string(link_attrs[SMC_NLA_LINK_PEER_GID])); return NL_OK; } static void fill_lgr_v2_common_struct(struct smc_v2_lgr_info *v2_lgr_info, struct nlattr **v2_lgr_attrs) { if (v2_lgr_attrs[SMC_NLA_LGR_V2_VER]) v2_lgr_info->smc_version = nla_get_u8(v2_lgr_attrs[SMC_NLA_LGR_V2_VER]); if (v2_lgr_attrs[SMC_NLA_LGR_V2_REL]) v2_lgr_info->peer_smc_release = nla_get_u8(v2_lgr_attrs[SMC_NLA_LGR_V2_REL]); if (v2_lgr_attrs[SMC_NLA_LGR_V2_OS]) v2_lgr_info->peer_os = nla_get_u8(v2_lgr_attrs[SMC_NLA_LGR_V2_OS]); if (v2_lgr_attrs[SMC_NLA_LGR_V2_NEG_EID]) snprintf((char*)v2_lgr_info->negotiated_eid, sizeof(v2_lgr_info->negotiated_eid), "%s", nla_get_string(v2_lgr_attrs[SMC_NLA_LGR_V2_NEG_EID])); if (v2_lgr_attrs[SMC_NLA_LGR_V2_PEER_HOST]) snprintf((char*)v2_lgr_info->peer_hostname, sizeof(v2_lgr_info->peer_hostname), "%s", nla_get_string(v2_lgr_attrs[SMC_NLA_LGR_V2_PEER_HOST])); v2_lgr_info->v2_lgr_info_received = 1; } static int fill_lgr_struct(struct smc_diag_lgr *lgr, struct nlattr **attrs) { struct nlattr *lgr_attrs[SMC_NLA_LGR_R_MAX + 1]; if (nla_parse_nested(lgr_attrs, SMC_NLA_LGR_R_MAX, attrs[SMC_GEN_LGR_SMCR], smc_gen_lgr_smcr_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_lgr_smcr_sock_policy\n"); return NL_STOP; } if (lgr_attrs[SMC_NLA_LGR_R_ID]) *(__u32*)lgr->lgr_id = nla_get_u32(lgr_attrs[SMC_NLA_LGR_R_ID]); if (lgr_attrs[SMC_NLA_LGR_R_ROLE]) lgr->lgr_role = nla_get_u8(lgr_attrs[SMC_NLA_LGR_R_ROLE]); if (lgr_attrs[SMC_NLA_LGR_R_TYPE]) lgr->lgr_type = nla_get_u8(lgr_attrs[SMC_NLA_LGR_R_TYPE]); if (lgr_attrs[SMC_NLA_LGR_R_VLAN_ID]) lgr->vlan_id = nla_get_u8(lgr_attrs[SMC_NLA_LGR_R_VLAN_ID]); if (lgr_attrs[SMC_NLA_LGR_R_CONNS_NUM]) lgr->conns_num = nla_get_u32(lgr_attrs[SMC_NLA_LGR_R_CONNS_NUM]); if (lgr_attrs[SMC_NLA_LGR_R_PNETID]) snprintf((char*)lgr->pnet_id, sizeof(lgr->pnet_id), "%s", nla_get_string(lgr_attrs[SMC_NLA_LGR_R_PNETID])); if (lgr_attrs[SMC_NLA_LGR_R_SNDBUF_ALLOC]) lgr->sndbuf_alloc = nl_attr_get_uint(lgr_attrs[SMC_NLA_LGR_R_SNDBUF_ALLOC]); if (lgr_attrs[SMC_NLA_LGR_R_RMB_ALLOC]) lgr->rmb_alloc = nl_attr_get_uint(lgr_attrs[SMC_NLA_LGR_R_RMB_ALLOC]); if (lgr_attrs[SMC_NLA_LGR_R_V2_COMMON]) { struct nlattr *v2_lgr_attrs[SMC_NLA_LGR_V2_MAX + 1]; if (nla_parse_nested(v2_lgr_attrs, SMC_NLA_LGR_V2_MAX, lgr_attrs[SMC_NLA_LGR_R_V2_COMMON], smc_gen_lgr_v2_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_lgr_v2_sock_policy\n"); return NL_STOP; } fill_lgr_v2_common_struct(&lgr->v2_lgr_info, v2_lgr_attrs); } if (lgr_attrs[SMC_NLA_LGR_R_V2]) { struct nlattr *v2_lgr_attrs[SMC_NLA_LGR_R_V2_MAX + 1]; if (nla_parse_nested(v2_lgr_attrs, SMC_NLA_LGR_R_V2_MAX, lgr_attrs[SMC_NLA_LGR_R_V2], smc_gen_lgr_r_v2_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_lgr_r_v2_sock_policy\n"); return NL_STOP; } if (v2_lgr_attrs[SMC_NLA_LGR_R_V2_DIRECT]) lgr->v2_lgr_info.smcr_direct = nla_get_u8(v2_lgr_attrs[SMC_NLA_LGR_R_V2_DIRECT]); } return NL_OK; } static int fill_lgr_smcd_struct(struct smcd_diag_dmbinfo_v2 *lgr, struct nlattr **attrs) { struct nlattr *lgr_attrs[SMC_NLA_LGR_D_MAX + 1]; if (nla_parse_nested(lgr_attrs, SMC_NLA_LGR_D_MAX, attrs[SMC_GEN_LGR_SMCD], smc_gen_lgr_smcd_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_lgr_smcd_sock_policy\n"); return NL_STOP; } if (lgr_attrs[SMC_NLA_LGR_D_ID]) lgr->v1.linkid = nla_get_u32(lgr_attrs[SMC_NLA_LGR_D_ID]); if (lgr_attrs[SMC_NLA_LGR_D_VLAN_ID]) lgr->vlan_id = nla_get_u8(lgr_attrs[SMC_NLA_LGR_D_VLAN_ID]); if (lgr_attrs[SMC_NLA_LGR_D_CONNS_NUM]) lgr->conns_num = nla_get_u32(lgr_attrs[SMC_NLA_LGR_D_CONNS_NUM]); if (lgr_attrs[SMC_NLA_LGR_D_PNETID]) snprintf((char*)lgr->pnet_id, sizeof(lgr->pnet_id), "%s", nla_get_string(lgr_attrs[SMC_NLA_LGR_D_PNETID])); if (lgr_attrs[SMC_NLA_LGR_D_SNDBUF_ALLOC]) lgr->sndbuf_alloc = nl_attr_get_uint(lgr_attrs[SMC_NLA_LGR_D_SNDBUF_ALLOC]); if (lgr_attrs[SMC_NLA_LGR_D_DMB_ALLOC]) lgr->dmb_alloc = nl_attr_get_uint(lgr_attrs[SMC_NLA_LGR_D_DMB_ALLOC]); if (lgr_attrs[SMC_NLA_LGR_D_V2_COMMON]) { struct nlattr *v2_lgr_attrs[SMC_NLA_LGR_V2_MAX + 1]; if (nla_parse_nested(v2_lgr_attrs, SMC_NLA_LGR_V2_MAX, lgr_attrs[SMC_NLA_LGR_D_V2_COMMON], smc_gen_lgr_v2_sock_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_lgr_v2_sock_policy\n"); return NL_STOP; } fill_lgr_v2_common_struct(&lgr->v2_lgr_info, v2_lgr_attrs); } return NL_OK; } static int show_lgr_smcr_info_lgr_details_first_loop = 1; static void show_lgr_smcr_info_lgr_details(struct smc_diag_lgr *lgr) { if (!show_lgr_smcr_info_lgr_details_first_loop) printf("\n"); /* separator line between link groups */ else show_lgr_smcr_info_lgr_details_first_loop = 0; printf("LG-ID : %08x\n", *(__u32*)lgr->lgr_id); printf("LG-Role : %s\n", lgr->lgr_role ? "SERV" : "CLNT"); printf("LG-Type : %s\n", smc_lgr_type(lgr->lgr_type)); printf("VLAN : %x\n", lgr->vlan_id); printf("PNET-ID : %s\n", trim_space((char *)lgr->pnet_id)); printf("Version : %d\n", (lgr->v2_lgr_info.v2_lgr_info_received ? lgr->v2_lgr_info.smc_version : 1)); if (lgr->v2_lgr_info.v2_lgr_info_received) { printf("Peer-Rel : %d\n", lgr->v2_lgr_info.peer_smc_release); printf("Peer-Host: %s\n", trim_space((char *)lgr->v2_lgr_info.peer_hostname)); printf("Peer-OS : %s\n", smc_peer_os(lgr->v2_lgr_info.peer_os)); printf("Direct : %s\n", (lgr->v2_lgr_info.smcr_direct ? "Yes" : "No")); printf("EID : %s\n", lgr->v2_lgr_info.negotiated_eid); } printf("#Conns : %d\n", lgr->conns_num); printf("Sndbuf : %lld B\n", lgr->sndbuf_alloc); printf("RMB : %lld B\n", lgr->rmb_alloc); } static int show_lgr_smcr_info(struct nlattr **attr) { struct smc_diag_linkinfo_v2 link = {0}; static struct smc_diag_lgr lgr = {0}; int rc = NL_OK; if (attr[SMC_GEN_LGR_SMCR]) rc = fill_lgr_struct(&lgr, attr); if (attr[SMC_GEN_LINK_SMCR]) rc = fill_link_struct(&link, attr); if (attr[SMC_GEN_LGR_SMCR] && show_links) return rc; if (filter_smcr_item(&link, &lgr)) return rc; if (!show_links && d_level >= SMC_DETAIL_LEVEL_V) { show_lgr_smcr_info_lgr_details(&lgr); /* clear the (static) lgr struct for next lgr */ memset(&lgr, 0, sizeof(lgr)); return rc; } printf("%08x ", *(__u32*)lgr.lgr_id); printf("%-8s ", lgr.lgr_role ? "SERV" : "CLNT"); printf("%-8s ", smc_lgr_type(lgr.lgr_type)); if (show_links) { if (strnlen((char*)link.netdev, sizeof(link.netdev)) > (IFNAMSIZ - 1)) printf("%-.15s ", link.netdev); else printf("%-15s ", link.netdev); printf("%-15s ", smc_link_state(link.link_state)); printf("%6d ", link.conn_cnt); if (d_level >= SMC_DETAIL_LEVEL_V) { printf("%08x ", ntohl(*(__u32*)link.link_uid)); printf("%08x ", ntohl(*(__u32*)link.peer_link_uid)); if (strnlen((char*)link.v1.ibname, sizeof(link.v1.ibname)) > SMC_MAX_IBNAME) printf("%-.8s ", link.v1.ibname); else printf("%-8s ", link.v1.ibname); printf("%4d ", link.v1.ibport); if (d_level >= SMC_DETAIL_LEVEL_VV) { printf("%-40s ", link.v1.gid); printf("%s ", link.v1.peer_gid); } } } else { printf("%#4x ", lgr.vlan_id); printf(" %6d ", lgr.conns_num); printf(" %-16s ", trim_space((char *)lgr.pnet_id)); } printf("\n"); return rc; } static int show_lgr_smcd_info_lgr_details_first_loop = 1; static void show_lgr_smcd_info_lgr_details(struct smcd_diag_dmbinfo_v2 *lgr) { if (!show_lgr_smcd_info_lgr_details_first_loop) printf("\n"); /* separator line between link groups */ else show_lgr_smcd_info_lgr_details_first_loop = 0; printf("LG-ID : %08x\n", lgr->v1.linkid); printf("VLAN : %x\n", lgr->vlan_id); printf("PNET-ID : %s\n", trim_space((char *)lgr->pnet_id)); printf("Version : %d\n", (lgr->v2_lgr_info.v2_lgr_info_received ? lgr->v2_lgr_info.smc_version : 1)); if (lgr->v2_lgr_info.v2_lgr_info_received) { printf("Peer-Rel : %d\n", lgr->v2_lgr_info.peer_smc_release); printf("Peer-Host: %s\n", trim_space((char *)lgr->v2_lgr_info.peer_hostname)); printf("Peer-OS : %s\n", smc_peer_os(lgr->v2_lgr_info.peer_os)); printf("EID : %s\n", lgr->v2_lgr_info.negotiated_eid); } printf("#Conns : %d\n", lgr->conns_num); printf("Sndbuf : %lld B\n", lgr->sndbuf_alloc); printf("DMB : %lld B\n", lgr->dmb_alloc); } static int show_lgr_smcd_info(struct nlattr **attr) { struct smcd_diag_dmbinfo_v2 lgr = {0}; int rc = NL_OK; if (attr[SMC_GEN_LGR_SMCD]) rc = fill_lgr_smcd_struct(&lgr, attr); if(filter_smcd_item(&lgr)) return rc; if (d_level >= SMC_DETAIL_LEVEL_V) { show_lgr_smcd_info_lgr_details(&lgr); return rc; } printf("%08x ", lgr.v1.linkid); printf("%#4x ", lgr.vlan_id); printf("%6d ", lgr.conns_num); printf("%-16s ", trim_space((char *)lgr.pnet_id)); printf("\n"); return rc; } static int handle_gen_lgr_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); static int header_printed = 0; int rc = NL_OK; if (!header_printed) { if (lgr_smcd) print_lgr_smcd_header(); else print_lgr_smcr_header(); header_printed = 1; } if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_LGR_SMCR] && !attrs[SMC_GEN_LINK_SMCR] && !attrs[SMC_GEN_LGR_SMCD]) return NL_STOP; if (lgr_smcr && (attrs[SMC_GEN_LGR_SMCR] || attrs[SMC_GEN_LINK_SMCR])) rc = show_lgr_smcr_info(&attrs[0]); if (lgr_smcd && attrs[SMC_GEN_LGR_SMCD]) rc = show_lgr_smcd_info(&attrs[0]); return rc; } static void handle_cmd_params(int argc, char **argv) { if (((argc == 1) && (contains(argv[0], "help") == 0)) || (argc > 4)) usage(); if (argc > 0) { if (contains(argv[0], "show") == 0) show_links=0; else if (contains(argv[0], "link-show") == 0) show_links=1; else PREV_ARG(); /* no object given, so use the default "show" */ } while (NEXT_ARG_OK()) { NEXT_ARG(); if (ibdev_entered) { snprintf(target_ibdev, sizeof(target_ibdev), "%s", argv[0]); ibdev_entered = 0; break; } else if (netdev_entered) { snprintf(target_ndev, sizeof(target_ndev), "%s", argv[0]); netdev_entered = 0; break; } else if (type_entered) { snprintf(target_type, sizeof(target_type), "%s", argv[0]); if (strncmp(target_type, "smcd", SMC_TYPE_STR_MAX) == 0) { lgr_smcd = 1; lgr_smcr = 0; } else if ((strnlen(target_type, sizeof(target_type)) < 4) || (strncmp(target_type, "smcr", SMC_TYPE_STR_MAX) != 0)) { print_type_error(); } type_entered = 0; break; } else if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "all") == 0) { all_entered=1; #if !defined(SMCD) && !defined(SMCR) } else if (contains(argv[0], "type") == 0) { type_entered=1; #endif #if !defined(SMCD) } else if (contains(argv[0], "ibdev") == 0) { ibdev_entered =1; } else if (contains(argv[0], "netdev") == 0) { netdev_entered =1; #endif } else if (!all_entered){ char *endptr = NULL; unmasked_trgt_lgid = (unsigned int)strtol(argv[0], &endptr, 16); if (argv[0] == endptr) /* string doesn't contain any digits */ unmasked_trgt_lgid = SMC_INVALID_LINK_ID; else target_lgid = (unmasked_trgt_lgid & SMC_MASK_LINK_ID); break; } else { usage(); } } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); } int invoke_lgs(int argc, char **argv, int detail_level) { int rc = EXIT_SUCCESS; d_level = detail_level; handle_cmd_params(argc, argv); if (lgr_smcd) rc = gen_nl_handle_dump(SMC_NETLINK_GET_LGR_SMCD, handle_gen_lgr_reply, NULL); else if (show_links) rc = gen_nl_handle_dump(SMC_NETLINK_GET_LINK_SMCR, handle_gen_lgr_reply, NULL); else rc = gen_nl_handle_dump(SMC_NETLINK_GET_LGR_SMCR, handle_gen_lgr_reply, NULL); return rc; } smc-tools-1.8.4/linkgroup.h000066400000000000000000000011311473052164700156340ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * User space program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef LINKGROUP_H_ #define LINKGROUP_H_ extern struct rtnl_handle rth; int invoke_lgs(int argc, char **argv, int detail_level); #endif /* LINKGROUP_H_ */ smc-tools-1.8.4/seid.c000066400000000000000000000123761473052164700145560ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "seid.h" static int enable_cmd = 0; static int disable_cmd = 0; static int show_cmd = 0; extern int smc_id; extern struct nl_sock *sk; const struct nla_policy smc_gen_seid_policy[SMC_NLA_SEID_TABLE_MAX + 1] = { [SMC_NLA_SEID_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_SEID_ENTRY] = { .type = NLA_NUL_STRING }, [SMC_NLA_SEID_ENABLED] = { .type = NLA_U8 }, }; static void usage(void) { fprintf(stderr, "Usage: smcd seid [show]\n" " smcd seid enable\n" " smcd seid disable\n" ); exit(-1); } /* arg is an (int *) */ static int is_seid_defined_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); int *is_seid = (int *)arg; if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_SEID_TABLE_MAX, (struct nla_policy *)smc_gen_seid_policy) < 0) { fprintf(stderr, "Error: Invalid data returned: smc_gen_seid_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_NLA_SEID_ENTRY]) *is_seid = 0; else *is_seid = 1; return NL_OK; } static int is_seid_defined(int *is_seid) { *is_seid = 0; return gen_nl_handle_dump(SMC_NETLINK_DUMP_SEID, is_seid_defined_reply, is_seid); } static int handle_gen_seid_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_NLA_SEID_TABLE_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); int rc = NL_OK; char *state; if (!show_cmd) return rc; if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_SEID_TABLE_MAX, (struct nla_policy *)smc_gen_seid_policy) < 0) { fprintf(stderr, "Error: invalid data returned: smc_gen_seid_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_NLA_SEID_ENTRY] || !attrs[SMC_NLA_SEID_ENABLED]) { printf("n/a\n"); return NL_STOP; } if (nla_get_u8(attrs[SMC_NLA_SEID_ENABLED])) state = "[enabled]"; else state = "[disabled]"; printf("%s %s\n", nla_get_string(attrs[SMC_NLA_SEID_ENTRY]), state); return rc; } int gen_nl_seid_handle(int cmd, char dump, int (*cb_handler)(struct nl_msg *msg, void *arg)) { int rc = EXIT_FAILURE, nlmsg_flags = 0; struct nl_msg *msg; nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); /* Allocate a netlink message and set header information. */ msg = nlmsg_alloc(); if (!msg) { nl_perror(NLE_NOMEM, "Error"); rc = EXIT_FAILURE; goto err; } if (dump) nlmsg_flags = NLM_F_DUMP; if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, cmd, SMC_GENL_FAMILY_VERSION)) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err; } /* Send message */ rc = nl_send_auto(sk, msg); if (rc < 0) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err; } /* Receive reply message, returns number of cb invocations. */ rc = nl_recvmsgs_default(sk); if (rc < 0) { /* For cmd "SEID disable" the kernel might return ENOENT when * no UEID is defined and the SEID cannot be disabled. * This is mapped to NLE_OBJ_NOTFOUND in libnl, lib/error.c. */ if (rc == -NLE_OPNOTSUPP) { fprintf(stderr, "Error: operation not supported by kernel\n"); } else if (cmd == SMC_NETLINK_DISABLE_SEID && rc == -NLE_OBJ_NOTFOUND) { fprintf(stderr, "Error: System EID cannot be disabled because no User EID is defined\n"); } else { nl_perror(rc, "Error"); } rc = EXIT_FAILURE; goto err; } nlmsg_free(msg); return EXIT_SUCCESS; err: nlmsg_free(msg); return rc; } static void handle_cmd_params(int argc, char **argv) { if (argc == 0) { show_cmd = 1; /* no object given, so use the default "show" */ return; } while (1) { if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "enable") == 0) { enable_cmd = 1; } else if (contains(argv[0], "disable") == 0) { disable_cmd = 1; } else if (contains(argv[0], "show") == 0) { show_cmd = 1; break; } else { usage(); } if (!NEXT_ARG_OK()) break; NEXT_ARG(); } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); /* Only single cmd expected */ if ((enable_cmd + disable_cmd + show_cmd) != 1) usage(); } int invoke_seid(int argc, char **argv, int detail_level) { int rc = EXIT_SUCCESS; handle_cmd_params(argc, argv); if (enable_cmd || disable_cmd) { int is_seid = 0; is_seid_defined(&is_seid); if (!is_seid) { printf("Error: System EID not available\n"); return EXIT_FAILURE; } } if (enable_cmd) { rc = gen_nl_seid_handle(SMC_NETLINK_ENABLE_SEID, 0, handle_gen_seid_reply); } else if (disable_cmd) { rc = gen_nl_seid_handle(SMC_NETLINK_DISABLE_SEID, 0, handle_gen_seid_reply); } else if (show_cmd) { rc = gen_nl_seid_handle(SMC_NETLINK_DUMP_SEID, 1, handle_gen_seid_reply); } else { printf("Error: unknown command\n"); /* we should never come here ... */ } return rc; } smc-tools-1.8.4/seid.h000066400000000000000000000010261473052164700145510ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef SEID_H_ #define SEID_H_ extern struct rtnl_handle rth; int invoke_seid(int argc, char **argv, int detail_level); #endif /* SEID_H_ */ smc-tools-1.8.4/smc-device.8000066400000000000000000000060361473052164700155720ustar00rootroot00000000000000.\" smc-device.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMC-DEVICE 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smc-device \- Print information about SMC devices .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smc .RI "[ " OPTIONS " ]" .B device .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smc device" .RI "[" .B "show" .RI "] [" .B "all" .RI "] [ " .B type .IR TYPE " ] [ " .B netdev .IR NETDEV " ] [ " .B ibdev .IR IBDEV " ] .ti -8 .IR TYPE " := [ " .BR smcr " | " .BR smcd " ]" .SH "DESCRIPTION" The .B smc device command displays SMC devices and their properties. Devices can be listed for SMC-R and SMC-D. The command operates with SMC-R type per default on all command levels. .SS smc device show - look at the device properties .TP .B all (default) Show all the devices. .TP .BI type " TYPE" List only the devices of the given type. .TP .BI netdev " NETDEV" List only the device with the given network device name. .TP .BI ibdev " IBDEV" List only the device ports with the given RoCE (InfiniBand) device name. .SH OUTPUT .SS "Net-Dev" Network device name. .SS "IB-Dev" RoCE (InfiniBand) device name. .SS "IB-P" InfiniBand port of the RoCE device. .SS "IB-State" State of the RoCE device port. .TP .I INACTIVE The RoCE device port is inactive. .TP .I ACTIVE The RoCE device port is active. .SS "Type" Type of the underlying PCI device. .TP .I RoCE_Express Underlying used device is RoCE Express. .TP .I RoCE_Express2 Underlying used device is RoCE Express 2. .TP .I RoCE_Express3 Underlying used device is RoCE Express 3. .TP .I ISM Underlying used device is ISM. .SS "Crit" Show whether the device is critical i.e. without failover possibility. .TP .I Yes In case of SMC-R, there is at least one linkgroup running on the device with state "SINGLE" or locally "ASYMMETRIC" which means the linkgroups do not have any fail-over device in case of a failure. In case of SMC-D, there is at least one linkgroup running on the ISM device. .TP .I No In case of SMC-R, there is no linkgroup running on the device with state "SINGLE" or locally "ASYMMETRIC" which means the linkgroup(s) have a fallback device in case of a failure. In case of SMC-D, there is no linkgroup running on the ISM device. .SS "FID" Functional ID of the PCI device. .SS "PCI-ID" ID of the PCI device. .SS "PCHID" Physical channel ID of the PCI device. .SS "#Links" Number of links(SMC-R)/linkgroups(SMC-D) on the device. .SS "PNET-ID" PNET-ID of the device. "*" means PNET-ID is set by the user. .SH "EXAMPLES" .br 1. Show all devices of the type smcd: .br \fB# smc device show all type smcd\fP .br 2. Show all devices of the type smcr: .br \fB# smc devices show all type smcr\fP .br 3. Shows all devices with RoCE (InfiniBand) device name "mlx4_0": .br \fB# smc device show ibdev mlx4_0\fP .br 4. Shows all devices with network device name "eth0": .br \fB# smc device show netdev eth0\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smc-linkgroup.8000066400000000000000000000104411473052164700163400ustar00rootroot00000000000000.\" smc-linkgroup.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMC-LINKGROUP 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smc-linkgroup \- Print information about SMC linkgroups and links .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smc .RI "[ " OPTIONS " ]" .B linkgroup .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smc linkgroup" " { " show " | " link-show " } [" .B "all " .RI "| " LG-ID " ] [ " .B type .IR TYPE " ] [ " .B netdev .IR NETDEV " ] [ " .B ibdev .IR IBDEV " ] .ti -8 .IR TYPE " := [ " .BR smcr " | " .BR smcd " ]" .SH "DESCRIPTION" The .B smc linkgroup command displays linkgroups, links and their properties. Links can be listed only for SMC-R linkgroups. The command operates with type smcr per default on all command levels. .SS smc linkgroup show - look at the linkgroup properties .TP .B all (default) Show all the linkgroups. .TP .I LG-ID Show the linkgroup with the id .I LG-ID .TP .BI type " TYPE" List only linkgroups of the given type. .TP .BI netdev " NETDEV" List linkgroups making use of the given network device only. .TP .BI ibdev " IBDEV" List only linkgroups of the given RoCE (infiniband) device. .SS smc linkgroup link-show - look at the link properties (SMC-R only) .TP .B all (default) Show all the links of the linkgroups. .TP .I LG-ID Show links of the linkgroup with the id .I LG-ID .TP .BI type " TYPE" List only links of the linkgroups of the given type. .TP .BI netdev " NETDEV" List only links of the linkgroups making use of the given network device. .TP .BI ibdev " IBDEV" List only links of the linkgroups of the given RoCE (InfiniBand) device. .SH OUTPUT .SS "LG-ID" ID of the linkgroup. .SS "LG-Role" Role of the linkgroup. .TP .I SERV The linkgroup has a SERVER role. .TP .I CLNT The linkgroup has a CLIENT role. .SS "LG-Type" Linkgroup type of the linkgroup. .TP .I NONE The linkgroup has the initial type. .TP .I SINGLE The linkgroup has only a single link, i.e. the local and the peer system can offer one device port only for this linkgroup, which means a link outage on any side cannot be covered. .TP .I SYM The linkgroup has two symmetric links, i.e. the local and the peer system can offer two device ports for this linkgroup, which means a link outage on any side can be covered. .TP .I ASYMP The linkgroup has asymmetric links, i.e. the peer system can offer one device port only for this linkgroup, which means a link outage on the peer side cannot be covered. .TP .I ASYML The linkgroup has asymmetric links, i.e. the local system can offer one device port only for this linkgroup, which means a link outage on the local side cannot be covered. .SS "VLAN" VLAN the linkgroup belongs to. .SS "#Conns" Number of connections(sockets) running on the link/linkgroup. .SS "PNET-ID" PNET-ID of the linkgroup. "*" means PNET-ID is set by the user. .SS "Net-Dev" Network device name corresponding to the link. .SS "Link-State" The state of the link. .TP .I LINK_UNUSED The link is not in use and in initial state. .TP .I LINK_INACTIVE The link is inactive and will go away. .TP .I LINK_ACTIVATING The link is being activated with the peer. .TP .I LINK_ACTIVE The link is active and operates on an established link with the peer. Data is being exchanged via RDMA. .SS "Link-UID" Unique identifier of the link. This identifier consists of linkgroup id and link id. .SS "Peer-UID" Unique identifier of the link on peer side. This identifier consists of linkgroup id and link id. .SS "IB-Dev" Name of the RoCE device used by the link. .SS "IB-P" Port of the RoCE device used by the link. .SS "Local-GID" GID of the RoCE port used by the link. .SS "Peer-GID" GID of the peer RoCE port used by the link. .SH "EXAMPLES" .br 1. Show all linkgroups of the type smcd: .br \fB# smc linkgroup show all type smcd\fP .br 2. Show all links of the linkgroups with type smcr: .br \fB# smc linkgroup link-show all type smcr\fP .br 3. Show all links with linkgroup id 40: .br \fB# smc linkgroup link-show 40\fP .br 4. Show all links on RoCE device "mlx4_0": .br \fB# smc linkgroup link-show ibdev mlx4_0\fP .br 5. Shows all links on network device "eth0": .br \fB# smc linkgroup link-show netdev eth0\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smc-preload.c000066400000000000000000000063241473052164700160340ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2016, 2018 * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define DLOPEN_FLAG RTLD_LAZY #ifndef AF_SMC #define AF_SMC 43 #endif #ifndef SMCPROTO_SMC #define SMCPROTO_SMC 0 /* SMC protocol, IPv4 */ #define SMCPROTO_SMC6 1 /* SMC protocol, IPv6 */ #endif int (*orig_socket)(int domain, int type, int protocol) = NULL; static void *dl_handle = NULL; static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; static void initialize(void); static int debug_mode = 0; #define GET_FUNC(x) \ if (dl_handle) { \ char *err; \ dlerror(); \ orig_ ## x=dlsym(dl_handle,#x); \ if ((!orig_ ## x)&&(err=dlerror())) { \ fprintf(stderr, "dlsym failed on " #x ": %s\n",err); \ orig_ ## x=&emergency_ ## x; \ } \ } else { \ orig_ ## x=&emergency_ ## x; \ } static void dbg_msg(FILE *f, const char *format, ...) { va_list vl; if (debug_mode) { va_start(vl, format); vfprintf(f, format, vl); va_end(vl); } } static int emergency_socket(int domain, int type, int protocol) { errno = EINVAL; return -1; } static void set_bufsize(int socket, int opt, const char *envname) { char *val, *end; int size; int rc; val = getenv(envname); if (!val) return; size = strtol(val, &end, 10); if (end != NULL) { switch (toupper(*end)) { case 'K': size *= 1024; break; case 'M': size *= 1048576; break; default: break; } } rc = setsockopt(socket, SOL_SOCKET, opt, &size, sizeof(size)); dbg_msg(stderr, "sockopt %d set to %d\n", opt, size, rc); } int socket(int domain, int type, int protocol) { int rc; if (!orig_socket) initialize(); /* check if socket is eligible for AF_SMC */ if ((domain == AF_INET || domain == AF_INET6) && // see kernel code, include/linux/net.h, SOCK_TYPE_MASK (type & 0xf) == SOCK_STREAM && (protocol == IPPROTO_IP || protocol == IPPROTO_TCP)) { dbg_msg(stderr, "libsmc-preload: map sock to AF_SMC\n"); if (domain == AF_INET) protocol = SMCPROTO_SMC; else /* AF_INET6 */ protocol = SMCPROTO_SMC6; domain = AF_SMC; } rc = (*orig_socket)(domain, type, protocol); if (rc != -1) { set_bufsize(rc, SO_SNDBUF, "SMC_SNDBUF"); set_bufsize(rc, SO_RCVBUF, "SMC_RCVBUF"); } return rc; } static void set_debug_mode(const char *var_name) { char *var_value; var_value = getenv(var_name); debug_mode = 0; if (var_value != NULL) debug_mode = (var_value[0] != '0'); } static void initialize(void) { pthread_mutex_lock(&mutex); if (orig_socket) { pthread_mutex_unlock(&mutex); return; } set_debug_mode("SMC_DEBUG"); dl_handle = dlopen(LIBC_SO, DLOPEN_FLAG); if (!dl_handle) dbg_msg(stderr, "dlopen failed: %s\n", dlerror()); GET_FUNC(socket); pthread_mutex_unlock(&mutex); } smc-tools-1.8.4/smc-tools.autocomplete000066400000000000000000000114421473052164700200220ustar00rootroot00000000000000_smc() { local cur prev opts COMPREPLY=() cur="${COMP_WORDS[COMP_CWORD]}" prev="${COMP_WORDS[COMP_CWORD-1]}" opts="device linkgroup info stats ueid -a -d -dd -v" opts_smcd="device linkgroup info stats ueid seid -a -d -v" opts_short="device linkgroup" opts_show="show link-show" opts_show_smcd="show" opts_stats="show reset" opts_ueid="show add del flush" opts_seid="show enable disable" opts_type="smcd smcr" opts_final="all netdev ibdev" opts_final_smcd="all" case "${prev}" in -v) if [ $1 = "smcr" ]; then COMPREPLY=( $(compgen -W "${opts_short}" -- ${cur})) fi return 0 ;; -vv) if [ $1 = "smcr" ]; then COMPREPLY=( $(compgen -W "${opts_short}" -- ${cur})) fi return 0 ;; device) if [ $1 = "smcr" ]; then COMPREPLY=( $(compgen -W "${opts_show}" -- ${cur})) else COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur})) fi return 0 ;; linkgroup) if [ $1 = "smcr" ]; then COMPREPLY=( $(compgen -W "${opts_show}" -- ${cur}) ) else COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur}) ) fi return 0 ;; info) COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur}) ) return 0 ;; show) if [ $1 = "smcr" ]; then COMPREPLY=( $(compgen -W "${opts_final}" -- ${cur}) ) else COMPREPLY=( $(compgen -W "${opts_final_smcd}" -- ${cur}) ) fi return 0 ;; link-show) COMPREPLY=( $(compgen -W "${opts_final}" -- ${cur}) ) return 0 ;; smcd) COMPREPLY=( $(compgen -W "${opts_smcd}" -- ${cur}) ) return 0 ;; smcr) COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) ) return 0 ;; stats) COMPREPLY=( $(compgen -W "${opts_stats}" -- ${cur}) ) return 0 ;; ueid) COMPREPLY=( $(compgen -W "${opts_ueid}" -- ${cur}) ) return 0 ;; seid) COMPREPLY=( $(compgen -W "${opts_seid}" -- ${cur}) ) return 0 ;; *) ;; esac } function _smc_pnet_complete_() { case "${COMP_WORDS[COMP_CWORD-1]}" in --interface*|-I*) COMPREPLY=($(compgen -W "$(ls -1 /sys/class/net/)" -- "${COMP_WORDS[COMP_CWORD]}")) return;; --ibdevice*|-D*) which smc_rnics >/dev/null if [ $? -eq 0 ]; then COMPREPLY=($(compgen -W "$(smc_rnics | tail -n +3 | awk '{print($3)}' | uniq)" -- "${COMP_WORDS[COMP_CWORD]}")) else COMPREPLY=($(compgen -W "$(ls -1 /sys/bus/pci/devices)" -- "${COMP_WORDS[COMP_CWORD]}")) fi return;; --ibport*|-P*) ;; esac COMPREPLY=($(compgen -W "--help --version --add --delete --show --flush --interface --ibdevice --ibport" -- "${COMP_WORDS[COMP_CWORD]}")) } function _smc_rnics_complete_() { case "${COMP_WORDS[COMP_CWORD-1]}" in --enable|-e) COMPREPLY=($(compgen -W "$(smc_rnics | grep -e "^ [[:space:]0-9a-f]\{2\} 0" | awk '{print($1)}')" -- "${COMP_WORDS[COMP_CWORD]}")) return;; --disable|-d) COMPREPLY=($(compgen -W "$(smc_rnics | grep -e "^ [[:space:]0-9a-f]\{2\} 1" | awk '{print($1)}')" -- "${COMP_WORDS[COMP_CWORD]}")) return;; esac COMPREPLY=($(compgen -W "--help --version --disable --enable --rawids" -- "${COMP_WORDS[COMP_CWORD]}")) } function _smc_chk_complete_() { case "${COMP_WORDS[COMP_CWORD-1]}" in --pnetid|-i) COMPREPLY=($(compgen -W "$(ip link show | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//') $(ip link show up | grep -e "^\s*altname" | awk '{print($2)}')" -- "${COMP_WORDS[COMP_CWORD]}")) return;; --connect|-C|--port|-p) COMPREPLY=() return;; esac COMPREPLY=($(compgen -W "$(ip link show up | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//') $(ip link show up | grep -e "^\s*altname" | awk '{print($2)}') --connect --help --version --debug --pnetid --port --server --static-analysis --live-test --ipv6" -- "${COMP_WORDS[COMP_CWORD]}")) } complete -W "--help --tgz --version" smc_dbg complete -W "--help --version --all --listening --debug --wide --smcd --smcr" smcss complete -F _smc smcd complete -F _smc smcr complete -F _smc_pnet_complete_ smc_pnet complete -F _smc_rnics_complete_ smc_rnics complete -F _smc_chk_complete_ smc_chk smc-tools-1.8.4/smc.8000066400000000000000000000040141473052164700143270ustar00rootroot00000000000000.\" smc.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMC 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smc \- Print information about SMC linkgroups, links, devices .SH SYNOPSIS .B smc .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " .BR help " }" .sp .IR OBJECT " := { " .BR device " | " info " | " linkgroup " | " stats " }" .sp .IR OPTIONS " := { " \fB\-V\fR[\fIersion\fR] | \fB\-a\fR[\fIbsolute\fR] | \fB\-v\fR[\fIerbose\fR] | \fB\-vv\fR[\fIerbose\fR]} .SH OPTIONS .TP .BR "\-V" , " -Version" Print the version of the .B smc utility and exit. .TP .BR "\-a", " \-absolute" Print absolute statistic value (valid only for stats). .TP .BR "\-v", " \-verbose" Print detailed information. .TP .BR "\-vv", " \-vverbose" Print more detailed information. .SH SMC - COMMAND SYNTAX .SS .I OBJECT .B device Device(s) as used by SMC .TP .B info Generic SMC information .TP .B linkgroup Linkgroup(s) or link(s) as used by SMC .TP .B stats SMC statistics .PP The names of all objects can be abbreviated down to a unique stem. For example, .B device can be abbreviated to .B dev or just .B d. Man pages in .B SEE ALSO section contain more information for individual objects. .SS .I COMMAND Specifies the action to perform on the object. The set of possible actions depends on the object type. As a rule, it is possible to .BR " show " or " link-show" objects, but some objects do not allow all of these operations. The .B help command is available for all objects. It prints out a list of available commands and argument syntax conventions. .sp If no command is given, a default command is assumed. .SH RETURN CODES Successful .IR smc commands return 0 and display the requested information. If an error occurs, .IR smc writes a message to stderr and completes with a return code other than 0. .P .SH SEE ALSO .BR af_smc (7), .BR smc-device (8), .BR smc-info (8), .BR smc-linkgroup (8) .BR smc-stats (8) smc-tools-1.8.4/smc.c000066400000000000000000000067161473052164700144150ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include #include #include "smctools_common.h" #include "libnetlink.h" #include "util.h" #include "linkgroup.h" #include "dev.h" #include "ueid.h" #include "seid.h" #include "info.h" #include "stats.h" static int option_detail = 0; #if defined(SMCD) char *myname = "smcd"; #elif defined(SMCR) char *myname = "smcr"; #else char *myname = "smc"; #endif static void version(void) { fprintf(stderr, "%s utility, smc-tools-%s\n", myname, RELEASE_STRING); exit(-1); } static void usage(void) { fprintf(stderr, "Usage: %s [ OPTIONS ] OBJECT {COMMAND | help}\n" #if defined(SMCD) "where OBJECT := {info | linkgroup | device | stats | ueid | seid}\n" " OPTIONS := {-v[ersion] | -d[etails] | -a[bsolute]}\n", myname); #else "where OBJECT := {info | linkgroup | device | stats | ueid}\n" " OPTIONS := {-v[ersion] | -d[etails] | -dd[etails] | -a[bsolute]}\n", myname); #endif } static int invoke_help(int argc, char **argv, int k) { usage(); return 0; } static const struct cmd { const char *cmd; int (*func)(int argc, char **argv, int option_detail); } cmds[] = { { "device", invoke_devs }, { "linkgroup", invoke_lgs }, { "info", invoke_info }, { "stats", invoke_stats }, { "ueid", invoke_ueid }, #if defined(SMCD) { "seid", invoke_seid }, #endif { "help", invoke_help }, { 0 } }; static int run_cmd(const char *argv0, int argc, char **argv) { const struct cmd *c; for (c = cmds; c->cmd; ++c) { if (contains(argv0, c->cmd) == 0) return -(c->func(argc-1, argv+1, option_detail)); } #if defined(SMCR) /* Special warning for those who mixed up smcd and smcr */ if (contains(argv0, "seid") == 0) { fprintf(stderr, "Error: Object \"%s\" is valid for SMC-D only, try \"%s help\".\n", argv0, myname); return EXIT_FAILURE; } #endif fprintf(stderr, "Error: Object \"%s\" is unknown, try \"%s help\".\n", argv0, myname); return EXIT_FAILURE; } int main(int argc, char **argv) { int rc = 0; while (argc > 1) { char *opt = argv[1]; if (strcmp(opt, "--") == 0) { argc--; argv++; break; } if (opt[0] != '-') break; if (opt[1] == '-') opt++; if ((strncmp(opt, "-ad", 3) == 0) || (strncmp(opt, "-da", 3) == 0)) { option_detail = SMC_OPTION_DETAIL_ABS; } else if (contains(opt, "-absolute") == 0) { option_detail = SMC_OPTION_ABS; } else if (contains(opt, "-version") == 0) { version(); } else if (contains(opt, "-details") == 0) { option_detail = SMC_DETAIL_LEVEL_V; } else if (contains(opt, "-ddetails") == 0) { option_detail = SMC_DETAIL_LEVEL_VV; } else if (contains(opt, "-help") == 0) { usage(); goto out; } else { fprintf(stderr, "Error: Option \"%s\" is unknown, try \"%s help\".\n", opt, myname); exit(-1); } argc--; argv++; } if (gen_nl_open(myname)) exit(1); if (argc > 1) { rc = run_cmd(argv[1], argc-1, argv+1); goto out; } usage(); out: gen_nl_close(); return rc; } smc-tools-1.8.4/smc_chk000077500000000000000000000353261473052164700150230ustar00rootroot00000000000000#!/bin/bash # Copyright IBM Corp. 2021 VERSION="1.8.4"; function usage() { echo; echo "Usage: smc_chk [OPTIONS] -C "; echo " smc_chk [OPTIONS] -S"; echo " smc_chk -i "; echo; echo "Check SMC setup"; echo; echo " -C, --connect connect to specified IP"; echo " -d, --debug show debug messages"; echo " -h, --help display this message"; echo " -i, --pnetid print PNET ID and exit"; echo " -p, --port use the next free port starting"; echo " with PORT (default: $PRT_DFT)"; echo " -S, --server start server only"; echo " -v, --version display version info"; echo " -6, --ipv6 IP address is IPv6"; echo; } function debug() { if [ $dbg -gt 0 ]; then echo "[DEBUG] $@"; fi } function get_free_port() { local i; for ((i=$1;; i=i+1)); do ss -tan | awk '{print($4)}' | sed 's/.*://' | sort | uniq | grep -w $i >/dev/null [ $? -ne 0 ] && break done echo $i; } # Params: # $1 Port # $2 Set to '-6' for IPv6 function run_server() { local i; cmd="smc_run $srv -p $1 $2"; debug "Starting server: $cmd"; $cmd >/dev/null 2>&1 & pidsrv=$!; for (( i=0; i<100; i++ )); do # wait 10s max ss -tln | awk '{print($4)}' | sed 's/.*://' | grep -w $1 >/dev/null [ $? -eq 0 ] && break; sleep 0.1; done } # Implicit params: # $1 IP # $2 Port # $3 Set to '-6' for IPv6 function run_client() { local mode; local i; cmd="smc_run $clt $1 -p $2 $3"; debug "Running client: $cmd"; $cmd >/dev/null 2>&1 & pidclt=$!; for (( i=0; i<100; i++ )); do # wait 10s max res="`smcss | awk -v id="$1:$2" '$5 == id {print($7" "$8)}'`"; [ "$res" != "" ] && break; sleep 0.1; done kill -INT $pidclt >/dev/null 2>&1; debug "Client result: $res"; if [ "$res" == "" ]; then echo " Failed, no connection" else mode=`echo $res | awk '{print($1)}'`; err_clt="`echo $res | awk '{print($2)}' | sed 's#/.*##'`"; if [ `echo $res | awk '{print($2)}' | grep -c /` -eq 1 ]; then err_srv="`echo $res | awk '{print($2)}' | sed 's#.*/##'`"; else err_srv=""; fi if [ "$mode" == "TCP" ]; then echo " Failed (TCP fallback), reasons:" res="`man smcss | grep $err_clt`"; if [ "$res" == "" ]; then res="$err_srv (Unkown error code)"; fi echo " Client: $res"; if [ "$err_srv" != "" ]; then res="`man smcss | grep $err_srv`"; if [ "$res" == "" ]; then res=" $err_srv (Unknown error code or non-Linux OS)"; fi echo " Server: $res"; fi else echo " Success, using ${mode:0:3}-${mode:3:1}"; fi fi } function is_python3_available() { if ! which python3 >/dev/null; then echo "Error: python3 is not available"; signal_handler; fi } function init_server() { if [ $init_srv -ne 0 ]; then return; fi init_srv=1; is_python3_available; port=`get_free_port $port`; port6=`get_free_port $(expr $port + 1)`; srv=`mktemp /tmp/echo-srv.XXXXXX`; cat <<-EOF > $srv #!/usr/bin/env python3 import argparse import signal import socket import sys def receiveSignal(signalNumber, frame): if conn: conn.close() s.close() sys.exit(0) signal.signal(signal.SIGINT, receiveSignal) conn = None parser = argparse.ArgumentParser(description='Echo server implemented in python3') parser.add_argument('-p', '--port', required=True, dest='port', action='store', help='listen port') parser.add_argument('-6', '--ipv6', dest='ipv6', action='store_true', help='IPv6 mode') args = parser.parse_args() host = '' # Symbolic name meaning all available interfaces if args.ipv6: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) else: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind((host, int(args.port))) s.listen(1) while True: conn, addr = s.accept() print('Connected from', addr) while True: data = conn.recv(1024) if not data: break conn.close() EOF chmod +x $srv; } function init_client() { if [ $init_clt -ne 0 ]; then return; fi init_clt=1; is_python3_available; clt=`mktemp /tmp/echo-clt.XXXXXX`; cat <<-EOF > $clt #!/usr/bin/env python3 import argparse import socket import signal import sys import time def receiveSignal(signalNumber, frame): s.close() sys.exit(0) signal.signal(signal.SIGINT, receiveSignal) parser = argparse.ArgumentParser(description='Echo client implemented in python3') parser.add_argument('-p', '--port', required=True, dest='port', action='store', help='target port') parser.add_argument(dest='dest', action='store', help='destination address') parser.add_argument('-6', '--ipv6', dest='ipv6', action='store_true', help='IPv6 mode') args = parser.parse_args() if args.ipv6: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) else: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((args.dest, int(args.port))) time.sleep(10) EOF chmod +x $clt; } function test_init() { init_client; init_server; } function test_deinit() { debug "Cleaning up PIDs: $pidsrv $pidsrv6 $pidclt"; kill -INT $pidsrv 2>/dev/null [ "$pidsrv6" != "" ] && kill -INT $pidsrv6 2>/dev/null kill -INT $pidclt 2>/dev/null [ "$clt" != "" ] && [ -e $clt ] && rm $clt; [ "$srv" != "" ] && [ -e $srv ] && rm $srv; } function signal_handler() { test_deinit; exit 1; } function test_iface() { local i; if [ $mode -eq $MODE_CONNECT ]; then echo " Live test (SMC-D and SMC-R)"; else echo " Live test (SMC-D and SMC-R, EXPERIMENTAL)"; fi if [ "$1" != "" ]; then debug "Determine IP for interface $1"; ip="`get_netmasks $1 | head -1 | sed 's#/.*##'`"; if [ "$ip" == "" ]; then echo " No usable IP address configured, skipping"; echo; return; fi fi if [[ $ip == *:* ]]; then run_client $ip $port6 "-6"; else run_client $ip $port; fi echo; } function get_netmasks() { # filter out link-locals ip addr show $1 | grep -we "inet[6]\?" | grep -v "scope link" | awk '{print($2)}'; } function set_pnetid() { debug "Determine PNET ID for $1"; smc_rnics | grep -e "$1\$" >/dev/null; if [ $? -eq 0 ]; then # PCI device - use smc_rnics for easy PNET_ID access debug "PCI device, retrieve PNET ID via smc_rnics"; pnetid="`smc_rnics | grep -e "$1\$" | awk '{print($7)}'`"; [ "$pnetid" != "" ] && return; fi if [ -e /sys/class/net/$1/device/portno ] && [ -e /sys/class/net/$1/device/chpid ]; then # CCW device debug "CCW device, retrieve PNET ID via sysfs"; portno=`cat /sys/class/net/$1/device/portno`; chpid=`cat /sys/class/net/$1/device/chpid`; chpid=${chpid,,}; pnetids="`cat /sys/devices/css0/chp0.$chpid/util_string | sed 's/\x0/\x40/g' | iconv -f IBM-1047 -t ASCII 2>/dev/null`"; (( idx=16*$portno+1 )) (( end=$idx+15 )) pnetid="`echo "$pnetids" | cut -c $idx-$end | sed 's/ //g'`"; [ "$pnetid" != "" ] && return; fi # Check for a software-defined PNET ID debug "No luck so far - try the SW PNET table"; pnetid="`smc_pnet | awk -v id="$1" '$2 == id {print($1)}'`"; if [ "$pnetid" != "" ]; then pnetid="$pnetid*"; fi debug "PNET ID is '$pnetid'"; } function is_smcd_available() { # Verify version availability via new 'smcd info' command smcd info | grep -e "^SMC-D Features:" | grep -w "$1" >/dev/null return $?; } # Returns ISM device with PNET ID == $1 function set_ism() { is_smcd_available "v1"; [ $? -ne 0 ] && return; ism="`smc_rnics | awk -v pn="$1" '$5 == "ISM" && $7 == pn {print($1)}' | sed 's/ $//'`"; } # Returns all ISMv2-eligible devices (PNET ID not set or same as *some* NIC) function set_ismv2() { local pnet; local i; is_smcd_available "v2"; [ $? -ne 0 ] && return; debug "Determine all PNET IDs in use"; all_pnets="`smc_pnet | awk '{print($1)}'`"; for i in `get_iface_realname`; do set_pnetid $i; all_pnets="$all_pnets"$'\n'"$pnetid"; done all_pnets="`echo "$all_pnets" | sort | uniq`"; debug "All pnets found: `echo $all_pnets | tr '\n' ' '`"; ismv2=""; while read line; do [ "$line" == "" ] && return; pnet=`echo $line | awk '{print($2)}'`; if [ "$pnet" != "n/a" ]; then echo "$all_pnets" | grep -w $pnet >/dev/null; [ $? -ne 0 ] && continue; fi ismv2="$ismv2 `echo $line | awk '{print($1)}'`"; done <<< $(smc_rnics | awk '$5 == "ISM" {print($1" "$7)}') ismv2="`echo $ismv2`"; # strip leading blank } function get_mode_param() { case $1 in $MODE_STATIC) echo "'-s/--static-analysis'";; $MODE_LIVE) echo "'-l/--live-test'";; $MODE_PPNETID) echo "'-i/--pnetid'";; $MODE_CONNECT) echo "'-C/--connect'";; $MODE_SERVER) echo "'-S/--server'";; esac } function set_mode() { if [ $mode -eq $1 ]; then return; fi if [ $mode -eq $MODE_LIVE ] && [ $1 -eq $MODE_STATIC ]; then mode=$MODE_ALL; return; fi if [ $mode -lt 0 ]; then mode=$1; else echo "Error: Cannot combine options `get_mode_param $mode` and `get_mode_param $1`"; exit 1; fi } function analyze_iface() { local smcd_configured=0; local tab=0; local i; local TAB1=25; local TAB2=25; echo " Static Analysis (SMC-D only, EXPERIMENTAL)"; set_pnetid $1; set_ism $pnetid; # SMC version echo -n " Configuration: "; if [ "$ism" != "" ]; then smcd_configured=1; tab=$TAB1; echo "SMC-Dv1 (ISMv1 FID(s): $ism)"; fi if [ "$ismv2" != "" ]; then smcd_configured=2; printf "%*sSMC-Dv2 (ISMv2 FID(s): %s\n" $tab "" "$ismv2)"; fi if [ $smcd_configured -eq 0 ]; then echo "SMC-D not configured"; echo; return; fi # PNET ID echo " PNET ID: $pnetid"; # (S) EID in use tab=0; smcd seid show >/dev/null 2>&1; if [ $? -eq 0 ]; then seid="`smcd seid show | awk '$2 == "[enabled]" {print($1)}'`"; if [ "$seid" != "" ]; then echo " Advertising SEID: $seid"; fi for i in `smcd ueid show`; do [ $tab -eq 0 ] && echo -n " Advertising UEIDs: "; printf "%*s%s\n" $tab " " $i; tab=$TAB2; done fi # Reachable IP subnets echo -n " Reachable subnets: "; tab=0; case $smcd_configured in 0) echo "None";; # Shouldn't happen, but... 1) for i in `get_netmasks $1`; do printf "%*s%s\n" $tab "" $i; tab=$TAB1; done;; 2) echo "Any";; esac echo; } # Returns the real name of interface $1 (in case $1 is an altname) # Call with $1 == "" for a list of all interfaces # Call with $1 == "up" for a list of all active interfaces function get_iface_realname() { ip link show $1 | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//'; } trap signal_handler SIGINT SIGTERM; pid=""; pidclt=""; pidsrv=""; pidsrv6=""; MODE_ALL=0; MODE_STATIC=1; MODE_LIVE=2; MODE_PPNETID=3; MODE_CONNECT=4; MODE_SERVER=5; MODE_DFT=$MODE_LIVE; PRT_DFT=37373 port=$PRT_DFT; ipv6=""; fb=$(tput bold 2>/dev/null) # bold font fn=$(tput sgr0 2>/dev/null) # normal font init_clt=0; init_srv=0; args=`getopt -u -o C:dhi:lp:sSv6 -l connect:,debug,help,port:,pnetid:,server,static-analysis,live-test,version,ipv6 -- $*`; [ $? -ne 0 ] && exit 2; set -- $args; tgt=""; mode=-1; dbg=0; rc=0; while [ $# -gt 0 ]; do case $1 in "-C" | "--connect" ) set_mode $MODE_CONNECT; ip="`getent ahosts $2 | awk '{print($1)}' | head -1`"; if [ "$ip" == "" ]; then echo "Error: Unknown destination '$2'"; exit 1; fi ifaces="`ip route get $ip 2>/dev/null | grep -oP '(?<=dev )\w+'`"; if [ "$ifaces" == "" ]; then echo "Error: No route to host: $2"; exit 1; fi shift;; "-d" | "--debug" ) let dbg++;; "-h" | "--help" ) usage; exit 0;; "-i" | "--pnetid" ) set_mode $MODE_PPNETID; tgt="$2"; shift;; "-l" | "--live-test" ) set_mode $MODE_LIVE;; "-p" | "--port" ) port="$2"; shift;; "-s" | "--static-analysis" ) set_mode $MODE_STATIC;; "-S" | "--server" ) set_mode $MODE_SERVER; ifaces="`ip link show up | head -1 | awk '{print($2)}' | sed s'/:$//'`";; "-v" | "--version" ) echo "smc_chk utility, smc-tools-$VERSION"; exit 0;; "-6" | "--ipv6" ) ipv6="-6";; "--" ) ;; * ) if [ $mode == $MODE_PPNETID ]; then echo "Error: Option -P/--print-PNETID takes no extra targets"; exit 1; fi if [ "$tgt" == "" ]; then tgt="$1"; else tgt="$tgt $1"; fi esac shift done if [ $mode -lt 0 ]; then usage; exit 0; fi if [ $mode -le $MODE_PPNETID ]; then if [ "$tgt" != "" ]; then ipv6=""; # we got an interface - if '-6' was specified, it is moot up="up"; for i in $tgt; do ip link show $i >/dev/null 2>&1; if [ $? -ne 0 ]; then echo "Error: Interface $i does not exist"; exit 1; fi if [ $mode -ne $MODE_PPNETID ]; then if [ `ip link show up $i | wc -l` -eq 0 ]; then echo "Error: $i is not an active interface"; exit 2; fi fi done ifaces="$tgt"; else ifaces="`get_iface_realname "up"`"; fi fi debug "Interfaces to check: $ifaces"; if [ $mode -eq $MODE_ALL ] || [ $mode -eq $MODE_LIVE ]; then test_init; run_server $port6 "-6"; # We cannot know whether we need to check an interface with IPv6, pidsrv6=$pidsrv; # so we start servers for both to be on the safe side run_server $port; fi for i in $ifaces; do i=`get_iface_realname $i`; case $mode in $MODE_ALL ) echo "Checking active link: ${fb}$i${fn}"; analyze_iface $i; test_iface $i;; $MODE_STATIC ) echo "Checking active link: ${fb}$i${fn}"; set_ismv2; analyze_iface $i;; $MODE_LIVE ) echo "Checking active link: ${fb}$i${fn}"; test_iface $i;; $MODE_PPNETID ) set_pnetid $i; [ "$pnetid" != "" ] && echo $pnetid;; $MODE_CONNECT ) echo "Test with target IP $ip and port $port"; port6=$port; init_client; test_iface;; $MODE_SERVER ) init_server $port; if [ "$ipv6" == "" ]; then run_server $port; p=$port; else run_server $port6 "-6"; p=$port6; fi echo "Server started on port $p"; wait $pidsrv;; esac done test_deinit; exit 0; smc-tools-1.8.4/smc_chk.8000066400000000000000000000041141473052164700151550ustar00rootroot00000000000000.\" Copyright IBM Corp. 2021 .TH SMC_CHK 8 "January 2021" "smc-tools" "Linux Programmer's Manual" .SH NAME smc_chk \- SMC support diagnostics .SH SYNOPSIS .nf .BI "smc_chk [OPTIONS] -C " IP .BI "smc_chk [OPTIONS] -S" .BI "smc_chk -i "INTERFACE .SH DESCRIPTION Use .B -C/--connect to obtain diagnostic information about the SMC support of a service that runs at a given .IR IP address. Otherwise, you can use .B -S/--server to start a server before connecting with .IR -C . Use .B -i/--pnetid to print the PNET ID of a specified .IR INTERFACE . .SH OPTIONS .TP .BI "\-C, \-\-connect " IP Test SMC-D and SMC-R connectivity to .IR IP . Use option .B -p/--port to specify a .IR PORT . .I IP can specify any service, including remote services. By default, .I IP is assumed to be in IPv4 format. Specify .B -6/--IPv6 if .I IP is an address in IPv6 format. .TP .BR "\-d, \-\-debug " Show debug messages. .TP .BR "\-h, \-\-help" Display a brief .B smc_chk usage information. .TP .BI "\-i, \-\-pnetid " INTERFACE Print the PNET ID of interface .I INTERFACE and exit. An appended asterisk * indicates that the PNET ID was defined via .BR smc_pnet . .TP .BI "\-p, \-\-port " PORT Use port .I PORT for any live tests. When starting a server via .BR "\-S, \-\-server" , and if .I PORT is already in use, the next free port is used. .TP .BR "\-S, \-\-server" Start a server for manual tests. Use option .B -p/--port to specify a port. .TP .BR "\-v, \-\-version" Display version information. .TP .BR "\-6, \-\-ipv6" .I IP address provided is in IPv6 format. .SH Examples .SS "Check whether a z/OS instance running at 192.168.37.1 is enabled for \ SMC using the 3270 console service running on port 23" smc_chk -C 192.168.37.1 -p 23 .SS "Print PNET ID of interface eth0" smc_chk -i eth0 .SS "Start server on port 45901 (or the next successive free port) for smc_chk -C to connect to" smc_chk -S -p 45901 .SH RETURN CODES On success, .B smc_chk returns 0. If an error occurs, a return code other than 0 is returned. .P .SH SEE ALSO .BR af_smc (7), .BR smc_pnet (8), .BR smc_run (8), .BR smcd (8), .BR smcr (8), .BR smcss (8) smc-tools-1.8.4/smc_dbg000077500000000000000000000042041473052164700150010ustar00rootroot00000000000000#!/bin/bash # Copyright IBM Corp. 2019 VERSION="1.8.4"; function usage() { echo; echo "Usage: smc_dbg [ OPTIONS ]"; echo; echo "Collect debug information"; echo; echo " -h, --help display this message"; echo " -t, --tgz generate .tgz file"; echo " -v, --version display version info"; echo; } function redirect() { if [ "$tgz" == "on" ]; then exec &>$tmpdir/$1; else echo; fi } tgz="off"; ARCH=`uname -m | cut -c1-4`; args=`getopt -u -o hvt -l help,version,tgz -- $*`; [ $? -ne 0 ] && exit 1; set -- $args; while [ $# -gt 0 ]; do case $1 in "-h" | "--help" ) usage; exit 0;; "-t" | "--tgz" ) tgz="on";; "-v" | "--version" ) echo "smc_dbg utility, smc-tools-$VERSION"; exit 0;; * ) esac shift; done if [ "$tgz" == "on" ]; then exec 3>&1 4>&2 tmpdir=`mktemp -d /tmp/smc_dbg-XXXXXX`; fi redirect version.txt; smcss -v smc_dbg -v smc_pnet -v smc_rnics -v smc_chk -v smcd -v smcr -v if [ "$ARCH" == "s390" ]; then redirect devices.txt; echo "CCW Devices:" printf " Device CHPID Port PNET ID\n"; echo " -------------------------------------------"; for device in `ls -1 /sys/bus/ccwgroup/devices`; do chpid=`cat /sys/bus/ccwgroup/devices/$device/chpid | tr [A-F] [a-f]`; osaport=`cat /sys/bus/ccwgroup/devices/$device/portno`; iface=`cat /sys/bus/ccwgroup/devices/$device/if_name`; printf " %8s %4s %-4s %s\n" $device 0x$chpid $osaport `smc_chk -i $iface`; done echo; echo "PCI Devices:" smc_rnics | sed 's/^/ /'; redirect smcss_smcd; smcss --smcd; fi redirect smcss_all.txt; smcss --all --debug; redirect smcss_smcr; smcss --smcr; redirect pnet_table.txt; smc_pnet --show; redirect smcr_links.txt; smcr -d linkgroup link-show; redirect smcd_lgs.txt; smcd -d linkgroup show; redirect smcd_info.txt; smcd info; redirect smcd_stats.txt; smcd -d stats; redirect smcr_stats.txt; smcr -d stats; if [ "$tgz" == "on" ]; then exec >&3 2>&4 cd /tmp; tar cvfz $tmpdir.tgz `basename $tmpdir` >/dev/null 2>&1; rm -rf $tmpdir; echo "Debug output written to $tmpdir.tgz"; fi exit 0; smc-tools-1.8.4/smc_pnet.8000066400000000000000000000107171473052164700153640ustar00rootroot00000000000000.\" smc_pnet.8 .\" .\" .\" Copyright IBM Corp. 2017, 2019 .\" Author(s): Thomas Richter .\" Ursula Braun .\" ---------------------------------------------------------------------- .\" .TH SMC_PNET 8 "January 2017" "smc-tools" "Linux Programmer's Manual" .SH NAME smc_pnet \- create, destroy, and change the SMC PNET table .SH SYNOPSIS .B smc_pnet { \fB\-a\fR | \fB\-\-add\fR \fI\fR } { \fB\-I\fR | \fB\-\-interface\fR } .P .B smc_pnet { \fB\-a\fR | \fB\-\-add\fR \fI\fR } { \fB\-D\fR | \fB\-\-ibdevice\fR } [ \fB\-P\fR | \fB\-\-ibport\fR ] .P .B smc_pnet { \fB\-a\fR | \fB\-\-add\fR \fI\fR } { \fB\-I\fR | \fB\-\-interface\fR } { \fB\-D\fR | \fB\-\-ibdevice\fR } [ \fB\-P\fR | \fB\-\-ibport\fR ] .P .B smc_pnet { \fB\-s\fR | \fB\-\-show\fR \fI\fR } .P .B smc_pnet { \fB\-d\fR | \fB\-\-delete\fR \fI\fR } .P .B smc_pnet { \fB\-f\fR | \fB\-\-flush\fR } .P .B smc_pnet { \fB\-v\fR | \fB\-\-version\fR } .P .B smc_pnet { \fB\-h\fR | \fB\-\-help\fR } .SH DESCRIPTION The SMC protocol requires grouping of standard Ethernet and RoCE networks or ISM devices. Such groups are called \fIPhysical Networks\fR (PNETs). The mapping is configured within a table called \fIpnet table\fR. Any available Ethernet interface can be combined with an available RDMA-capable network interface card (RNIC) or a DMA-capable ISM device, if they belong to the same Converged Ethernet fabric. To configure mapping of a RoCE Adapter port or an ISM device to a standard Ethernet interface, both devices need to have the same PNET ID; either hardware-defined or user-defined using the pnet table. Hardware-defined PNET IDs cannot be overwritten. .P The .B smc_pnet command configures the pnet table. .SH OPTIONS By default, .B smc_pnet shows all entries of the pnet table. .TP .IR defines a name for a grouping of Ethernet interface and RNICs or ISM devices. A PNET ID consists of up to 16 alphanumeric uppercase characters without blanks. .TP .BR "\-a, \-\-add" creates a new PNET ID definition to the pnet table (if it does not already exist). Only one PNET ID can be defined for a certain Ethernet interface, a certain InfiniBand device port or a certain ISM device. Adding more than one PNET ID fails. Hardware defined PNET IDs cannot be overwritten. .TP .BR "\-s, \-\-show" shows a certain PNET ID definition in the pnet table. .TP .BR "\-d, \-\-delete" deletes an existing PNET ID definition from the pnet table. .TP .BR "\-f, \-\-flush" removes all PNET ID definitions from the pnet table. .TP .BR "\-I, \-\-interface " specifies the name of the Ethernet interface to be added for a certain PNET ID definition. .TP .BR "\-D, \-\-ibdevice " specifies the ID of the InfiniBand device or ISM device. .TP .BR "\-P, \-\-ibport " specifies the port number of the InfiniBand device port. Valid numbers are 1 or 2. The default value is 1. .TP .BR "\-v, \-\-version" displays smc_pnet program version. .TP .BR "\-h, \-\-help" displays a brief smc_pnet usage information. .SH EXAMPLES .B Define PNET ID ABC for the ethernet device names encf500 and bond0, and define .B PNET ID ABC for the InfiniBand device ID 0001:00:00.0 (port 2) and the ISM .B device ID 0004:00:00.0: .RS 4 .PP .nf $ smc_pnet \-a ABC \-I encf500 $ smc_pnet \-a ABC \-I bond0 $ smc_pnet \-a ABC \-D 0001:00:00:00.0 \-P 2 $ smc_pnet \-a ABC \-D 0004:00:00:00.0 .RE .PP . .B Show all pnet table entries: .RS 4 .PP .nf $ smc_pnet ABC encf500 n/a 255 ABC bond0 n/a 255 ABC n/a 0001:00:00.0 2 ABC n/a 0004:00:00.0 1 .RE .PP . .B Define PNET ID XYZ for the ethernet interface name vlan0201 and the InfiniBand .B device ID 0001:00:00.0 (port 1): .RS 4 .PP $ smc_pnet \-a XYZ \-I vlan0201 \-D 0001:00:00.0 \-P 1 .RE .PP . .B Show all entries for PNET ID XYZ: .RS 4 .PP .nf $ smc_pnet \-s XYZ XYZ vlan0201 n/a 255 XYZ n/a 0001:00:00.0 1 .RE .PP . .B Delete all pnet table entries with PNET ID named ABC: .RS 4 .PP $ smc_pnet \-d ABC .RE .PP . .B Delete all entries in the pnet table: .RS 4 .PP $ smc_pnet \-f .RE .PP . . .SH RETURN CODES Successful \fBsmc_pnet\fR commands return 0. If an error occurs, \fBsmc_pnet\fR writes a message to stderr and completes with a return code other than 0. .P .SH SEE ALSO .BR af_smc (7), .BR smc_chk (8) .BR smc_rnics (8), .BR smc_run (8), .BR smcd (8), .BR smcr (8), .BR smcss (8) smc-tools-1.8.4/smc_pnet.c000066400000000000000000000206731473052164700154410ustar00rootroot00000000000000/* * Shared Memory Communications over RDMA (SMC-R) and RoCE * * Copyright IBM Corp. 2017 * * Author(s): Thomas Richter * * User space program for SMC-R PNET Table manipulation with generic netlink. * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "smctools_common.h" static char *progname; static struct pnetentry { char *pnetid; /* Pnetid */ char *ethname; /* Ethernet device name */ char *ibname; /* Infiniband/ISM device name */ int ibport; /* Infiniband device port number */ unsigned char cmd; /* Command to execute */ } pnetcmd; static void _usage(FILE *dest) { fprintf(dest, "Usage: %s [ OPTIONS ] [pnetid]\n" "\t-h, --help this message\n" "\t-v, --version show version information\n" "\t-a, --add add a pnetid entry, requires interface or ib/ism device\n" "\t-d, --delete delete a pnetid entry\n" "\t-s, --show show a pnetid entry\n" "\t-f, --flush flush the complete pnet table\n" "\t-I, --interface Ethernet interface name of a pnetid entry\n" "\t-D, --ibdevice Infiniband/ISM device name of a pnetid entry\n" "\t-P, --ibport Infiniband device port (default: 1)\n" "\t\n" "\tno OPTIONS show complete pnet table\n", progname); } static void help(void) __attribute__((noreturn)); static void help(void) { _usage(stdout); exit(EXIT_SUCCESS); } static void usage(void) __attribute__((noreturn)); static void usage(void) { _usage(stderr); exit(EXIT_FAILURE); } static int convert(char *string) { unsigned long no; char *endp; no = strtoul(string, &endp, 0); if (*endp != '\0' || no > 2) { fprintf(stderr, "%s invalid ib port:%s\n", progname, string); usage(); } return no; } static const struct option long_opts[] = { { "interface", 1, 0, 'I' }, { "ibdevice", 1, 0, 'D' }, { "ibport", 1, 0, 'P' }, { "flush", 0, 0, 'f' }, { "add", 0, 0, 'a'}, { "show", 0, 0, 's'}, { "delete", 0, 0, 'd'}, { "version", 0, 0, 'v' }, { "help", 0, 0, 'h' }, { NULL, 0, NULL, 0} }; static struct nla_policy smc_pnet_policy[SMC_PNETID_MAX + 1] = { [SMC_PNETID_NAME] = { .type = NLA_STRING, .maxlen = 17 }, [SMC_PNETID_ETHNAME] = { .type = NLA_STRING, .maxlen = 16 }, [SMC_PNETID_IBNAME] = { .type = NLA_STRING, .maxlen = 64 }, [SMC_PNETID_IBPORT] = { .type = NLA_U8, .maxlen = 1 } }; /* Netlink library call back handler to be called on data reception. */ static int cb_handler(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_PNETID_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); if (genlmsg_validate(hdr, 0, SMC_PNETID_MAX, smc_pnet_policy) || genlmsg_parse(hdr, 0, attrs, SMC_PNETID_MAX, smc_pnet_policy) < 0) { fprintf(stderr, "%s: invalid data returned\n", progname); nl_msg_dump(msg, stderr); return NL_STOP; } printf("%s %s %s %d\n", nla_get_string(attrs[SMC_PNETID_NAME]), nla_get_string(attrs[SMC_PNETID_ETHNAME]), nla_get_string(attrs[SMC_PNETID_IBNAME]), nla_get_u8(attrs[SMC_PNETID_IBPORT])); return NL_OK; } static int genl_command(void) { int rc = EXIT_FAILURE, id, nlmsg_flags = 0; struct nl_sock *sk; struct nl_msg *msg; /* Allocate a netlink socket and connect to it */ sk = nl_socket_alloc(); if (!sk) { nl_perror(NLE_NOMEM, progname); return rc; } rc = genl_connect(sk); if (rc) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out1; } id = genl_ctrl_resolve(sk, SMCR_GENL_FAMILY_NAME); if (id < 0) { rc = EXIT_FAILURE; if (id == -NLE_OBJ_NOTFOUND) fprintf(stderr, "%s: SMC module not loaded\n", progname); else nl_perror(id, progname); goto out2; } nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); /* Allocate a netlink message and set header information. */ msg = nlmsg_alloc(); if (!msg) { nl_perror(NLE_NOMEM, progname); rc = EXIT_FAILURE; goto out2; } if ((pnetcmd.cmd == SMC_PNETID_DEL || pnetcmd.cmd == SMC_PNETID_GET) && !pnetcmd.pnetid) /* List all */ nlmsg_flags = NLM_F_DUMP; if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, id, 0, nlmsg_flags, pnetcmd.cmd, SMCR_GENL_FAMILY_VERSION)) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } switch (pnetcmd.cmd) { /* Start message construction */ case SMC_PNETID_ADD: if (pnetcmd.ethname) rc = nla_put_string(msg, SMC_PNETID_ETHNAME, pnetcmd.ethname); if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } if (pnetcmd.ibname) rc = nla_put_string(msg, SMC_PNETID_IBNAME, pnetcmd.ibname); if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } if (pnetcmd.ibname) rc = nla_put_u8(msg, SMC_PNETID_IBPORT, pnetcmd.ibport); if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } /* Fall through intended */ case SMC_PNETID_DEL: case SMC_PNETID_GET: if (!pnetcmd.pnetid) /* List all */ break; rc = nla_put_string(msg, SMC_PNETID_NAME, pnetcmd.pnetid); if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } } /* Send message */ rc = nl_send_auto(sk, msg); if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } /* Receive reply message, returns number of cb invocations. */ rc = nl_recvmsgs_default(sk); /* Kernel commit a9d8b0b1e3d689346b016316bd91980d60c6885d * introduced a misbehavior that a FLUSH of an empty table * returned -ENOENT. Fix it in smc-tools as long as kernel patch did'nt * land in the distros. */ if (pnetcmd.cmd == SMC_PNETID_FLUSH && rc != -NLE_OBJ_NOTFOUND) rc = 0; if (rc < 0) { nl_perror(rc, progname); rc = EXIT_FAILURE; goto out3; } rc = EXIT_SUCCESS; out3: nlmsg_free(msg); out2: nl_close(sk); out1: nl_socket_free(sk); return rc; } int main(int argc, char **argv) { char *slash; int rc, ch; progname = (slash = strrchr(argv[0], '/')) ? slash + 1 : argv[0]; while ((ch = getopt_long(argc, argv, "I:D:P:fasdhv", long_opts, NULL )) != EOF) { switch (ch) { case 'f': if (pnetcmd.cmd) usage(); pnetcmd.cmd = SMC_PNETID_FLUSH; break; case 's': if (pnetcmd.cmd) usage(); pnetcmd.cmd = SMC_PNETID_GET; pnetcmd.pnetid = optarg; break; case 'd': if (pnetcmd.cmd) usage(); pnetcmd.cmd = SMC_PNETID_DEL; pnetcmd.pnetid = optarg; break; case 'a': if (pnetcmd.cmd) usage(); pnetcmd.cmd = SMC_PNETID_ADD; pnetcmd.pnetid = optarg; break; case 'I': pnetcmd.ethname = optarg; break; case 'D': pnetcmd.ibname = optarg; break; case 'P': pnetcmd.ibport = convert(optarg); break; case 'v': printf("smc_pnet utility, smc-tools-%s\n", RELEASE_STRING); exit(0); case 'h': help(); case '?': default: usage(); } } if (optind + 1 < argc) { fprintf(stderr, "%s too many parameters\n", progname); usage(); } if (optind + 1 == argc) pnetcmd.pnetid = argv[optind]; if (!pnetcmd.cmd) { if (optind < argc) { fprintf(stderr, "%s: parameters without option\n", progname); usage(); } pnetcmd.cmd = SMC_PNETID_GET; } if (pnetcmd.cmd == SMC_PNETID_FLUSH) { if (optind < argc) { fprintf(stderr, "%s: -f takes no parameters\n", progname); usage(); } } if (pnetcmd.cmd == SMC_PNETID_ADD) { if (!pnetcmd.ethname && !pnetcmd.ibname) { fprintf(stderr, "%s: interface or device missing\n", progname); usage(); } if (!pnetcmd.ibport) pnetcmd.ibport = 1; } if (pnetcmd.cmd == SMC_PNETID_GET || pnetcmd.cmd == SMC_PNETID_DEL) { if (pnetcmd.ethname) { fprintf(stderr, "%s: interface %s ignored\n", progname, pnetcmd.ethname); pnetcmd.ethname = NULL; } if (pnetcmd.ibname) { fprintf(stderr, "%s: device %s ignored\n", progname, pnetcmd.ibname); pnetcmd.ibname = NULL; } if (pnetcmd.ibport) { fprintf(stderr, "%s: ibport %d ignored\n", progname, pnetcmd.ibport); pnetcmd.ibport = 0; } } rc = genl_command(); return rc; } smc-tools-1.8.4/smc_rnics000077500000000000000000000147561473052164700154000ustar00rootroot00000000000000#!/bin/bash # Copyright IBM Corp. 2018, 2022 VERSION="1.8.4"; function usage() { echo; echo "Usage: smc_rnics [ OPTIONS ] [ FID ]"; echo; echo "List RNICs"; echo; echo " -a, --all include disabled devices in output"; echo " -d, --disable disable the specified FID"; echo " -e, --enable enable the specified FID"; echo " -h, --help display this message"; echo " -I, --IB-dev display IB-dev instead of netdev attributes"; echo " -r, --rawids display 'type' as raw vendor/device IDs"; echo " -v, --version display version info"; echo; } function print_header() { if [ $IBdev -eq 0 ]; then printf " FID Power PCI_ID PCHID Type PPrt PNET_ID Net-Dev\n"; else printf " FID Power PCI_ID PCHID Type IPrt PNET_ID IB-Dev\n"; fi echo '------------------------------------------------------------------------------------------'; } function get_softset_pnet_id() { local res="n/a"; local line; local id; local iface; local dev; local prt; while read -r line; do read id iface dev prt <<< $line; if [[ ("$iface" != "n/a" && "$iface" == "$int") || ("$dev" != "n/a" && "$dev" == "$addr") ]]; then if [ "$prt" != "255" ] && [ "$prt" != "$iport" ]; then continue; fi res="$id*"; fi done <<< "$(smc_pnet)" echo "$res"; } function get_pnet_from_port() { local idx; local end; local lport=$port; local iport; local res; if [ "$lport" == "" ]; then echo ""; return; fi if [ "$lport" == "n/a" ] || [ "$dev_type" != "RoCE_Express" ]; then lport=0; else [ $IBdev -ne 0 ] && let lport=$lport-1; fi (( iport=$lport+1 )) (( idx=16*$lport+1 )) (( end=$idx+15 )) res="`echo "$pnetids" | cut -c $idx-$end | sed 's/ //g'`"; if [ "$res" == "" ]; then res="`get_softset_pnet_id`"; fi echo $res; } function print_rnic() { printf "%8x %-5s %-12s %-4s %-14s %-4s %-17s %s\n" "$((16#$fid))" "$power" "$addr" "$pchid" "$dev_type" "$port" "`get_pnet_from_port`" "$int"; (( printed++ )); } function set_RoCE_dev_and_port() { dev_type="$1"; if [ -e port ]; then port=`cat port`; if [ $port -eq 0 ]; then port="n/a"; else if [ $IBdev -eq 0 ]; then let port=$port-1; else port=1; fi fi fi; } function set_by_firmware_lvl() { local iface; local name; local lvl; name="Mlx_$id"; which ethtool >/dev/null 2>&1; if [ $? -eq 0 ] && [ "$int" != "n/a" ] && [ -d "net" ]; then iface="`ls -1 net | head -1`"; lvl="`ethtool -i $iface | grep -e "^firmware-version:" | awk '{print($2)}'`"; if [ "${lvl%%.*}" == "22" ]; then name="RoCE_Express3"; fi fi set_RoCE_dev_and_port $name; } function print_rnics() { # iterate over slots, as powered-off devices won't show elsewhere for fid in `ls -1 /sys/bus/pci/slots`; do cd /sys/bus/pci/slots/$fid; fid="$fid"; if [ "$target" != "" ] && [ "$fid" != "$target" ]; then continue; fi power=`cat power`; interfaces=""; port="n/a"; addr=""; int=""; if [ $power -eq 0 ]; then # device not yet hotplugged if [ $all -ne 0 ]; then dev_type=""; pchid=""; pnet=""; port=""; print_rnic; fi continue; fi # device is hotplugged - locate it for dev in `ls -1 /sys/bus/pci/devices`; do cd /sys/bus/pci/devices/$dev; if [ "`cat function_id`" == "0x$fid" ]; then addr=$dev; break; fi done if [ "$addr" == "" ]; then echo "Error: No matching device found for FID $fid" >&2; continue; fi cd /sys/bus/pci/devices/$addr; id=`cat device`; vend=`cat vendor`; dev_type="${vend#0x}:${id#0x}"; if [ $rawIDs -eq 0 ]; then case "$vend" in "0x1014" ) # IBM case "$id" in "0x04ed") dev_type="ISM"; int="n/a";; *) continue; esac;; "0x15b3" ) # Mellanox case "$id" in "0x1003" | \ "0x1004") dev_type="RoCE_Express";; "0x1016") set_RoCE_dev_and_port "RoCE_Express2";; "0x101e") set_by_firmware_lvl;; *) set_RoCE_dev_and_port "Mlx_$id";; esac;; *) [ $all -eq 0 ] && continue esac fi pchid="`cat pchid | sed 's/^0x//'`"; pnetids="`cat util_string | sed 's/\x0/\x40/g' | iconv -f IBM-1047 -t ASCII`"; if [ $IBdev -eq 0 ]; then if [ -d "net" ]; then interfaces="`ls -1 net`"; else int="n/a"; print_rnic; continue; fi # one device can have multiple interfaces (one per port) for int in $interfaces; do cd /sys/bus/pci/devices/$addr/net/$int; if [ "$dev_type" == "RoCE_Express" ] && [ -e dev_port ]; then port=`cat dev_port`; fi print_rnic; done else if [ -d "infiniband" ]; then int="`ls -1 infiniband`"; else int="n/a"; print_rnic; continue; fi # only one IB interface per card cd /sys/bus/pci/devices/$addr/infiniband/$int for port in `ls -1 ports`; do print_rnic; done fi done } function format_fid() { res="${1#0x}"; if [[ ! "$res" =~ ^[[:xdigit:]]+$ ]]; then printf "Error: '%s' is not a valid FID\n" "$res" >&2; exit 3; fi res="`printf "%08x" $((16#$res))`"; } args=`getopt -u -o hIrvae:d: -l all,enable:,disable:,help,IB-dev,rawids,version -- $*`; [ $? -ne 0 ] && exit 2; set -- $args; action="print"; rawIDs=0; all=0; target=""; IBdev=0; printed=0; while [ $# -gt 0 ]; do case $1 in "-a" | "--all" ) all=1;; "-e" | "--enable" ) action="enable"; fid=$2; shift;; "-d" | "--disable" ) action="disable"; fid=$2; shift;; "-h" | "--help" ) usage; exit 0;; "-I" | "--IB-dev" ) IBdev=1;; "-r" | "--rawids" ) rawIDs=1;; "-v" | "--version" ) echo "smc_rnics utility, smc-tools-$VERSION"; exit 0;; "--" ) ;; * ) format_fid "$1"; target="$res"; esac shift; done if [ "`uname -m`" != "s390x" ] && [ "`uname -m`" != "s390" ]; then printf "Error: s390/s390x supported only\n" >&2; exit 1; fi if [ "$action" != "print" ]; then if [ "$target" != "" ]; then usage; exit 4; fi format_fid "$fid"; fid="$res"; ufid=`printf "%x" $((16#$fid))`; # representation without leading zeros if [ ! -d /sys/bus/pci/slots/$fid ]; then echo "Error: FID $ufid does not exist" >&2; exit 5; fi power=`cat /sys/bus/pci/slots/$fid/power 2>/dev/null`; val=0; [ "$action" == "enable" ] && val=1; if [ $power -eq $val ]; then echo "Error: FID $ufid is already ${action}d" >&2; exit 6; fi echo $val > /sys/bus/pci/slots/$fid/power 2>/dev/null; if [ $? -ne 0 ]; then echo "Error: Failed to $action FID $ufid" >&2; exit 7; fi exit 0; fi print_header; print_rnics | sort -k 1; if [ "$target" != "" ] && [ $printed -eq 0 ]; then exit 8; fi exit 0; smc-tools-1.8.4/smc_rnics.8000066400000000000000000000044001473052164700155240ustar00rootroot00000000000000.\" Copyright IBM Corp. 2018 .TH SMC_RNICS 8 "October 2018" "smc-tools" "Linux Programmer's Manual" .SH NAME smc_rnics \- list, enable and disable (R)NICS as used by SMC-R and SMC-D .SH SYNOPSIS .B smc_rnics .RB [ \-ahrv ] .RB [ \-d .IR FID ] .RB [ \-e .IR FID ] .RI [ FID ] .SH DESCRIPTION The SMC protocol requires an (R)NIC for the (R)DMA traffic. Use .B smc_rnics to handle, and to identify hotplugged (R)NICs. .SH OPTIONS By default, .B smc_rnics shows all enabled (R)NICs in the system. .TP .I FID Limit output to the specified function ID .RI ( FID ). .TP .BR "\-a, \-\-all" Include disabled and unknown devices in output .TP .BR "\-d, \-\-disable " \fIFID Set (R)NIC .I FID offline. .TP .BR "\-e, \-\-enable " \fIFID Set (R)NIC .I FID online. Note that devices other than (R)NICs are not displayed by default and therefore, once enabled, will not appear in further output unless option .BR -a or .BR -r is specified. .TP .BR "\-h, \-\-help" Display a brief .B smc_rnics usage information. .TP .BR "\-I, \-\-IB-dev" Display IB device information. .TP .BR "\-r, \-\-rawids" Display raw PCI vendor and device codes in column. Note that this will also include unknown devices. .TP .BR "\-v, \-\-version" Display version information. .SH OUTPUT .SS "FID" Function ID. .SS "Power" Indicates whether the PCI slot is on (1) or off (0). See option .B -e on how to enable a device that is offline. .SS "PCI_ID" PCI ID in BDF (Bus:Device.Function) notation. .SS "PCHID" Physical channel identifier, or virtual channel identifier (VCHID) for ISM devices. .SS "Type" Device type in human readable form. See option .B -r to switch to display of PCI vendor and device instead. .SS "PPrt" Corresponding physical port of an RNIC, if applicable. Starts counting at 0. .SS "IPrt" Corresponding Infiniband port of an RNIC, if applicable. Starts counting at 1. .SS "PNET_ID" Physical network ID. Has an asterisk .B * appended if defined via .BR smc_pnet . .SS "Net-Dev" Network interface in Linux, if applicable. .SS "IB-Dev" Infiniband interface in Linux, if applicable. .SH RETURN CODES On success, .B smc_rnics returns 0. If an error occurs, a return code other than 0 is returned. .P .SH SEE ALSO .BR af_smc (7), .BR smc_pnet (8), .BR smc_run (8), .BR smcd (8), .BR smcr (8), .BR smcss (8) smc-tools-1.8.4/smc_run000077500000000000000000000037271473052164700150620ustar00rootroot00000000000000#!/bin/bash # # SMC Tools - Shared Memory Communication Tools # # Copyright IBM Corp. 2017, 2018 # # All rights reserved. This program and the accompanying materials # are made available under the terms of the Eclipse Public License v1.0 # which accompanies this distribution, and is available at # http://www.eclipse.org/legal/epl-v10.html # LIB_NAME="libsmc-preload.so" VERSION="1.8.4"; function usage() { echo; echo "Usage: smc_run [ OPTIONS ] COMMAND"; echo; echo "Run COMMAND using SMC for TCP sockets"; echo; echo " -d enable debug mode"; echo " -h display this message"; echo " -r request receive buffer size in Bytes"; echo " -t request transmit buffer size in Bytes"; echo " -v display version info"; } function check_size() { if [[ ! "$1" =~ ^[0-9]+[k|K|m|M]?$ ]]; then echo "Error: Invalid buffer size specified: '$1'"; exit 1; fi return } function adjust_core_net_max() { case ${2: -1} in k | K) (( OPTARG=${2%?}*1024 ));; m | M) (( OPTARG=${2%?}*1048576 ));; *) esac if [ `sysctl -n net.core.$1_max` -lt $OPTARG ]; then sysctl -w net.core.$1_max=$OPTARG >/dev/null; fi } # # Verify command line arguments and specify the preload library debug mode # if necessary. # SMC_DEBUG=0; while getopts "dhr:t:v" opt; do case $opt in d) SMC_DEBUG=1;; h) usage; exit 0;; r) check_size $OPTARG; adjust_core_net_max rmem $OPTARG; export SMC_RCVBUF=$OPTARG;; t) check_size $OPTARG; adjust_core_net_max wmem $OPTARG; export SMC_SNDBUF=$OPTARG;; v) echo "smc_run utility, smc-tools-$VERSION"; exit;; \?) echo "`basename "$0"`: Error: Invalid option: -$OPTARG"; exit 1;; esac done shift $(expr $OPTIND - 1); if [ $# -eq 0 ]; then echo "`basename "$0"`: Error: Missing command parameter"; exit 1; fi export SMC_DEBUG; # # Execute the specified command. # export LD_PRELOAD=$LD_PRELOAD:$LIB_NAME; exec "$@" exit $?; smc-tools-1.8.4/smc_run.8000066400000000000000000000046071473052164700152230ustar00rootroot00000000000000.\" smc_run.8 .\" .\" .\" Copyright IBM Corp. 2017 .\" Author(s): Ursula Braun .\" ---------------------------------------------------------------------- .\" .TH SMC_RUN 8 "January 2017" "smc-tools" "Linux Programmer's Manual " .SH NAME smc_run \- start a TCP socket program with the capability to use SMC as networking protocol .SH SYNOPSIS .B smc_run .RB [ \-dhrtv ] [ \-r .IR SIZE ] .RB [-t .IR SIZE ] .I program .I parameters .SH DESCRIPTION .B smc_run starts a .IR program specified as argument with its .IR parameters , allowing to use the SMC protocol for program-used TCP socket connections. .br The script specifies libsmc-preload.so as a preload shared library for the Linux program loader, and may adjust transmit and receive buffer sizes to .I SIZE by setting socket options SO_RCVBUF and SO_SNDBUF respectively. .I SIZE can be specified in Bytes or using metric prefixes k and m, e.g. 8k...1024k/1m. .br The preload library libsmc-preload.so intercepts a few TCP socket calls and triggers the equivalent execution through SMC. .br Note: If it is not possibile to use .IR smc_run , the libsmc-preload.so may be installed as apreload library via environment variable LD_PRELOAD. Use environment varibles SMC_SNDBUF and SMC_RCVBUF to request specific transmit and receive buffer sizes respectively. Supports metric prefixes k and m. The following options can be specified: .TP .BR "\-d" Display additional diagnostic messages during the program execution. .TP .BR "\-h" Display a brief usage information. .TP .BR "\-r " \fISIZE Request receive buffer size .IR SIZE . .br .BR Notes : .RS .IP \[bu] 2 May be overridden by the application. .IP \[bu] Increases net.core.rmem_max if necessary. .RE .TP .BR "\-t " \fISIZE Request transmit buffer size .IR SIZE . .br .BR Notes : .RS .IP \[bu] 2 May be overridden by the application. .IP \[bu] Increases net.core.wmem_max if necessary. .RE .TP .BR "\-v" Display version information. .SH RETURN CODES On success, the .IR smc_run command returns 0. If an error occurs .IR smc_run writes a message to stdout and completes with a return code other than 0. .TP .B 1 An invalid option was specified. .P .SH EXAMPLES .B Run program foo using SMC, requesting a receive buffer size of 512KB .RS 4 .PP $ smc_run -r 512k ./foo .P .SH SEE ALSO .BR af_smc (7), .BR smc_chk (8) .BR smc_pnet (8), .BR smc_rnics (8), .BR smcd (8), .BR smcr (8), .BR smcss (8), .BR tcp (7) smc-tools-1.8.4/smcd-device.8000066400000000000000000000065331473052164700157400ustar00rootroot00000000000000.\" smcd-device.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMCD-DEVICE 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-device \- Print information about SMC-D devices smcr-device \- Print information about SMC-R devices .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B device .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smcd device" .RI "[" .B "show" .RI "] [" .B "all" .RI "] " .ti -8 .B smcr .RI "[ " OPTIONS " ]" .B device .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smcr device" .RI "[" .B "show" .RI "] [" .B "all" .RI "] [ " .B netdev .IR NETDEV " ] [ " .B ibdev .IR IBDEV " ] .SH "DESCRIPTION" The .B smcd device command displays SMC-D devices and their properties. The .B smcr device command displays SMC-R devices and their properties. .SS smcd,smcr device show inspect the device properties .TP .B all (default) List all devices. .TP .BI netdev " NETDEV" .B SMC-R only: limit the command output to the device with the specified network device name. .TP .BI ibdev " IBDEV" .B SMC-R only: limit the command output to the device port with the specified RoCE device name. .SH OUTPUT .SS "Net-Dev" Network device name. .SS "IB-Dev" RoCE (InfiniBand) device name. .SS "IB-P" InfiniBand port of the RoCE device. According to the InfiniBand conventions, the port count starts at 1. Consequently, devices where each port is represented as a separate device will indicate the port as the first port for all ports. .SS "IB-State" State of the RoCE device port. .TP .I INACTIVE The RoCE device port is inactive. .TP .I ACTIVE The RoCE device port is active. .SS "Type" Type of the underlying PCI device. .TP .I RoCE_Express Underlying used device is RoCE Express. .TP .I RoCE_Express2 Underlying used device is RoCE Express 2. .TP .I ISM Underlying used device is ISM. .SS "Crit / InUse" Show whether the device is critical i.e. without failover possibility(SMC-R) or whether the device is in use.(SMC-D) .TP .I Yes For SMC-R, there is at least one link group running on the device with state "SINGLE" or locally "ASYMMETRIC" which means one or more link groups without a failover device in case of a link failure. In case of SMC-D, there is at least one link group running on the ISM device. .TP .I No For SMC-R, there is no link group running on the device with state "SINGLE" or locally "ASYMMETRIC" which means the link group(s) have a fallback device in case of a failure. In case of SMC-D, there is no link group running on the ISM device. .SS "FID" Function ID of the PCI device. .SS "PCI-ID" ID of the PCI device. .SS "PCHID" Physical channel ID of the PCI device. .SS "#Links" Number of links (SMC-R)/link groups (SMC-D) on the device. .SS "PNET-ID" PNET ID of the device. Leading asterisk "*" means PNET ID is set by the user. (e.g "*PNET-ID") .SH "EXAMPLES" .br .HP 2 1. Show all SMC-D devices: .br \fB# smcd device show all\fP .br .HP 2 2. Show all SMC-R devices: .br \fB# smcr devices show all\fP .br .HP 2 3. Show all SMC-R devices with RoCE (InfiniBand) device name "mlx4_0": .br \fB# smcr device show ibdev mlx4_0\fP .br .HP 2 4. Show all SMC-R devices with network device name "eth0": .br \fB# smcr device show netdev eth0\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd-info.8000066400000000000000000000020511473052164700154230ustar00rootroot00000000000000.\" smcd-info.8 .\" .\" .\" Copyright IBM Corp. 2021 .\" ---------------------------------------------------------------------- .\" .TH SMCD-INFO 8 "January 2021" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-info \- Print generic information about SMC smcr-info \- Print generic information about SMC .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B info .RI " { " .BR show " | " .BR help " }" .sp .ti -8 .B smcr .RI "[ " OPTIONS " ]" .B info .RI " { " .BR show " | " .BR help " } " .sp .SH "DESCRIPTION" The .B smcd info and .B smcr info commands display generic SMC information from the kernel and the hardware. .SS smcd,smcr info show show the information output .TP .SS Kernel Capabilities Shows the Linux kernel's SMC capabilities independent of any hardware prerequisites .TP .SS Hardware Capabilities Shows the hardware's capabilities independent of support in Linux .SH "EXAMPLES" .HP 2 1. Show the SMC information .br \fB# smcd info\fP .br \fB# smcr info\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd-linkgroup.8000066400000000000000000000106451473052164700165120ustar00rootroot00000000000000.\" smcd-linkgroup.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMCD-LINKGROUP 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-linkgroup \- Print information about SMC-D link groups smcr-linkgroup \- Print information about SMC-R link groups and links .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B linkgroup .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smcd linkgroup" " [ " show " ] [" .B "all " .RI "| " LG-ID " .RI ] .ti -8 .B smcr .RI "[ " OPTIONS " ]" .B linkgroup .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .BR "smcr linkgroup" " { " show " | " link-show " } [" .B "all " .RI "| " LG-ID " ] [ " .B netdev .IR NETDEV " ] [ " .B ibdev .IR IBDEV " ] .SH "DESCRIPTION" The .B smcd linkgroup command displays SMC-D link groups and their properties. The .B smcr linkgroup command displays SMC-R link groups and links with their properties. .SS smcd,smcr linkgroup show inspect the link group properties .TP .B all (default) Show all link groups. .TP .I LG-ID Show the link group with the ID .I LG-ID .TP .BI netdev " NETDEV" .B SMC-R only: list the link groups for the specified network device. .TP .BI ibdev " IBDEV" .B SMC-R only: list the link groups for the specified RoCE device. .SS smcr linkgroup link-show SMC-R only: Inspect the link properties .TP .B all (default) List all links of all link groups. .TP .I LG-ID Show links of the link group with the id. .I LG-ID .TP .BI netdev " NETDEV" List the links of the link groups for the specified network device. .TP .BI ibdev " IBDEV" List the links of the link groups for the specified RoCE device. .SH OUTPUT .SS "LG-ID" ID of the link group. .SS "LG-Role" Role of the link group. .TP .I SERV The link group has a SERVER role. .TP .I CLNT The link group has a CLIENT role. .SS "LG-Type" Type of the link group. .TP .I NONE The link group has the initial type. .TP .I SINGLE The link group has only a single link, i.e. the local and the peer system can offer one device port only for this link group, which means a link outage on either side cannot be recovered. .TP .I SYM The link group has two symmetric links, i.e. the local and the peer system can offer two device ports for this link group, which means a link outage on either side can be recovered. .TP .I ASYMP The link group has asymmetric links, i.e. the peer system can offer one device port only for this link group, which means a link outage on the peer side cannot be recovered. .TP .I ASYML The link group has asymmetric links, i.e. the local system can offer one device port only for this link group, which means a link outage on the local side cannot be recovered. .SS "VLAN" The VLAN to which the link group belongs. .SS "#Conns" Number of connections(sockets) running on the link or link group. .SS "PNET-ID" PNET ID of the link group. Asterisk "*" means PNET ID is set by the user. .SS "Net-Dev" Network device name corresponding to the link. .SS "Link-State" The state of the link. .TP .I LINK_UNUSED The link is not in use and in initial state. .TP .I LINK_INACTIVE The link is inactive and will go away. .TP .I LINK_ACTIVATING The link is being activated with the peer. .TP .I LINK_ACTIVE The link is active and operates on an established link with the peer. Data is being exchanged via RDMA. .SS "Link-UID" Unique identifier of the link. This identifier consists of link group id and link id. .SS "Peer-UID" Unique identifier of the link on the peer side. This identifier consists of link group id and link id. .SS "IB-Dev" Name of the RoCE device used by the link. .SS "IB-P" Port of the RoCE device used by the link. .SS "Local-GID" GID of the RoCE port used by the link. .SS "Peer-GID" GID of the peer RoCE port used by the link. .SH "EXAMPLES" .HP 2 1. Show all SMC-D link groups: .br \fB# smcd linkgroup show all\fP .br .HP 2 2. Show all SMC-D link groups with link group id 50: .br \fB# smcd linkgroup show 50\fP .br .HP 2 3. Show all SMC-R links: .br \fB# smcr linkgroup link-show all\fP .br .HP 2 4. Show all SMC-R links with link group ID 40: .br \fB# smcr linkgroup link-show 40\fP .br .HP 2 5. Show all SMC-R links on RoCE device "mlx4_0": .br \fB# smcr linkgroup link-show ibdev mlx4_0\fP .br .HP 2 6. Show all SMC-R links on network device "eth0": .br \fB# smcr linkgroup link-show netdev eth0\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd-seid.8000066400000000000000000000016511473052164700154210ustar00rootroot00000000000000.\" smcd-seid.8 .\" .\" .\" Copyright IBM Corp. 2021 .\" ---------------------------------------------------------------------- .\" .TH SMCD-SEID 8 "January 2021" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-seid \- Control the system EID .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B seid .RI " { " .BR show " | " .BR enable " | " .BR disable " | " .BR help " }" .sp .SH "DESCRIPTION" Use the .B smcd seid command to control the system enterprise ID (EID). You can disable the system EID while at least one user defined EID entry exists. A disabled system EID is automatically enabled when the last user defined EID entry is deleted. .SS smcd seid show show the system EID .SS smcd seid enable enable the system EID .SS smcd seid disable disable the system EID .SH "EXAMPLES" .HP 2 1. Disable the system EID .br \fB# smcd seid disable\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd-stats.8000066400000000000000000000117431473052164700156360ustar00rootroot00000000000000.TH SMCD-STATS 8 "June 2021" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-stats \- Print statistics about SMC-D usage smcr-stats \- Print statistics about SMC-R usage .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B stats .RI " { " COMMAND " | " .BR help " }" .sp .ti -8 .B smcr .RI "[ " OPTIONS " ]" .B stats .RI " { " COMMAND " | " .BR help " }" .sp .SH DESCRIPTION The smcd stats and smcr stats commands display statistics about SMC-D and SMC-R, respectively. Data includes number of connections, number of failed connections, buffer information, and special calls. Statistics are collected at various data points in the smc kernel module. Therefore, minor inconsistencies in the data can occur and are to be expected. .SH COMMANDS .TP .BR "show " (default) Display statistics on respective SMC mode usage. Values displayed are cumulative since last reset, unless option .B -a/--absolute is specified. .TP .BR reset Display current statistics and reset all counters to zero. .TP .BR json Display current statistics in JSON format. .SH OPTIONS .TP .B \-d, \-\-details Display additional fields and information in the output. .TP .B \-a, \-\-absolute Display values since smc module load, ignoring any resets. .SH OUTPUT The .B -d/--details option displays a number of additional fields as indicated in the following. It also breaks up a number of counters in section .I Connections Summary into separate counts for client and server. .SS "Total connections handled" Total number of connections handled by the SMC module. Includes .I TCP fallback connections as well as .IR "handshake errors" . .SS "SMC connections" Number of connections that successfully entered the respective SMC mode. .B -d/--details breaks up connections by SMC versions. .SS Handshake errors Number of connections that failed due to errors during the handshaking phase, e.g. peer no longer responding. .SS Avg requests per SMC conn Average number of requests sent and received (including .IR "special socket calls" ) per SMC connection. .SS TCP fallback Number of connections that fell back to TCP/IP. .SS Data transmitted Amount of data sent (TX) or received (RX) in Bytes. .SS Total requests Total number of individual send (TX) or receive (RX) requests handled. Includes requests that ended with errors or did not transfer any data. .SS Buffer full Number of occurrences where the respective send buffer (TX) could not contain all data to be sent, or did not contain as much data as requested in a receive call (RX). .SS Buffer full (remote) (\-\-details only) Number of occurrences where the peer's receive buffer was exceeded by writing data. That is, requests that fill the buffer up to the last bit are not included in the count. .SS Buffer too small (\-\-details only) Number of occurrences where a send request was larger than the local send buffer's total capacity. .SS Buffer too small (remote) (\-\-details only) Number of occurrences where a send request exceeded the total capacity of the peer's receive buffer. .SS Buffer downgrades (\-\-details only) Number of occurrences where a buffer of the requested size could not be allocated for a new connection, and a smaller buffer was used. .SS Buffer reuses (\-\-details only) Number of occurrences where a buffer was provided as requested for a new connection by reusing a buffer from a previous connection. .SS Bufs Histogram of buffer sizes for all connections, including .I buffer downgrades and .IR "buffer reuses" . The histogram scale presents exact buffer sizes. .SS Reqs Histogram of request sizes. The histogram scale includes upper boundaries of request sizes. Counts reflect requested send sizes for TX, and actual receive sizes for RX. Difference to .I "Total requests" is due to requests not transferring any data and/or erroneous requests. .SS Special socket calls Summarizes the total number of sockets calls that require special handling in SMC. To categorize these calls into individual counters as follows, use the .I -d/\-\-details option. .TP .I cork Counts occurrences of sockopt TCP_CORK enablements. That is, does not reflect the number of send requests with TCP_CORK enabled. .TP .I nodelay Counts occurrences of sockopt TCP_NODELAY enablements. That is, does not reflect the number of send requests with TCP_CORK enabled. .TP .I sendpage Counts occurrences of the AF_SMC implementation of the sendpage call. .TP .I splice Counts number of calls of the splice() system call. .TP .I urgent data Counts number of send and receive calls with MSG_OOB set. .SH "EXAMPLES" .HP 2 1. Show SMC-D statistics: .br \fB# smcd stats\fP .br .HP 2 2. Show detailed SMC-R statistics: .br \fB# smcr -d stats show\fP .br .HP 2 3. Show detailed SMC-R statistics and reset SMC-R statistics counters: .br \fB# smcr -d stats reset\fP .br .HP 2 4. Show detailed SMC-D statistics since module load in JSON format: .br \fB# smcd -da stats json\fP .br .HP 2 .HP 2 5. Show SMC-R statistics since module load: .br \fB# smcr -a stats\fP .br .HP 2 .P .SH SEE ALSO .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd-ueid.8000066400000000000000000000026241473052164700154240ustar00rootroot00000000000000.\" smcd-ueid.8 .\" .\" .\" Copyright IBM Corp. 2021 .\" ---------------------------------------------------------------------- .\" .TH SMCD-UEID 8 "January 2021" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd-ueid \- Manage user defined EIDs smcr-ueid \- Manage user defined EIDs .SH "SYNOPSIS" .sp .ad l .in +8 .ti -8 .B smcd .RI "[ " OPTIONS " ]" .B ueid .RI " { " .BR show " | " .BR add " | " .BR del " | " .BR flush " | " .BR help " }" .sp .ti -8 .B smcr .RI "[ " OPTIONS " ]" .B ueid .RI " { " .BR show " | " .BR add " | " .BR del " | " .BR flush " | " .BR help " }" .sp .SH "DESCRIPTION" Use the .B smcd ueid and .B smcr ueid commands to manage user defined enterprise IDs (EIDs). .SS smcd,smcr ueid show show all user defined EID entries .SS smcd,smcr ueid add [ueid] add an entry with the specified EID For [ueid], specify up to 32 uppercase alphabetic characters (A-Z), numerals (0-9), hyphens (-), and dots (.). The first character must be alphanumeric, and dots must not be consecutive. An invalid [ueid] is rejected. .SS smcd,smcr ueid del [ueid] delete the specified user defined EID .SS smcd,smcr ueid flush delete all user defined EIDs .SH "EXAMPLES" .HP 2 1. Add a user defined EID .br \fB# smcd ueid add LOCATION-WEST\fP .br .HP 2 2. Show the user defined EIDs .br \fB# smcd ueid show\fP .br \fBLOCATION-WEST\fP .br \fBLOCAL-WEST\fP .br .SH SEE ALSO .br .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcd.8000066400000000000000000000045151473052164700145010ustar00rootroot00000000000000.\" smcd.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMCD 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smcd \- Print information about SMC-D link groups and devices .SH SYNOPSIS .B smcd .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " .BR help " }" .sp .IR OBJECT " := { " .BR info " | " linkgroup " | " device " | " stats " | " ueid " | " seid " }" .sp .IR OPTIONS " := { " \fB\-v\fR[\fIersion\fR] | \fB\-d\fR[\fIetails\fR] | \fB\-a\fR[\fIbsolute\fR]} .SH OPTIONS .TP .BR "\-v" , " -version" Print the version of the .B smcd utility and exit. .TP .BR "\-d", " \-details" Print detailed information (valid only for stats). .TP .BR "\-a", " \-absolute" Print absolute statistic value (valid only for stats). .SH SMCD - COMMAND SYNTAX .SS .I OBJECT .TP .B device One or more SMC-D devices. .TP .B info Generic SMC information. .TP .B linkgroup One or more SMC-D link groups or links. .TP .B stats SMC-D statistics. .TP .B ueid Work with User defined Enterprise IDs (UEID). .TP .B seid Maintain the System defined Enterprise ID (SEID). .PP The names of all objects can be abbreviated down to a unique stem. For example .B device can be abbreviated to .B dev or just .B d. For more information about individual objects see the man pages in .B SEE ALSO section. .SS .I COMMAND Specifies the action to perform on the object. The set of possible actions depends on the object type. For most objects you can specify the .BR " show " or " link-show" command. Use the .B help command for an object to print information about the available actions and the specific syntax for that object. .sp If no command is given, a default command is assumed. .SH RETURN CODES Successful .IR smcd commands return 0 and display the requested link group or device information. If an error occurs, .IR smcd writes a message to stderr and completes with a return code other than 0. Possible error messages to stderr in case of non-zero return code: .TP .BR "SMC module not loaded" Either kernel is not supporting the .IR smcd tool or the smc kernel module is not loaded. .P .SH SEE ALSO .BR af_smc (7), .BR smcr (8), .BR smcd-device (8), .BR smcd-info (8), .BR smcd-linkgroup (8), .BR smcd-stats (8), .BR smcd-ueid (8), .BR smcd-seid (8) smc-tools-1.8.4/smcr.8000066400000000000000000000045201473052164700145130ustar00rootroot00000000000000.\" smcr.8 .\" .\" .\" Copyright IBM Corp. 2020 .\" Author(s): Guvenc Gulce .\" ---------------------------------------------------------------------- .\" .TH SMCR 8 "June 2020" "smc-tools" "Linux Programmer's Manual" .SH NAME smcr \- Print information about SMC-R link groups, links and devices .SH SYNOPSIS .B smcr .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " .BR help " }" .sp .IR OBJECT " := { " .BR info " | " linkgroup " | " device " | " stats " | " ueid " }" .sp .IR OPTIONS " := { " \fB\-v\fR[\fIersion\fR] | \fB\-a\fR[\fIbsolute\fR] | \fB\-d\fR[\fIetails\fR] | \fB\-dd\fR[\fIetails\fR] } .SH OPTIONS .TP .BR "\-v" , " -version" Print the version of the .B smcr utility and exit. .TP .BR "\-a", " \-absolute" Print absolute statistic value (valid only for stats). .TP .BR "\-d", " \-details" Print detailed information. .TP .BR "\-dd", " \-ddetails" Print more detailed information. .SH SMCR - COMMAND SYNTAX .SS .I OBJECT .TP .B device One or more SMC-R devices. .TP .B info Generic SMC information. .TP .B linkgroup One or more SMC-R link groups or links. .TP .B stats SMC-R statistics. .TP .B ueid Work with User defined Enterprise IDs (UEID). .PP The names of all objects can be abbreviated down to a unique stem. For example .B device can be abbreviated to .B dev or just .B d. For more information about individual objects see the man pages in .B SEE ALSO section. .SS .I COMMAND Specifies the action to perform on the object. The set of possible actions depends on the object type. For most objects you can specify the .BR " show " or " link-show" command. Use the .B help command for an object to print information about the available actions and the specific syntax for that object. .sp If no command is given, a default command is assumed. .SH RETURN CODES Successful .IR smcr commands return 0 and display the requested link group, link or device information. If an error occurs, .IR smcr writes a message to stderr and completes with a return code other than 0. Possible error messages to stderr in case of non-zero return code: .TP .BR "SMC module not loaded" Either kernel is not supporting the .IR smcr tool or the smc kernel module is not loaded. .P .SH SEE ALSO .BR af_smc (7), .BR smcd (8), .BR smcr (8), .BR smcr-device (8), .BR smcr-info (8), .BR smcr-linkgroup (8) .BR smcr-stats (8), .BR smcr-ueid (8) smc-tools-1.8.4/smcss.8000066400000000000000000000176241473052164700147100ustar00rootroot00000000000000.\" smcss.8 .\" .\" .\" Copyright IBM Corp. 2017, 2018 .\" Author(s): Ursula Braun .\" ---------------------------------------------------------------------- .\" .TH SMCSS 8 "June 2018" "smc-tools" "Linux Programmer's Manual" .SH NAME smcss \- print information about the AF_SMC sockets and link groups .SH SYNOPSIS .B smcss .RB [ \-\-debug | \-d ] .RB [ \-\-smcd | \-D ] .RB [ \-\-wide | \-W ] .P .B smcss .RB { \-\-all | -a } .RB [ \-\-debug | \-d ] .RB [ \-\-smcd | \-D ] .RB [ \-\-wide | \-W ] .P .B smcss .RB [ \-\-debug | \-d ] .RB [ \-\-smcr | \-R ] .RB [ \-\-wide | \-W ] .P .B smcss .RB { \-\-all | -a } .RB [ \-\-debug | \-d ] .RB [ \-\-smcr | \-R ] .RB [ \-\-wide | \-W ] .P .B smcss .RB { \-\-listening | \-l } .RB [ \-\-wide | \-W ] .P .B smcss .RB { \-\-version | \-v } .P .B smcss .RB { \-\-help | \-h } .SH DESCRIPTION .B smcss prints information about the Linux AF_SMC sockets and link groups. The type of information printed is controlled by the first argument, as follows: .TP .BR (none) .br displays a list of connecting, closing, or connected SMC sockets with basic information. .TP .BR "\-a, \-\-all" displays all types of SMC sockets: listening, opening, closing, and connected. .TP .BR "\-l, \-\-listening" shows listening sockets only. These are omitted by default. .SH OPTIONS .TP .BR "\-d, \-\-debug" displays additional debug information, such as shutdown state. .TP .BR "\-D, \-\-smcd displays additional SMC-D specific information. Shows SMC-D sockets only. .TP .BR "\-h, \-\-help" displays usage information. .TP .BR "\-R, \-\-smcr displays additional SMC-R specific information. Shows SMC-R sockets only. .TP .BR "\-v, \-\-version" displays program version. .TP .BR "\-W, \-\-wide" do not truncate IP addresses. .SH OUTPUT .SS "State" The state of the socket. The state can be one of these values: .TP .I INIT The SMC socket is being initialized. It is not connected nor listening yet. .TP .I CLOSED The SMC socket is closed. It is not connected nor listening anymore. .TP .I LISTEN The SMC socket is a listening socket, waiting for incoming connection requests. .TP .I ACTIVE The SMC socket has an established connection. In this state, the TCP connection is fully established, rendezvous processing has been completed, and SMC peers can exchange data via RDMA. .TP .I PEERCLW1 No further data will be sent to the peer. .TP .I PEERCLW2 No further data will be sent to or received from the peer. .TP .I APPLCLW1 No further data will be received from the peer. .TP .I APPLCLW2 No further data will be received from or sent to the peer. .TP .I APPLFINCLW The peer has closed the socket. .TP .I PEERFINCLW The socket is closed locally. .TP .I PEERABORTW The socket was abnormally closed locally. .TP .I PROCESSABORT The peer has closed the socket abnormally. .SS "UID" User ID of the SMC socket. .SS "Inode" Inode attribute of the socket. .SS "Local Address" Address and port number of the local end of the SMC socket. The displayed address is truncated when it ends with '..'. .BR "-W, --wide" can be used to display addresses untruncated. .SS "Peer Address" Address and port number of the remote end of the socket. Analogous to "Local Address". .SS "Intf" When the socket is explicitly bound with setsockopt option SO_BINDTODEVICE then Intf shows the interface number of the Ethernet device to which the socket is bound. .SS "Mode" .TP .I SMCD The SMC socket uses SMC-D for data exchange. .TP .I SMCR The SMC socket uses SMC-R for data exchange. .TP .I TCP The SMC socket uses the TCP protocol for data exchange, because an SMC connection could not be established. .TP In case of an error during connection setup, respective error codes will be displayed in the format /. .HP Linux error codes: .TP 13 .I 0x01010000 Out of memory .TP 13 .I 0x02010000 Timeout while waiting for confirm link message over RDMA device .TP .I 0x02020000 Timeout while waiting for RDMA device to be added .TP .I 0x03000000 Configuration error .TP .I 0x03010000 Peer does not support SMC .TP .I 0x03020000 Connection uses IPsec .TP .I 0x03030000 No SMC devices found (R and D) .TP .I 0x03030001 No ISM device for SMC-D found .TP .I 0x03030002 No RDMA device for SMC-R found .TP .I 0x03030003 Hardware has no ISMv2 support .TP .I 0x03030004 Peer sent no SMCv2 extension .TP .I 0x03030005 Peer sent no SMC-Dv2 extension .TP .I 0x03030006 Peer sent no ISMv2 SEID .TP .I 0x03030007 No SMC-Dv2 device found, but required .TP .I 0x03030008 Peer sent no UEID .TP .I 0x03040000 SMC modes mismatch (R or D) .TP .I 0x03050000 Peer has eyecatcher in RMBE .TP .I 0x03060000 Fastopen sockopt not supported .TP .I 0x03070000 IP prefix / subnet mismatch .TP .I 0x03080000 Error retrieving VLAN ID of IP device .TP .I 0x03090000 Error while registering VLAN ID on ISM device .TP .I 0x030a0000 No active SMC-R link in link group .TP .I 0x030b0000 SMC-R link from server not found .TP .I 0x030c0000 SMC version mismatch .TP .I .I 0x030d0000 SMC-D connection limit reached .TP .I .TP .I 0x030e0000 SMC-Rv2 connection found no route to peer .TP .I 0x030f0000 SMC-Rv2 connection mismatch direct/indirect with peer 0x04000000 Synchronization error .TP .I 0x05000000 Peer declined during handshake .TP .I 0x09990000 Internal error .TP .I 0x09990001 rtoken handling failed .TP .I 0x09990002 RDMA link failed .TP .I 0x09990003 RMB registration failed .SS "ShutD" .TP .I <-> The SMC socket has not been shut down. .TP .I R-> The SMC socket is shut down one-way and cannot receive data. .TP .I <-W The SMC socket is shut down one-way and cannot send data. .TP .I R-W The SMC socket is shut down in both ways and cannot receive or send data. .SS "Token" Unique ID of the SMC socket connection. .SS "Sndbuf" Size of the to-be-sent window of the SMC socket connection. .SS "Rcvbuf" Size of the receiving window of the SMC socket connection (filled by peer). .SS "Peerbuf" Size of the peer receiving window of the SMC socket connection (to fill during RDMA-transfer). .SS "rxprod-Cursor" Describes the current cursor location of the "Rcvbuf" for data to be received from the peer. .SS "rxcons-Cursor" Describes the current cursor location of the "Peerbuf" for data sent to peer and confirmed by the peer. .SS "rxFlags" SMC socket connection flags set by and received from the peer. .SS "txprod-Cursor" Describes the current cursor location of the "Peerbuf" for data sent to peer. .SS "txcons-Cursor" Describes the current cursor location of the "Rcvbuf" for data received from the peer and confirmed to the peer. .SS "txFlags" SMC socket connection flags set locally and sent to the peer. .SS "txprep-Cursor" Describes the current cursor location of the "Sndbuf" for data to be sent. The data is to be moved to the "Peerbuf". .SS "txsent-Cursor" Describes the current cursor location of the "Sndbuf" for data sent. The data was moved to the "Peerbuf". .SS "txfin-Cursor" Describes the current cursor location of the "Sndbuf" for data sent and send completion confirmed. The data was moved to the "Peerbuf" and completion was confirmed. .SS "Role" .TP .I CLNT The link group of the SMC socket is used for client connections. .TP .I SERV The link group of the SMC socket is used for server connections. .SS "IB-Device" Name of the RoCE device used by the link group to which the SMC socket belongs. .SS "Port" Port of the RoCE device used by the link group to which the SMC socket belongs. .SS "Linkid" Unique link ID of the link within the link group to which the SMC socket belongs. .SS "GID" Gid of the RoCE port used by the link group to which the SMC socket belongs. .SS "Peer-GID" Gid of the Foreign RoCE port used by the link group to which the SMC socket belongs. .SS "VLAN" tbd. .SH RETURN CODES Successful .IR smcss commands return 0 and display the requested socket state table or link group information. If an error occurs, .IR smcss writes a message to stderr and completes with a return code other than 0. .P .SH SEE ALSO .BR af_smc (7), .BR smc_chk (8) .BR smc_rnics (8), .BR smc_run (8), .BR smc_pnet (8), .BR smcd (8), .BR smcr (8) smc-tools-1.8.4/smcss.c000066400000000000000000000252121473052164700147530ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2017, 2018 * * Author(s): Ursula Braun * * User space program for SMC Socket display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include #include #include #include "smctools_common.h" #include "libnetlink.h" #define ADDR_LEN_SHORT 23 static char *progname; int show_debug; int show_smcr; int show_smcd; int show_wide; int listening = 0; int all = 0; static void print_header(void) { printf("State "); printf("UID "); printf("Inode "); printf("Local Address "); printf("Peer Address "); printf("Intf "); printf("Mode "); if (show_debug) { printf("Shutd "); printf("Token "); printf("Sndbuf "); printf("Rcvbuf "); printf("Peerbuf "); printf("rxprod-Cursor "); printf("rxcons-Cursor "); printf("rxFlags "); printf("txprod-Cursor "); printf("txcons-Cursor "); printf("txFlags "); printf("txprep-Cursor "); printf("txsent-Cursor "); printf("txfin-Cursor "); } if (show_smcr) { printf("Role "); printf("IB-device "); printf("Port "); printf("Linkid "); printf("GID "); printf("Peer-GID"); } if (show_smcd) { printf("GID "); printf("Token "); printf("Peer-GID "); printf("Peer-Token "); printf("Linkid"); } printf("\n"); } static const char *smc_state(unsigned char x) { static char buf[16]; switch (x) { case 1: return "ACTIVE"; case 2: return "INIT"; case 7: return "CLOSED"; case 10: return "LISTEN"; case 20: return "PEERCLOSEWAIT1"; case 21: return "PEERCLOSEWAIT2"; case 22: return "APPCLOSEWAIT1"; case 23: return "APPCLOSEWAIT2"; case 24: return "APPFINCLOSEWAIT1"; case 25: return "PEERFINCLOSEWAIT"; case 26: return "PEERABORTWAIT"; case 27: return "PROCESSABORT"; default: sprintf(buf, "%#x?", x); return buf; } } /* format one sockaddr / port */ static void addr_format(char *buf, size_t buf_len, size_t short_len, __be32 addr[4], int port) { char addr_buf[INET6_ADDRSTRLEN + 1], port_buf[16]; int addr_len, port_len; int af; /* There was an upstream discussion about the content of the * diag_family field. Originally it was AF_SMC, but was changed with * IPv6 support to indicate AF_INET or AF_INET6. Upstream complained * later that there is no way to separate AF_INET from AF_SMC diag msgs. * We now change back the value of the diag_family field to be always * AF_SMC. We now 'parse' the IP address type. * Note that smc_diag.c in kernel always clears the whole addr field * before the ip address is copied into and we can rely on that here. */ if (addr[1] == 0 && addr[2] == 0 && addr[3] == 0) af = AF_INET; else af = AF_INET6; if (buf_len < 20) return; /* no space for errmsg */ if (!inet_ntop(af, addr, addr_buf, sizeof(addr_buf))) { strcpy(buf, "(inet_ntop error)"); return; } sprintf(port_buf, "%d", port); addr_len = strlen(addr_buf); port_len = strlen(port_buf); if (!show_wide && (addr_len + 1 + port_len > short_len)) { if (buf_len < short_len + 1) { strcpy(buf, "(buf to small)"); return; } /* truncate addr string */ addr_len = short_len - 1 - port_len - 2; strncpy(buf, addr_buf, addr_len); buf[addr_len] = '\0'; strcat(buf, ".."); /* indicate truncation */ strcat(buf, ":"); strcat(buf, port_buf); } else { if (buf_len < addr_len + 1 + port_len + 1) { strcpy(buf, "(buf to small)"); return; } snprintf(buf, buf_len, "%s:%s", addr_buf, port_buf); } } static void show_one_smc_sock(struct nlmsghdr *nlh) { struct smc_diag_msg *r = NLMSG_DATA(nlh); struct rtattr *tb[SMC_DIAG_MAX + 1]; unsigned long long inode; char txtbuf[128]; parse_rtattr(tb, SMC_DIAG_MAX, (struct rtattr *)(r+1), nlh->nlmsg_len - NLMSG_LENGTH(sizeof(*r))); if (listening) { if ( r->diag_state != 10) return; } else { if (!all && (r->diag_state == 10 || r->diag_state == 2)) return; } if (show_smcr && r->diag_mode != SMC_DIAG_MODE_SMCR) return; /* show only SMC-R sockets */ if (show_smcd && r->diag_mode != SMC_DIAG_MODE_SMCD) return; /* show only SMC-D sockets */ printf("%-14s ", smc_state(r->diag_state)); printf("%05d ", r->diag_uid); inode = r->diag_inode; printf("%07llu ", inode); if (r->diag_state == 2) /* INIT state */ goto newline; addr_format(txtbuf, sizeof(txtbuf), ADDR_LEN_SHORT, r->id.idiag_src, ntohs(r->id.idiag_sport)); printf("%-*s ", (int)MAX(ADDR_LEN_SHORT, strlen(txtbuf)), txtbuf); if (r->diag_state == 10) /* LISTEN state */ goto newline; addr_format(txtbuf, sizeof(txtbuf), ADDR_LEN_SHORT, r->id.idiag_dst, ntohs(r->id.idiag_dport)); printf("%-*s ", (int)MAX(ADDR_LEN_SHORT, strlen(txtbuf)), txtbuf); printf("%04x ", r->id.idiag_if); if (r->diag_state == 7) /* CLOSED state */ goto newline; if (r->diag_mode == SMC_DIAG_MODE_FALLBACK_TCP) { printf("TCP "); /* when available print local and peer fallback reason code */ if (tb[SMC_DIAG_FALLBACK] && tb[SMC_DIAG_FALLBACK]->rta_len >= sizeof(struct smc_diag_fallback)) { struct smc_diag_fallback fallback; fallback = *(struct smc_diag_fallback *)RTA_DATA(tb[SMC_DIAG_FALLBACK]); printf("0x%08x", fallback.reason); if (fallback.peer_diagnosis) printf("/0x%08x", fallback.peer_diagnosis); } goto newline; } else if (r->diag_mode == SMC_DIAG_MODE_SMCD) printf("%4s ", "SMCD"); else printf("%4s ", "SMCR"); if (show_debug) { if (tb[SMC_DIAG_SHUTDOWN] && tb[SMC_DIAG_SHUTDOWN]->rta_len >= sizeof(__u8)) { unsigned char mask; mask = *(__u8 *)RTA_DATA(tb[SMC_DIAG_SHUTDOWN]); printf(" %c-%c ", mask & 1 ? 'R' : '<', mask & 2 ? 'W' : '>'); } if (tb[SMC_DIAG_CONNINFO] && tb[SMC_DIAG_CONNINFO]->rta_len >= sizeof(struct smc_diag_conninfo)) { struct smc_diag_conninfo cinfo; cinfo = *(struct smc_diag_conninfo *)RTA_DATA(tb[SMC_DIAG_CONNINFO]); printf("%08x ", cinfo.token); printf("%08x ", cinfo.sndbuf_size); printf("%08x ", cinfo.rmbe_size); printf("%08x ", cinfo.peer_rmbe_size); printf("%04x:%08x ", cinfo.rx_prod.wrap, cinfo.rx_prod.count); printf("%04x:%08x ", cinfo.rx_cons.wrap, cinfo.rx_cons.count); printf("%02x:%02x ", cinfo.rx_prod_flags, cinfo.rx_conn_state_flags); printf("%04x:%08x ", cinfo.tx_prod.wrap, cinfo.tx_prod.count); printf("%04x:%08x ", cinfo.tx_cons.wrap, cinfo.tx_cons.count); printf("%02x:%02x ", cinfo.tx_prod_flags, cinfo.tx_conn_state_flags); printf("%04x:%08x ", cinfo.tx_prep.wrap, cinfo.tx_prep.count); printf("%04x:%08x ", cinfo.tx_sent.wrap, cinfo.tx_sent.count); printf("%04x:%08x ", cinfo.tx_fin.wrap, cinfo.tx_fin.count); } } if (show_smcr) { if (tb[SMC_DIAG_LGRINFO] && tb[SMC_DIAG_LGRINFO]->rta_len >= sizeof(struct smc_diag_lgrinfo)) { struct smc_diag_lgrinfo linfo; linfo = *(struct smc_diag_lgrinfo *)RTA_DATA(tb[SMC_DIAG_LGRINFO]); printf("%4s ", linfo.role ? "SERV" : "CLNT"); printf("%-15s ", linfo.lnk[0].ibname); printf("%02x ", linfo.lnk[0].ibport); printf("%02x ", linfo.lnk[0].link_id); printf("%-40s ", linfo.lnk[0].gid); printf("%s", linfo.lnk[0].peer_gid); } } if (show_smcd) { if (tb[SMC_DIAG_DMBINFO] && tb[SMC_DIAG_DMBINFO]->rta_len >= sizeof(struct smcd_diag_dmbinfo)) { struct smcd_diag_dmbinfo dinfo; dinfo = *(struct smcd_diag_dmbinfo *)RTA_DATA(tb[SMC_DIAG_DMBINFO]); printf("%016llx ", dinfo.my_gid); printf("%016llx ", dinfo.token); printf("%016llx ", dinfo.peer_gid); printf("%016llx ", dinfo.peer_token); printf("%08x ", dinfo.linkid); } } newline: printf("\n"); } static int smc_show_netlink() { struct rtnl_handle rth; unsigned char cmd = 0; int rc = 0; if ((rc = rtnl_open(&rth))) return EXIT_FAILURE; rth.dump = MAGIC_SEQ; if (show_debug) cmd |= (1<<(SMC_DIAG_CONNINFO-1)); if (show_smcr) cmd |= (1<<(SMC_DIAG_LGRINFO-1)); if (show_smcd) cmd |= (1<<(SMC_DIAG_DMBINFO-1)); if ((rc = sockdiag_send(rth.fd, cmd))) goto exit; print_header(); rc = rtnl_dump(&rth, show_one_smc_sock); exit: rtnl_close(&rth); return rc; } static const struct option long_opts[] = { { "all", 0, 0, 'a' }, { "debug", 0, 0, 'd' }, { "listening", 0, 0, 'l' }, { "smcd", 0, 0, 'D' }, { "smcr", 0, 0, 'R' }, { "version", 0, 0, 'v' }, { "wide", 0, 0, 'W' }, { "help", 0, 0, 'h' }, { NULL, 0, NULL, 0} }; static void _usage(FILE *dest) { fprintf(dest, "Usage: %s [ OPTIONS ]\n" "\t-h, --help this message\n" "\t-v, --version show version information\n" "\t-a, --all show all sockets\n" "\t-l, --listening show listening sockets\n" "\t-d, --debug show debug socket information\n" "\t-W, --wide do not truncate IP addresses\n" "\t-D, --smcd show detailed SMC-D information (shows only SMC-D sockets)\n" "\t-R, --smcr show detailed SMC-R information (shows only SMC-R sockets)\n" "\tno OPTIONS show all connected sockets\n", progname); } static void help(void) __attribute__((noreturn)); static void help(void) { _usage(stdout); exit(EXIT_SUCCESS); } static void usage(void) __attribute__((noreturn)); static void usage(void) { _usage(stderr); exit(EXIT_FAILURE); } int main(int argc, char *argv[]) { char *slash; int ch; progname = (slash = strrchr(argv[0], '/')) ? slash + 1 : argv[0]; while ((ch = getopt_long(argc, argv, "aldDRhvW", long_opts, NULL)) != EOF) { switch (ch) { case 'a': all++; break; case 'l': listening++; break; case 'd': show_debug++; break; case 'D': show_smcd++; break; case 'R': show_smcr++; break; case 'v': printf("smcss utility, smc-tools-%s\n", RELEASE_STRING); exit(0); case 'W': show_wide++; break; case 'h': help(); case '?': default: usage(); } } if (show_smcr && show_smcd) { fprintf(stderr, "--smcd together with --smcr is not supported\n"); usage(); } if (listening && show_debug) { fprintf(stderr, "--listening together with --debug is not supported\n"); usage(); } if (listening && all) { fprintf(stderr, "--listening together with --all is not supported\n"); usage(); } if (listening && show_smcr) { fprintf(stderr, "--listening together with --smcr is not supported\n"); usage(); } if (listening && show_smcd) { fprintf(stderr, "--listening together with --smcd is not supported\n"); usage(); } return smc_show_netlink(); } smc-tools-1.8.4/smctools_common.h000066400000000000000000000340511473052164700170440ustar00rootroot00000000000000/* * smc-tools/smctools_common.h * * Copyright IBM Corp. 2017 * * Author(s): Ursula Braun (ubraun@linux.ibm.com) * * Copyright IBM Corp. 2017 * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef SMCTOOLS_COMMON_H #define SMCTOOLS_COMMON_H #define STRINGIFY_1(x) #x #define STRINGIFY(x) STRINGIFY_1(x) #define RELEASE_STRING "1.8.4" #define PF_SMC 43 #include /*********************************************************** * Mimic definitions in kernel/include/uapi/linux/smc.h ***********************************************************/ #define SMC_MAX_PNETID_LEN 16 /* Max. length of PNET id */ #define SMC_LGR_ID_SIZE 4 #define SMC_MAX_IBNAME 8 #define SMC_MAX_HOSTNAME_LEN 32 /* Max length of hostname */ #define SMC_MAX_EID_LEN 32 /* Max length of eid */ #define SMC_MAX_UEID 8 /* Max number of eids */ #define SMC_MAX_PORTS 2 /* Max # of ports per ib device */ #define SMC_PCI_ID_STR_LEN 16 /* Max length of pci id string */ /* Netlink SMC_PNETID attributes */ enum { SMC_PNETID_UNSPEC, SMC_PNETID_NAME, SMC_PNETID_ETHNAME, SMC_PNETID_IBNAME, SMC_PNETID_IBPORT, __SMC_PNETID_MAX, SMC_PNETID_MAX = __SMC_PNETID_MAX - 1 }; enum { /* SMC PNET Table commands */ SMC_PNETID_GET = 1, SMC_PNETID_ADD, SMC_PNETID_DEL, SMC_PNETID_FLUSH }; #define SMCR_GENL_FAMILY_NAME "SMC_PNETID" #define SMCR_GENL_FAMILY_VERSION 1 /* Use for accessing non-socket information like */ /* SMC links, linkgroups and devices */ #define SMC_GENL_FAMILY_NAME "SMC_GEN_NETLINK" #define SMC_GENL_FAMILY_VERSION 1 /* SMC_GENL_FAMILY commands */ enum { SMC_NETLINK_GET_SYS_INFO = 1, SMC_NETLINK_GET_LGR_SMCR, SMC_NETLINK_GET_LINK_SMCR, SMC_NETLINK_GET_LGR_SMCD, SMC_NETLINK_GET_DEV_SMCD, SMC_NETLINK_GET_DEV_SMCR, SMC_NETLINK_GET_STATS, SMC_NETLINK_GET_FBACK_STATS, SMC_NETLINK_DUMP_UEID, SMC_NETLINK_ADD_UEID, SMC_NETLINK_REMOVE_UEID, SMC_NETLINK_FLUSH_UEID, SMC_NETLINK_DUMP_SEID, SMC_NETLINK_ENABLE_SEID, SMC_NETLINK_DISABLE_SEID, }; /* SMC_GENL_FAMILY top level attributes */ enum { SMC_GEN_UNSPEC, SMC_GEN_SYS_INFO, /* nest */ SMC_GEN_LGR_SMCR, /* nest */ SMC_GEN_LINK_SMCR, /* nest */ SMC_GEN_LGR_SMCD, /* nest */ SMC_GEN_DEV_SMCD, /* nest */ SMC_GEN_DEV_SMCR, /* nest */ SMC_GEN_STATS, /* nest */ SMC_GEN_FBACK_STATS, /* nest */ __SMC_GEN_MAX, SMC_GEN_MAX = __SMC_GEN_MAX - 1 }; /* SMC_GEN_SYS_INFO attributes */ enum { SMC_NLA_SYS_UNSPEC, SMC_NLA_SYS_VER, /* u8 */ SMC_NLA_SYS_REL, /* u8 */ SMC_NLA_SYS_IS_ISM_V2, /* u8 */ SMC_NLA_SYS_LOCAL_HOST, /* string */ SMC_NLA_SYS_SEID, /* string */ SMC_NLA_SYS_IS_SMCR_V2, /* u8 */ __SMC_NLA_SYS_MAX, SMC_NLA_SYS_MAX = __SMC_NLA_SYS_MAX - 1 }; /* SMC_NLA_LGR_D_V2_COMMON and SMC_NLA_LGR_R_V2_COMMON nested attributes */ enum { SMC_NLA_LGR_V2_VER, /* u8 */ SMC_NLA_LGR_V2_REL, /* u8 */ SMC_NLA_LGR_V2_OS, /* u8 */ SMC_NLA_LGR_V2_NEG_EID, /* string */ SMC_NLA_LGR_V2_PEER_HOST, /* string */ __SMC_NLA_LGR_V2_MAX, SMC_NLA_LGR_V2_MAX = __SMC_NLA_LGR_V2_MAX - 1 }; /* SMC_NLA_LGR_R_V2 nested attributes */ enum { SMC_NLA_LGR_R_V2_UNSPEC, SMC_NLA_LGR_R_V2_DIRECT, /* u8 */ __SMC_NLA_LGR_R_V2_MAX, SMC_NLA_LGR_R_V2_MAX = __SMC_NLA_LGR_R_V2_MAX - 1 }; /* SMC_GEN_LGR_SMCR attributes */ enum { SMC_NLA_LGR_R_UNSPEC, SMC_NLA_LGR_R_ID, /* u32 */ SMC_NLA_LGR_R_ROLE, /* u8 */ SMC_NLA_LGR_R_TYPE, /* u8 */ SMC_NLA_LGR_R_PNETID, /* string */ SMC_NLA_LGR_R_VLAN_ID, /* u8 */ SMC_NLA_LGR_R_CONNS_NUM, /* u32 */ SMC_NLA_LGR_R_V2_COMMON, /* nest */ SMC_NLA_LGR_R_V2, /* nest */ SMC_NLA_LGR_R_NET_COOKIE, /* u64 */ SMC_NLA_LGR_R_PAD, /* flag */ SMC_NLA_LGR_R_BUF_TYPE, /* u8 */ SMC_NLA_LGR_R_SNDBUF_ALLOC, /* uint */ SMC_NLA_LGR_R_RMB_ALLOC, /* uint */ __SMC_NLA_LGR_R_MAX, SMC_NLA_LGR_R_MAX = __SMC_NLA_LGR_R_MAX - 1 }; /* SMC_GEN_LINK_SMCR attributes */ enum { SMC_NLA_LINK_UNSPEC, SMC_NLA_LINK_ID, /* u8 */ SMC_NLA_LINK_IB_DEV, /* string */ SMC_NLA_LINK_IB_PORT, /* u8 */ SMC_NLA_LINK_GID, /* string */ SMC_NLA_LINK_PEER_GID, /* string */ SMC_NLA_LINK_CONN_CNT, /* u32 */ SMC_NLA_LINK_NET_DEV, /* string */ SMC_NLA_LINK_UID, /* u32 */ SMC_NLA_LINK_PEER_UID, /* u32 */ SMC_NLA_LINK_STATE, /* u32 */ __SMC_NLA_LINK_MAX, SMC_NLA_LINK_MAX = __SMC_NLA_LINK_MAX - 1 }; /* SMC_GEN_LGR_SMCD attributes */ enum { SMC_NLA_LGR_D_UNSPEC, SMC_NLA_LGR_D_ID, /* u32 */ SMC_NLA_LGR_D_GID, /* u64 */ SMC_NLA_LGR_D_PEER_GID, /* u64 */ SMC_NLA_LGR_D_VLAN_ID, /* u8 */ SMC_NLA_LGR_D_CONNS_NUM, /* u32 */ SMC_NLA_LGR_D_PNETID, /* string */ SMC_NLA_LGR_D_CHID, /* u16 */ SMC_NLA_LGR_D_PAD, /* flag */ SMC_NLA_LGR_D_V2_COMMON, /* nest */ SMC_NLA_LGR_D_EXT_GID, /* u64 */ SMC_NLA_LGR_D_PEER_EXT_GID, /* u64 */ SMC_NLA_LGR_D_SNDBUF_ALLOC, /* uint */ SMC_NLA_LGR_D_DMB_ALLOC, /* uint */ __SMC_NLA_LGR_D_MAX, SMC_NLA_LGR_D_MAX = __SMC_NLA_LGR_D_MAX - 1 }; /* SMC_NLA_DEV_PORT nested attributes */ enum { SMC_NLA_DEV_PORT_UNSPEC, SMC_NLA_DEV_PORT_PNET_USR, /* u8 */ SMC_NLA_DEV_PORT_PNETID, /* string */ SMC_NLA_DEV_PORT_NETDEV, /* string */ SMC_NLA_DEV_PORT_STATE, /* u8 */ SMC_NLA_DEV_PORT_VALID, /* u8 */ SMC_NLA_DEV_PORT_LNK_CNT, /* u32 */ __SMC_NLA_DEV_PORT_MAX, SMC_NLA_DEV_PORT_MAX = __SMC_NLA_DEV_PORT_MAX - 1 }; /* SMC_GEN_DEV_SMCD and SMC_GEN_DEV_SMCR attributes */ enum { SMC_NLA_DEV_UNSPEC, SMC_NLA_DEV_USE_CNT, /* u32 */ SMC_NLA_DEV_IS_CRIT, /* u8 */ SMC_NLA_DEV_PCI_FID, /* u32 */ SMC_NLA_DEV_PCI_CHID, /* u16 */ SMC_NLA_DEV_PCI_VENDOR, /* u16 */ SMC_NLA_DEV_PCI_DEVICE, /* u16 */ SMC_NLA_DEV_PCI_ID, /* string */ SMC_NLA_DEV_PORT, /* nest */ SMC_NLA_DEV_PORT2, /* nest */ SMC_NLA_DEV_IB_NAME, /* string */ __SMC_NLA_DEV_MAX, SMC_NLA_DEV_MAX = __SMC_NLA_DEV_MAX - 1 }; /* SMC_NLA_STATS_T_TX(RX)_RMB_SIZE nested attributes */ /* SMC_NLA_STATS_TX(RX)PLOAD_SIZE nested attributes */ enum { SMC_NLA_STATS_PLOAD_PAD, SMC_NLA_STATS_PLOAD_8K, /* u64 */ SMC_NLA_STATS_PLOAD_16K, /* u64 */ SMC_NLA_STATS_PLOAD_32K, /* u64 */ SMC_NLA_STATS_PLOAD_64K, /* u64 */ SMC_NLA_STATS_PLOAD_128K, /* u64 */ SMC_NLA_STATS_PLOAD_256K, /* u64 */ SMC_NLA_STATS_PLOAD_512K, /* u64 */ SMC_NLA_STATS_PLOAD_1024K, /* u64 */ SMC_NLA_STATS_PLOAD_G_1024K, /* u64 */ __SMC_NLA_STATS_PLOAD_MAX, SMC_NLA_STATS_PLOAD_MAX = __SMC_NLA_STATS_PLOAD_MAX - 1 }; /* SMC_NLA_STATS_T_TX(RX)_RMB_STATS nested attributes */ enum { SMC_NLA_STATS_RMB_PAD, SMC_NLA_STATS_RMB_SIZE_SM_PEER_CNT, /* u64 */ SMC_NLA_STATS_RMB_SIZE_SM_CNT, /* u64 */ SMC_NLA_STATS_RMB_FULL_PEER_CNT, /* u64 */ SMC_NLA_STATS_RMB_FULL_CNT, /* u64 */ SMC_NLA_STATS_RMB_REUSE_CNT, /* u64 */ SMC_NLA_STATS_RMB_ALLOC_CNT, /* u64 */ SMC_NLA_STATS_RMB_DGRADE_CNT, /* u64 */ __SMC_NLA_STATS_RMB_MAX, SMC_NLA_STATS_RMB_MAX = __SMC_NLA_STATS_RMB_MAX - 1 }; /* SMC_NLA_STATS_SMCD_TECH and _SMCR_TECH nested attributes */ enum { SMC_NLA_STATS_T_PAD, SMC_NLA_STATS_T_TX_RMB_SIZE, /* nest */ SMC_NLA_STATS_T_RX_RMB_SIZE, /* nest */ SMC_NLA_STATS_T_TXPLOAD_SIZE, /* nest */ SMC_NLA_STATS_T_RXPLOAD_SIZE, /* nest */ SMC_NLA_STATS_T_TX_RMB_STATS, /* nest */ SMC_NLA_STATS_T_RX_RMB_STATS, /* nest */ SMC_NLA_STATS_T_CLNT_V1_SUCC, /* u64 */ SMC_NLA_STATS_T_CLNT_V2_SUCC, /* u64 */ SMC_NLA_STATS_T_SRV_V1_SUCC, /* u64 */ SMC_NLA_STATS_T_SRV_V2_SUCC, /* u64 */ SMC_NLA_STATS_T_SENDPAGE_CNT, /* u64 */ SMC_NLA_STATS_T_SPLICE_CNT, /* u64 */ SMC_NLA_STATS_T_CORK_CNT, /* u64 */ SMC_NLA_STATS_T_NDLY_CNT, /* u64 */ SMC_NLA_STATS_T_URG_DATA_CNT, /* u64 */ SMC_NLA_STATS_T_RX_BYTES, /* u64 */ SMC_NLA_STATS_T_TX_BYTES, /* u64 */ SMC_NLA_STATS_T_RX_CNT, /* u64 */ SMC_NLA_STATS_T_TX_CNT, /* u64 */ SMC_NLA_STATS_T_RX_RMB_USAGE, /* uint */ SMC_NLA_STATS_T_TX_RMB_USAGE, /* uint */ __SMC_NLA_STATS_T_MAX, SMC_NLA_STATS_T_MAX = __SMC_NLA_STATS_T_MAX - 1 }; /* SMC_GEN_STATS attributes */ enum { SMC_NLA_STATS_PAD, SMC_NLA_STATS_SMCD_TECH, /* nest */ SMC_NLA_STATS_SMCR_TECH, /* nest */ SMC_NLA_STATS_CLNT_HS_ERR_CNT, /* u64 */ SMC_NLA_STATS_SRV_HS_ERR_CNT, /* u64 */ __SMC_NLA_STATS_MAX, SMC_NLA_STATS_MAX = __SMC_NLA_STATS_MAX - 1 }; /* SMC_GEN_FBACK_STATS attributes */ enum { SMC_NLA_FBACK_STATS_PAD, SMC_NLA_FBACK_STATS_TYPE, /* u8 */ SMC_NLA_FBACK_STATS_SRV_CNT, /* u64 */ SMC_NLA_FBACK_STATS_CLNT_CNT, /* u64 */ SMC_NLA_FBACK_STATS_RSN_CODE, /* u32 */ SMC_NLA_FBACK_STATS_RSN_CNT, /* u16 */ __SMC_NLA_FBACK_STATS_MAX, SMC_NLA_FBACK_STATS_MAX = __SMC_NLA_FBACK_STATS_MAX - 1 }; /* SMC_NETLINK_UEID attributes */ enum { SMC_NLA_EID_TABLE_UNSPEC, SMC_NLA_EID_TABLE_ENTRY, /* string */ __SMC_NLA_EID_TABLE_MAX, SMC_NLA_EID_TABLE_MAX = __SMC_NLA_EID_TABLE_MAX - 1 }; /* SMC_NETLINK_SEID attributes */ enum { SMC_NLA_SEID_UNSPEC, SMC_NLA_SEID_ENTRY, /* string */ SMC_NLA_SEID_ENABLED, /* u8 */ __SMC_NLA_SEID_TABLE_MAX, SMC_NLA_SEID_TABLE_MAX = __SMC_NLA_SEID_TABLE_MAX - 1 }; /*********************************************************** * Mimic definitions in kernel/include/uapi/linux/smc_diag.h ***********************************************************/ #include #include #include #define SMC_DIAG_EXTS_PER_CMD 16 /* Sequence numbers */ enum { MAGIC_SEQ = 123456, MAGIC_SEQ_V2, MAGIC_SEQ_V2_ACK, }; /* Request structure */ struct smc_diag_req { __u8 diag_family; __u8 pad[2]; __u8 diag_ext; /* Query extended information */ struct inet_diag_sockid id; }; /* Request structure v2 */ struct smc_diag_req_v2 { __u8 diag_family; __u8 pad[2]; __u8 diag_ext; /* Query extended information */ struct inet_diag_sockid id; __u32 cmd; __u32 cmd_ext; __u8 cmd_val[8]; }; /* Base info structure. It contains socket identity (addrs/ports/cookie) based * on the internal clcsock, and more SMC-related socket data */ struct smc_diag_msg { __u8 diag_family; __u8 diag_state; __u8 diag_mode; __u8 diag_shutdown; struct inet_diag_sockid id; __u32 diag_uid; __u64 diag_inode; }; /* Mode of a connection */ enum { SMC_DIAG_MODE_SMCR, SMC_DIAG_MODE_FALLBACK_TCP, SMC_DIAG_MODE_SMCD, }; /* GET_SOCK_DIAG command extensions */ enum { SMC_DIAG_NONE, SMC_DIAG_CONNINFO, SMC_DIAG_LGRINFO, SMC_DIAG_SHUTDOWN, SMC_DIAG_DMBINFO, SMC_DIAG_FALLBACK, __SMC_DIAG_MAX, }; /* V2 Commands */ enum { SMC_DIAG_GET_LGR_INFO = SMC_DIAG_EXTS_PER_CMD, SMC_DIAG_GET_DEV_INFO, SMC_DIAG_GET_SYS_INFO, __SMC_DIAG_EXT_MAX, }; /* SMC_DIAG_GET_LGR_INFO command extensions */ enum { SMC_DIAG_LGR_INFO_SMCR = 1, SMC_DIAG_LGR_INFO_SMCR_LINK, SMC_DIAG_LGR_INFO_SMCD, }; /* SMC_DIAG_GET_DEV_INFO command extensions */ enum { SMC_DIAG_DEV_INFO_SMCD = 1, SMC_DIAG_DEV_INFO_SMCR, }; /* SMC_DIAG_GET_SYS_INFO command extensions */ enum { SMC_DIAG_SYS_INFO = 1, }; #define SMC_DIAG_MAX (__SMC_DIAG_MAX - 1) #define SMC_DIAG_EXT_MAX (__SMC_DIAG_EXT_MAX - 1) /* SMC_DIAG_CONNINFO */ #define IB_DEVICE_NAME_MAX 64 struct smc_diag_cursor { __u16 reserved; __u16 wrap; __u32 count; }; struct smc_diag_conninfo { __u32 token; /* unique connection id */ __u32 sndbuf_size; /* size of send buffer */ __u32 rmbe_size; /* size of RMB element */ __u32 peer_rmbe_size; /* size of peer RMB element */ /* local RMB element cursors */ struct smc_diag_cursor rx_prod; /* received producer cursor */ struct smc_diag_cursor rx_cons; /* received consumer cursor */ /* peer RMB element cursors */ struct smc_diag_cursor tx_prod; /* sent producer cursor */ struct smc_diag_cursor tx_cons; /* sent consumer cursor */ __u8 rx_prod_flags; /* received producer flags */ __u8 rx_conn_state_flags; /* recvd connection flags*/ __u8 tx_prod_flags; /* sent producer flags */ __u8 tx_conn_state_flags; /* sent connection flags*/ /* send buffer cursors */ struct smc_diag_cursor tx_prep; /* prepared to be sent cursor */ struct smc_diag_cursor tx_sent; /* sent cursor */ struct smc_diag_cursor tx_fin; /* confirmed sent cursor */ }; struct smc_v2_lgr_info { __u8 v2_lgr_info_received; __u8 smc_version; __u8 peer_smc_release; __u8 peer_os; /* peer operating system */ __u8 negotiated_eid[SMC_MAX_EID_LEN + 1]; __u8 peer_hostname[SMC_MAX_HOSTNAME_LEN + 1]; /* SMC-R v2 specific */ __u8 smcr_direct; }; /* unused struct smc_system_info { __u8 smc_version; __u8 smc_release; __u8 ueid_count; __u8 smc_ism_is_v2; __u32 reserved; __u8 local_hostname[SMC_MAX_HOSTNAME_LEN]; __u8 seid[SMC_MAX_EID_LEN]; __u8 ueid[SMC_MAX_UEID][SMC_MAX_EID_LEN]; }; */ /* SMC_DIAG_LINKINFO */ struct smc_diag_linkinfo { __u8 link_id; /* link identifier */ __u8 ibname[IB_DEVICE_NAME_MAX]; /* name of the RDMA device */ __u8 ibport; /* RDMA device port number */ __u8 gid[40]; /* local GID */ __u8 peer_gid[40]; /* peer GID */ }; struct smc_diag_linkinfo_v2 { struct smc_diag_linkinfo v1; __u32 conn_cnt; __u8 netdev[IFNAMSIZ]; __u8 link_uid[4]; __u8 peer_link_uid[4]; __u32 link_state; }; struct smc_diag_lgrinfo { struct smc_diag_linkinfo lnk[1]; __u8 role; }; struct smc_diag_fallback { __u32 reason; __u32 peer_diagnosis; }; struct smcd_diag_dmbinfo { /* SMC-D Socket internals */ __u32 linkid; /* Link identifier */ __u64 peer_gid; /* Peer GID */ __u64 my_gid; /* My GID */ __u64 token; /* Token of DMB */ __u64 peer_token; /* Token of remote DMBE */ }; struct smcd_diag_dmbinfo_v2 { struct smcd_diag_dmbinfo v1; __u8 pnet_id[SMC_MAX_PNETID_LEN]; __u32 conns_num; __u16 chid; __u8 vlan_id; __u64 sndbuf_alloc; __u64 dmb_alloc; struct smc_v2_lgr_info v2_lgr_info; }; struct smc_diag_dev_info { __u8 pnet_id[SMC_MAX_PORTS][SMC_MAX_PNETID_LEN]; __u8 pnetid_by_user[SMC_MAX_PORTS]; __u32 use_cnt; __u8 is_critical; __u32 pci_fid; __u16 pci_pchid; __u16 pci_vendor; __u16 pci_device; __u8 pci_id[SMC_PCI_ID_STR_LEN]; __u8 dev_name[IB_DEVICE_NAME_MAX]; __u8 netdev[SMC_MAX_PORTS][IFNAMSIZ]; __u8 port_state[SMC_MAX_PORTS]; __u8 port_valid[SMC_MAX_PORTS]; __u32 lnk_cnt_by_port[SMC_MAX_PORTS]; /* # lnks per port */ }; struct smc_diag_lgr { __u8 lgr_id[SMC_LGR_ID_SIZE]; __u8 lgr_role; __u8 lgr_type; __u8 pnet_id[SMC_MAX_PNETID_LEN]; __u8 vlan_id; __u32 conns_num; __u64 sndbuf_alloc; __u64 rmb_alloc; struct smc_v2_lgr_info v2_lgr_info; }; #endif /* SMCTOOLS_COMMON_H */ smc-tools-1.8.4/stats.c000066400000000000000000001017161473052164700147650ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2021 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "stats.h" #if defined(SMCD) static int is_smcd = 1; #else static int is_smcd = 0; #endif static int d_level = 0; static int is_abs = 0; static int show_cmd = 0; static int reset_cmd = 0; static int json_cmd = 0; static int cache_file_exists = 0; struct smc_stats smc_stat; /* kernel values, might contain merged values */ struct smc_stats smc_stat_c; /* cache file values */ struct smc_stats smc_stat_org; /* original kernel values */ struct smc_stats_rsn smc_rsn; struct smc_stats_rsn smc_rsn_c; struct smc_stats_rsn smc_rsn_org; FILE *cache_fp = NULL; char *cache_file_path = NULL; static char* j_output[65] = {"SMC_INT_TX_BUF_8K", "SMC_INT_TX_BUF_16K", "SMC_INT_TX_BUF_32K", "SMC_INT_TX_BUF_64K", "SMC_INT_TX_BUF_128K", "SMC_INT_TX_BUF_256K", "SMC_INT_TX_BUF_512K", "SMC_INT_TX_BUF_1024K", "SMC_INT_TX_BUF_G_1024K", "SMC_INT_RX_BUF_8K", "SMC_INT_RX_BUF_16K", "SMC_INT_RX_BUF_32K", "SMC_INT_RX_BUF_64K", "SMC_INT_RX_BUF_128K", "SMC_INT_RX_BUF_256K", "SMC_INT_RX_BUF_512K", "SMC_INT_RX_BUF_1024K", "SMC_INT_RX_BUF_G_1024K", "SMC_USR_TX_BUF_8K", "SMC_USR_TX_BUF_16K", "SMC_USR_TX_BUF_32K", "SMC_USR_TX_BUF_64K", "SMC_USR_TX_BUF_128K", "SMC_USR_TX_BUF_256K", "SMC_USR_TX_BUF_512K", "SMC_USR_TX_BUF_1024K", "SMC_USR_TX_BUF_G_1024K", "SMC_USR_RX_BUF_8K", "SMC_USR_RX_BUF_16K", "SMC_USR_RX_BUF_32K", "SMC_USR_RX_BUF_64K", "SMC_USR_RX_BUF_128K", "SMC_USR_RX_BUF_256K", "SMC_USR_RX_BUF_512K", "SMC_USR_RX_BUF_1024K", "SMC_USR_RX_BUF_G_1024K", "SMC_INT_TX_BUF_SIZE_SM_PEER_CNT", "SMC_INT_TX_BUF_SIZE_SM_CNT", "SMC_INT_TX_BUF_FULL_PEER_CNT", "SMC_INT_TX_BUF_FULL_CNT", "SMC_INT_TX_BUF_REUSE_CNT", "SMC_INT_TX_BUF_ALLOC_CNT", "SMC_INT_TX_BUF_DGRADE_CNT", "SMC_INT_RX_BUF_SIZE_SM_PEER_CNT", "SMC_INT_RX_BUF_SIZE_SM_CNT", "SMC_INT_RX_BUF_FULL_PEER_CNT", "SMC_INT_RX_BUF_FULL_CNT", "SMC_INT_RX_BUF_REUSE_CNT", "SMC_INT_RX_BUF_ALLOC_CNT", "SMC_INT_RX_BUF_DGRADE_CNT", "SMC_CLNT_V1_SUCC_CNT", "SMC_CLNT_V2_SUCC_CNT", "SMC_SRV_V1_SUCC_CNT", "SMC_SRV_V2_SUCC_CNT", "SMC_SENDPAGE_CNT", "SMC_URG_DATA_CNT", "SMC_SPLICE_CNT", "SMC_CORK_CNT", "SMC_NDLY_CNT", "SMC_RX_BYTES", "SMC_TX_BYTES", "SMC_RX_CNT", "SMC_TX_CNT", "SMC_RX_RMB_USAGE", "SMC_TX_RMB_USAGE" }; static struct nla_policy smc_gen_stats_policy[SMC_NLA_STATS_MAX + 1] = { [SMC_NLA_STATS_PAD] = { .type = NLA_UNSPEC }, [SMC_NLA_STATS_SMCD_TECH] = { .type = NLA_NESTED }, [SMC_NLA_STATS_SMCR_TECH] = { .type = NLA_NESTED }, [SMC_NLA_STATS_CLNT_HS_ERR_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_SRV_HS_ERR_CNT] = { .type = NLA_U64 }, }; static struct nla_policy smc_gen_stats_fback_policy[SMC_NLA_FBACK_STATS_MAX + 1] = { [SMC_NLA_FBACK_STATS_PAD] = { .type = NLA_UNSPEC }, [SMC_NLA_FBACK_STATS_TYPE] = { .type = NLA_U8 }, [SMC_NLA_FBACK_STATS_SRV_CNT] = { .type = NLA_U64 }, [SMC_NLA_FBACK_STATS_CLNT_CNT] = { .type = NLA_U64 }, [SMC_NLA_FBACK_STATS_RSN_CODE] = { .type = NLA_U32 }, [SMC_NLA_FBACK_STATS_RSN_CNT] = { .type = NLA_U16 }, }; static struct nla_policy smc_gen_stats_tech_policy[SMC_NLA_STATS_T_MAX + 1] = { [SMC_NLA_STATS_T_PAD] = { .type = NLA_UNSPEC }, [SMC_NLA_STATS_T_TX_RMB_SIZE] = { .type = NLA_NESTED }, [SMC_NLA_STATS_T_RX_RMB_SIZE] = { .type = NLA_NESTED }, [SMC_NLA_STATS_T_TXPLOAD_SIZE] = { .type = NLA_NESTED }, [SMC_NLA_STATS_T_RXPLOAD_SIZE] = { .type = NLA_NESTED }, [SMC_NLA_STATS_T_CLNT_V1_SUCC] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_CLNT_V2_SUCC] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_SRV_V1_SUCC] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_SRV_V2_SUCC] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_SENDPAGE_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_SPLICE_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_CORK_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_URG_DATA_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_NDLY_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_RX_BYTES] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_TX_BYTES] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_RX_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_T_TX_CNT] = { .type = NLA_U64 }, }; static struct nla_policy smc_gen_stats_rmb_policy[SMC_NLA_STATS_RMB_MAX + 1] = { [SMC_NLA_STATS_RMB_PAD] = { .type = NLA_UNSPEC }, [SMC_NLA_STATS_RMB_SIZE_SM_PEER_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_SIZE_SM_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_FULL_PEER_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_FULL_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_REUSE_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_ALLOC_CNT] = { .type = NLA_U64 }, [SMC_NLA_STATS_RMB_DGRADE_CNT] = { .type = NLA_U64 }, }; static struct nla_policy smc_gen_stats_pload_policy[SMC_NLA_STATS_PLOAD_MAX + 1] = { [SMC_NLA_STATS_PLOAD_PAD] = { .type = NLA_UNSPEC }, [SMC_NLA_STATS_PLOAD_8K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_16K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_32K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_64K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_128K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_256K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_512K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_1024K] = { .type = NLA_U64 }, [SMC_NLA_STATS_PLOAD_G_1024K] = { .type = NLA_U64 }, }; static void usage(void) { fprintf(stderr, #if defined(SMCD) "Usage: smcd stats [show | reset]\n" #elif defined(SMCR) "Usage: smcr stats [show | reset]\n" #else "Usage: smc stats [show | reset]\n" #endif ); exit(-1); } static char* get_fbackstr(int code) { char* str; switch (code) { case SMC_CLC_DECL_PEERNOSMC: str = "PEER_NO_SMC"; break; case SMC_CLC_DECL_MEM: str = "MEMORY"; break; case SMC_CLC_DECL_TIMEOUT_CL: str = "TIMEOUT_CL"; break; case SMC_CLC_DECL_TIMEOUT_AL: str = "TIMEOUT_AL"; break; case SMC_CLC_DECL_CNFERR: str = "CNF_ERR"; break; case SMC_CLC_DECL_IPSEC: str = "IPSEC"; break; case SMC_CLC_DECL_NOSMCDEV: str = "NOSMCDEV"; break; case SMC_CLC_DECL_NOSMCDDEV: str = "NOSMCDDEV"; break; case SMC_CLC_DECL_NOSMCRDEV: str = "NOSMCRDEV"; break; case SMC_CLC_DECL_NOISM2SUPP: str = "NOISM2SUPP"; break; case SMC_CLC_DECL_NOV2EXT: str = "NOV2EXT"; break; case SMC_CLC_DECL_PEERDECL: str = "PEERDECL"; break; case SMC_CLC_DECL_SYNCERR: str = "SYNCERR"; break; case SMC_CLC_DECL_MAX_DMB: str = "MAX_DMB"; break; case SMC_CLC_DECL_VERSMISMAT: str = "VERSMISMAT"; break; case SMC_CLC_DECL_NOSRVLINK: str = "NOSRVLINK"; break; case SMC_CLC_DECL_NOSEID: str = "NOSEID"; break; case SMC_CLC_DECL_NOSMCD2DEV: str = "NOSMCD2DEV"; break; case SMC_CLC_DECL_MODEUNSUPP: str = "MODEUNSUPP"; break; case SMC_CLC_DECL_RMBE_EC: str = "RMBE_EC"; break; case SMC_CLC_DECL_OPTUNSUPP: str = "OPTUNSUPP"; break; case SMC_CLC_DECL_DIFFPREFIX: str = "DIFFPREFIX"; break; case SMC_CLC_DECL_GETVLANERR: str = "GETVLANERR"; break; case SMC_CLC_DECL_ISMVLANERR: str = "ISMVLANERR"; break; case SMC_CLC_DECL_NOACTLINK: str = "NOACTLINK"; break; case SMC_CLC_DECL_NOV2DEXT: str = "NOV2DEXT"; break; default: str = "[unknown]"; break; } return str; } static void bubble_sort(struct smc_stats_fback *fback) { struct smc_stats_fback temp; int i, j; for (i = 0; i < SMC_MAX_FBACK_RSN_CNT; i++) { for (j = 0; j < SMC_MAX_FBACK_RSN_CNT - 1; j++) { if (fback[j + 1].count > fback[j].count) { temp = fback[j]; fback[j] = fback[j + 1]; fback[j + 1] = temp; } } } } static void print_fback_details(struct smc_stats_fback *fback, int is_srv) { int caption_printed = 0; char *fback_str = NULL; int i, count = 0; bubble_sort(fback); for (i = 0; i < SMC_MAX_FBACK_RSN_CNT; i++) { if (fback[i].fback_code != 0) { if (!caption_printed) { caption_printed = 1; if (is_srv) printf(" Server\n"); else printf(" Client\n"); } fback_str = get_fbackstr(fback[i].fback_code); printf(" %-12s %12d\n", fback_str, fback[i].count); count++; if (count == 3) break; } } } static void print_fbackstr() { struct smc_stats_fback *server, *client; server = smc_rsn.srv; client = smc_rsn.clnt; print_fback_details(server, 1); print_fback_details(client, 0); } static void fillbuffer(struct smc_stats_memsize *mem, char buf[][7]) { get_abbreviated(mem->buf[SMC_BUF_8K], 6, buf[SMC_BUF_8K]); get_abbreviated(mem->buf[SMC_BUF_16K], 6, buf[SMC_BUF_16K]); get_abbreviated(mem->buf[SMC_BUF_32K], 6, buf[SMC_BUF_32K]); get_abbreviated(mem->buf[SMC_BUF_64K], 6, buf[SMC_BUF_64K]); get_abbreviated(mem->buf[SMC_BUF_128K], 6, buf[SMC_BUF_128K]); get_abbreviated(mem->buf[SMC_BUF_256K], 6, buf[SMC_BUF_256K]); get_abbreviated(mem->buf[SMC_BUF_512K], 6, buf[SMC_BUF_512K]); get_abbreviated(mem->buf[SMC_BUF_G_1024K] + mem->buf[SMC_BUF_1024K], 6, buf[SMC_BUF_1024K]); } static void print_as_json() { int size, i; __u64 *src; size = sizeof(struct smc_stats_tech) / sizeof(__u64); if (is_smcd) { src = (__u64 *)&smc_stat.smc[SMC_TYPE_D]; printf("{\"SMCD\": {"); } else { src = (__u64 *)&smc_stat.smc[SMC_TYPE_R]; printf("{\"SMCR\": {"); } for (i = 0; i < size; i++) { printf("\"%s\":%llu",j_output[i] ,*src); if (i != size - 1) printf(","); src++; } printf("}}\n"); } static void print_as_text() { __u64 smc_conn_cnt = 0, special_calls = 0, total_req_cn = 0; __u64 total_conn = 0, fback_count = 0, hshake_err_cnt = 0; float buf_small = 0, buf_small_r = 0, buf_rx_full = 0; __u64 smc_c_cnt_v1 = 0, smc_c_cnt_v2 = 0; float buf_full = 0, buf_full_r = 0; struct smc_stats_tech *tech; float avg_req_p_conn = 0; char buf[SMC_BUF_MAX][7]; char temp_str[7]; int tech_type; if (is_smcd) { printf("SMC-D Connections Summary\n"); tech_type = SMC_TYPE_D; } else { printf("SMC-R Connections Summary\n"); tech_type = SMC_TYPE_R; } tech = &smc_stat.smc[tech_type]; smc_c_cnt_v1 = tech->clnt_v1_succ_cnt + tech->srv_v1_succ_cnt; smc_c_cnt_v2 = tech->clnt_v2_succ_cnt + tech->srv_v2_succ_cnt; smc_conn_cnt += tech->clnt_v1_succ_cnt; smc_conn_cnt += tech->clnt_v2_succ_cnt; smc_conn_cnt += tech->srv_v1_succ_cnt; smc_conn_cnt += tech->srv_v2_succ_cnt; total_conn += smc_conn_cnt; hshake_err_cnt += smc_stat.clnt_hshake_err_cnt; hshake_err_cnt += smc_stat.srv_hshake_err_cnt; total_conn += hshake_err_cnt; fback_count += smc_rsn.clnt_fback_cnt; fback_count += smc_rsn.srv_fback_cnt; total_conn += fback_count; total_req_cn = tech->rx_cnt + tech->tx_cnt; if (smc_conn_cnt) avg_req_p_conn = total_req_cn / (double)smc_conn_cnt; special_calls += tech->cork_cnt; special_calls += tech->ndly_cnt; special_calls += tech->sendpage_cnt; special_calls += tech->splice_cnt; special_calls += tech->urg_data_cnt; if (tech->tx_cnt) { buf_full = tech->rmb_tx.buf_full_cnt / (double)tech->tx_cnt * 100; buf_full_r = tech->rmb_tx.buf_full_peer_cnt / (double)tech->tx_cnt * 100; buf_small = tech->rmb_tx.buf_size_small_cnt / (double)tech->tx_cnt * 100; buf_small_r = tech->rmb_tx.buf_size_small_peer_cnt / (double)tech->tx_cnt * 100; } if (tech->rx_cnt) buf_rx_full = tech->rmb_rx.buf_full_cnt / (double)tech->rx_cnt * 100; printf(" Total connections handled %12llu\n", total_conn); if (d_level) { printf(" SMC connections %12llu (client %llu, server %llu)\n", smc_conn_cnt, tech->clnt_v1_succ_cnt + tech->clnt_v2_succ_cnt, tech->srv_v1_succ_cnt + tech->srv_v2_succ_cnt); printf(" v1 %12llu\n", smc_c_cnt_v1); printf(" v2 %12llu\n", smc_c_cnt_v2); } else { printf(" SMC connections %12llu\n", smc_conn_cnt); } if (d_level) { printf(" Handshake errors %12llu (client %llu, server %llu)\n", hshake_err_cnt, smc_stat.clnt_hshake_err_cnt, smc_stat.srv_hshake_err_cnt); } else { printf(" Handshake errors %12llu\n", hshake_err_cnt); } printf(" Avg requests per SMC conn %14.1f\n", avg_req_p_conn); if (d_level) { printf(" TCP fallback %12llu (client %llu, server %llu)\n", fback_count, smc_rsn.clnt_fback_cnt, smc_rsn.srv_fback_cnt); print_fbackstr(); } else { printf(" TCP fallback %12llu\n", fback_count); } printf("\n"); printf("RX Stats\n"); get_abbreviated(smc_stat.smc[tech_type].rx_bytes, 6, temp_str); printf(" Data transmitted (Bytes) %14llu (%s)\n", smc_stat.smc[tech_type].rx_bytes, temp_str); printf(" Total requests %12llu\n", tech->rx_cnt); get_abbreviated(smc_stat.smc[tech_type].rx_rmbuse, 6, temp_str); printf(" Buffer usage (Bytes) %12llu (%s)\n", tech->rx_rmbuse, temp_str); printf(" Buffer full %12llu (%.2f%%)\n", tech->rmb_rx.buf_full_cnt, buf_rx_full); if (d_level) { printf(" Buffer downgrades %12llu\n", tech->rmb_rx.dgrade_cnt); printf(" Buffer reuses %12llu\n", tech->rmb_rx.reuse_cnt); } fillbuffer(&tech->rx_rmbsize, buf); printf(" 8KB 16KB 32KB 64KB 128KB 256KB 512KB >512KB\n"); printf(" Bufs %6s %6s %6s %6s %6s %6s %6s %6s\n", buf[SMC_BUF_8K], buf[SMC_BUF_16K], buf[SMC_BUF_32K], buf[SMC_BUF_64K], buf[SMC_BUF_128K], buf[SMC_BUF_256K], buf[SMC_BUF_512K], buf[SMC_BUF_1024K]); fillbuffer(&tech->rx_pd, buf); printf(" Reqs %6s %6s %6s %6s %6s %6s %6s %6s\n", buf[SMC_BUF_8K], buf[SMC_BUF_16K], buf[SMC_BUF_32K], buf[SMC_BUF_64K], buf[SMC_BUF_128K], buf[SMC_BUF_256K], buf[SMC_BUF_512K], buf[SMC_BUF_1024K]); printf("\n"); printf("TX Stats\n"); get_abbreviated(smc_stat.smc[tech_type].tx_bytes, 6, temp_str); printf(" Data transmitted (Bytes) %14llu (%s)\n", smc_stat.smc[tech_type].tx_bytes, temp_str); printf(" Total requests %12llu\n", tech->tx_cnt); get_abbreviated(smc_stat.smc[tech_type].tx_rmbuse, 6, temp_str); printf(" Buffer usage (Bytes) %12llu (%s)\n", tech->tx_rmbuse, temp_str); printf(" Buffer full %12llu (%.2f%%)\n", tech->rmb_tx.buf_full_cnt, buf_full); printf(" Buffer full (remote) %12llu (%.2f%%)\n", tech->rmb_tx.buf_full_peer_cnt, buf_full_r); printf(" Buffer too small %12llu (%.2f%%)\n", tech->rmb_tx.buf_size_small_cnt, buf_small); printf(" Buffer too small (remote) %12llu (%.2f%%)\n", tech->rmb_tx.buf_size_small_peer_cnt, buf_small_r); if (d_level) { printf(" Buffer downgrades %12llu\n", tech->rmb_tx.dgrade_cnt); printf(" Buffer reuses %12llu\n", tech->rmb_tx.reuse_cnt); } fillbuffer(&tech->tx_rmbsize, buf); printf(" 8KB 16KB 32KB 64KB 128KB 256KB 512KB >512KB\n"); printf(" Bufs %6s %6s %6s %6s %6s %6s %6s %6s\n", buf[SMC_BUF_8K], buf[SMC_BUF_16K], buf[SMC_BUF_32K], buf[SMC_BUF_64K], buf[SMC_BUF_128K], buf[SMC_BUF_256K], buf[SMC_BUF_512K], buf[SMC_BUF_1024K]); fillbuffer(&tech->tx_pd, buf); printf(" Reqs %6s %6s %6s %6s %6s %6s %6s %6s\n", buf[SMC_BUF_8K], buf[SMC_BUF_16K], buf[SMC_BUF_32K], buf[SMC_BUF_64K], buf[SMC_BUF_128K], buf[SMC_BUF_256K], buf[SMC_BUF_512K], buf[SMC_BUF_1024K]); printf("\n"); printf("Extras\n"); printf(" Special socket calls %12llu\n", special_calls); if (d_level) { printf(" cork %12llu\n", tech->cork_cnt); printf(" nodelay %12llu\n", tech->ndly_cnt); printf(" sendpage %12llu\n", tech->sendpage_cnt); printf(" splice %12llu\n", tech->splice_cnt); printf(" urgent data %12llu\n", tech->urg_data_cnt); } } static int show_tech_pload_info(struct nlattr **attr, int type, int direction) { struct nlattr *tech_pload_attrs[SMC_NLA_STATS_PLOAD_MAX + 1]; struct smc_stats_memsize *tmp_memsize; uint64_t trgt = 0; int rc = NL_OK; int tech_type; if (type == SMC_NLA_STATS_SMCD_TECH) tech_type = SMC_TYPE_D; else tech_type = SMC_TYPE_R; if (direction == SMC_NLA_STATS_T_TXPLOAD_SIZE) tmp_memsize = &smc_stat.smc[tech_type].tx_pd; else if (direction == SMC_NLA_STATS_T_RXPLOAD_SIZE) tmp_memsize = &smc_stat.smc[tech_type].rx_pd; else if (direction == SMC_NLA_STATS_T_TX_RMB_SIZE) tmp_memsize = &smc_stat.smc[tech_type].tx_rmbsize; else if (direction == SMC_NLA_STATS_T_RX_RMB_SIZE) tmp_memsize = &smc_stat.smc[tech_type].rx_rmbsize; else return NL_STOP; if (nla_parse_nested(tech_pload_attrs, SMC_NLA_STATS_PLOAD_MAX, attr[direction], smc_gen_stats_pload_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_stats_pload_policy\n"); return NL_STOP; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_8K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_8K]); tmp_memsize->buf[SMC_BUF_8K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_16K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_16K]); tmp_memsize->buf[SMC_BUF_16K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_32K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_32K]); tmp_memsize->buf[SMC_BUF_32K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_64K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_64K]); tmp_memsize->buf[SMC_BUF_64K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_128K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_128K]); tmp_memsize->buf[SMC_BUF_128K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_256K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_256K]); tmp_memsize->buf[SMC_BUF_256K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_512K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_512K]); tmp_memsize->buf[SMC_BUF_512K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_1024K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_1024K]); tmp_memsize->buf[SMC_BUF_1024K] = trgt; } if (tech_pload_attrs[SMC_NLA_STATS_PLOAD_G_1024K]) { trgt = nla_get_u64(tech_pload_attrs[SMC_NLA_STATS_PLOAD_G_1024K]); tmp_memsize->buf[SMC_BUF_G_1024K] = trgt; } return rc; } static int show_tech_rmb_info(struct nlattr **attr, int type, int direction) { struct nlattr *tech_rmb_attrs[SMC_NLA_STATS_RMB_MAX + 1]; struct smc_stats_rmbcnt *tmp_rmb_stats; uint64_t trgt = 0; int rc = NL_OK; int tech_type; if (type == SMC_NLA_STATS_SMCD_TECH) tech_type = SMC_TYPE_D; else tech_type = SMC_TYPE_R; if (direction == SMC_NLA_STATS_T_TX_RMB_STATS) tmp_rmb_stats = &smc_stat.smc[tech_type].rmb_tx; else tmp_rmb_stats = &smc_stat.smc[tech_type].rmb_rx; if (nla_parse_nested(tech_rmb_attrs, SMC_NLA_STATS_RMB_MAX, attr[direction], smc_gen_stats_rmb_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_stats_rmb_policy\n"); return NL_STOP; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_REUSE_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_REUSE_CNT]); tmp_rmb_stats->reuse_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_ALLOC_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_ALLOC_CNT]); tmp_rmb_stats->alloc_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_SIZE_SM_PEER_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_SIZE_SM_PEER_CNT]); tmp_rmb_stats->buf_size_small_peer_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_SIZE_SM_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_SIZE_SM_CNT]); tmp_rmb_stats->buf_size_small_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_FULL_PEER_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_FULL_PEER_CNT]); tmp_rmb_stats->buf_full_peer_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_FULL_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_FULL_CNT]); tmp_rmb_stats->buf_full_cnt = trgt; } if (tech_rmb_attrs[SMC_NLA_STATS_RMB_DGRADE_CNT]) { trgt = nla_get_u64(tech_rmb_attrs[SMC_NLA_STATS_RMB_DGRADE_CNT]); tmp_rmb_stats->dgrade_cnt = trgt; } return rc; } static int fill_tech_info(struct nlattr **attr, int type) { struct nlattr *tech_attrs[SMC_NLA_STATS_T_MAX + 1]; uint64_t trgt = 0; int tech_type; if (type == SMC_NLA_STATS_SMCD_TECH) tech_type = SMC_TYPE_D; else tech_type = SMC_TYPE_R; if (nla_parse_nested(tech_attrs, SMC_NLA_STATS_T_MAX, attr[type], smc_gen_stats_tech_policy)) { fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_stats_fback_policy\n"); return NL_STOP; } if (tech_attrs[SMC_NLA_STATS_T_SRV_V1_SUCC]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_SRV_V1_SUCC]); smc_stat.smc[tech_type].srv_v1_succ_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_SRV_V2_SUCC]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_SRV_V2_SUCC]); smc_stat.smc[tech_type].srv_v2_succ_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_CLNT_V1_SUCC]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_CLNT_V1_SUCC]); smc_stat.smc[tech_type].clnt_v1_succ_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_CLNT_V2_SUCC]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_CLNT_V2_SUCC]); smc_stat.smc[tech_type].clnt_v2_succ_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_RX_BYTES]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_RX_BYTES]); smc_stat.smc[tech_type].rx_bytes = trgt; } if (tech_attrs[SMC_NLA_STATS_T_TX_BYTES]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_TX_BYTES]); smc_stat.smc[tech_type].tx_bytes = trgt; } if (tech_attrs[SMC_NLA_STATS_T_RX_RMB_USAGE]) { trgt = nl_attr_get_uint(tech_attrs[SMC_NLA_STATS_T_RX_RMB_USAGE]); smc_stat.smc[tech_type].rx_rmbuse = trgt; } if (tech_attrs[SMC_NLA_STATS_T_TX_RMB_USAGE]) { trgt = nl_attr_get_uint(tech_attrs[SMC_NLA_STATS_T_TX_RMB_USAGE]); smc_stat.smc[tech_type].tx_rmbuse = trgt; } if (tech_attrs[SMC_NLA_STATS_T_RX_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_RX_CNT]); smc_stat.smc[tech_type].rx_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_TX_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_TX_CNT]); smc_stat.smc[tech_type].tx_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_SENDPAGE_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_SENDPAGE_CNT]); smc_stat.smc[tech_type].sendpage_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_SPLICE_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_SPLICE_CNT]); smc_stat.smc[tech_type].splice_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_CORK_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_CORK_CNT]); smc_stat.smc[tech_type].cork_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_NDLY_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_NDLY_CNT]); smc_stat.smc[tech_type].ndly_cnt = trgt; } if (tech_attrs[SMC_NLA_STATS_T_URG_DATA_CNT]) { trgt = nla_get_u64(tech_attrs[SMC_NLA_STATS_T_URG_DATA_CNT]); smc_stat.smc[tech_type].urg_data_cnt = trgt; } if (show_tech_rmb_info(tech_attrs, type, SMC_NLA_STATS_T_TX_RMB_STATS) != NL_OK) goto errout; if (show_tech_rmb_info(tech_attrs, type, SMC_NLA_STATS_T_RX_RMB_STATS) != NL_OK) goto errout; if (show_tech_pload_info(tech_attrs, type, SMC_NLA_STATS_T_TXPLOAD_SIZE) != NL_OK) goto errout; if (show_tech_pload_info(tech_attrs, type, SMC_NLA_STATS_T_RXPLOAD_SIZE) != NL_OK) goto errout; if (show_tech_pload_info(tech_attrs, type, SMC_NLA_STATS_T_TX_RMB_SIZE) != NL_OK) goto errout; if (show_tech_pload_info(tech_attrs, type, SMC_NLA_STATS_T_RX_RMB_SIZE) != NL_OK) goto errout; return NL_OK; errout: return NL_STOP; } static int handle_gen_stats_reply(struct nl_msg *msg, void *arg) { struct nlattr *stats_attrs[SMC_NLA_STATS_MAX + 1]; struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); uint64_t trgt = 0; int rc = NL_OK; if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "%s: invalid data returned\n", "smc"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_STATS]) return NL_STOP; if (nla_parse_nested(stats_attrs, SMC_NLA_STATS_MAX, attrs[SMC_GEN_STATS], smc_gen_stats_policy)) { fprintf(stderr, "failed to parse nested attributes!\n"); return NL_STOP; } memset(&smc_stat, 0, sizeof(smc_stat)); if (stats_attrs[SMC_NLA_STATS_CLNT_HS_ERR_CNT]) { trgt = nla_get_u64(stats_attrs[SMC_NLA_STATS_CLNT_HS_ERR_CNT]); smc_stat.clnt_hshake_err_cnt = trgt; } if (stats_attrs[SMC_NLA_STATS_SRV_HS_ERR_CNT]) { trgt = nla_get_u64(stats_attrs[SMC_NLA_STATS_SRV_HS_ERR_CNT]); smc_stat.srv_hshake_err_cnt = trgt; } if (stats_attrs[SMC_NLA_STATS_SMCR_TECH]) rc = fill_tech_info(&stats_attrs[0], SMC_NLA_STATS_SMCR_TECH); if (stats_attrs[SMC_NLA_STATS_SMCD_TECH]) rc = fill_tech_info(&stats_attrs[0], SMC_NLA_STATS_SMCD_TECH); return rc; } static int fback_array_last_pos(struct smc_stats_fback *fback) { int k; for (k = 0; k < SMC_MAX_FBACK_RSN_CNT; k++) if (fback[k].fback_code == 0) return k; return SMC_MAX_FBACK_RSN_CNT - 1; } static int handle_gen_fback_stats_reply(struct nl_msg *msg, void *arg) { struct nlattr *stats_fback_attrs[SMC_NLA_FBACK_STATS_MAX + 1]; struct nlattr *attrs[SMC_GEN_MAX + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); struct smc_stats_fback *smc_fback; unsigned short type = 0, last_pos; uint64_t trgt64 = 0; int rc = NL_OK; int trgt = 0; if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, (struct nla_policy *)smc_gen_net_policy) < 0) { fprintf(stderr, "%s: invalid data returned\n", "smc"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_GEN_FBACK_STATS]) return NL_STOP; if (nla_parse_nested(stats_fback_attrs, SMC_NLA_FBACK_STATS_MAX, attrs[SMC_GEN_FBACK_STATS], smc_gen_stats_fback_policy)) { fprintf(stderr, "failed to parse nested attributes!\n"); return NL_STOP; } if (stats_fback_attrs[SMC_NLA_FBACK_STATS_SRV_CNT]) { trgt64 = nla_get_u64(stats_fback_attrs[SMC_NLA_FBACK_STATS_SRV_CNT]); smc_rsn.srv_fback_cnt = trgt64; } if (stats_fback_attrs[SMC_NLA_FBACK_STATS_SRV_CNT]) { trgt64 = nla_get_u64(stats_fback_attrs[SMC_NLA_FBACK_STATS_CLNT_CNT]); smc_rsn.clnt_fback_cnt = trgt64; } if (stats_fback_attrs[SMC_NLA_FBACK_STATS_TYPE]) { type = nla_get_u8(stats_fback_attrs[SMC_NLA_FBACK_STATS_TYPE]); if (type) smc_fback = smc_rsn.srv; else smc_fback = smc_rsn.clnt; last_pos = fback_array_last_pos(smc_fback); if (stats_fback_attrs[SMC_NLA_FBACK_STATS_RSN_CODE]) { trgt = nla_get_u32(stats_fback_attrs[SMC_NLA_FBACK_STATS_RSN_CODE]); smc_fback[last_pos].fback_code = trgt; } if (stats_fback_attrs[SMC_NLA_FBACK_STATS_RSN_CNT]) { trgt = nla_get_u16(stats_fback_attrs[SMC_NLA_FBACK_STATS_RSN_CNT]); smc_fback[last_pos].count = trgt; } } return rc; } static void handle_cmd_params(int argc, char **argv) { memset(&smc_rsn, 0, sizeof(smc_rsn)); if (argc == 0) { show_cmd = 1; /* no object given, so use the default "show" */ return; } while (1) { if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "show") == 0) { show_cmd = 1; break; } else if (contains(argv[0], "reset") == 0) { reset_cmd = 1; break; } else if (contains(argv[0], "json") == 0) { json_cmd = 1; break; }else { usage(); } if (!NEXT_ARG_OK()) break; NEXT_ARG(); } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); } static void read_cache_file(FILE *fp) { int count = 0, idx = 0, rc, size_fback = 0; int size, val_err, val_cnt; unsigned long long val; __u64 *trgt, *fbck_cnt; char buf[4096]; int *trgt_fbck; /* size without fallback reasons */ size = sizeof(smc_stat_c) / sizeof(__u64); trgt = (__u64 *)&smc_stat_c; size_fback = size + 2*SMC_MAX_FBACK_RSN_CNT; trgt_fbck = (int *)&smc_rsn_c; fbck_cnt = (__u64 *)&smc_rsn_c.srv_fback_cnt; while (fgets(buf, sizeof(buf), fp) != NULL) { if (count < size) { rc = sscanf(buf, "%d%llu", &idx, &val); if (rc < 2) { perror("Error: parsing cache file(stats)"); exit(-1); } if (idx != count) { perror("Error: unexpected value in cache file"); exit(-1); } *trgt = val; trgt++; } else if (count < size_fback) { rc = sscanf(buf, "%d%d%d", &idx, &val_err, &val_cnt); if (rc < 3) { perror("Error: parsing cache file (fback stats)"); exit(-1); } if (idx > SMC_MAX_FBACK_RSN_CNT * 2) { perror("Error: unexpected value in cache file (fback stats)"); exit(-1); } *trgt_fbck = val_err; trgt_fbck++; *trgt_fbck = val_cnt; trgt_fbck++; } else if (count < size_fback + 2) { rc = sscanf(buf, "%llu", &val); if (rc < 1) { perror("Error: parsing cache file(fback counters)"); exit(-1); } *fbck_cnt = val; fbck_cnt++; } else { perror("Error: cache file corrupt"); exit(-1); } cache_file_exists = 1; count++; } } static int get_fback_err_cache_count(struct smc_stats_fback *fback, int trgt) { int i; for (i = 0; i < SMC_MAX_FBACK_RSN_CNT; i++) { if (fback[i].fback_code == trgt) return fback[i].count; } return 0; } /* Check whether there were wrap arounds or really old data in the cache */ static int is_data_consistent () { int size, i, size_fback, val_err, val_cnt, cache_cnt; struct smc_stats_fback *kern_fbck; __u64 *kernel, *cache; size = sizeof(smc_stat) / sizeof(__u64); kernel = (__u64 *)&smc_stat; cache = (__u64 *)&smc_stat_c; for (i = 0; i < size; i++) { if (*kernel < *cache) return 0; kernel++; cache++; } size_fback = 2 * SMC_MAX_FBACK_RSN_CNT; kern_fbck = (struct smc_stats_fback *)&smc_rsn; for (i = 0; i < size_fback; i++) { val_err = kern_fbck->fback_code; if (i < SMC_MAX_FBACK_RSN_CNT) cache_cnt = get_fback_err_cache_count(smc_rsn_c.srv, val_err); else cache_cnt = get_fback_err_cache_count(smc_rsn_c.clnt, val_err); val_cnt = kern_fbck->count; kern_fbck++; if (val_cnt < cache_cnt) return 0; } if ((smc_rsn.srv_fback_cnt < smc_rsn_c.srv_fback_cnt) || (smc_rsn.clnt_fback_cnt < smc_rsn_c.clnt_fback_cnt)) return 0; return 1; } static void merge_cache () { int size, i, size_fback, val_err, cache_cnt; struct smc_stats_fback *kern_fbck; __u64 *kernel, *cache; if (!is_data_consistent()) { unlink(cache_file_path); return; } size = sizeof(smc_stat) / sizeof(__u64); kernel = (__u64 *)&smc_stat; cache = (__u64 *)&smc_stat_c; for (i = 0; i < size; i++) *(kernel++) -= *(cache++); size_fback = 2 * SMC_MAX_FBACK_RSN_CNT; kern_fbck = (struct smc_stats_fback *)&smc_rsn; for (i = 0; i < size_fback; i++) { val_err = kern_fbck->fback_code; if (i < SMC_MAX_FBACK_RSN_CNT) cache_cnt = get_fback_err_cache_count(smc_rsn_c.srv, val_err); else cache_cnt = get_fback_err_cache_count(smc_rsn_c.clnt, val_err); kern_fbck->count -= cache_cnt; kern_fbck++; } smc_rsn.srv_fback_cnt -= smc_rsn_c.srv_fback_cnt; smc_rsn.clnt_fback_cnt -= smc_rsn_c.clnt_fback_cnt; } static void open_cache_file() { int fd; cache_file_path = malloc(128); sprintf(cache_file_path, "/tmp/.smcstats.u%d", getuid()); fd = open(cache_file_path, O_RDWR|O_CREAT|O_NOFOLLOW, 0600); if (fd < 0) { perror("Error: open cache file"); exit(-1); } if ((cache_fp = fdopen(fd, "r+")) == NULL) { perror("Error: cache file r+"); exit(-1); } if (flock(fileno(cache_fp), LOCK_EX)) { perror("Error: cache file lock"); exit(-1); } } static void init_cache_file() { open_cache_file(); read_cache_file(cache_fp); } static void fill_cache_file() { int size, i, val_err, val_cnt; int *fback_src; __u64 *src; if (ftruncate(fileno(cache_fp), 0) < 0) perror("Error: ftruncate"); size = sizeof(smc_stat) / sizeof(__u64); src = (__u64 *)&smc_stat_org; for (i = 0; i < size; i++) { fprintf(cache_fp, "%-12d%-16llu\n",i ,*src); src++; } fback_src = (int*)&smc_rsn_org; size = 2 * SMC_MAX_FBACK_RSN_CNT; for (i = 0; i < size; i++) { val_err = *(fback_src++); val_cnt = *(fback_src++); fprintf(cache_fp, "%-12d%-16d%-16d\n",i , val_err, val_cnt); } fprintf(cache_fp, "%16llu\n", smc_rsn_org.srv_fback_cnt); fprintf(cache_fp, "%16llu\n", smc_rsn_org.clnt_fback_cnt); } int invoke_stats(int argc, char **argv, int option_details) { if (option_details == SMC_DETAIL_LEVEL_V || option_details == SMC_DETAIL_LEVEL_VV) { d_level = 1; } else if (option_details == SMC_OPTION_ABS) { is_abs = 1; } else if (option_details == SMC_OPTION_DETAIL_ABS) { is_abs = 1; d_level = 1; } handle_cmd_params(argc, argv); if (!is_abs) init_cache_file(); if (gen_nl_handle_dump(SMC_NETLINK_GET_FBACK_STATS, handle_gen_fback_stats_reply, NULL)) goto errout; if (gen_nl_handle_dump(SMC_NETLINK_GET_STATS, handle_gen_stats_reply, NULL)) goto errout; memcpy(&smc_stat_org, &smc_stat, sizeof(smc_stat_org)); memcpy(&smc_rsn_org, &smc_rsn, sizeof(smc_rsn_org)); if (!is_abs && cache_file_exists) merge_cache(); if (!json_cmd) print_as_text(); else print_as_json(); if (reset_cmd) { unlink(cache_file_path); if (cache_fp) { fclose(cache_fp); cache_fp = NULL; } free(cache_file_path); cache_file_path = NULL; open_cache_file(); fill_cache_file(); } errout: if (cache_fp) fclose(cache_fp); free(cache_file_path); return 0; } smc-tools-1.8.4/stats.h000066400000000000000000000103141473052164700147630ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2021 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef SMC_SYSTEM_H_ #define SMC_SYSTEM_H_ #define SMC_CLC_DECL_MEM 0x01010000 /* insufficient memory resources */ #define SMC_CLC_DECL_TIMEOUT_CL 0x02010000 /* timeout w4 QP confirm link */ #define SMC_CLC_DECL_TIMEOUT_AL 0x02020000 /* timeout w4 QP add link */ #define SMC_CLC_DECL_CNFERR 0x03000000 /* configuration error */ #define SMC_CLC_DECL_PEERNOSMC 0x03010000 /* peer did not indicate SMC */ #define SMC_CLC_DECL_IPSEC 0x03020000 /* IPsec usage */ #define SMC_CLC_DECL_NOSMCDEV 0x03030000 /* no SMC device found (R or D) */ #define SMC_CLC_DECL_NOSMCDDEV 0x03030001 /* no SMC-D device found */ #define SMC_CLC_DECL_NOSMCRDEV 0x03030002 /* no SMC-R device found */ #define SMC_CLC_DECL_NOISM2SUPP 0x03030003 /* hardware has no ISMv2 support */ #define SMC_CLC_DECL_NOV2EXT 0x03030004 /* peer sent no clc v2 extension */ #define SMC_CLC_DECL_NOV2DEXT 0x03030005 /* peer sent no clc SMC-Dv2 ext. */ #define SMC_CLC_DECL_NOSEID 0x03030006 /* peer sent no SEID */ #define SMC_CLC_DECL_NOSMCD2DEV 0x03030007 /* no SMC-Dv2 device found */ #define SMC_CLC_DECL_MODEUNSUPP 0x03040000 /* smc modes do not match (R or D)*/ #define SMC_CLC_DECL_RMBE_EC 0x03050000 /* peer has eyecatcher in RMBE */ #define SMC_CLC_DECL_OPTUNSUPP 0x03060000 /* fastopen sockopt not supported */ #define SMC_CLC_DECL_DIFFPREFIX 0x03070000 /* IP prefix / subnet mismatch */ #define SMC_CLC_DECL_GETVLANERR 0x03080000 /* err to get vlan id of ip device*/ #define SMC_CLC_DECL_ISMVLANERR 0x03090000 /* err to reg vlan id on ism dev */ #define SMC_CLC_DECL_NOACTLINK 0x030a0000 /* no active smc-r link in lgr */ #define SMC_CLC_DECL_NOSRVLINK 0x030b0000 /* SMC-R link from srv not found */ #define SMC_CLC_DECL_VERSMISMAT 0x030c0000 /* SMC version mismatch */ #define SMC_CLC_DECL_MAX_DMB 0x030d0000 /* SMC-D DMB limit exceeded */ #define SMC_CLC_DECL_SYNCERR 0x04000000 /* synchronization error */ #define SMC_CLC_DECL_PEERDECL 0x05000000 /* peer declined during handshake */ #define SMC_CLC_DECL_INTERR 0x09990000 /* internal error */ #define SMC_CLC_DECL_ERR_RTOK 0x09990001 /* rtoken handling failed */ #define SMC_CLC_DECL_ERR_RDYLNK 0x09990002 /* ib ready link failed */ #define SMC_CLC_DECL_ERR_REGRMB 0x09990003 /* reg rmb failed */ #define SMC_TYPE_R 0 #define SMC_TYPE_D 1 #define SMC_SERVER 1 #define SMC_CLIENT 0 #define SMC_MAX_FBACK_RSN_CNT 30 enum { SMC_BUF_8K, SMC_BUF_16K, SMC_BUF_32K, SMC_BUF_64K, SMC_BUF_128K, SMC_BUF_256K, SMC_BUF_512K, SMC_BUF_1024K, SMC_BUF_G_1024K, SMC_BUF_MAX, }; struct smc_stats_fback { int fback_code; int count; }; struct smc_stats_rsn { struct smc_stats_fback srv[SMC_MAX_FBACK_RSN_CNT]; struct smc_stats_fback clnt[SMC_MAX_FBACK_RSN_CNT]; __u64 srv_fback_cnt; __u64 clnt_fback_cnt; }; struct smc_stats_rmbcnt { __u64 buf_size_small_peer_cnt; __u64 buf_size_small_cnt; __u64 buf_full_peer_cnt; __u64 buf_full_cnt; __u64 reuse_cnt; __u64 alloc_cnt; __u64 dgrade_cnt; }; struct smc_stats_memsize { __u64 buf[SMC_BUF_MAX]; }; struct smc_stats_tech { struct smc_stats_memsize tx_rmbsize; struct smc_stats_memsize rx_rmbsize; struct smc_stats_memsize tx_pd; struct smc_stats_memsize rx_pd; struct smc_stats_rmbcnt rmb_tx; struct smc_stats_rmbcnt rmb_rx; __u64 clnt_v1_succ_cnt; __u64 clnt_v2_succ_cnt; __u64 srv_v1_succ_cnt; __u64 srv_v2_succ_cnt; __u64 sendpage_cnt; __u64 urg_data_cnt; __u64 splice_cnt; __u64 cork_cnt; __u64 ndly_cnt; __u64 rx_bytes; __u64 tx_bytes; __u64 rx_cnt; __u64 tx_cnt; __u64 rx_rmbuse; __u64 tx_rmbuse; }; struct smc_stats { struct smc_stats_tech smc[2]; __u64 clnt_hshake_err_cnt; __u64 srv_hshake_err_cnt; }; int invoke_stats(int argc, char **argv, int detail_level); #endif /* SMC_SYSTEM_H_ */ smc-tools-1.8.4/ueid.c000066400000000000000000000152501473052164700145520ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include #include #include "smctools_common.h" #include "util.h" #include "libnetlink.h" #include "ueid.h" static int add_cmd = 0; static int del_cmd = 0; static int flush_cmd = 0; static int show_cmd = 0; static char target_eid[SMC_MAX_EID_LEN + 1] = {0}; extern int smc_id; extern struct nl_sock *sk; const struct nla_policy smc_gen_ueid_policy[SMC_NLA_EID_TABLE_MAX + 1] = { [SMC_NLA_EID_TABLE_UNSPEC] = { .type = NLA_UNSPEC }, [SMC_NLA_EID_TABLE_ENTRY] = { .type = NLA_NUL_STRING }, }; static void usage(void) { fprintf(stderr, "Usage: smcd ueid [show]\n" " smcd ueid add \n" " smcd ueid del \n" " smcd ueid flush\n" ); exit(-1); } static int gen_nl_ueid_handle(int cmd, char *ueid, int (*cb_handler)(struct nl_msg *msg, void *arg)) { int rc = EXIT_FAILURE, nlmsg_flags = 0; struct nl_msg *msg; nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); /* Allocate a netlink message and set header information. */ msg = nlmsg_alloc(); if (!msg) { nl_perror(NLE_NOMEM, "Error"); rc = EXIT_FAILURE; goto err; } if (cmd == SMC_NETLINK_DUMP_UEID) nlmsg_flags = NLM_F_DUMP; if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, cmd, SMC_GENL_FAMILY_VERSION)) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err; } if (ueid && ueid[0]) { rc = nla_put_string(msg, SMC_NLA_EID_TABLE_ENTRY, ueid); if (rc < 0) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err; } } /* Send message */ rc = nl_send_auto(sk, msg); if (rc < 0) { nl_perror(rc, "Error"); rc = EXIT_FAILURE; goto err; } /* Receive reply message, returns number of cb invocations. */ rc = nl_recvmsgs_default(sk); if (rc < 0) { /* For cmd "UEID remove" the kernel might return ENOENT when * the specified UEID is not in the list. * This is mapped to NLE_OBJ_NOTFOUND in libnl, lib/error.c. */ if (rc == -NLE_OPNOTSUPP) { fprintf(stderr, "Error: operation not supported by kernel\n"); } else if (cmd == SMC_NETLINK_REMOVE_UEID) { if (rc == -NLE_OBJ_NOTFOUND) { fprintf(stderr, "Error: specified User EID is not defined\n"); } else if (rc == -NLE_AGAIN) { fprintf(stderr, "Info: the System EID was activated because the last User EID was removed\n"); } else { fprintf(stderr, "Error: specified User EID is not defined\n"); } } else if (cmd == SMC_NETLINK_ADD_UEID) { if (rc == -NLE_INVAL) { fprintf(stderr, "Error: specified User EID was rejected by the kernel\n"); } else if (rc == -NLE_NOMEM) { fprintf(stderr, "Error: kernel reported an out of memory condition\n"); } else if (rc == -NLE_RANGE) { fprintf(stderr, "Error: specified User EID was rejected because the maximum number of User EIDs is reached\n"); } else if (rc == -NLE_EXIST) { fprintf(stderr, "Error: specified User EID is already defined\n"); } else { nl_perror(rc, "Error"); } } else { nl_perror(rc, "Error"); } rc = EXIT_FAILURE; goto err; } nlmsg_free(msg); return EXIT_SUCCESS; err: nlmsg_free(msg); return rc; } static int handle_gen_ueid_reply(struct nl_msg *msg, void *arg) { struct nlattr *attrs[SMC_NLA_EID_TABLE_ENTRY + 1]; struct nlmsghdr *hdr = nlmsg_hdr(msg); int rc = NL_OK; if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_EID_TABLE_ENTRY, (struct nla_policy *)smc_gen_ueid_policy) < 0) { fprintf(stderr, "Error: invalid data returned: smc_gen_ueid_policy\n"); nl_msg_dump(msg, stderr); return NL_STOP; } if (!attrs[SMC_NLA_EID_TABLE_ENTRY]) return NL_STOP; printf("%s\n", nla_get_string(attrs[SMC_NLA_EID_TABLE_ENTRY])); return rc; } static char ueid_valid(char *ueid) { char *end = ueid + SMC_MAX_EID_LEN; while (--end >= ueid && isspace(*end)) ; if (end < ueid) { fprintf(stderr, "Error: Invalid User EID specified: EID is empty\n"); return 0; } if (!isalnum(*ueid)) { fprintf(stderr, "Error: Invalid User EID specified: first character must be alphanumeric\n"); return 0; } if (strstr(ueid, "..")) { fprintf(stderr, "Error: Invalid User EID specified: consecutive dots not allowed\n"); return 0; } while (ueid <= end) { if ((!isalnum(*ueid) || islower(*ueid)) && *ueid != '.' && *ueid != '-') { fprintf(stderr, "Error: Invalid User EID specified: unsupported character: '%c'\n", *ueid); fprintf(stderr, " Supported characters are: A-Z, 0-9, '.' and '-'\n"); return 0; } ueid++; } return 1; } static void set_eid(char *eid) { if (strlen(eid) > SMC_MAX_EID_LEN) { fprintf(stderr, "Error: Invalid User EID specified: EID is longer than 32 characters\n"); exit(-1); } /* pad to 32 byte using blanks */ sprintf(target_eid, "%-32s", eid); if (!ueid_valid(target_eid)) exit(-1); } static void handle_cmd_params(int argc, char **argv) { if (argc == 0) { show_cmd = 1; /* no object given, so use the default "show" */ return; } while (1) { if (add_cmd) { set_eid(argv[0]); break; } else if (del_cmd) { set_eid(argv[0]); break; } else if (contains(argv[0], "help") == 0) { usage(); } else if (contains(argv[0], "add") == 0) { add_cmd = 1; } else if (contains(argv[0], "del") == 0) { del_cmd = 1; } else if (contains(argv[0], "flush") == 0) { flush_cmd = 1; break; } else if (contains(argv[0], "show") == 0) { show_cmd = 1; break; } else { usage(); } if (!NEXT_ARG_OK()) break; NEXT_ARG(); } /* Too many parameters or wrong sequence of parameters */ if (NEXT_ARG_OK()) usage(); /* Only single cmd expected */ if ((add_cmd + del_cmd + flush_cmd + show_cmd) != 1) usage(); /* eid required for command */ if (!target_eid[0] && (add_cmd || del_cmd)) usage(); } int invoke_ueid(int argc, char **argv, int detail_level) { int rc = EXIT_SUCCESS; handle_cmd_params(argc, argv); if (add_cmd) { rc = gen_nl_ueid_handle(SMC_NETLINK_ADD_UEID, target_eid, handle_gen_ueid_reply); } else if (del_cmd) { rc = gen_nl_ueid_handle(SMC_NETLINK_REMOVE_UEID, target_eid, handle_gen_ueid_reply); } else if (flush_cmd) { rc = gen_nl_ueid_handle(SMC_NETLINK_FLUSH_UEID, NULL, handle_gen_ueid_reply); } else if (show_cmd) { rc = gen_nl_ueid_handle(SMC_NETLINK_DUMP_UEID, NULL, handle_gen_ueid_reply); } else { printf("Error: Unknown command\n"); /* we should never come here ... */ } return rc; } smc-tools-1.8.4/ueid.h000066400000000000000000000010261473052164700145530ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef UEID_H_ #define UEID_H_ extern struct rtnl_handle rth; int invoke_ueid(int argc, char **argv, int detail_level); #endif /* UEID_H_ */ smc-tools-1.8.4/util.c000066400000000000000000000063341473052164700146040ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #include #include #include #include #include "util.h" void print_unsup_msg(void) { fprintf(stderr, "Error: Kernel does not support this parameter !\n"); exit(-1); } void print_type_error(void) { fprintf(stderr, "Error: You entered an invalid type. Possible values are smcd and smcr !\n"); exit(-1); } char* trim_space(char *str) { char *end; while (isspace(*str)) { str = str + 1; } /* remove trailing whitespace */ end = str + strlen((const char*)str) - 1; while (end > str && isspace(*end)) { end = end - 1; } *(end+1) = '\0'; return str; } int contains(const char *prfx, const char *str) { if (!*prfx) return 1; while (*str && *prfx == *str) { prfx++; str++; } return !!*prfx; } static void determine_mag_factor(int leading_places, char *magnitude, double *factor) { if (leading_places < 7) { *magnitude = 'K'; *factor = 1000; } else if (leading_places < 10) { *magnitude = 'M'; *factor = 1000000; } else if (leading_places < 13) { *magnitude = 'G'; *factor = 1000000000; } else { // this is quite expensive, hence we avoid if possible *factor = pow(1000, leading_places/3); if (leading_places < 16) *magnitude = 'T'; else if (leading_places < 19) *magnitude = 'P'; else *magnitude = '?'; } } static void determine_digs(int leading_places, int max_digs, int *num_full_digs, int *num_places) { *num_full_digs = leading_places % 3; if (*num_full_digs == 0) *num_full_digs = 3; *num_places = max_digs - *num_full_digs - 2; if (*num_places <= 0) { *num_places = 0; *num_full_digs = max_digs - 1; } } int get_abbreviated(uint64_t num, int max_digs, char *res) { int num_full_digs, leading_places; char magnitude; int num_places; double factor; char tmp[128]; if (num == 0) { snprintf(res, max_digs + 1, "0"); return 1; } leading_places = sprintf(tmp, "%lld", (long long int)num); if (leading_places < 4) { snprintf(res, max_digs + 1, "%lu", num); return leading_places; } determine_digs(leading_places, max_digs, &num_full_digs, &num_places); determine_mag_factor(leading_places, &magnitude, &factor); double tmpnum = num / factor; if (tmpnum + 5 * pow(10, -1 - num_places) >= pow(10, num_full_digs)) { if (num_places > 0) // just strip down one decimal place, // e.g. 9.96... with 1.1 format would result in // 10.0 otherwise num_places--; else { // indicate that we need one more leading place // e.g. 999.872 with 3.0 format digits would result // in 1.000K otherwise leading_places++; determine_digs(leading_places, max_digs, &num_full_digs, &num_places); determine_mag_factor(leading_places, &magnitude, &factor); tmpnum = num / factor; } } snprintf(res, max_digs + 1, "%*.*lf%c", max_digs - 1, num_places, num / factor, magnitude); return 0; } smc-tools-1.8.4/util.h000066400000000000000000000021421473052164700146020ustar00rootroot00000000000000/* * SMC Tools - Shared Memory Communication Tools * * Copyright IBM Corp. 2020 * * Author(s): Guvenc Gulce * * Userspace program for SMC Information display * * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html */ #ifndef UTIL_H_ #define UTIL_H_ #include #include #define SMC_DETAIL_LEVEL_V 1 #define SMC_DETAIL_LEVEL_VV 2 #define SMC_OPTION_ABS -1 #define SMC_OPTION_DETAIL_ABS -2 #define SMC_TYPE_STR_MAX 5 #define NEXT_ARG() do { argv++; argc--; } while(0) #define NEXT_ARG_OK() (argc - 1 > 0) #define PREV_ARG() do { argv--; argc++; } while(0) void print_unsup_msg(void); void print_type_error(void); char* trim_space(char *str); int get_abbreviated(uint64_t num, int max_digs, char *res); int contains(const char *prfx, const char *str); static inline int is_str_empty(char *str) { if (str && str[0] == '\0') return 1; else return 0; } #endif /* UTIL_H_ */