HP 4X - DDR InfiniBand Mezzanine HCA Using Manual
Using infiniband for a scalable compute infrastructure
Hide thumbs
Also See for 4X - DDR InfiniBand Mezzanine HCA:
- Frequently asked questions (4 pages) ,
- Read this first manual (7 pages) ,
- Read this first manual (10 pages)
Table of Contents
Quick Links
Using InfiniBand for a
scalable compute infrastructure
technology brief, 3
rd
Abstract .............................................................................................................................................. 2
Introduction ......................................................................................................................................... 2
InfiniBand technology ........................................................................................................................... 4
InfiniBand components ...................................................................................................................... 5
InfiniBand software architecture ......................................................................................................... 5
InfiniBand hardware architecture ....................................................................................................... 8
Link operation .................................................................................................................................. 9
Scale-out clusters built on InfiniBand and HP technology ......................................................................... 11
Conclusion ........................................................................................................................................ 13
Appendix A: Glossary ........................................................................................................................ 14
For more information .......................................................................................................................... 15
Call to action .................................................................................................................................... 15
edition
MPI ............................................................................................................................................. 7
IPoIB ........................................................................................................................................... 7
RDMA-based protocols .................................................................................................................. 7
RDS ............................................................................................................................................ 8
Table of Contents
Related Manuals for HP 4X - DDR InfiniBand Mezzanine HCA
Summary of Contents for HP 4X - DDR InfiniBand Mezzanine HCA
-
Page 1: Table Of Contents
RDMA-based protocols ........................7 RDS ............................8 InfiniBand hardware architecture ....................... 8 Link operation ..........................9 Scale-out clusters built on InfiniBand and HP technology ................. 11 Conclusion ............................13 Appendix A: Glossary ........................14 For more information .......................... 15 Call to action ............................ 15... -
Page 2: Abstract
(HPC) but also for financial services and Oracle-based database applications. InfiniBand (IB) is one of the most important technologies that enable the adoption of cluster computing. This technology brief describes InfiniBand as an interconnect technology used in cluster computing, provides basic technical information, and explains the advantages of implementing InfiniBand-based scale-out architectures. - Page 3 Scale-out systems allow infrastructure architects to meet performance and cost goals, but interconnect performance, scalability, and reliability are key areas that must be carefully considered. A cluster infrastructure works best when built with an interconnect technology that scales easily, reliably, and economically with system expansion.
-
Page 4: Infiniband Technology
Packets can contain data payloads of up to four kilobytes. The packets are passed to the network layer, which selects a route to the destination node and, if necessary, attaches the route information to the packets. The data link layer attaches a local identifier (LID) to the packet for communication at the subnet level. -
Page 5: Infiniband Components
The layered software architecture of the HCA allows writing code without specific hardware in mind. The functionality of an HCA is defined by its verb set, which is a table of commands used by the application programming interface (API) of the operating system being run. A number of services and software protocols are available (Figure 4) and, depending on type, can be implemented from user space or from the kernel. - Page 6 Open Fabrics Verbs and API Provider Hardware-Specific Driver Hardware InfiniBand HCA As indicated in Figure 4, InfiniBand supports a variety of upper level protocols (ULPs) and libraries that have evolved since the introduction of InfiniBand. The key protocols are discussed in the following pages.
-
Page 7: Mpi
InfiniBand. The iSER protocol provides iSCSI manageability to RDMA storage operations. NFS – The Network File System (NFS) is a storage protocol that has evolved since its inception in the 1980s, undergoing several generations of development while remaining network-independent. With... -
Page 8: Rds
The bi-directional links contain dedicated send and receive lanes for full duplex operation. For quad data rate (QDR) operation, each lane has a signaling rate of 10 Gbps. Bandwidth is increased by adding more lanes per link. InfiniBand interconnect types include 1x, 4x, or 12x wide full-duplex links (Figure 5)., The 4x is the most popular configuration and provides a theoretical full-... -
Page 9: Link Operation
4X DDR products is widespread, and deployment of QDR systems is expected to increase. Link operation Each link can be divided (multiplexed) into a set of virtual lanes, similar to highway lanes (Figure 7). Each virtual lane provides flow control and allows a pair of devices to communicate autonomously. - Page 10 When a connection between two channel adapters is established, one of the following transport layer communication protocols is selected: Reliable connection (RC) – data transfer between two entities using receive acknowledgment Unreliable connection (UC) – same as RC but without acknowledgement (rarely used) Reliable datagram (RD) –...
-
Page 11: Scale-Out Clusters Built On Infiniband And Hp Technology
The only operation available for the receive queue is Post Receive Buffer transfer, which identifies a buffer that a client may send to or receive from using a Send or RDMA Write data transfer. Scale-out clusters built on InfiniBand and HP technology In the past few years, scale-out cluster computing has become a mainstream architecture for high performance computing. - Page 12 HP Oracle Database Machine HP BladeSystem for Oracle Optimized Warehouse (OOW) HP Cluster Platforms are built around specific hardware and software platforms and offer a choice of interconnects. For example, the HP Cluster Platform CL3000BL uses the HP BL2x220c G5, BL280c G6, and BL460c blade servers as the compute node with a choice of GbE or InfiniBand interconnects.
-
Page 13: Conclusion
While zero-copy (RDMA) protocols have been applied to TCP/IP networks such as Ethernet, RDMA is a core capability of InfiniBand architecture. Flow control support is native to the HCA design, and the latency time for InfiniBand data transfers is generally less—approaching 1 microsecond for MPI pingpong latency—than that for... -
Page 14: Appendix A: Glossary
Description Application Programming Interface: software routine or object written in support of a language Double Data Rate: for InfiniBand, clock rate of 5.0 Gbps (2.5 Gbps x 2) Gigabit Ethernet: Ethernet network operating at 1 Gbps or greater Host Channel Adapter: hardware interface connecting a server node to the IB network... -
Page 15: For More Information
[email protected] © 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty.