Guidelines for OpenVMS Cluster Configurations

The six interconnects described in this chapter share some general characteristics. Table 4-1 describes these characteristics.

Table 4-1 Interconnect Characteristics

Characteristic	Description
Throughput	The quantity of data transferred across the interconnect. Some interconnects require more processor overhead than others. For example, Ethernet and FDDI interconnects require more processor overhead than do CI or DSSI. Larger packet sizes allow higher data-transfer rates (throughput) than do smaller packet sizes.
Cable length	Interconnects range in length from 3 m to 40 km.
Maximum number of nodes	The number of nodes that can connect to an interconnect varies among interconnect types. Be sure to consider this when configuring your OpenVMS Cluster system.
Supported systems and storage	Each OpenVMS Cluster node and storage subsystem requires an adapter to connect the internal system bus to the interconnect. First consider the storage and processor I/O performance, then the adapter performance, when choosing an interconnect type.

4.2 Comparison of Interconnect Types

Table 4-2 shows key statistics for a variety of interconnects.

Table 4-2 Comparison of Interconnect Types

Attribute	CI	DSSI	FDDI	SCSI	MEMORY CHANNEL	Ethernet
Maximum throughput (Mb/s)	140	32	100	160	800	10
Hardware-assisted data link¹	Yes	Yes	No	No	No	No
Connection to storage	Direct and MSCP served	Direct and MSCP served	MSCP served	Direct and MSCP served	MSCP served	MSCP served
Topology	Radial coaxial cable	Bus	Dual ring of trees	Bus	Radial copper cable	Linear coaxial cable
Maximum nodes	32²	8³	96 4	8--16 5	4	96 4
Maximum length	45 m	6 m 6	40 km	25m	3 m	2800 m

You can use multiple interconnects to achieve the following benefits:

• Failover
  If one interconnect or adapter fails, the node communications automatically move to another interconnect.
• MSCP server load balancing
  In a multiple MSCP server configuration, an OpenVMS Cluster performs load balancing to automatically choose the best path. This reduces the chances that a single adapter could cause an I/O bottleneck. Depending on your configuration, multiple paths from one node to another node may transfer more information than would a single path.
  Reference: See Section 7.7.3 for an example of dynamic MSCP load balancing.

A mixed interconnect is a combination of two or more different types of interconnects in an OpenVMS Cluster system. You can use mixed interconnects to combine the advantages of each type and to expand your OpenVMS Cluster system. For example, an Ethernet cluster that requires more storage can expand with the addition of CI, DSSI, or SCSI connections.

4.5 Interconnects Supported by Alpha and VAX Systems

Table 4-3 shows the OpenVMS Cluster interconnects supported by Alpha and VAX systems. You can also refer to the most recent OpenVMS Cluster SPD to see the latest information on supported interconnects.

Table 4-3 Cluster Interconnects Supported by Systems

Systems	CI	DSSI	SCSI	FDDI	Ethernet	MEMORY CHANNEL
AlphaServer 8400, 8200	X	X	X	X¹	X	X
AlphaServer 4100, 2100, 2000	X	X	X	X¹	X¹	X
AlphaServer 1000		X	X	X	X¹	X
AlphaServer 400		X	X	X	X¹
AlphaStation series			X	X	X¹
DEC 7000/10000	X	X		X¹	X
DEC 4000		X		X	X¹
DEC 3000			X	X¹	X¹
DEC 2000				X	X¹
VAX 6000/7000/10000	X	X		X	X
VAX 4000, MicroVAX 3100		X		X	X¹
VAXstation 4000				X	X¹

As Table 4-3 shows, OpenVMS Clusters support a wide range of interconnects: CI, DSSI, SCSI, FDDI, Ethernet, and MEMORY CHANNEL. This power and flexibility means that almost anything will work well. The most important factor to consider is how much I/O you need, as explained in Chapter 2.

In most cases, the I/O requirements will be less than the capabilities of any one OpenVMS Cluster interconnect. Ensure that you have a reasonable surplus I/O capacity, then choose your interconnects based on other needed features.

4.6 MEMORY CHANNEL Interconnect

MEMORY CHANNEL is a high-performance cluster interconnect technology for PCI-based Alpha systems. With the benefits of very low latency, high bandwidth, and direct memory access, MEMORY CHANNEL complements and extends the unique ability of OpenVMS Clusters to work as a single, virtual system.

Three hardware components are required by a node to support a MEMORY CHANNEL connection:

• A PCI-to-MEMORY CHANNEL adapter
• A link cable (3 m or 10 feet long)
• A port in a MEMORY CHANNEL hub (except for a two-node configuration in which the cable connects just two PCI adapters)

A MEMORY CHANNEL hub is a desktop-PC sized unit that provides a connection among systems. For its initial release in OpenVMS Version 7.1, MEMORY CHANNEL supports up to four Alpha nodes per hub. You can configure systems with two MEMORY CHANNEL adapters in order to provide failover in case an adapter fails. Each adapter must be connected to a different hub.

A MEMORY CHANNEL hub is not required in clusters that comprise only two nodes. In a two-node configuration, one PCI adapter is configured, using module jumpers, as a virtual hub.

4.6.1 Advantages

MEMORY CHANNEL technology provides the following features:

• Offers excellent price/performance.
  With several times the CI bandwidth, MEMORY CHANNEL provides a 100 MB/s interconnect with minimal latency. MEMORY CHANNEL architecture is designed for the industry-standard PCI bus.
• Requires no change to existing applications.
  MEMORY CHANNEL works seamlessly with existing cluster software, so that no change is necessary for existing applications. The new MEMORY CHANNEL drivers, PMDRIVER and MCDRIVER, integrate with the System Communications Services layer of OpenVMS Clusters in the same way that existing port drivers do. Higher layers of cluster software are unaffected.
• Offloads CI, DSSI, and the LAN in SCSI clusters.
  You cannot connect storage directly to MEMORY CHANNEL, but you can use it to make maximum use of each interconnect's strength.
  While MEMORY CHANNEL is not a replacement for CI and DSSI, when used in combination with those interconnects, it offloads their node-to-node traffic. This enables them to be dedicated to storage traffic, optimizing communications in the entire cluster.
  When used in a cluster with SCSI and LAN interconnects, MEMORY CHANNEL offloads node-to-node traffic from the LAN, enabling it to handle more TCP/IP or DECnet traffic.
• Provides fail-separately behavior.
  When a system failure occurs, MEMORY CHANNEL nodes behave like any failed node in an OpenVMS Cluster. The rest of the cluster continues to perform until the failed node can rejoin the cluster.
• Will provide a system programming interface (SPI).
  A future version of OpenVMS Alpha Version 7.1 will include a customer-accessible system programming interface, called the SPI.

The MEMORY CHANNEL interconnect has a very high maximum throughput of 100 MB/s. If a single MEMORY CHANNEL is not sufficient, up to two interconnects (and two MEMORY CHANNEL hubs) can share throughput.

4.6.3 Supported Adapter

The MEMORY CHANNEL adapter is CCMAA-AA. It connects to the PCI bus.

Reference: For complete information about each adapter's features and order numbers, see the Digital Systems and Options Catalog, order number EC-I6601-10.

To access the most recent Digital Systems and Options Catalog on the World Wide Web, use the following URL:

http://www.digital.com/info/soc 

The SCSI interconnect is an industry standard interconnect that supports one or more computers, peripheral devices, and interconnecting components. SCSI is a single-path, daisy-chained, multidrop bus. It is a single 8-bit or 16-bit data path with byte parity for error detection. Both inexpensive single-ended and differential signaling for longer distances are available.

In an OpenVMS Cluster, multiple Alpha computers on a single SCSI interconnect can simultaneously access SCSI disks. This type of configuration is called multihost SCSI connectivity. A second type of interconnect is required for node-to-node communication. For multihost access to SCSI storage, the following components are required:

• SCSI host adapter that is supported in a multihost configuration (see Table 4-6)
• SCSI interconnect
• Terminators, one for each end of the SCSI interconnect
• Storage devices that are supported in a multihost configuration (RZnn; refer to the OpenVMS Cluster SPD [29.78.nn])

For larger configurations, the following components are available:

• Storage controllers (HSZnn)
• Bus isolators (DWZZA or DWZZB) to convert single-ended to differential signaling and to effectively double the SCSI interconnect length

Reference: For a detailed description of how to connect SCSI configurations, see Appendix A.

4.7.1 Advantages

The SCSI interconnect offers the following advantages:

• Lowest cost, shared direct access to storage
  Because SCSI is an industry standard and is used extensively throughout the industry, it is available from many manufacturers at competitive prices.
• Scalable configuration to achieve high performance at a moderate price
  You can choose:
  • Width of SCSI interconnect
    Narrow (8 bits) or wide (16 bits)
  • Transmission mode
    Single-ended signaling, the most common and least expensive, or differential signaling, which provides higher signal integrity and allows a longer SCSI interconnect
  • Signal speed (standard mode or fast mode)
  • Number of nodes sharing the SCSI bus (two or three)
  • Number of shared SCSI buses to which a node can connect (six is the maximum)
  • Storage type and size (RZnn or HSZ40)
  • Computer type and size (AlphaStation or AlphaServer)

Throughput for the SCSI interconnect is shown in Table 4-4.

Table 4-4 Maximum Data Transfer Rates in Megabytes per Second

Mode	Narrow (8-Bit)	Wide (16-Bit)
Standard	5	10
Fast	10	20

4.7.3 SCSI Interconnect Distances

The maximum length of the SCSI interconnect is determined by the signaling method used in the configuration and, for single-ended signaling, by the data transfer rate.

There are two types of electrical signaling for SCSI interconnects: single-ended and differential. Both types can operate in standard mode or fast mode. For differential signaling, the maximum SCSI interconnect possible is the same whether you use standard or fast mode.

Table 4-5 summarizes how the type of signaling method affects SCSI interconnect distances.

Table 4-5 Maximum SCSI Interconnect Distances

Signaling Technique	Rate of Data Transfer	Maximum Cable Length
Single ended	Standard	6 m¹
Single ended	Fast	3 m
Differential	Standard or Fast	25 m

Table 4-6 shows SCSI adapters with the internal buses and computers they support.

Table 4-6 SCSI Adapters

Adapter	Internal Bus	Supported Computers
Embedded (NCR-810 based)/KZPAA¹	PCI	AlphaServer 400
		AlphaServer 1000
		AlphaServer 2000
		AlphaServer 2100
		AlphaStation 200
		AlphaStation 250
		AlphaStation 400
		AlphaStation 600
KZPSA²	PCI	Supported on all Alpha computers that support KZPSA in single-host configurations.³
KZTSA²	TURBOchannel	DEC 3000

Reference: For complete information about each adapter's features and order number, see the Digital Systems and Options Catalog, order number EC-I6601-10.

To access the most recent Digital Systems and Options Catalog on the World Wide Web, use the following URL:

http://www.digital.com/info/soc 

The CI interconnect is a radial bus through which OpenVMS Cluster systems communicate. It comprises the following components:

• CI host adapter.
• Star coupler---A passive device that serves as a common connection point for signals between OpenVMS nodes and HSC or HSJ controllers that are connected by the CI.
• Optional star coupler expander (CISCE)---Consists of two amplifiers, one for each of its two paths.
• CI cable.

The CI interconnect offers the following advantages:

• High speed
  Suitable for larger processors and I/O-intensive applications.
• Efficient access to large amounts of storage
  HSC and HSJ controllers can connect large numbers of disk and tape drives to the OpenVMS Cluster system, with direct access from all OpenVMS nodes on the CI.
• Minimal CPU overhead for communication
  CI adapters are intelligent interfaces that perform much of the work required for communication among OpenVMS nodes and storage. The CI topology allows all nodes attached to a CI bus to communicate directly with the HSC and HSJ controllers on the same CI bus.
• High availability through redundant, independent data paths
  Each CI adapter is connected with two pairs of CI cables. If a single CI cable connection fails, failover automatically occurs.
• Multiple access paths to disks and tapes
  Dual HSC and HSJ controllers and dual-ported devices create alternative paths to storage devices.

The CI interconnect has a high maximum throughput. CI adapters use high-performance microprocessors that perform many of the processing activities usually performed by the CPU. As a result, they consume minimal CPU processing power.

Because the effective throughput of the CI bus is high, a single CI interconnect is not likely to be a bottleneck in a large OpenVMS Cluster configuration. If a single CI is not sufficient, multiple CI interconnects can increase throughput.

4.8.3 Supported Adapters and Bus Types

The following are CI adapters and internal buses that each supports:

• CIPCA (PCI/EISA)
• CIXCD (XMI)
• CIBCA--B (VAXBI)

Reference: For complete information about each adapter's features and order numbers, see the Digital Systems and Options Catalog, order number EC-I6601-10.

To access the most recent Digital Systems and Options Catalog on the World Wide Web, use the following URL:

http://www.digital.com/info/soc 

You can configure multiple CI adapters on some OpenVMS nodes. Multiple star couplers can be used in the same OpenVMS Cluster.

With multiple CI adapters on a node, adapters can share the traffic load. This reduces I/O bottlenecks and increases the total system I/O throughput. Table 4-7 lists the limits for multiple CI adapters per system.

Table 4-7 Maximum CI Adapters per System

System	CIPCA	CIBCA--A	CIBCA--B	CIXCD
AlphaServer 8400	26			10
AlphaServer 8200	26
AlphaServer 4100, 2100, 2000	3--4
DEC 7000/10000				10
VAX 6000	1	4	4
VAX 7000, 10000				10

Reference: For more extensive information about the CIPCA adapter, see Appendix C.

4.8.5 Configuration Guidelines for CI Clusters

Use the following guidelines when configuring systems in a CI cluster:

• The maximum number of nodes that you can connect to a star coupler is 32. Up to 16 of these nodes can be OpenVMS systems, and the remainder can be HSJ and HSC storage controllers.
• The number of star couplers is limited by the number of CI adapters configured on a system.
• Dual porting of devices between HSJ and HSC controllers is supported as long as they are connected to the same or separate star couplers. Dual porting of devices between HSJ and HSC controllers and local controllers is not supported.
• Different types of CI adapters cannot be combined in the same system.
• You can use multiple CI adapters for redundancy and throughput. You can increase throughput by connecting additional CI adapters to separate star couplers; throughput does not increase substantially when you connect a second CI adapter to the same star coupler.

DSSI is a single-path, daisy-chained, multidrop bus. It provides a single, 8-bit parallel data path with both byte parity and packet checksum for error detection.

4.9.1 Advantages

DSSI offers the following advantages:

• High reliability
• Shared, direct access to storage at a lower cost than CI
• Direct communication between systems and storage
• High-performance, intelligent storage controllers with embedded caches

DSSI storage often resides in the same cabinet as the CPUs. For these configurations, the whole system may need to be shut down for service, unlike configurations and interconnects with separately housed systems and storage devices.

4.9.3 Throughput

The maximum throughput is 32 Mb/s.

DSSI has highly intelligent adapters that require minimal CPU processing overhead.

4.9.4 DSSI Adapter Types

There are two types of DSSI adapters:

• Embedded adapter, which is part of the system.
• Optional adapter, which you can purchase separately and add to the system.

The following are DSSI adapters and internal bus that each supports:

• KFESA (EISA)
• KFESB (EISA)
• KFPSA (PCI)
• KFMSA (XMI)---VAX only
• KFMSB (XMI)---Alpha only
• KFQSA (Q-bus)
• N710 (embedded)
• SHAC (embedded)
• EDA640 (embedded)

Reference: For complete information about each adapter's features and order numbers, see the Digital Systems and Options Catalog, order number EC-I6601-10.

To access the most recent Digital Systems and Options Catalog on the World Wide Web, use the following URL:

http://www.digital.com/info/soc 

DSSI configurations use HSD intelligent controllers to connect disk drives to an OpenVMS Cluster. HSD controllers serve the same purpose with DSSI as HSJ controllers serve with CI: they enable you to configure more storage.

Alternatively, DSSI configurations use integrated storage elements (ISEs) connected directly to the DSSI bus. Each ISE contains either a disk and disk controller or a tape and tape controller.

4.9.7 Multiple DSSI Adapters

Multiple DSSI adapters are supported for some systems, enabling higher throughput than with a single DSSI bus.

Table 4-8 lists the limitations for multiple DSSI adapters. You can also refer to the most recent OpenVMS Cluster SPD for the latest information about DSSI adapters.

Table 4-8 DSSI Adapters per System

System	Embedded	KFPSA&185;	KFQSA&178;	KFESA	KFESB	KFMSA&179;	KFMSB&179;
AlphaServer 8400	-	4	-	-	-	-	12
AlphaServer 8200, 4100	-	4	-	-	-	-	-
AlphaServer 2100	-	4	-	-	-	-	-
AlphaServer 2000, 1000	-	4	-	-	4	-	-
DEC 4000 (embedded N710)	2	-	-	-	-	-	-
DEC 7000/10000	-	-	-	-	-	-	12
MicroVAX II, 3500, 3600, 3800, 3900	-	-	2	-	-	-	-
MicroVAX 3300/3400 (embedded EDA640)	1	-	2	-	-	-	-
VAX 4000 Model 105A (embedded SHAC 4)	1 + 1 4	-	2 5	-	-	-	-
VAX 4000 Model 200 (embedded SHAC 4)	1	-	2	-	-	-	-
VAX 4000 Model 300, 400, 500, 600	2	-	2	-	-	-	-
VAX 4000 Model 505A/705A (embedded SHAC³)	2 + 2 6	-	2	-	-	-	-
VAX 6000	-	-	-	-	-	6	-
VAX 7000	-	-	-	-	-	12	-

The following configuration guidelines apply to all DSSI clusters:

• Each DSSI interconnect can have up to eight nodes attached; four can be systems and the rest can be storage devices. Each of the following counts as a DSSI node:
  • DSSI adapter
  • Any member of the HSDxx family of DSSI and SCSI controllers
  • Any RF, TF, or EF integrated storage element (ISE)
  
  In some cases, physical cabling and termination limitations may restrict the number of systems to two or three that may be connected to a DSSI interconnect. For example:
  • Some variants of the DSSI adapter terminate the bus; for example, the N710. For this reason, only two DEC 4000 systems can be configured on a DSSI interconnect.
  • The size of a DEC or VAX 10000 system generally limits the number of systems that can be connected to a DSSI interconnect.
• Each DSSI adapter in a single system must be connected to a different DSSI bus.
• Configure VAX 6000, VAX 7000, and VAX 10000 systems with KFMSA adapters.
• Configure DEC 7000 and DEC 10000 systems with KFMSB adapters.
• Configure PCI-based AlphaServer systems with KFPSA adapters. EISA adapters (KFESA/KFESB) adapters can also be configured on most AlphaServer systems, but use of KFPSA is recommended whenever possible.
• Dual porting of devices between HSD controllers is supported as long as they are connected to the same or separate DSSI interconnects. Dual porting of devices between HSD controllers and local controllers is not supported.
• All systems connected to the same DSSI bus must have a common power or ground.

  6318P002.HTM
  OSSG Documentation
  26-NOV-1996 11:20:11.41

Copyright © Digital Equipment Corporation 1996. All Rights Reserved.

Legal