Volume Shadowing for OpenVMS

Assisted merge operations are usually too short to be noticeable. Improved performance is possible during the assisted copy operation because it consumes less CPU and interconnect resources. Although the primary purpose of the performance assists is to reduce the system resources required to perform a copy or merge operation, in some circumstances you may also observe improved read and write I/O performance.

Volume Shadowing for OpenVMS supports both assisted and unassisted shadow sets in the same OpenVMS Cluster configuration. Whenever you create a shadow set, add members to an existing shadow set, or boot a system, the shadowing software reevaluates each device in the changed configuration to determine whether it is capable of supporting either the copy assist or the minimerge. Enhanced performance is possible only as long as all shadow set members are configured on controllers that support performance assist capabilities. If any shadow set member is connected to a controller without these capabilities, the shadowing software disables the performance assist for the shadow set.

When the correct revision levels of software are installed, the copy assist and minimerge are enabled by default, and are fully managed by the shadowing software.

8.2.2 Effects on Performance

The copy assist and minimerge are designed to reduce the time needed to do copy and merge operations. In fact, you may notice significant time reductions on systems that have little or no user I/O occurring during the assisted copy or merge operation. Data availability is also improved because copy operations quickly make data consistent across the shadow set.

Minimerge Performance Improvements

The minimerge feature provides a significant reduction in the time needed to perform merge operations. By using controller-based write logs, it is possible to avoid the total volume scan required by earlier merge algorithms and to merge only those areas of the shadow set where write activity is known to be in progress.

Unassisted merge operations often take several hours, depending on user I/O rates. Minimerge operations typically complete in a few seconds and are usually undetectable by users.

The exact time taken to complete a minimerge depends on the amount of outstanding write activity to the shadow set when the merge process is initiated, and on the number of shadow set members undergoing a minimerge simultaneously. Even under the heaviest write activity, a minimerge should complete in less than 1 minute. Additionally, minimerge operations consume minimal compute and I/O bandwidth.

Copy Assist Performance Improvements

Table 8-1 shows the approximate time required to perform copy operations on shadow sets that consist of two RA82 disks connected to an HSC70 controller on a VAX 8700 computer. The table shows assisted and unassisted copy times measured for one, two, three, and four concurrent copy operations, where all source and target disks are on separate HSC requesters.

Note that the copy times in Table 8-1 reflect the best possible performance measurements taken on a controlled test system with no user I/O load. Copy times vary according to each configuration and generally take longer on systems supporting user I/O. Performance benefits are also enhanced by having the source and target disks on different HSC requesters.

Table 8-1 Copy Times (In Minutes) for Two-Member RA82 Shadow Sets

Type of Shadowing	1 Set	2 Sets	3 Sets	4 Sets
Phase II: Assisted copy	9	17	25	34
Phase II: Unassisted copy with identical data	24	28	34	41
Phase II: Unassisted Copy with different data	91	120	157	200

Table 8-1 shows that, for a single copy operation, a phase II assisted copy is significantly faster than any other. Note that, as the number of simultaneous copies grows, the timing proportions change. This is caused by differing levels of parallelism in the various copying algorithms. Also, the overall copy times and their proportions differ with the disk type because each disk varies in total blocks and data transfer speed.

Additionally, Table 8-1 shows that unassisted phase II copies vary significantly in time, based on the similarity of data on the source and target disks. To make blocks with different data consistent, extra I/O operations are required. Thus, unassisted copy operations take significantly longer to make a shadow set consistent when the disk members contain different data. The phase II assisted copy operation does not depend on the similarity of data on the disks.

In addition, the following subsection describes how you can control the number of assisted copies an HSC controller can perform.

Determining the DCD Connection Limit

The default DCD connection limit of 4 is based on measurements obtained from tests of simultaneous assisted copies on several types of disks. In most cases, if more than four simultaneous copies are required, lower copy times are achieved when the additional copies are unassisted. For example, if six copies are necessary, four are assisted and two are unassisted. The default setting assumes that any unassisted copies will be performed on disks with nearly identical data.

You may need to increase the DCD connection limit under the following circumstances:

• Copy operations are generally performed on disks with dissimilar data. In this case, unassisted copies take significantly longer than assisted copies. Therefore, setting a higher DCD connection limit would be advantageous.
• Good user I/O performance is more important than total elapsed time for the copy operations. Assisted copy operations conserve system I/O resources, thereby permitting good user I/O performance.

The best mix of assisted and unassisted copies can vary based on the CPU, the HSC, and the disks involved because each environment has a different performance profile.

Setting the Number of Copy Assists Per HSC

You can use the HSC command SET SERVER DISK/DCD_CONNECTION_LIMIT to control the mix of assisted and unassisted copies performed by a given HSC.

Follow these steps for each HSC controller on which you want to change the DCD connection limit.

1. Press Ctrl/C to get to the HSC> prompt.
2. At the HSC prompt, enter the following commands to set the DCD connection limit to 6:

   HSC> RUN SETSHO
   SETSHO> SET SERVER DISK/DCD_CONNECTION_LIMIT=6
   SETSHO> 
   

Sections 8.1 and 8.2 describe the performance impacts on a shadow set in steady state and while a copy or merge operation is in progress. In general, performance during steady state compares with that of a nonshadowed disk. Performance is affected when a copy or a merge operation is in progress to a shadow set. In the case of copy operations, you control when the operations are performed.

However, merge operations are not started because of user or program actions. They are started automatically when a CPU fails. In this case, the shadowing software reduces the utilization of system resources and the effects on user activity by throttling itself dynamically. Minimerge operations consume few resources and complete rapidly with little or no effect on user activity.

The actual resources that are utilized during a copy or merge operation depend on the access path to the member units of a shadow set, which in turn depends on the way the shadow set is configured. By far, the resources that are consumed most during both operations are the adapter and interconnect I/O bandwidths.

You can control resource utilization by setting the SHADOW_MAX_COPY system parameter to an appropriate value on a CPU based on the type of CPU and the adapters on the machine. SHADOW_MAX_COPY is a dynamic system parameter that controls the number of concurrent copy or merge threads that can be active on a single CPU. If the number of copy threads that start up on a particular CPU is more than the value of the SHADOW_MAX_COPY parameter on that CPU, only the number of threads specified by SHADOW_MAX_COPY will be allowed to proceed. The other copy threads are stalled until one of the active copy threads completes.

For example, assume that the SHADOW_MAX_COPY parameter is set to 3. If you mount four shadow sets that all need a copy operation, only three of the copy operations can proceed; the fourth copy operation must wait until one of the first three operations completes. Because copy operations use I/O bandwidth, this parameter provides a way to limit the number of concurrent copy operations and avoid saturating interconnects or adapters in the system. The value of SHADOW_MAX_COPY can range from 0 to 200. The default value is 4.

Chapter 3 explains how to set the SHADOW_MAX_COPY parameter. Keep in mind that, once you arrive at a good value for the parameter on a node, you should also reflect this change by editing the MODPARAMS.DAT file so that when invoking AUTOGEN, the changed value takes effect.

In addition to setting the SHADOW_MAX_COPY parameter, the following list provides some general guidelines to control resource utilization and the effects on system performance when shadow sets are in transient states.

• Create or add members to shadow sets when your system is lightly loaded.
• The amount of data that a system can transfer during copy operations varies depending on the type of disks, interconnect, controller, the number of units in the shadow set, and the shadow set configuration on the system. For example, approximately 5% to 15% of the Ethernet or CI bandwidth might be consumed for each copy operation (for disks typically configured in CI or Ethernet environments).
• While creating unassisted three-member shadow sets, add all members at the same time in a single mount command rather than in two separate mount commands. Adding several members at once optimizes the copy operations by starting a single copy thread that reads from the source members once, and performs write I/O requests to the target members in parallel.
• For satellite nodes in a mixed-interconnect or local arean OpenVMS Cluster system, set the system parameter SHADOW_MAX_COPY to a value of 0 for nodes that do not have local disks as shadow set members.
• Do not use the MOUNT/CLUSTER command to mount every shadow set across the cluster unless all nodes must access the set. Instead, use the MOUNT/SYSTEM command to mount the shadow sets on only those nodes that need to access a particular set. This reduces the chances of a shadow set going into a merge state. Because a shadow set goes into a merge state only when a node that has the set mounted fails, you can reduce the chances of this happening by limiting the number of nodes that mount a shadow set, especially when there is no need for a node to access the shadow sets.
• Because a copy operation can occur only on nodes that have the shadow set mounted, create and mount shadow sets on the nodes that are local (have direct access) to the shadow set members. This allows the copy threads to run on these nodes, resulting in faster copy operations with fewer resources utilized.
• If you have shadow sets configured across nodes that are accessed through the MSCP server, you might need to increase the value of the MSCP_BUFFER system parameter in order to avoid I/O fragmentation. You can determine whether you need to increase the MSCP_BUFFER parameter by running AUTOGEN with FEEDBACK specified for the served nodes. You can use the feedback reports to observe how efficient the current setting of MSCP_BUFFER is.
• Dual-pathed and dual-ported shadowed disks in a OpenVMS Cluster system can provide additional coverage against the failure of nodes that are directly connected to shadowed disks. This type of configuration provides higher data availability with reasonable performance characteristics.
• Use the preferred path option to ensure dual-ported drives are accessed via the same controller so that the shadowing software will perform assisted copy operations.

Digital's StorageWorks RAID Software for OpenVMS provides ways to configure and use disk drives so that they achieve improved I/O performance. RAID (redundant arrays of independent disks) uses striping technology to chunk data and distribute it across multiple drives. RAID software is available in various levels, one of which is volume shadowing. Table 8-2 describes RAID levels.

Table 8-2 RAID Levels

RAID Level	Description
Level 0	Striping with no redundancy.
Level 1	Shadowing.
Levels 0 + 1	Striping and shadowing together.
Level 3	Striped data with dedicated parity drive. Drives are rotationally synchronized.
Level 5	Striped data and parity.
Level 6	Striped data and parity with two parity drives.

Shadowing striped drives can increase both performance and availability, because you can achieve faster response time with striping and data redundancy with shadowing. In addition to shadowing striped sets, you can also stripe shadow sets. Each strategy offers different advantages and tradeoffs in terms of availability, performance, and cost.

For more information about RAID 0, see the POLYCENTER product documentation set. For more information about RAID 5, see the StorageWorks RAID 5 Software for OpenVMS User's Guide.

As of OpenVMS AXP and OpenVMS VAX Versions 6.2, phase I Volume Shadowing was not available. You can migrate from phase I (controller-based) shadowing to phase II (host-based) shadowing using the information in this appendix, which contains:

• An overview of phase II shadowing features
• How to migrate from phase I to phase II
• A more detailed comparison of phase I and phase II shadowing

Phase II shadowing is designed to fully replace phase I with significantly enhanced features, such as:

• State-of-the-art failover and recovery features and superior data availability
• Support for many more disk controllers and devices in a wider range of configurations
• Support for the HSC60 and HSC90 cache option
• Simplified system management of shadow sets
• Increased availability of shadowed disks
• Improved data reliability
• Support for a large number of shadow sets on a standalone or OpenVMS Cluster system
• Opportunities to use the controller performance assists (see Chapter 6) for enhanced performance

Migrating from phase I to phase II requires dismounting and remounting disks and adjusting shadowing parameters and bootstrap values. This section contains instructions for this process.

A.2.1 Migrating Nonsystem Disk Shadow Sets

To migrate a nonsystem disk shadow set from phase I to phase II shadowing, dismount the phase I shadow set and remount it using the phase II mount semantics. No other preparation is necessary unless you are migrating a system disk shadow set or unless you are preparing to use the copy and merge performance assists. Section A.2.4 includes a sample migration session to help you migrate to phase II volume shadowing.

A.2.2 Migrating System Disk Shadow Sets

To migrate system disk shadow sets to phase II shadowing, you need to reset all system parameters for shadowing, change the VMB registers, and then reboot all systems that are using this shadow set.

A.2.3 Adjusting Parameters and Bootstrap Values

When you migrate the shadow sets to phase II volume shadowing, you must:

• Set the SHADOWING parameter to 2 to indicate phase II.
• Set the SHADOW_SYS_DISK parameter to 1 if the system disk is shadowed.
• Set the SHADOW_SYS_UNIT parameter to the desired virtual unit number.
• Remove any MOUNT commands for the system disk shadow set if you are migrating your system disk to a phase II shadow set. See also Section 3.3 for information about booting from a system disk shadow set.
• If the system disk was a phase I shadow set, change the value in the high-order word (bits <31:16>) of the VMB (primary bootstrap) R3 to zeros.

  ---

  Note

  The high-order word (bits <31:16>) of console register R3 should be set to zeros for phase II volume shadowing. This is input for VMB, the primary bootstrap image for OpenVMS, which is passed via the BOOT command or the boot command file. The low-order word (bits <15:0>) of register R3 should be set to the same value on nodes that are booting off a particular system disk shadow set.

  ---

• Prior to rebooting, edit SYSTARTUP_VMS.COM and other command files to change the syntax of all MOUNT command lines to reflect phase II volume shadowing naming conventions.
• Reboot the system or OpenVMS Cluster.

If you cannot reboot the entire OpenVMS Cluster system at the same time, you must dismount the phase I shadow sets and remount them as phase II shadow sets one at a time (see Section A.2.4).

See Section 3.2 for more information about shadowing parameters.

A.2.4 Sample User Session

Examples

Examples

$ DISMOUNT/NOUNL $255$DUS45 (1)
$ SHOW DEVICES $255$DUS45 (2) 
Device                  Device           Error    Volume         Free Trans Mnt
 Name                   Status           Count     Label        BlocksCount Cnt
$255$DUS45:      (DUD)  Online               0
$ SHOW DEVICES $255$DUA45 (3)
Device                  Device           Error    Volume         Free Trans Mnt
 Name                   Status           Count     Label        BlocksCount Cnt
$255$DUA45:       (MUD)  Online               0
$ MOUNT/CLU DSA34/SHAD=$255$DUA45 VOLUMELABEL LOGICALNAME (4)
%MOUNT-I-MOUNTED, VOLUMELABEL mounted on _DSA34:
%MOUNT-I-SHDWMEMSUCC, _$255$DUA45: (DUD) is now a valid member of the shadow set

The following list describes the elements in

Examples

1. Dismounts the virtual unit $255$DUS45 to dissolve the phase I shadow set.
2. Shows that $255$DUS45 is on line, but it no longer represents a shadow set.
3. Shows that $255$DUA45 is on line, but it is not a shadow set member.
4. Mounts $255$DUA45 as a phase II shadow set represented by the DSA34 virtual unit.

Although there are many differences between the two shadowing phases, the most noticeable differences are:

• Phase II shadowing allows greater flexibility in configurations, especially in the types of controllers and disks supported. (Note that phase II requires that KDM70 disk controllers run a minimum of Version 3.0 microcode.)
  See Chapter 2 for information about shadow set configurations.
• Phase II shadowing allows more shadow sets in a configuration.
  As with phase I shadowing, a phase II shadow set can have a maximum of three shadow set members. However, phase II shadowing allows more shadow sets in standalone and in OpenVMS Cluster systems. See the most recent Volume Shadowing for OpenVMS Software Product Description for information about the maximum number of shadow sets supported. (For OpenVMS Version 7.1 systems, see SPD 25.01.xx.)

  
  

• The manner in which a shadow set is served in an OpenVMS Cluster configuration.
  For a phase II shadow set, the member units (not the virtual unit) are served to the OpenVMS Cluster system. All systems that mount a phase II shadow set manage that shadow set collectively.

Phase II volume shadowing allows shadowing on the same configurations as phase I. In addition, it provides distributed shadowing in a controller-independent manner. Phase II volume shadowing supports clusterwide shadowing of all DSA disks that are compliant with the MSCP protocol or SCSI devices having identical physical geometry. The disks can be on a single system or located anywhere in a OpenVMS Cluster system. Note, however, that you need not have OpenVMS Cluster software to run volume shadowing on a single, standalone node.

Shadowing also provides full support for all SCSI devices and for some third-party SCSI devices. Volume Shadowing for OpenVMS can support third-party devices that implement READL (read long) and WRITEL (write long) commands because phase II shadowing software uses the optional SCSI READL and WRITEL commands.

Phase II volume shadowing supports clusterwide shadowing of all DSA and supported SCSI devices. Phase II is not limited to HSC disks; rather, it extends volume shadowing capabilities to all DSA disks, including those on local adapters, all Digital Small Systems Interconnect (DSSI RF-series) disk devices on any VAX computer, all interfaces (CI, Ethernet, mixed interconnect), FDDI, and across MSCP servers.

The underlying design of phase II volume shadowing is different from phase I, but the user interface for the two shadowing products is similar. The OpenVMS Mount utility commands, SHOW commands, and system services for both shadowing products have only a few differences:

• Phase II shadowing uses the DSAn: format to name virtual units.
• You must set the SHADOWING parameter value to 2 to run phase II shadowing on each node. (See Chapter 3 for more information.)
• The Mount utility provides an additional qualifier (/INCLUDE) for phase II shadowing. The /INCLUDE qualifier is useful for automating the task of rebuilding former shadow sets. (See Chapter 4 and Chapter 5 for more information.)
• Nodes that mount a phase II shadow set must have a Volume Shadowing for OpenVMS license installed. For further information, see (regist_sect) or contact your local Digital sales office.

Application I/O semantics for shadow set manipulation are the same for both phase I and phase II shadowing.

A.3.1 Interdependencies Between Operating System Versions and Volume Shadowing Phases

Following are some dependencies to consider when you think about migrating to phase II volume shadowing:

• A VAX node that currently runs VMS Version 5.4 or earlier can use only VAX Volume Shadowing (phase I). Such nodes cannot access a phase II shadow set.
• Any phase I shadow sets that you have mounted are not accessible from OpenVMS Version 6.2 systems.
• Only phase II is supported as of OpenVMS AXP and OpenVMS VAX Versions 6.2.
• An OpenVMS Cluster that contains both Alpha and VAX computers has version interoperability requirements that are described in OpenVMS Cluster Version 7.1 Software Software Product Description 29.78.xx.

This appendix lists volume shadowing status messages that are displayed on the console device. For other system messages that are related to volume shadowing, use the Help Message utility. For information about the HELP/MESSAGE command and qualifiers, see DCL help (type HELP HELP/MESSAGE at the DCL prompt). Messages that can occur before a system is fully functional are also included in OpenVMS System Messages: Companion Guide for Help Message Users.

B.1 Mount Verification Messages

The following mount verification messages have approximately the same meaning for shadow sets as they do for regular disks. They are sent to the system console (OPA0) and to any operator terminals that are enabled to receive disk operator messages.

• virtual-unit: is off line. Mount verification in progress.
• virtual-unit: has completed mount verification.
• virtual-unit: has aborted mount verification.

The following OPCOM message is returned in response to shadow set operations. This message results when the shadowing code detects that the boot device is no longer in the system disk shadow set. If the boot device is not added back into the system disk shadow set, the system may not reboot, and the dump may be lost if the system crashes.

virtual-unit: does not contain the member named to VMB. System may not reboot.

• The boot device is dismounted or failed out of the system disk shadow set.
• Shadowing finds the boot device missing from the system disk shadow set membership during any dismount operations on the system disk.

• Mount the boot device back into the shadow set as soon as possible.
• If you cannot mount the boot device back into the shadow set, change the device name in VMB (primary bootstrap) so the system can reboot when necessary.

Shadow server operations can display the following status messages on the system console (OPA0) and on terminals enabled to receive operator messages.

  5423P008.HTM
  OSSG Documentation
  22-NOV-1996 13:03:48.04

Copyright © Digital Equipment Corporation 1996. All Rights Reserved.

Legal