HP OpenVMS I/O User's Reference Manual |
Optional Features for Improving I/O Performance |
|
|
| |
Fast Path (Alpha and I64 Only)
Fast Path addresses these factors as follows:
Using Fast Path features does not require source-code changes. It does require major changes to device drivers, so it has been implemented only for the newer high-performance devices. These currently service many CI, Fibre Channel, parallel SCSI, and LAN devices.
Supported Ports for Each Version of OpenVMS Alpha and I64 lists the supported ports for each OpenVMS Alpha version.
Prior to OpenVMS Alpha Version 7.3-1, all hardware interrupts took place on the primary CPU. Interrupts from Fast Path enabled devices would have to be redirected from the primary CPU to a ''preferred'' CPU. However, this redirection still involved the primary CPU, and also incurred interprocessor overhead.
Starting with OpenVMS Alpha Version 7.3-1, hardware interrupts that are targetted for a ''preferred'' CPU go directly to the ''preferred'' CPU, thereby eliminating any I/O processing in the primary CPU. This major Fast Path enhancement is known as distributed interrupts.
| This feature is available on Fibre Channel, CI, and some SCSI ports on AlphaServer DS20, ES40/45, and GS series systems. |
Using
Fast Path Features ![]()
Preferred CPU Selection
All Fast Path ports are assignable to CPUs. You can set a system parameter specifying the set of CPUs that are allowed to serve as preferred CPUs. This set is called the set of allowable CPUs. At any point in time, the set of CPUs that currently can have ports assigned to them, called the set of usable CPUs, is the intersection of the set of allowable CPUs, and the current set of running CPUs.
Each Fast Path Port is initially assigned to a CPU by the FASTPATH_SERVER process that runs at port initialization time. This process executes an automatic assignment algorithm that spreads Fast Path ports evenly among the usable CPUs. The FASTPATH_SERVER process also runs whenever a secondary CPU is started, and whenever the set of system parameters specifying the allowable CPUs is changed.
If the primary CPU is in the set of allowable CPUs, the initial distribution will be biased against the primary CPU in that a port will only be assigned to the primary after ports have been assigned to each of the other usable CPUs.
To identify a device or port's current preferred CPU, you can use either $GETDVI or the SHOW DEVICE/FULL command. To identify the Fast Path ports currently assigned to a CPU, you use the SHOW CPU /FULL command.
You can directly assign a Fast Path port to a CPU, or request the system to automatically select the port's preferred CPU from a specific set of CPUs. To do this, you either issue a $QIO or use the SET DEVICE/PREFERRED_CPU command. This will also set the port's User Preferred CPU to be the selected CPU.
You can clear the port's User Preferred CPU by issuing either a $QIO, or by using the SET DEVICE/NOPREFERRED CPU DCL command.
You can redistribute the system assignable Fast Path ports across a subset of the set of usable CPUs by calling the $IO_FASTPATH system service.
Optimizing Application Performance
Processes running on a port's preferred CPU have an inherent advantage when issuing I/O to a port in that the overhead to assign the I/O to the preferred CPU can be avoided. An application process can use the $PROCESS_AFFINITY system service to assign itself to the preferred CPU of the device to which the majority of its I/O is sent.
With proper attention to assignment, a process's execution need never leave the preferred CPU. This presents a scalable process and I/O scheme for maximizing multiprocessor system operation. Like most RISC systems, Alpha system performance is highly dependent on the performance of CPU memory caches. Process assignment and preferred CPU assignment are two keys to minimizing the memory stalls in the application and in the operating system, thereby maximizing multiprocessor system throughput.
Managing
Fast Path ![]()
This section describes how to manage Fast Path.
Fast Path System Parameters ![]()
There are three FAST_PATH system parameters:
These parameters can be used to control Fast Path as follows:
Identifying and Setting a Port's Preferred
CPU ![]()
Following are the commands used to identify and set a preferred
CPU for a port.
DCL SHOW DEVICE/FULL or $GETDVI DVI$_PREFERRED_CPU | ||
To identify the preferred CPU for any Fast Path-capable device when Fast Path is enabled, use the DCL command SHOW DEVICE/FULL to display -- whether or not the device supports Fast Path -- the current preferred CPU ID and, if set, the User Preferred CPU ID for a port or disk device. Alternatively, the $GETDVI system service or the DCL F$GETDVI lexical function will return the preferred CPU for a given device or file. The $GETDVI system service item code is DVI$_PREFERRED_CPU, and the F$GETDVI item code string argument is PREFERRED_CPU. The return argument is a 32-bit CPU bit mask with a bit set indicating the preferred CPU. A return argument containing a bit mask of zero indicates that no preferred CPU exists, either because Fast Path is disabled or the device is not a Fast Path-capable device. The return argument serves as a CPU bit mask input argument to the $PROCESS_AFFINITY system service. The argument can be used to assign an application process to the optimal preferred CPU. For an application seeking optimal Fast Path benefits, you can code each application process to identify and run on the preferred CPU where the majority of the process' I/O activity occurs. A high-availability feature of OpenVMS Cluster Systems is that dual-pathed devices automatically fail over to a secondary path, if the primary path becomes inoperable. Because a Fast Path device could fail over to another path or port, and thereby, to another preferred CPU, an application should occasionally reissue the $GETDVI in a timer thread to check that process assignment is optimal. | ||
DCL SHOW CPU /FULL | You can use this DCL command to identify whether a CPU is enabled for use as a preferred CPU, and the current set of ports assigned to that CPU. | |
DCL SET /PREFERRED_CPU and /NOPREFERRED_CPU | ||
These commands allow you to specify a CPU or a set of candidate CPUs from which the operating system will choose the CPU to assign to the Fast Path port. The chosen CPU is called the preferred CPU for this Fast Path port. The Fast Path port's interrupt I/O completion processing and I/O initiation processing will be performed on this preferred CPU. In addition to selecting the preferred CPU, the User Preferred CPU will be set for this port. Setting the User Preferred CPU prevents the port from being reassigned to another CPU unless the User Preferred CPU is being stopped. The qualifier can be negated. When the /NOPREFERRED_CPUS qualifier is specified, the User Preferred CPU will be cleared for the port, but it still remains a Fast Path port, and the current preferred CPU will not be changed. If both /PREFERRED_CPUS and /NOPREFERRED_CPUS are specified on the same command line, /NOPREFERRED_CPUS is ignored. | ||
$QIO IO$_SETPRFPATH ! IO$M_PREFERRED_CPU [!IO$M_SYS_ASSIGNABLE] | ||
You can change the assignment of a Fast Path port to a CPU by issuing a $QIO IO$_SETPRFPATH (Set Preferred Path) to the port device, for example, PNA0. The IO$M_PREFERRED_CPU modifier must be set, and the $QIO argument P1 must be set to either 0 or the address of a 32-bit CPU bit mask with a bit set indicating the new preferred CPU. On return from the I/O, the port and its associated devices are all assigned to a new preferred CPU. Note that explicitly setting the preferred CPU overrides any default assignment of Fast Path ports to CPUs. This interface allows you the flexibility to load balance I/O activity over multiple CPUs in an SMP system. This is important because I/O activity can change over the course of a day or week. The $QIO passes in either a set containing one or more candidate CPUs, or 0 as a wildcard value indicating the set of usable CPUs. If the candidate set contains only one CPU, you are explicitly designating the new preferred CPU. If the candidate set contains multiple CPUs, you are requesting use of the automatic preferred CPU assignment algorithm to select a suitable CPU from the candidate set. Including the IO$M_SYS_ASSIGNABLE modifier inhibits setting the selected CPU as the device's User Preferred CPU. The $QIO or the SET DEVICE/PREFERRED_CPU command will make a best effort to assign the port to a CPU. However, it is possible for this request to return failure for the following reasons:
| ||
$IO_FASTPATH | The $IO_FASTPATH system service performs operations
on the set of Fast Path devices and CPUs enabled for Fast Path use.
The $IO_FASTPATHW system service completes synchronously. That is,
it returns after the operation is complete. The FP$K_BALANCE_PORTS function code specifies that the system service is to distribute the set of system assignable Fast Path ports across the intersection of a caller-supplied set of candidate CPUs. | |
Fast Path Restrictions ![]()
Fast Path restrictions include the following:
Special
Considerations for Fast Path on Multi-RAD Systems ![]()
On systems supporting multiple resource affinity domains (RADs),
the best performance for Fast Path ports is usually obtained by
setting the Fast Path preferred CPU assignment to a CPU within the
same RAD as the port.
The FASTPATH_SERVER restricts its distribution of ports accordingly whenever possible. If a port should be within a RAD without available Fast Path CPUs, the system will set the preferred CPU to the primary CPU.
Because you can override this assignment by the methods described in this chapter, care should be taken that reassignment does not sacrifice the performance improvements provided by localizing activity to a single RAD.
|
|