The vSphere Pluggable Storage Architecture
The vSphere Pluggable Storage Architecture is essentially a collection of plugins that reside inside the VMkernel layer of an ESXi host. The following figure shows a graphical representation of all the components of the PSA. The top-level plugin in the PSA is the Multipathing Plugin (MPP). The MPP defines how vSphere will manage and access storage including load balancing, path selection, and failover. The MPP itself can be provided by the storage vendor (IE EMC PowerPath) or you may also use the VMware provided Native Multipathing Plugin (NMP).
So essentially, the VMware provided NMP is in itself a MPP. The NMP is loaded by default for all storage devices, however, it can be overridden and replaced by installing a third-party MPP. Within each MPP, including the VMware NMP are two subplugins; Storage Array Type Plugin (SATP) and Path Selection Plugin (PSP). The SATP handles the details about path failover, whereas the PSP handles the details around load balancing and which physical path to use to issue an I/O request.
The VMware Pluggable Storage Architecture
Confused yet? I know the PSA is a lot to take in but it is essential to understand when you are troubleshooting storage issues. Let's have a look at each individual plugin included in the default VMware NMP in a little more detail to better understand the role it plays.
Pluggable Storage Architecture (PSA) roles and commands
The PSA performs two essential tasks as it pertains to storage:
- Discover which storage devices are available on a host
- Assign predefined claim rules associated with an MPP to take control of the storage device. Claim rules are explained in more detail in Chapter 3, Troubleshooting Storage Visibility.
In order to view a list of the PSA plugins, we use the storage core namespace of the esxcli command:
esxcli storage core plugin list
Multipathing Plugin – the VMware Native Multipathing Plugin roles and commands
The NMP/MPP performs the following functions:
- The MPP claims a physical path to the device, that is, SATP
- NMP comes with its own set of claim rules that associate certain SATP with a PSP
- Exports a logical device to the physical path contained in the PSP
To list devices controlled by the NMP with their respective SATP and PSP information, use the storage nmp namespace of esxcli, as outlined:
esxcli storage nmp device list
Storage Array Type Plugin roles and commands
The SATP plugin, which is a subplugin of the overall MPP, performs the following functions:
- Monitors the state of paths to the physical storage system
- Determines when a physical path is to be declared failed or down
- Handles the switching of physical paths after a path failure has occurred
- VMware provides a number of SATP plugins depending on which supported storage array is being used and also some generic active-active/active-passive SATP's for unknown storage arrays
To list the currently loaded SATP plugins along with their default PSP information, run the storage nmp namespace with esxcli.
esxcli storage nmp satp list
To change the default PSP associated with a given SATP, you can use the esxcli storage nmp satp set –b <boottime> -P <Default PSP> -s <SATP> command similar to the one shown in following screenshot:
Associate a default PSP with a SATP via esxcli
Path Selection Plugin roles and commands
The PSP, which is a subplugin of the overall MPP, provides the PSA with the following functionality:
- Responsible for choosing a path to issue an I/O request.
- Differs from SATP in terms that the PSP is a load balancing mechanism and deals with only active paths. The SATP determines which paths are active/standby/failed.
- VMware provides three default PSP plugins; Fixed, Most Recently Used, and Round Robin.
- The VMware NMP will select a default PSP based on which SATP plugin has been loaded for the storage array.
To list all of the available PSP plugins, you can use the storage nmp psp namespace of esxcli as shown:
esxcli storage nmp psp list
More information in regards to each of the default policies that VMware provides is listed in the following table:
To list certain configuration of the different PSP's on certain devices, you can use the esxcli storage nmp psp <PSP Namespace> deviceconfig get –d <device identifier command similar to the one shown in the following screenshot. On the flip side, you can set certain parameters by replacing get with set.
Retrieving configuration from a devices PSP via esxcli
Although we have used the ESXi Shell to obtain all of the information mentioned previously, we should note that it is possible to retrieve and change some of the information from within the vSphere Client as well. Most of these operations are done in the Storage section of the Configuration tab of a host.