How to manage PSA claimrules and SATP rules with esxcli

I will not try to explain PSA here, there is enough resources to read about it (if you asked me for recommendation I’d suggest Cormac Hogan’s excellent blog) and I don’t feel expert enough to “increase entropy” on that subject.

Instead I’d like to focus on what used to be particularly confusing to me: the claimrules, SATP rules and how to manage them with esxcli.

PSA behaviour can be controlled with two sets of rules (at least I know two of them – maybe there is more 😉 ) first there is set of “claimrules” that decide which PSA plugin (NMP or 3rd party) will handle any particular storage device. Once that is decided there is another set of rules – “SATP rules” which decide which SATP exactly (out of the set of SATPs available in selected MP plugin) will handle the I/O.

Initially I was confusing these two sets because esxcli syntax to manipulate them is very similar and I was easily loosing track of what is going on with PSA on my vSphere host.

Just to set the focus – all listings that will follow come from vSphere 5.5 host in my homelab.

So let’s have a look at “claimrules” first.

As you can see I only have Native Multipathing Plugin (NMP) installed, the claimrule list is also pretty generic, with either NMP or MASK_PATH plugins applied and MASK_PATH is used to hide some specific (unsupported?) SAN device from vSphere. These claimrules are checked in descending order (so if you configure more than one rule that could match a device the one that is listed first will apply).
You can also see a “catch all” rule at the bottom of the list, it is there to make sure that any device detected yet not matched with previous rules is handled with NMP.

I also show information about specific storage path (which happens to be iSCSI LUN that I share from OpenFiler appliance) because I want to show how to change claimrules to mask this particular LUN.

I use esxcli storage core claimrule add command here to create new claimrule, let vSphere autoassign the rule number, then I specify type of “target” and provide as much details as possible that will allow device to be matched (so I declare the protocol needs to be iSCSI, coming from specific iqn with LUN 2)
I want this LUN to be masked, so I select MASK_PATH plugin.
The rule shows immediately in the list, yet only in “file” class, which means it is saved, but vmkernel is not using it at the moment and we need to (re)load the claimrules.
The final step after loading claimrules (our claimrule is listed twice now, both in “file” and “runtime” class) is to reclaim the device (to activate our new rule) and… we’re ready.
Information about the path shows now that plugin has changed to MASK_PATH and the path state is DEAD!

You have to believe me that this device disappears from vSphere Client immediately after what we just did.

OK, let’s try to revert this now and make LUN 2 visible again.

Now “unmasking” a LUN that we’ve hidden with MASK_PATH plugin can be a bit tricky, at the beginning we need to remove the rule (using just the rule number) from wherever it is stored, but as you can see “remove” operation does not immediately deactivate the claimrule (it is still listed with “runtime” class).
This is solved by (re)loading the whole claimrule set, but subsequent attempt to reclaim the device fails with “Unknown Device” error…
Well it makes sense, we’ve just masked it a moment ago, so t10 identifier we use disappeared from our system…

At first I thought host reboot is the only solution to make that LUN visible again, but after a few minutes of playing around with esxcli I discovered there is a way to avoid a reboot.

We can not reclaim a device, but we can set the storage path (using runtime identifier) that “leads” to device to be “unclaimed” and then rescan our storage adapters so that default “catch all” claimrule is matched and our LUN gets handled by NMP again, you can see all the steps listed above.

The thing worth to remember is that claimrules are really easy – it is just about which MP plugin to assign to storage devices.

SATP rules on the other hand can get really complicated, cause you can have many SATPs as “sub-plugins” to MP plugin that is selected. NMP itself (so the PSA plugin that you get “out-of-box”, just with “vanilla” vSphere installation) has 13 of these SATPs, as a result SATP rule set gets much bigger and if you consider that each SAN can require some specific options to be provided by the means of SATP rules… Well I suggest you just to execute

 esxcli storage nmp satp rule list

from your host’s console and admire this whole list scrolling.

Yet the esxcli syntax to manipulate these SATP rules is very similar to what we exercised with claimrules (and this is what was confusing me a lot when I first started to use them).

As an example I will show you how to “make” an SSD out of traditional, magnetic hard disk (something you can probably hear about at every VMware training, also “tier-1” bloggers like Duncan or William have posted about it like ages ago).

The only thing we do here is to add a rule that assigns VMW_SATP_LOCAL and sets “enable_ssd” option for our mpx.vmhba1:C0:T1:L0 device. Luckily we don’t need to (re)load anything just reclaim the device – and here we go, our magnetic drive will be considered SSD by vSphere from now on (you can see “Is SSD” property has value of “true”).

To be honest – reclaiming device just for SATP rule change also used to confuse me a lot – even though we only modify some SATP option it looks like whole PSA “stack” needs to be reset for this operation (including initial MP plugin selection for device we are working on).

The other observation I have – it seems like SATP rules are applied in different manner than “claimrules”, I’ve written above that “claimrules” are applied in descending order (and they have numbers!), as for SATP I think these rules are applied according to “best match” policy (same way like network gateway to forward traffic to is selected if you have multiple TCP/IP static routes defined). If you look at SATP rules list for VMW_SATP_LOCAL above there are at least two rules (“transport = sata” configured by default and our “device = mpx.vmhba1:C0:T1:L0” rule) that match, yet our trick works, so it seems “best match” policy is used (cause our rule is more precise, so makes better match than generic “transport” class rule).
If not “best match” policy, maybe the order in which rule classes are presented in listing above (Device > Vendor > Model > Driver > Transport) is the key: so if you include device name in your rule definition it overrides all rules with matching vendor or driver?
Oh and have you noticed? SATP rules don’t have numbers!

But again – this is only an “educated guess” based on some of my experiments, I wasn’t able to find any documentation that would prove me right or wrong and explain in what order SATP rules are really applied.

OK, it is fun to make a fool of vmkernel for a while, but just having SSD enabled for a spindle hard drive will not give you any performance boost, in fact it will give you nothing but trouble if you continue like this.
The only real-life scenario you might want to use this trick is when you have SSD from some “out-of-mainstream” vendor and capabilities of this disk are not recognized properly by vSphere, then you have to configure this SATP rule to get the performance you paid for.

Since I used this rule against magnetic disk, I’m going to revert my settings to their original state.

Removing SATP rules is not as easy as with claimrules – you can not use rule number (they don’t have numbers, right?), so there is a need to provide the exact syntax the rule was created with.
Once the rule is removed – just reclaim the device again and everything is back to normal.

I hope you find this lengthy post useful, any comments are welcome!

Sebastian Baryło

Successfully jumping to conclusions since 2001.

  • Guest

    Small digression. Would you be able to configure PSA related settings via host profile? It might be useful if you think about VCAP examination. Really.

    • sbarylo

      PSA settings can be controlled via host profile of course, still you need to configure them first (with esxcli, powercli or even gui) on your reference host and then extract (save) the profile. At least this is how I am using host profiles and this is how VMware recommends to do it (just because profile editor is not very ergonomic tool I guess 😉 )
      I don’t quite get it how does it relate to VCAP exam?