IBM only supports a queue depth of 1 when attaching to XIV with the default algorithm of round_robin. Usually round_robin or load_balance is the best choice, but since IBM is only supporting a queue depth of 1 at this time, there is a performance penalty for asynchronous I/Os. This looks to have been fixed in 5.3.10 (APAR IZ42730) and 6.1 (APAR IZ43146), but is still broken (probably never to be fixed) in earlier releases.
So, IBM recommendation is to split up your storage needs into a number of LUNs matching the number of paths to your XIV, use the fail_over algorithm with a larger queue depth, and assign a different path the highest priority for each LUN. This is kind of a poor man’s load balancing. It’s not that bad, other than having to look at 4 or more hdisks for every LUN, and having to figure out what path to give the highest priority for each one!
IBM doesn’t really see this as a problem, but it’s a huge pain to do correctly in an enterprise.
So, how do we start? First, figure out what hdisk you’re talking about, then run:
lspath -H -l hdiskx -F "status name parent path_id connection" status name parent path_id connection Enabled hdiskx fscsi0 0 50050763061302fb,4010401200000000 Enabled hdiskx fscsi0 1 50050763060302fb,4010401200000000 Enabled hdiskx fscsi1 2 50050763060802fb,4010401200000000 Enabled hdiskx fscsi1 3 50050763061802fb,4010401200000000
We need the parent device and the connection bit (WWN,LUN#) to specify just a single path. Then run:
lspath -AHE -l hdiskx -p fscsi0 -w "50050763061302fb,4010401200000000" attribute value description user_settable scsi_id 0x20400 SCSI ID False node_name 0x5005076306ffc2fb FC Node Name False priority 1 Priority True
That shows you the priority of this path. You can see it’s still the default of 1. You can check the other paths too.
The goal is to spread out the load between all the available paths. To do this, we must create 4 LUNs. If we need 4GB, we need 4 1GB LUNs. Then we can give each one a different primary path. So, in this example, we should run:
chpath -l hdiskx -p fscsi0 -w 50050763061302fb,4010401200000000 -a priority=1 chpath -l hdiskx -p fscsi0 -w 50050763060302fb,4010401200000000 -a priority=2 chpath -l hdiskx -p fscsi1 -w 50050763060802fb,4010401200000000 -a priority=3 chpath -l hdiskx -p fscsi1 -w 50050763061802fb,4010401200000000 -a priority=4
The first command isn’t really necessary, but I was on a roll. Now, we have to change the algorithm for the hdisk and set the queue depth:
chdev -l hdiskx -a algorithm=fail_over -a queue_depth=32
Make sure to stager the next one so that path 1 gets a priority of 1, 2 gets 2… and 0 gets a priority of 4. Rinse and repeat until you have 4 LUNs each with a different primary path.
Now wasn’t that easy. Oh, and when you add more disks, be sure to keep them distributed as evenly as possible.