Red Hat Knowledgebase

Resolution:

Release Found: Red Hat Enterprise Linux 4 and Red Hat Enterprise Linux 5

Limitation:
For this article, /dev/sda will be used as an example of the internal disk. Note that dm multipath is only supported by Red Hat Global Support Services on Red Hat Enterprise Linux 4 Update 2 and subsequent versions thereafter and on Red Hat Enterprise Linux 5.

Some machines have local SCSI cards for their internal disks. In these cases, device mapper is not recommended for seaching multipathed devices.

Determine which disks are the internal disks and mark them as the ones to blacklist. Prior to blacklisting these devices, notice that multipath -v2 may show the local disk in its multipath map:

[root@rh4cluster1 ~]# multipath -v2
create: SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1
[size=33 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 0:0:0:0 sda  8:0     

device-mapper ioctl cmd 9 failed: Invalid argument
device-mapper ioctl cmd 14 failed: No such device or address
create: 3600a0b80001327d80000006d43621677
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:0 sdb  8:16    
  \_ 3:0:0:0 sdf  8:80    

create: 3600a0b80001327510000009a436215ec
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:1 sdc  8:32    
  \_ 3:0:0:1 sdg  8:96    

create: 3600a0b80001327d800000070436216b3
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:2 sdd  8:48    
  \_ 3:0:0:2 sdh  8:112   

create: 3600a0b80001327510000009b4362163e
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:3 sde  8:64    
  \_ 3:0:0:3 sdi  8:128   

As highlighted in the example above, device mapper has mapped /dev/sda in its multipath maps. In order to stop this from happening, edit the /etc/multipath.conf file.

• Solution 1:

  One solution is to use the devnode keyword in the blacklist section:

  devnode_blacklist {
       ...
       devnode "^sda$"
       ...
  }
  

  If the machine has more than one internal disk, eg /dev/sda and /dev/sdb, then the line would read:

       devnode "^sd[ab]$"
  

  Note: It is important to add the $ symbol to the pattern match otherwise devices names from /dev/sdaa to /dev/sdaz will also be blacklisted because they match simply sda. This would become an issue if the machine has more than 26 /dev/sd* devices, which can happen if there are, say, 7 or more LUNs assigned to the machine with 4 paths per LUN.

  
  

• Solution 2:

  However there is no guarantee that the internal disk will always be assigned the device name /dev/sda. A better solution is to use the wwid keyword to blacklist the internal disk because the disk's WWID/serial number will never change. The WWID/serial number of /dev/sda is shown in the above multipath output. It may also be found by running multipath -ll or by using the scsi_id command, like so:

  [root@rh4cluster1 ~]# scsi_id -g -u -s /block/sda
  SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1
  

  So, using the wwid keyword to blacklist /dev/sda, on Red Hat Enterprise Linux 4 the blacklist section will look similar to this:

  devnode_blacklist {
        wwid SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1
        wwid 26353900f02796769
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][0-9]*"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
  }
  

  On Red Hat Enterprise Linux 5, the blacklist section will look similar to this:

  blacklist {
        wwid SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1
        wwid 26353900f02796769
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][0-9]*"
  }
  

To apply the new configuration, run these commands:

multipath -F
multipath -v2

The local disks should no longer be listed in the new multipath maps:

[root@rh4cluster1 ~]# multipath -F
[root@rh4cluster1 ~]# multipath -v2
create: 3600a0b80001327d80000006d43621677
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:0 sdb  8:16    
  \_ 3:0:0:0 sdf  8:80    

create: 3600a0b80001327510000009a436215ec
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:1 sdc  8:32    
  \_ 3:0:0:1 sdg  8:96    

create: 3600a0b80001327d800000070436216b3
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:2 sdd  8:48    
  \_ 3:0:0:2 sdh  8:112   

create: 3600a0b80001327510000009b4362163e
[size=12 GB][features="0"][hwhandler="0"]
\_ round-robin 0 
  \_ 2:0:0:3 sde  8:64    
  \_ 3:0:0:3 sdi  8:128  

To view the list of devices are being blacklisted, check the output of the multipath -v3 -ll command:

[root@rh4cluster1 ~]# multipath -v3 -ll | grep blacklist
...
sda blacklisted
...

Note: On Red Hat Enterprise Linux 4, wwid values may not show up in this output list; only the blacklisted devnode values are displayed. On Red Hat Enterprise Linux 5, both wwid and devnode blacklisted devices are displayed.