Tag: scsimgr

HPUX: vmunix: Evpd inquiry page 83h/80h failed or the current page 83h/80h data do not match the previous known page 83h/80h data on LUN id 0x0 probed beneath the target path (class = tgtpath, instance = 27)

Looking at the logfile /var/adm/syslog/syslog.log I saw the messages

Mar 26 17:06:17 hpux vmunix: Evpd inquiry page 83h/80h failed or the current page 83h/80h data do not match the previous known page 83h/80h data on LUN id 0x0 probed beneath the target path (class = tgtpath, instance = 37) The lun path is (class = lunpath, instance 33).Run ‘scsimgr replace_wwid’ command to validate the change
Mar 26 17:06:47 hpux vmunix: Evpd inquiry page 83h/80h failed or the current page 83h/80h data do not match the previous known page 83h/80h data on LUN id 0x0 probed beneath the target path (class = tgtpath, instance = 27) The lun path is (class = lunpath, instance 25).Run ‘scsimgr replace_wwid’ command to validate the change

Running the command to validate the change

root@hpux:~ # scsimgr replace_wwid -C lunpath -I 27
scsimgr:WARNING: Performing replace_wwid on the resource may have some impact on system operation.
Do you really want to replace? (y/[n])? y
Binding of LUN path 0/2/0/0/0/0.0x5d8d385c1a8e4000.0x0 with new LUN validated successfully

root@hpux:~ # scsimgr -f replace_wwid -C lunpath -I 37
Binding of LUN path 0/4/0/0/0/1.0x5d8d385c1a8e4010.0x0 with new LUN validated successfully

Advertisement

HP-UX: UXMON:Critical multipath error detected. Please see /var/opt/OV/log/OpC/scsi_mon.log for details.

I received this problem where the disk was showing some multipath erros on an HP-UX server

UXMON:Critical multipath error detected. Please see /var/opt/OV/log/OpC/scsi_mon.log for details.

Node : hpux.setaoffice.com
Node Type : Itanium 64/32(HTTPS)
Severity : critical
OM Server Time: 2016-05-24 11:35:09
Message : UXMON:Critical multipath error detected. Please see /var/opt/OV/log/OpC/scsi_mon.log for details.
Msg Group : OS
Application : scsimon
Object : No
Event Type :
Instance Name :
Instruction : No

root@hpux:~ # cat /var/opt/OV/log/OpC/scsi_mon.log
Tue May 24 12:35:07 2016 : Critical /dev/rdisk/disk670 has failed lunpaths! Please check with scsimgr -p lun_map -D /dev/rdisk/disk670

root@hpux:~ # scsimgr -p lun_map -D /dev/rdisk/disk670
lunpath:647:38/0/0/2/0/0/0.0x50060e800574f200.0x405c000000000000:fibre_channel:FAILED:FAILED
lunpath:632:38/0/0/2/0/0/1.0x50060e800574f210.0x405c000000000000:fibre_channel:FAILED:FAILED
lunpath:617:36/0/0/2/0/0/0.0x50060e800574f200.0x405c000000000000:fibre_channel:FAILED:FAILED
lunpath:602:36/0/0/2/0/0/1.0x50060e800574f210.0x405c000000000000:fibre_channel:FAILED:FAILED
lunpath:33:36/0/0/2/0/0/0.0x50060e800574f200.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:35:36/0/0/2/0/0/1.0x50060e800574f210.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:37:38/0/0/2/0/0/0.0x50060e800574f200.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:39:38/0/0/2/0/0/1.0x50060e800574f210.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE

Removing invalid paths

root@hpux:~ # rmsf -H 38/0/0/2/0/0/0.0x50060e800574f200.0x405c000000000000
root@hpux:~ # rmsf -H 38/0/0/2/0/0/1.0x50060e800574f210.0x405c000000000000
root@hpux:~ # rmsf -H 36/0/0/2/0/0/0.0x50060e800574f200.0x405c000000000000
root@hpux:~ # rmsf -H 36/0/0/2/0/0/1.0x50060e800574f210.0x405c000000000000

Checking LUN paths

root@hpux:~ # scsimgr -p lun_map -D /dev/rdisk/disk670
lunpath:33:36/0/0/2/0/0/0.0x50060e800574f200.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:35:36/0/0/2/0/0/1.0x50060e800574f210.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:37:38/0/0/2/0/0/0.0x50060e800574f200.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:39:38/0/0/2/0/0/1.0x50060e800574f210.0x40a5000000000000:fibre_channel:ACTIVE:ACTIVE

Checking another disk

root@hpux:~ # scsimgr -p lun_map -D /dev/rdisk/disk31
lunpath:61:0/2/1/0.0x21230002ac001673.0x4001000000000000:fibre_channel: FAILED:AUTH_FAILED
lunpath:62:0/2/1/0.0x21230002ac001673.0x4002000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:71:0/5/1/0.0x20240002ac001673.0x4001000000000000:fibre_channel: FAILED:AUTH_FAILED
lunpath:72:0/5/1/0.0x20240002ac001673.0x4002000000000000:fibre_channel:ACTIVE:ACTIVE

root@hpux:~ # ioscan -m lun /dev/rdisk/disk31
Class I Lun H/W Path Driver S/W State H/W Type Health Description
=======================================================================
disk 31 64000/0xfa00/0x8c esdisk CLAIMED DEVICE limited 3PARdataVV
0/2/1/0.0x21230002ac001673.0x4001000000000000
0/2/1/0.0x21230002ac001673.0x4002000000000000
0/5/1/0.0x20240002ac001673.0x4001000000000000
0/5/1/0.0x20240002ac001673.0x4002000000000000
/dev/disk/disk31 /dev/rdisk/disk31

2 Failed LUN paths

root@hpux:~ # scsimgr get_info -D /dev/rdisk/disk31|more

STATUS INFORMATION FOR LUN : /dev/rdisk/disk31

Generic Status Information

SCSI services internal state = ONLINE
Device type = Direct_Access
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x50002ac0031a1673
Serial number = ” 1405747″
Vendor id = “3PARdata”
Product id = “VV ”
Product revision = “3131”
Other properties = “”
SPC protocol revision = 6
Open count (includes chr/blk/pass-thru/class) = 1
Raw open count (includes class/pass-thru) = 0
Pass-thru opens = 0
LUN path count = 4
Active LUN paths = 2
Standby LUN paths = 0
Failed LUN paths = 2
Maximum I/O size allowed = 2097152
Preferred I/O size = 2097152
Outstanding I/Os = 0
I/O load balance policy = round_robin
Path fail threshold time period = 0
Transient time period = 120
Tracing buffer size = 1024
LUN Path used when policy is path_lockdown = NA
LUN access type = NA
Asymmetric logical unit access supported = No
Asymmetric states supported = NA
Preferred paths reported by device = No
Preferred LUN paths = 0

Driver esdisk Status Information :

Capacity in number of blocks = 213909504
Block size in bytes = 512
Number of active IOs = 0
Special properties =
Maximum number of IO retries = 45
IO transfer timeout in secs = 30
FORMAT command timeout in secs = 86400
START UNIT command timeout in secs = 60
Timeout in secs before starting failing IO = 120
IO infinite retries = false

Validating disk paths for disk31

root@hpux:~ # scsimgr -f replace_wwid -D /dev/rdisk/disk31
scsimgr: Successfully validated binding of LUN paths with new LUN.

The invalid paths were removed

root@hpux:~ # ioscan -m lun /dev/rdisk/disk31
Class I Lun H/W Path Driver S/W State H/W Type Health Description
======================================================================
disk 31 64000/0xfa00/0x8c esdisk CLAIMED DEVICE online 3PARdataVV
0/2/1/0.0x21230002ac001673.0x4002000000000000
0/5/1/0.0x20240002ac001673.0x4002000000000000
/dev/disk/disk31 /dev/rdisk/disk31

DMESG-UNCLASSIFIED: ia64dsk: The disk for dev_t bc4e0100 appears to have grown since the partition t

Node : hpux.setaoffice.com
Node Type : Itanium 64/32(HTTPS)
Severity : major
OM Server Time: 2016-01-02 21:07:51
Message : DMESG-UNCLASSIFIED: ia64dsk: The disk for dev_t bc4e0100 appears to have grown since the partition table was written.
Msg Group : OS
Application : HPUX_dmesg
Object : dmesg_UNCLASSIFIED
Event Type :
not_found

Instance Name :
not_found

Instruction : No

Checking which disks are alarming

root@hpux:~ # dmesg | grep ia64dsk
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc420100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.
ia64dsk: The disk for dev_t bc430100 appears to have grown since the partition table was written.

Checking system device

root@hpux:~ # ls -ltraR /dev | grep 420100
crw-r—– 1 bin sys 188 0x420100 Feb 12 2015 c66t0d1
brw-r—– 1 bin sys 31 0x420100 Feb 12 2015 c66t0d1

Checking hardware path

root@hpux:~ # ioscan -kfnC disk /dev/dsk/c66t0d1
Class I H/W Path Driver S/W State H/W Type Description
==================================================================
disk 48 0/2/1/1.10.234.64.0.0.1 sdisk CLAIMED DEVICE 3PARdataVV
/dev/dsk/c66t0d1 /dev/dsk/c66t0d1s2 /dev/rdsk/c66t0d1 /dev/rdsk/c66t0d1s2
/dev/dsk/c66t0d1s1 /dev/dsk/c66t0d1s3 /dev/rdsk/c66t0d1s1 /dev/rdsk/c66t0d1s3

root@hpux:~ # ioscan -m hwpath -H 0/2/1/1.10.234.64.0.0.1
Lun H/W Path Lunpath H/W Path Legacy H/W Path
====================================================================
64000/0xfa00/0x9a
0/2/1/1.0x21230002ac001673.0x4001000000000000 0/2/1/1.10.218.65.0.0.1
0/2/1/1.10.234.64.0.0.1

root@hpux:~ # scsimgr -f replace_wwid -H 64000/0xfa00/0x9a
scsimgr: Successfully validated binding of LUN paths with new LUN.

root@hpux:~ # ls -ltraR /dev | grep 430100
crw-r—– 1 bin sys 188 0x430100 Feb 12 2015 c67t0d1
brw-r—– 1 bin sys 31 0x430100 Feb 12 2015 c67t0d1

root@hpux:~ # ioscan -kfnC disk /dev/dsk/c67t0d1
Class I H/W Path Driver S/W State H/W Type Description
==================================================================
disk 59 0/5/1/0.20.218.64.0.0.1 sdisk CLAIMED DEVICE 3PARdataVV
/dev/dsk/c67t0d1 /dev/dsk/c67t0d1s2 /dev/rdsk/c67t0d1 /dev/rdsk/c67t0d1s2
/dev/dsk/c67t0d1s1 /dev/dsk/c67t0d1s3 /dev/rdsk/c67t0d1s1 /dev/rdsk/c67t0d1s3

root@hpux:~ # ioscan -m hwpath -H 0/5/1/0.20.218.64.0.0.1
Lun H/W Path Lunpath H/W Path Legacy H/W Path
====================================================================
64000/0xfa00/0x9a
0/5/1/0.0x20240002ac001673.0x4001000000000000 0/5/1/0.20.218.64.0.0.1

root@hpux:~ # scsimgr -f replace_wwid -H 64000/0xfa00/0x9a
scsimgr: Successfully validated binding of LUN paths with new LUN.

The legacy lun path (b 32 – t 13 – l 0) registration failed because it has been re-mapped from its original LUN (default dev 0xb000018) to a different LUN (default dev 0xc0000bd).

Checking /var/adm/syslog/syslog.log I see that lunpath, instance 81 is showing an error message

root@hp-ux:~ # grep vmunix /var/adm/syslog/syslog.log | grep “Nov 9”
Nov 9 04:37:14 hp-ux vmunix: class : lunpath, instance 81
Nov 9 04:37:14 hp-ux vmunix: The legacy lun path (b 32 – t 13 – l 0)
Nov 9 04:37:14 hp-ux vmunix: registration failed because it has been
Nov 9 04:37:14 hp-ux vmunix: re-mapped from its original LUN (default dev 0xb000018)
Nov 9 04:37:14 hp-ux vmunix: to a different LUN (default dev 0xc0000bd).
Nov 9 04:37:14 hp-ux vmunix: The administrator has to close the original LUN and
Nov 9 04:37:14 hp-ux vmunix: then validate this LUN re-mapping using the scsimgr
Nov 9 04:37:14 hp-ux vmunix: command:
Nov 9 04:37:14 hp-ux vmunix: scsimgr [-f] replace_leg_dsf -D /dev/rdsk/cxtydz

(b 32 – t 13 – l 0)
scsimgr [-f] replace_leg_dsf -D /dev/rdsk/c32t13d0

Let’s discover the hardware path for this instance

root@hp-ux:~ # scsimgr -p get_attr -C lunpath -I 81 -a hw_path
0/2/0/0/0/1.0x5001438017928fe6.0x0

Pick the LUN H/W Path

root@hp-ux:~ # ioscan -m hwpath -H 0/2/0/0/0/1.0x5001438017928fe6.0x0
Lun H/W Path Lunpath H/W Path Legacy H/W Path
====================================================================
64000/0xfa00/0xbd
0/2/0/0/0/1.0x5001438017928fe6.0x0

And run the following command

root@hp-ux:~ # scsimgr -f replace_wwid -H 64000/0xfa00/0xbd
scsimgr: Successfully validated binding of LUN paths with new LUN.

In another case, there was a missing disk. I removed the disk

root@hp-ux:~ # ioscan -fnNC disk
Class I H/W Path Driver S/W State H/W Type Description
===================================================================
disk 2 64000/0xfa00/0x0 esdisk CLAIMED DEVICE HP EH0146FARUB
/dev/disk/disk2 /dev/rdisk/disk2
/dev/disk/disk2_p1 /dev/rdisk/disk2_p1
/dev/disk/disk2_p2 /dev/rdisk/disk2_p2
/dev/disk/disk2_p3 /dev/rdisk/disk2_p3
disk 3 64000/0xfa00/0x1 esdisk CLAIMED DEVICE HP EH0146FARUB
/dev/disk/disk3 /dev/rdisk/disk3
/dev/disk/disk3_p1 /dev/rdisk/disk3_p1
/dev/disk/disk3_p2 /dev/rdisk/disk3_p2
/dev/disk/disk3_p3 /dev/rdisk/disk3_p3
disk 5 64000/0xfa00/0x2 esdisk CLAIMED DEVICE TEAC DVD-ROM DW-224EV
/dev/disk/disk5 /dev/rdisk/disk5
disk 14 64000/0xfa00/0xa esdisk CLAIMED DEVICE HP HSV340
/dev/disk/disk14 /dev/rdisk/disk14
disk 19 64000/0xfa00/0x46 esdisk NO_HW DEVICE NETAPP LUN C-Mode
/dev/disk/disk19 /dev/rdisk/disk19

root@hp-ux:~ # rmsf -H 64000/0xfa00/0x46

Disk showing Failed LUN paths in HP-UX

Displaying Lun paths to the LUN

root@hp-ux:~ # scsimgr -p lun_map -D /dev/rdisk/disk31
lunpath:61:0/2/1/0.0x21230002ac001673.0x4001000000000000:fibre_channel:FAILED:AUTH_FAILED
lunpath:62:0/2/1/0.0x21230002ac001673.0x4002000000000000:fibre_channel:ACTIVE:ACTIVE
lunpath:71:0/5/1/0.0x20240002ac001673.0x4001000000000000:fibre_channel:FAILED:AUTH_FAILED
lunpath:72:0/5/1/0.0x20240002ac001673.0x4002000000000000:fibre_channel:ACTIVE:ACTIVE

Displaying the LUN to the lunpath mapping

root@hp-ux:~ # ioscan -m lun /dev/rdisk/disk31
Class I Lun H/W Path Driver S/W State H/W Type Health Description
=======================================================================
disk 31 64000/0xfa00/0x8c esdisk CLAIMED DEVICE limited 3PARdataVV
0/2/1/0.0x21230002ac001673.0x4001000000000000
0/2/1/0.0x21230002ac001673.0x4002000000000000
0/5/1/0.0x20240002ac001673.0x4001000000000000
0/5/1/0.0x20240002ac001673.0x4002000000000000
/dev/disk/disk31 /dev/rdisk/disk31

root@hp-ux:~ # scsimgr get_info -D /dev/rdisk/disk31

STATUS INFORMATION FOR LUN : /dev/rdisk/disk31

Generic Status Information

SCSI services internal state = ONLINE
Device type = Direct_Access
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x50002ac0031a1673
Serial number = ” 1405747″
Vendor id = “3PARdata”
Product id = “VV ”
Product revision = “3131”
Other properties = “”
SPC protocol revision = 6
Open count (includes chr/blk/pass-thru/class) = 1
Raw open count (includes class/pass-thru) = 0
Pass-thru opens = 0
LUN path count = 4
Active LUN paths = 2
Standby LUN paths = 0
Failed LUN paths = 2
Maximum I/O size allowed = 2097152
Preferred I/O size = 2097152
Outstanding I/Os = 0
I/O load balance policy = round_robin
Path fail threshold time period = 0
Transient time period = 120
Tracing buffer size = 1024
LUN Path used when policy is path_lockdown = NA
LUN access type = NA
Asymmetric logical unit access supported = No
Asymmetric states supported = NA
Preferred paths reported by device = No
Preferred LUN paths = 0

Driver esdisk Status Information :

Capacity in number of blocks = 213909504
Block size in bytes = 512
Number of active IOs = 0
Special properties =
Maximum number of IO retries = 45
IO transfer timeout in secs = 30
FORMAT command timeout in secs = 86400
START UNIT command timeout in secs = 60
Timeout in secs before starting failing IO = 120
IO infinite retries = false

I saw two failed LUN paths. To validate disk paths to the disk I use scsimgr

root@hp-ux:~ # scsimgr -f replace_wwid -D /dev/rdisk/disk31
scsimgr: Successfully validated binding of LUN paths with new LUN.

The invalid ones were removed

root@hp-ux:~ # ioscan -m lun /dev/rdisk/disk31
Class I Lun H/W Path Driver S/W State H/W Type Health Description
======================================================================
disk 31 64000/0xfa00/0x8c esdisk CLAIMED DEVICE online 3PARdataVV
0/2/1/0.0x21230002ac001673.0x4002000000000000
0/5/1/0.0x20240002ac001673.0x4002000000000000
/dev/disk/disk31 /dev/rdisk/disk31