Advertisements

Clustered Linux server showing device-mapper: multipath: Failing path in /var/log/messages

I have a disk presented to 4 servers.

Everyday we receive a notification saying that a specific multipathed disk lost all paths.

The disk is showing a SCSI reservation conflict – SCSI persistent reservations provide the capability to control the access of each node to shared storage devices

May 11 13:35:04 linux kernel: sd 0:0:0:38: reservation conflict
May 11 13:35:04 linux kernel: sd 0:0:0:38: [sdag] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
May 11 13:35:04 linux kernel: sd 0:0:0:38: [sdag] CDB: Write(10): 2a 00 00 00 14 50 00 00 08 00

May 11 13:35:04 linux kernel: end_request: critical nexus error, dev sdag, sector 5200
May 11 13:35:04 linux kernel: device-mapper: multipath: Failing path 66:0. <————————sdag
May 11 13:35:04 linux kernel: sd 1:0:0:38: reservation conflict
May 11 13:35:04 linux kernel: sd 1:0:0:38: [sdeh] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
May 11 13:35:04 linux kernel: sd 1:0:0:38: [sdeh] CDB: Write(10): 2a 00 00 00 14 50 00 00 08 00
May 11 13:35:04 linux kernel: end_request: critical nexus error, dev sdeh, sector 5200
May 11 13:35:04 linux kernel: device-mapper: multipath: Failing path 128:144. <———————-sdeh
May 11 13:35:04 linux multipathd: 66:0: mark as failed
May 11 13:35:04 linux multipathd: PP0_oraarch_disk_001: remaining active paths: 3

May 11 13:35:04 linux kernel: sd 0:0:1:38: reservation conflict
May 11 13:35:04 linux kernel: sd 0:0:1:38: [sdcc] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
May 11 13:35:04 linux kernel: sd 0:0:1:38: [sdcc] CDB: Write(10): 2a 00 00 00 14 50 00 00 08 00
May 11 13:35:04 linux kernel: end_request: critical nexus error, dev sdcc, sector 5200
May 11 13:35:04 linux kernel: device-mapper: multipath: Failing path 69:0.<—————————sdcc
May 11 13:35:04 linux kernel: sd 1:0:1:38: reservation conflict
May 11 13:35:04 linux kernel: sd 1:0:1:38: [sdgg] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
May 11 13:35:04 linux kernel: sd 1:0:1:38: [sdgg] CDB: Write(10): 2a 00 00 00 14 50 00 00 08 00
May 11 13:35:04 linux kernel: end_request: critical nexus error, dev sdgg, sector 5200
May 11 13:35:04 linux kernel: device-mapper: multipath: Failing path 131:192.<————————-sdgg
May 11 13:35:04 linux kernel: end_request: critical nexus error, dev dm-209, sector 5200

May 11 13:35:05 linux multipathd: 128:144: mark as failed
May 11 13:35:05 linux multipathd: PP0_oraarch_disk_001: remaining active paths: 2
May 11 13:35:05 linux multipathd: 69:0: mark as failed
May 11 13:35:05 linux multipathd: PP0_oraarch_disk_001: remaining active paths: 1
May 11 13:35:05 linux multipathd: 131:192: mark as failed

PP0_oraarch_disk_001 (350002ad05071374b) dm-209 3PARdata,VV
size=300G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
|- 0:0:0:38 sdag 66:0 active undef running
|- 1:0:0:38 sdeh 128:144 active undef running
|- 0:0:1:38 sdcc 69:0 active undef running
`- 1:0:1:38 sdgg 131:192 active undef running

Since this is opening support tickets, I removed monitoring for this disk

Advertisements
%d bloggers like this: