Advertisements

LVM – Disk removed and appears message read failed after 0 of 4096 at 4096 or similar

In this server when I ran pvs it was showing a lot of input/output errors

root@linux:~ # pvs
/dev/sdq: read failed after 0 of 4096 at 0: Input/output error
/dev/sdq1: read failed after 0 of 2048 at 0: Input/output error
/dev/mpath/disk2: read failed after 0 of 4096 at 80530571264: Input/output error
/dev/mpath/disk2: read failed after 0 of 4096 at 80530628608: Input/output error
/dev/mpath/disk2: read failed after 0 of 4096 at 0: Input/output error
/dev/mpath/disk2: read failed after 0 of 4096 at 4096: Input/output error
/dev/mpath/disk2p1: read failed after 0 of 512 at 80525328384: Input/output error
/dev/mpath/disk2p1: read failed after 0 of 512 at 80525447168: Input/output error
/dev/mpath/disk2p1: read failed after 0 of 512 at 0: Input/output error
/dev/mpath/disk2p1: read failed after 0 of 512 at 4096: Input/output error
/dev/mpath/disk2p1: read failed after 0 of 2048 at 0: Input/output error
/dev/devvg/u01lv: read failed after 0 of 4096 at 21474770944: Input/output error
/dev/devvg/u01lv: read failed after 0 of 4096 at 21474828288: Input/output error
/dev/devvg/u01lv: read failed after 0 of 4096 at 0: Input/output error
/dev/devvg/u01lv: read failed after 0 of 4096 at 4096: Input/output error
/dev/sde: read failed after 0 of 4096 at 0: Input/output error
/dev/sde1: read failed after 0 of 2048 at 0: Input/output error
/dev/sdw: read failed after 0 of 4096 at 0: Input/output error
/dev/sdw1: read failed after 0 of 2048 at 0: Input/output error
/dev/sdk: read failed after 0 of 4096 at 0: Input/output error

A disk was removed and it was not removed cleanly

root@linux:~ # multipath -ll disk2
sde: checker msg is “tur checker reports path is down”
sdk: checker msg is “tur checker reports path is down”
sdq: checker msg is “tur checker reports path is down”
sdw: checker msg is “tur checker reports path is down”
disk2 (360000000000000000000000000000000) dm-25 HP,P2000 G3 FC
[size=75G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=0][enabled]
\_ 1:0:0:5 sde 8:64 [failed][faulty]
\_ 1:0:1:5 sdk 8:160 [failed][faulty]
\_ 2:0:0:5 sdq 65:0 [failed][faulty]
\_ 2:0:1:5 sdw 65:96 [failed][faulty]

Removed the block devices

root@linux:~ # echo 1 > /sys/block/sde/device/delete
root@linux:~ # echo 1 > /sys/block/sdk/device/delete
root@linux:~ # echo 1 > /sys/block/sdq/device/delete
root@linux:~ # echo 1 > /sys/block/sdw/device/delete

I was able to remove the device that device-mapper used but I was not able to remove the device with the partition

root@linux:~ # dmsetup remove disk2
root@linux:~ # dmsetup remove disk2p1
device-mapper: remove ioctl failed: Device or resource busy
Command failed

I had a filesystem using the device /dev/devvg/u01lv so I had to remove it before I was able to remove the device with the partition

root@linux:~ # dmsetup remove devvg-u01lv

root@linux:~ # dmsetup remove disk2p1

Advertisements
%d bloggers like this: