[ovirt-users] Storage domain experienced a high latency

Wed Feb 8 16:11:24 UTC 2017

Hi,

got a new FC Storage (EMC Unity 300F) which is seen by my Hosts additional to my old Storage for Migration.

New Storage has only on PATH until Migration is done.

I already have a few VMs running on the new Storage without Problem.

But after starting some VMs (don't really no whats the difference to working ones), the Path for new Storage fails.

Engine tells me: Storage Domain <storagedomain> experienced a high latency of 22.4875 seconds from host <host>

Where can I start looking?

In /var/log/messages I found:

Feb  8 09:03:53 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: sdd - emc_clariion_checker: Active path is healthy.

Feb  8 09:03:53 ovirtnode01 multipathd: 8:48: reinstated

Feb  8 09:03:53 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: remaining active paths: 1

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 8

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 5833475

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 5833475

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168

Feb  8 09:03:53 ovirtnode01 kernel: Buffer I/O error on dev dm-207, logical block 97, async page read

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 0

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 0

Feb  8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168

Feb  8 09:03:53 ovirtnode01 kernel: device-mapper: multipath: Reinstating path 8:48.

Feb  8 09:03:53 ovirtnode01 kernel: sd 3:0:0:22: alua: port group 01 state A preferred supports tolUsNA

Feb  8 09:03:53 ovirtnode01 sanlock[5192]: 2017-02-08 09:03:53+0100 151809 [11772]: s59 add_lockspace fail result -202

Feb  8 09:04:05 ovirtnode01 multipathd: dm-33: remove map (uevent)

Feb  8 09:04:05 ovirtnode01 multipathd: dm-33: devmap not registered, can't remove

Feb  8 09:04:05 ovirtnode01 multipathd: dm-33: remove map (uevent)

Feb  8 09:04:06 ovirtnode01 multipathd: dm-34: remove map (uevent)

Feb  8 09:04:06 ovirtnode01 multipathd: dm-34: devmap not registered, can't remove

Feb  8 09:04:06 ovirtnode01 multipathd: dm-34: remove map (uevent)

Feb  8 09:04:08 ovirtnode01 multipathd: dm-33: remove map (uevent)

Feb  8 09:04:08 ovirtnode01 multipathd: dm-33: devmap not registered, can't remove

Feb  8 09:04:08 ovirtnode01 multipathd: dm-33: remove map (uevent)

Feb  8 09:04:08 ovirtnode01 kernel: dd: sending ioctl 80306d02 to a partition!

Feb  8 09:04:24 ovirtnode01 sanlock[5192]: 2017-02-08 09:04:24+0100 151840 [15589]: read_sectors delta_leader offset 2560 rv -202 /dev/f9b70017-0a34-47bc-bf2f-dfc70200a347/ids

Feb  8 09:04:34 ovirtnode01 sanlock[5192]: 2017-02-08 09:04:34+0100 151850 [15589]: f9b70017 close_task_aio 0 0x7fd78c0008c0 busy

Feb  8 09:04:39 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: sdd - emc_clariion_checker: Read error for WWN 60060160422143002a38935800ae2760.  Sense data are 0x0/0x0/0x0.

Feb  8 09:04:39 ovirtnode01 multipathd: checker failed path 8:48 in map 360060160422143002a38935800ae2760

Feb  8 09:04:39 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: remaining active paths: 0

Feb  8 09:04:39 ovirtnode01 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort command issued nexus=3:0:22 --  1 2002.

Feb  8 09:04:39 ovirtnode01 kernel: device-mapper: multipath: Failing path 8:48.

Feb  8 09:04:40 ovirtnode01 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort command issued nexus=3:0:22 --  1 2002.

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: 8 callbacks suppressed

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 0

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280

Feb  8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-10, sector 0

multipath -ll output for this Domain

360060160422143002a38935800ae2760 dm-10 DGC     ,VRAID

size=2.0T features='1 retain_attached_hw_handler' hwhandler='1 alua' wp=rw

`-+- policy='service-time 0' prio=50 status=active

  `- 3:0:0:22 sdd 8:48  active ready  running

Thx Christian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170208/07a92cf4/attachment.html>