--_000_6A17C71B52524C408E7AAF69103E9E490FAE98C4fabamailserverf_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, got a new FC Storage (EMC Unity 300F) which is seen by my Hosts additional = to my old Storage for Migration. New Storage has only on PATH until Migration is done. I already have a few VMs running on the new Storage without Problem. But after starting some VMs (don't really no whats the difference to workin= g ones), the Path for new Storage fails. Engine tells me: Storage Domain <storagedomain> experienced a high latency = of 22.4875 seconds from host <host> Where can I start looking? In /var/log/messages I found: Feb 8 09:03:53 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: = sdd - emc_clariion_checker: Active path is healthy. Feb 8 09:03:53 ovirtnode01 multipathd: 8:48: reinstated Feb 8 09:03:53 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: = remaining active paths: 1 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 8 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 5833475 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 5833475 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967168 Feb 8 09:03:53 ovirtnode01 kernel: Buffer I/O error on dev dm-207, logical= block 97, async page read Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967168 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967280 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967280 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 0 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 0 Feb 8 09:03:53 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967168 Feb 8 09:03:53 ovirtnode01 kernel: device-mapper: multipath: Reinstating p= ath 8:48. Feb 8 09:03:53 ovirtnode01 kernel: sd 3:0:0:22: alua: port group 01 state = A preferred supports tolUsNA Feb 8 09:03:53 ovirtnode01 sanlock[5192]: 2017-02-08 09:03:53+0100 151809 = [11772]: s59 add_lockspace fail result -202 Feb 8 09:04:05 ovirtnode01 multipathd: dm-33: remove map (uevent) Feb 8 09:04:05 ovirtnode01 multipathd: dm-33: devmap not registered, can't= remove Feb 8 09:04:05 ovirtnode01 multipathd: dm-33: remove map (uevent) Feb 8 09:04:06 ovirtnode01 multipathd: dm-34: remove map (uevent) Feb 8 09:04:06 ovirtnode01 multipathd: dm-34: devmap not registered, can't= remove Feb 8 09:04:06 ovirtnode01 multipathd: dm-34: remove map (uevent) Feb 8 09:04:08 ovirtnode01 multipathd: dm-33: remove map (uevent) Feb 8 09:04:08 ovirtnode01 multipathd: dm-33: devmap not registered, can't= remove Feb 8 09:04:08 ovirtnode01 multipathd: dm-33: remove map (uevent) Feb 8 09:04:08 ovirtnode01 kernel: dd: sending ioctl 80306d02 to a partiti= on! Feb 8 09:04:24 ovirtnode01 sanlock[5192]: 2017-02-08 09:04:24+0100 151840 = [15589]: read_sectors delta_leader offset 2560 rv -202 /dev/f9b70017-0a34-4= 7bc-bf2f-dfc70200a347/ids Feb 8 09:04:34 ovirtnode01 sanlock[5192]: 2017-02-08 09:04:34+0100 151850 = [15589]: f9b70017 close_task_aio 0 0x7fd78c0008c0 busy Feb 8 09:04:39 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: = sdd - emc_clariion_checker: Read error for WWN 60060160422143002a38935800ae= 2760. Sense data are 0x0/0x0/0x0. Feb 8 09:04:39 ovirtnode01 multipathd: checker failed path 8:48 in map 360= 060160422143002a38935800ae2760 Feb 8 09:04:39 ovirtnode01 multipathd: 360060160422143002a38935800ae2760: = remaining active paths: 0 Feb 8 09:04:39 ovirtnode01 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort co= mmand issued nexus=3D3:0:22 -- 1 2002. Feb 8 09:04:39 ovirtnode01 kernel: device-mapper: multipath: Failing path = 8:48. Feb 8 09:04:40 ovirtnode01 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort co= mmand issued nexus=3D3:0:22 -- 1 2002. Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: 8 callbacks suppres= sed Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967168 Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967280 Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 0 Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967168 Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 4294967280 Feb 8 09:04:42 ovirtnode01 kernel: blk_update_request: I/O error, dev dm-1= 0, sector 0 multipath -ll output for this Domain 360060160422143002a38935800ae2760 dm-10 DGC ,VRAID size=3D2.0T features=3D'1 retain_attached_hw_handler' hwhandler=3D'1 alua' = wp=3Drw `-+- policy=3D'service-time 0' prio=3D50 status=3Dactive `- 3:0:0:22 sdd 8:48 active ready running Thx Christian --_000_6A17C71B52524C408E7AAF69103E9E490FAE98C4fabamailserverf_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
</p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: sd 3:0:0:22: alua: port group 01 state A preferred supports tolUs= NA<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 sanlock[5192]: 2017-02-08 09:03:53+0100 151809 [11772]: s59 add_locks=
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} span.E-MailFormatvorlage17 {mso-style-type:personal-compose; font-family:"Calibri",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 2.0cm 70.85pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3D"DE-AT" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span lang=3D"EN-US">Hi,<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">got a new FC Storage (EMC Unity= 300F) which is seen by my Hosts additional to my old Storage for Migration= .<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">New Storage has only on PATH un= til Migration is done.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">I already have a few VMs runnin= g on the new Storage without Problem.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">But after starting some VMs (do= n’t really no whats the difference to working ones), the Path for new= Storage fails.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Engine tells me: Storage Domain= <storagedomain> experienced a high latency of 22.4875 seconds from h= ost <host><o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Where can I start looking?<o:p>= </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">In /var/log/messages I found:<o= :p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 multipathd: 360060160422143002a38935800ae2760: sdd - emc_clariion_checker= : Active path is healthy.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 multipathd: 8:48: reinstated<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 multipathd: 360060160422143002a38935800ae2760: remaining active paths: 1<= o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 8<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 5833475<o:p></o:= p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 5833475<o:p></o:= p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: Buffer I/O error on dev dm-207, logical block 97, async page read= <o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 0<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 0<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:03:53 ovirtnode0= 1 kernel: device-mapper: multipath: Reinstating path 8:48.<o:p></o:p></span= pace fail result -202<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:05 ovirtnode0= 1 multipathd: dm-33: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:05 ovirtnode0= 1 multipathd: dm-33: devmap not registered, can't remove<o:p></o:p></span><= /p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:05 ovirtnode0= 1 multipathd: dm-33: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:06 ovirtnode0= 1 multipathd: dm-34: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:06 ovirtnode0= 1 multipathd: dm-34: devmap not registered, can't remove<o:p></o:p></span><= /p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:06 ovirtnode0= 1 multipathd: dm-34: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:08 ovirtnode0= 1 multipathd: dm-33: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:08 ovirtnode0= 1 multipathd: dm-33: devmap not registered, can't remove<o:p></o:p></span><= /p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:08 ovirtnode0= 1 multipathd: dm-33: remove map (uevent)<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:08 ovirtnode0= 1 kernel: dd: sending ioctl 80306d02 to a partition!<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:24 ovirtnode0= 1 sanlock[5192]: 2017-02-08 09:04:24+0100 151840 [15589]: read_sectors = delta_leader offset 2560 rv -202 /dev/f9b70017-0a34-47bc-bf2f-dfc70200a347/= ids<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:34 ovirtnode0= 1 sanlock[5192]: 2017-02-08 09:04:34+0100 151850 [15589]: f9b70017 clos= e_task_aio 0 0x7fd78c0008c0 busy<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:39 ovirtnode0= 1 multipathd: 360060160422143002a38935800ae2760: sdd - emc_clariion_checker= : Read error for WWN 60060160422143002a38935800ae2760. Sense data are= 0x0/0x0/0x0.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:39 ovirtnode0= 1 multipathd: checker failed path 8:48 in map 360060160422143002a38935800ae= 2760<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:39 ovirtnode0= 1 multipathd: 360060160422143002a38935800ae2760: remaining active paths: 0<= o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:39 ovirtnode0= 1 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort command issued nexus=3D3:0:2= 2 -- 1 2002.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:39 ovirtnode0= 1 kernel: device-mapper: multipath: Failing path 8:48.<o:p></o:p></span></p=
<p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:40 ovirtnode0= 1 kernel: qla2xxx [0000:11:00.0]-801c:3: Abort command issued nexus=3D3:0:2= 2 -- 1 2002.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: 8 callbacks suppressed<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 0<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967168<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 4294967280<o:p><= /o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Feb 8 09:04:42 ovirtnode0= 1 kernel: blk_update_request: I/O error, dev dm-10, sector 0<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">multipath -ll output for this D= omain<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">360060160422143002a38935800ae27= 60 dm-10 DGC ,VRAID<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">size=3D2.0T features=3D'1 retai= n_attached_hw_handler' hwhandler=3D'1 alua' wp=3Drw<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">`-+- policy=3D'service-time= 0' prio=3D50 status=3Dactive<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"> `- 3:0:0:22 sdd 8:48&nbs= p; active ready running<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Thx Christian<o:p></o:p></span>= </p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> </div> </body> </html> --_000_6A17C71B52524C408E7AAF69103E9E490FAE98C4fabamailserverf_--