[Users] [rhev 3] add new domain fails: Could not retrieve LUNs
--_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_ Content-Type: multipart/alternative; boundary="_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_" --_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, I'm trying to configure a FCP storage domain on RHEV 3. I try to add a new domain from the console, but it can't find any LUNs: "Co= uld not retrieve LUNs, please check your storage" [cid:image002.jpg@01CD8150.7C76A710] Here is the output from /var/log/rhevm/rhevm.log: ------------------------------------ 2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationVa= lueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with nul= l version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge= tDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand= (vdsId =3D 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), log id= : 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br= okerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br= okerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError an= d error message VDSGenericException: VDSErrorException: Failed to GetDevice= ListVDS, error =3D Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Br= okerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbro= ker.vdsbroker.GetDeviceListVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRp= c lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.v= dsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.B= rokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBa= se] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Except= ion: VDSErrorException: VDSGenericException: VDSErrorException: Failed to G= etDeviceListVDS, error =3D Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge= tDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSComman= d, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceL= istQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception = message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSEr= rorException: VDSGenericException: VDSErrorException: Failed to GetDeviceLi= stVDS, error =3D Error block device action: () ---------------------------------------------- First question: do LUNs have to be visible from RHEV-H or RHEV-M? Currently they are visible only from the hypervisor. ---------------------------------------- [root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=3D650G features=3D'1 queue_if_no_path' hwhandler=3D'1 emc' wp=3Drw |-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy=3D'round-robin 0' prio=3D0 status=3Denabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=3D205G features=3D'1 queue_if_no_path' hwhandler=3D'0' wp=3Drw `-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------ Our SAN device is Clariion AX150. Is it compatible with ovirt? vdsClient -s 0 getDeviceList gives me: Error block device action: () Could it be due to SPM turned off? (I have only one host) [root@pittor06vhxd020 log]# ps axu | grep -i spm root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i sp= m How can I turn it on? I know the command but I don't know what paramaters a= ppend spmStart <spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostI= D> <version> Thank you very much for any hints. AS Alberto Scotto [Blue] Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it ________________________________ -- The information transmitted is intended for the person or entity to which i= t is addressed and may contain confidential and/or privileged material. Any= review, retransmission, dissemination or other use of, or taking of any ac= tion in reliance upon, this information by persons or entities other than t= he intended recipient is prohibited. If you received this in error, please = contact the sender and delete the material from any computer. --_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
<style> <!-- @font-face {font-family:Tahoma} p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"} a:link, span.MsoHyperlink {color:blue; text-decoration:underline} a:visited, span.MsoHyperlinkFollowed {color:purple; text-decoration:underline} p {margin-right:0cm; margin-left:0cm; font-size:12.0pt; font-family:"Times New Roman","serif"} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {margin:0cm; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif"} span.EmailStyle17 {font-family:"Calibri","sans-serif"; color:windowtext} span.BalloonTextChar {font-family:"Tahoma","sans-serif"} .MsoChpDefault {font-family:"Calibri","sans-serif"} @page WordSection1 {margin:70.85pt 2.0cm 2.0cm 2.0cm} div.WordSection1 {} --> </style> </head> <body lang=3D"IT" link=3D"blue" vlink=3D"purple"> <div class=3D"WordSection1"> <p><span lang=3D"EN-US">Hi all,</span></p> <p><span lang=3D"EN-US">I'm trying to configure a FCP storage domain on RHE= V 3.</span></p> <p><span lang=3D"EN-US">I try to add a new domain from the console, but it = can't find any LUNs: "Could not retrieve LUNs, please check your stora= ge"</span></p> <p><img width=3D"755" height=3D"565" id=3D"Picture_x0020_2" src=3D"cid:imag= e002.jpg@01CD8150.7C76A710"></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">Here is the output from /var/log/rhevm/rhevm.log:</= span></p> <p><span lang=3D"EN-US"> ------------------------------------</span></= p> <p><span lang=3D"EN-US">2007-06-29 21:50:07,811 WARN [org.ovirt.engin= e.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetCon= figurationValueQuery with null version, using default general for version<b= r> 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbrok= er.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCo= mmand(vdsId =3D 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), l= og id: 60bdafe6<br> 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br= okerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method<br=
2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br= okerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError an= d error message VDSGenericException: VDSErrorException: Failed to GetDevice= ListVDS, error =3D Error block device action: ()<br> 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbrok= er.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.v= dsbroker.vdsbroker.GetDeviceListVDSCommand return value</span></p> <p><span lang=3D"EN-US">Class Name: org.ovirt.engine.core.vdsbroker.vdsbrok= er.LUNListReturnForXmlRpc<br> lunList &n= bsp; Null<br> mStatus &n= bsp; Class Name= : org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc<br> mCode &nbs= p; = 600<br> mMessage &= nbsp; Error block dev= ice action: ()</span></p> <p><span lang=3D"EN-US"> 2007-06-29 21:50:08,727 INFO [org.ovirt= .engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) V= ds: pittor06vhxd020<br> 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBa= se] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Except= ion: VDSErrorException: VDSGenericException: VDSErrorException: Failed to G= etDeviceListVDS, error =3D Error block device action: ()<br> 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbrok= er.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSC= ommand, log id: 60bdafe6<br> 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceL= istQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception = message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSEr= rorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error =3D Error block devic= e action: ()</span></p> <p><span lang=3D"EN-US">----------------------------------------------</spa= n></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">First question: do LUNs have to be visible from RHE= V-H or RHEV-M?</span></p> <p><span lang=3D"EN-US">Currently they are visible only from the hypervisor= .</span></p> <p><span lang=3D"EN-US">----------------------------------------</span></p> <p><span lang=3D"EN-US">[root@pittor06vhxd020 log]# multipath -ll<br> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5<br> size=3D650G features=3D'1 queue_if_no_path' hwhandler=3D'1 emc' wp=3Drw<br> |-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive<br> | |- 2:0:0:0 sda 8:0 &= nbsp; active ready running<br> | |- 2:0:1:0 sdd 8:48 = active ready running<br> | |- 3:0:0:0 sde 8:64 = active ready running<br> | `- 3:0:1:0 sdf 8:80 = active ready running<br> `-+- policy=3D'round-robin 0' prio=3D0 status=3Denabled<br> |- 2:0:2:0 sdb 8:16 &= nbsp; active ready running<br> |- 2:0:3:0 sdc 8:32 &= nbsp; active ready running<br> |- 3:0:2:0 sdg 8:96 &= nbsp; active ready running<br> `- 3:0:3:0 sdh 8:112 = active ready running<br> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME<br> size=3D205G features=3D'1 queue_if_no_path' hwhandler=3D'0' wp=3Drw<br> `-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive<br> `- 0:0:1:0 cciss!c0d1 104:16 active ready running<br> ------------------------------------------------------</span></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">Our SAN device is Clariion AX150. Is it compatible = with ovirt?</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New"">vdsCl= ient -s 0 getDeviceList</span><span lang=3D"EN-US"> gives me:</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New"">Error= block device action: ()</span></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">Could it be due to SPM turned off? (I have only one= host)</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New"">[root= @pittor06vhxd020 log]# ps axu | grep -i spm</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New"">root&= nbsp; 16068 0.0 0.0 7888 &nb= sp; 868 pts/1 R+ 00:04 0:00 g= rep -i spm</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New""> = ;</span></p> <p><span lang=3D"EN-US">How can I turn it on? I know the command but I don&= #8217;t know what paramaters append</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New"">spmSt= art</span></p> <p><span lang=3D"EN-US" style=3D"font-family:"Courier New""> = ; <spUUID> <prevID> <pre= vLVER> <recoveryMode> <scsiFencing> <maxHostID> <ve= rsion></span></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">Thank you very much for any hints.</span></p> <p><span lang=3D"EN-US"> </span></p> <p><span lang=3D"EN-US">AS</span></p> </div> <br> <br> <div align=3D"left"> <p style=3D"font-family:Calibri,Sans-Serif; font-size:10pt"><span style=3D"= color:#000000; font-weight:bold">Alberto Scotto</span> <span style=3D"color:#808080"></span><br> <br> <span style=3D"color:#000000"><img border=3D"0" alt=3D"Blue" src=3D"cid:dfb= 35e5d2df7434e8338047441010e53" style=3D"margin:0px"> </span><br> <span style=3D"color:#808080">Via Cardinal Massaia, 83<br> 10147 - Torino - ITALY <br> phone: +39 011 29100 <br> <a href=3D"al.scotto@reply.it" target=3D"" style=3D"color:blue; text-decora= tion:underline">al.scotto@reply.it</a> <br> <a title=3D"" href=3D"www.reply.it" target=3D"" style=3D"color:blue; text-d= ecoration:underline">www.reply.it</a> </span><br> </p> </div> <br> <hr> <font face=3D"Arial" color=3D"Gray" size=3D"1"><br> --<br> The information transmitted is intended for the person or entity to which i= t is addressed and may contain confidential and/or privileged material. Any= review, retransmission, dissemination or other use of, or taking of any ac= tion in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If= you received this in error, please contact the sender and delete the mater= ial from any computer.<br> </font> </body> </html> --_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_-- --_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_ Content-Type: image/jpeg; name="image002.jpg" Content-Description: image002.jpg Content-Disposition: inline; filename="image002.jpg"; size=28792; creation-date="Thu, 23 Aug 2012 14:54:39 GMT"; modification-date="Thu, 23 Aug 2012 14:54:39 GMT" Content-ID: <image002.jpg@01CD8150.7C76A710> Content-Transfer-Encoding: base64 /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAoHBwkHBgoJCAkLCwoMDxkQDw4ODx4WFxIZJCAmJSMg IyIoLTkwKCo2KyIjMkQyNjs9QEBAJjBGS0U+Sjk/QD3/2wBDAQsLCw8NDx0QEB09KSMpPT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT3/wAARCAI1AvMDASIA AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3 ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEA AwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSEx BhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElK U1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3 uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD0FgGU huQetVjp6Ek+dcj2Eppby9iskDzMFT1IJ/lRb3sdzbyTR/Mqc5HfjP8AWgBv9np/z3uf+/po/s6P /nvc/wDf01QS4e5MZa9e3mcbgpOFHoMd6nuNUlg0hLoRoXMgjcsSET5tpc4BO3im7rcSdyx/Z0f/ AD3uf+/po/s6P/nvc/8Af01H/a9tDb77qaNWXAkMeWUEruyDjkY5zUba/ZxzbZRNFH5Hn+ZJEygD dtA5HUmi4y8lvGkHlDdt65J5JznOah/s6P8A573P/f01WHiPS2SNluciToRGxx823njj5uOe9XrW 6ivbdZoGLIxI5UqQQcEEHkEEHikBF/Z0f/Pe5/7+mj+zk/573P8A39NW6KLgVP7Oj/573P8A39NH 9nR/897n/v6at0UXArxWUcUok3yuy9N75AqxRRQAVBNaJM+4vKh77H25qeigCp/Z0f8Az2uf+/pq yqLHEEQAKq4AHYU6kPQ/SgCrEA3lKehGfyFWHcqVCgEnPXgVBB96H/dP8qfOcPHj/a/lQA7e56eX z/tH/CjdJ/dj/wC+j/hXKvourW8MaWE6IqRbQAxDKSwLKDxweoPbmp47LXvtcBkvSbcBBJ8/zHj5 +AMcFRj/AHmp2A6Lzjv2Zh3/AN3fz+VLuk9I/wDvo/4VhXOn3MuoTyQ2sEbvKsiXhILrhQNuMZ/X FQxafq7qvmXdxEFwQv2ksc5Xdlsc5G8j0yKLAdGXkAyRGB6lj/hTZZ2hjLyKu0YzgnPJx6Vzb2Gu G4UeeWgEXlkPLncfUj16c/rV2O3ure2n+0zyyiQIfncttbdzj0GMUWA2ndg5VQuQMkscUbpPSP8A 76P+FVtRjaaO4hTG6SEqM+pzWPJYaosjzJeT8AlYxKdv3hgbcf3c8euKEgOh3Sekf/fR/wAKTfIO 0f8A30f8K5iK01xreFhLOpODh7o5Rvl+ZuPmXr8narltp93F9oM0sszz2vllpJiwD88AdhzRYDaS VpBlDCwHGVfNLuk9I/8Avo/4VzH9magih7W1SzGxUeG2uBGZCA3zbguByRx1OKsLp+rCQSNfSs+Q xHm4TIYcbcdNu7j1xTsBv75M4xHn/eP+FIJiJkicDLgkFTnpjP8AOubs9O1toQl7dupMgJZZsuFy uQG/BumOvataxWaOSyjuXLyrHIGYnJPTv3pWA0mbaBgZJpvmE9l/Olk6r9f6VUWGRUwBg55wQMj6 4pAWvMb0X86VnwBgcn1qrFC8ZBboFI4NWG6J9P6UAL5hPQL+dG9vRfzqr9nkQDYRnABxxS+VP3kO frQBZ3t6L+dHmEckDH1qsI5/7/Puc59/anIkq7/MbIPSgC3RRRQAUUUUAFFFFABRRRQAUUUUAFFF FABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAGcd/wBaKKKAIbi3FwoHmSRkfxRnBot7dbeMqGZyxyzO clvrU1IThSfQZoAxbzw0t5eJMb2dI0wBEoHAHYHtWm9oDarBBLLbhcbXiI3D8wR+YquurRtcCEB9 +4KcxsByQOGPB61k+MtQvLNLKO0ZlWV28wocNgYxgj60272TFZLUvwaKIrtjgLBHb+REdwZmJzli CMA8kd+p7cUreHLR7dYGknKBChyy/NlgwJ4xkMARjA9sVjeHdQvP7Ygt5Wu2inVy32gcZUZG05Nb H9p3cc8ssi232KO6Nu2AwkUZwGz0PJHGBQ1Z2GTpo0CIFLyMQqrn5VyA24cKAOvtVq2tktYmSMsQ 0jyHce7MWP6mskeJ7a8aBNLPntJcLC5KEhVKlt3HqFOP/rVbtdfsLzf5Mj5jLBlZORtGT+hpAaNF ZUOtiXUUthBIyyOypIqkgAKGy3p1rVoAKKKKACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lUsqRzJ slj3r6GoofvQ/wC6f5VLmmgIPsFn/wA+ifl/9ej7BZ/8+ifl/wDXqYH97/wH+tZFn4ntblUeeKW0 jlhaaN5iuHRTg9CcEZHB9aYGj9gs/wDn0T8v/r0fYLP/AJ9E/L/69VX8R6Sixs1/CBIMr15GcdMc c8c96eNd01hBsvIm+0AmLGfnx/LkY5oAn+wWf/Pon5f/AF6ctlaIwZbVAwOQcdKqQa/p1xLDCt3G J5gpSPOScjIGRx0B79qrzeKLKHVpdOdZftEUsUZGBg+Z0I56DvQI15oYrgATQhwOmah+wWf/AD6J +X/16qnxFpIjZzfw7VbYTz1wTxxzwCePSgeI9JMPnDUIfL3bd3PXG706Y5z0xSGWvsFn/wA+ifl/ 9ej7BZ/8+ifl/wDXqcMGAIIIIyCO9LmnYRX+wWf/AD6J+X/16PsFn/z6J+X/ANerGaM0AV/7Ps/+ fRPy/wDr1JDbwW7EwwBCeCQKkzSZosA4kEYKk0mE/wCedJmjNKwwwn/POnEgjBUmm5ozRYAwn/PO jCf886M0E8UWAMJ/zzpcIP4KoXmp/YPscS28txLcsUjSMqOQuTksQOgplv4h024jtiLlUe4XKRvw w6gg9gcgj3xxRYDU3/7JpN/+yazIPEek3MyRQahC7vwoGfm4z1xjoD+VH/CQ6aVDrdxtF8+584C7 Rk5B56c8UWA1N/8Asmjf/smqC6rBPYy3ViftaxEhljIByOo+bGDUFl4gtLu1SaU/ZS6owjlYbtr/ AHDwT1wfyosBrb/Y0b/9k1lr4i0l7d51v4TEmNzDPfkcYyRweR6GtBJFkjV42DIwBVlOQR60WAk3 /wCyaN/+yaZmmyH5PxH86LAS7/8AZNG//ZNZt7rMVnqVvZGGaWadS42beFBAJ5IJ65wMnANRW/iX S7iON/tSx+a7IgkBBJDbc+wzjk+tFgNff/smjf8A7JrMHiHSiAft0WC/lg4OC2cYzj14z0pTr+li SVDfQ74QS454wQD255IHHeiwGlv/ANk0b/8AZNZl9rUNnpX9oxRvd2w5LQsvA6Z+YjvxjrQ2u2EM Je7uI7ZlZlkSRgSjKASDjI4BB49aLAae/wD2TRv/ANk1nf27pnlxyfbYgkrFUJyMkdR046jr61fz RYQ7f/smjf8A7JpuaTNFhj9/+yaN/wDsmmZpc0WAdv8A9k0b/wDZNRjmZR6j+orJsvE9ndxTzMPI ghYqzySITkNtxtUlhk9MjmiwG1v/ANk0b/8AZNZb+ItJQKWv4QGQSDqflJwCeOOQRz3pY/EWlSo7 R30TKg3NjPAzt6Y554470WA09/8Asmjf/sms601uyvr021rL5r+SswZVO0qSQOfwq/miwh2//ZNG /wD2TTM0ZosMfv8A9k0b/wDZNVp7mKzguLid9kUS73b0AHNU11yGOES38E9gjFRGZwD5hYZAG0nn jp1FFgNXf/smjf7Gs06/parMxvoQICBIcn5cnHpzzxxnmnxazp89zHbxXkTzSoHRATllOcH9D+VF hF/f/smjf/smseTxBDDfSQSW8yxRzpbvcZXYHYZUYznHIGcUsniXS0aIC6VzJOIPkUnaxGeeOBjv RYZrb/8AZNLv/wBk1Sh1ayuLWe5guUeG3z5rDPyYGefw5qCLWozbme6t57OHarJJMBhw3TG0k59u vNFgNTf/ALJo3/7JrM/4SHSvMVBfws7LvULk5GNwxgegJx7Uy38S6Vc20MwvEQSxecqyAqQuMnP4 A8e1FgNbf/smjf8A7JrMu9ctLbSRqKsZrdmVUKcbixwOWwAPc8VGPElgJpIrh5LaSKJJZFlQ/KGP A4zk/SiwjX3/AOyaN/8AsmsqLxHpk161ql0vmAJgkHa+8EqAe/ANSwa3p11cRwQXkUk0qCREUnLK QSD+QNFgNDf/ALJo3/7JphPBrN1PWotIS0EsbO1ySifvEQAhcnJYgCiwzV3/AOyaN/8Asmsxtf02 MSebdxxtEwSRTklGIzt44Jx6Z9elTRarYz3f2aG6iefbuCKc5GAevToQfxosIu7/APZNG/8A2TWQ fENql9eW8qyRx2YzPcNjYnAPIzuA54OMEg1L/b2meXG/22LbI/lqTnlsgY6cdR19RRYZpb/9k0b/ APZNZza7pq+fuvYv3B2yYydpzjHTnnjjNXIpo54klidXjcblZTkEetFgJd/+yaN/+yaikP7s/h/O n55osIdu/wBk/pRTc0UWGSUUUUgKq6fEs4k3zNg5CNISoP0qh4g0E64tvtnWJoST8yFgQcehHpWz RQJq+hytvpNn4avYL+/vLeJV3RqVjYbiw6HJPapP7U8NfanlfVfMDyGUwvIxjDngnbioPiJ/yB7T /r4/9lNc7DZQLoVvLJbRMZot244yTz+P5VrGCkrsm7vZHUQ6h4ZhgSEapvijbciO7EJ8pXA46YYj 8qgSbwskRRNWkUnq4lYNjbt2529McY9hXEaXBBcsVuGO7y8xoHCeY3HG48DjP1q1PopSKORJ40Dq 7usx2mIK2Pm4+g471TpxI9pI7dNW8NxSxyRamEaN942s3PyhcHjkYA4q7/wleh/9BKH8m/wrzg6H cgoGeAGRyqfMTuwAcjAPGCDmlfQLuN3SRoEdS2FMnL7Rk7eOaPZx7h7SXY9HXxVojMFXUYSScAYb k/lWt3ryGfSLrTZ7V7lUAeVRgNkg5Bwfz7V6+33j9aznFR2LhJy3EoooqCwooooAKQ9DS0h6GgCr D96H/dP8qkzUcP3of90/yqby1zjfyOcZFNMQwH97/wAB/rWEvhO3+ypBNdzyrDH5duSqgw/MGyMD 5jkDr6V0PlDruNHlD+8ad0BgHwrbO8sklzO8szb5Xwo3t5gkzgDA5UD6Ux/CNs9zDKbu4xFJ5gQh SM7y/GRxycHHUAV0XlD+8aPKGcbjn0oug1MO18M21oYtk8zeW8LjOOTGpVe3+0c0l54Wsr7UVvpX lFwtwtwrLgYKrt2/Q4B+ord8of3jSKiOu5JNw9VIIougOdtvB9tbSFzd3EjZBywUHhWXkgcnDHk8 02TwXYyCMmeXfGqIGZEbhUCdCCOQPwPSul8of3jQYgOrEflRdBqRqAihVGAowB7Uuaf5Q/vGjyh/ eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/ eNF0AzNGeKf5Q/vGjyh/eNF0Bm3unG+NnLHcyW01qxeN0VW5K7TkMMdDVIeE7NAqRTTpCRH5sYwf OZCWViSMg5JJx1rf8kf3jR5Q/vGi6AwI/CtrGqBbm4+URAEEA/u1ZQenB+Y1BF4KtI4JIjdXDeYG DNtUE7k2E8DrjnPrXTeUP7xo8of3jRdAULfTo7ZbwI7n7W+984+X5QvH4Cs0+G/9P051kHkWFqYU yfmkbGFLDGMDk/jXQ+UP7xo8of3jRdAcpb+CYjY2qXt00l1BGkayCNWVAob5QpGCPmPXnpXSW8K2 1tFBH9yNQg4A4H0qfyh/eNHlD+8ad0AzNNkPy/iP51L5Q/vGjyR/eNK6Ay9U0dNWkhM1xKsUTK/l qqkEqwIIJGVPGMg9Kp/8IrB8wN3cbJMrIuFG9C/mbCcdAxPPXBxXQeUP7xo8of3jRdBqc7P4Rtp7 S2tjd3CxW5YhcKQxZ9+cEcEHgEc4NUo/B0087pf3ERtU3/Z1jXJTMokGQRjjHQ5zmuv8of3jR5Q/ vGi6DUy30aF9COlGRxEV2l1VVY85JwBgc+1ZuqeGZdRu9VuBJFG11AsMIySFORucjHUgKO/Sum8o f3jR5Q/vGi6A52TwhaTXK3Ms8j3BdnkkaND5mSpxgjC/dAyOetb+af5Q/vGjyh/eNF0AzNGaf5Q/ vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0BGGxMp9B/UVmroMEemx2kUsiGKf7RHMAu5X3FgemDyT1rW8 kepo8of3jRdAYEfhW1jjlUXFwxmRFkY4yxWQyFunUsT7elJJ4Ut5FO27uEYlzkbT96TzCCMcjPGD 1FdB5Q/vGjyh/eNF0Gpj6ToMWkSI0NxK4WAQFWVQGAZmB4HB+YjjitXNP8of3jR5Q/vGi6AZmjNP 8of3jR5Q/vGi6Aq3NvFe29xbTruimTY4zjIIqmNGZ/sxub+5nNtMk0e5VUAqCAMAe/J61reSP7xo 8of3jRdAc83hK1eF4nuJnj8tookZUIiRn3kdPm59aW18J2lpe2Nyk87PZR7E3YywwRyepHPTpwK6 Dyh/eNHlD+8aLoDGl0CKe8mlkuZjDNMtw9vtUKXUAKd2M4GAcetVbfwhBbDKX10ZDKshkwoYlVK8 4HJIY5PXPNdH5Q/vGjyh/eNF0Bg2HhWzs4J4pZJrkTEFix2Z+Xbztxkkdc555p0/hwXditrdX9xM sTI0JdExHs6cYw3HBzW55Q/vGjyh/eNF0BiReGrWJlKyygCSOTACgZRSo4A4+8aqJ4Ls1GGuZXzC IWLRpuOFKAg4yCAe3XFdN5Q/vGjyh/eNF0BRnsfO01LNJ3jCKqbwqtuA4wQQQQR1FZR8IW4t0hiv blI1jRCuFIfaWIyCOnzHjp09K6Pyh/eNHlD+8aLoDnU8JQRJEsV7cL5Qi2napwYwQD07hiCKfaeF LSzvrG6SeYvZxCJMhRuGMfMQMn6dK3/KH940eUP7xougGE8Gqk9hFdz2U8hO61LMi4BDbl2nNXvK H940eSP7xougMG88L2t4koaWQM9010GKK2xmTYQARgjA71JYeHLTTtTe9tyQWB+QonBICk7sZ6L0 zjk1teUP7xo8of3jRdAc7rnhj+1vtMoun894XihDqAse4AHJA3EexzyabceD4LpzJNe3DSPJ5kh2 ph2+XBxjAICAZHOCa6Tyh/eNHlD+8aLoDn5fCsE1sLdrucwxy+bBGyIyxNuLHgj5vvEc9q2LS3Sz s4beP7kSBRwBnHsOPyqx5Q/vGjyh/eNF0BFIfkP4fzp2af5IP8Ro8of3jRdAM/z0op/kj+8aKLoC SiiipGFFFFAHO+M9LvNW023isYvNdJtzDcBgbSO9c+NI8SDTIrL+zYykS7FfzwDj3Gea9BPQ/Sq4 I3R4ckFcn56uM2lYnl1uec2vhTxBaNmKxQ8Yw7RsPyJqz/Ynig532qSZ3Z8wxtkMckHPbPOK7PVZ bqDTHmsrlYnjbq0YkDAsBjk1DL4itrK7lspvPnlt0PmSxxYBYLuIx2OPw5xmn7Rk+yRyX9ieKfMD m3ViN3DNGR8wAIxnpgAYq1FpWviKVp7FpZ2DBPmiCpuULkHORx2FdJ/wkcK+Z51ndxCNXLFlU/Mq 7ioweTt59O2c0Sa6VuYkaCSEL5hmSXaGwI96kHOOf84o9o+w1TXc45/DXiC5uopLiyj3K6szgxhm wRySDzXpJOWNYS+KoWVZxbTC0EMkkshxlChxtxnJ/D1qa08RQXk1tGltcI05YAyBVC7TjqT8x74X JqZSchxiomvRRRUlBRRRQAUh6GlpD0NAFaD70P8Aun+VZWqWlyNQvL21id3W1WPaBnzlO/co9xkE f/XrVh+9D/un+VWqAMK0k1BL+CJjcBV2qYjABEItg534zu3ds/h3puqSauguHtJJFTzxGuId2yPb 94AAlvm46Hit+igDnY7vVo5pDIbidVSJ8x25VAMqHAUruLYyeCfoDUFyl/ekXOL2DDLhkiw+wTA4 2kH+HnGK6migDHtJLy4g1OKZrjcNwilMewcg8ICoPHHJznsapWaajaC2tYjcAoEVY/s6iJlK5dmY Dhsk8cdBwea6WigDn4rnVb4wov2q2UiNZXa3AIbDb8bhjqBzyKhZtSubi0W5+15WSFvLWAeWyj7z M2OGz2z+FdNRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRVS9u5LbHlwyTE/wAMe3P15NAF uqcctjd3LLHIjzRnGAxBH09fwqC4uXm0K+kIKOscgHYjC+3eq+23uDFDOghWJQyBOOeeQw5HSqSA 15oI54jFKgaNuqnv3qGTTbWWeWZ42LzDEgEjBX4xyoOCcd8ZrOt764lsreOOch57qSBZ3UMwVS2O OhbA71YS9lttVSyndp9yqqvtCncQxJOP93tUgS3mk291azxhFR5Q/wA+M4Zl2k4+lIND0/7OIWtw 688s7FskYJ3E56D147VRn8Rt9nd7ayYv9nE8fmyKokz1C+uO5HSnyeJUjlnjNhdsYAQ7KuV3jGU3 dM84znmgC4NFsAip9nyoLHl2Od33sknnPvSjSbTdAzJI5t23R+ZM77T68k5I7Z6UWmom5vJbaS3a CSNQ2HcEt64A6gHjd0NXaACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsP3of90/yq1QAUUUUA FFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAU UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRR RQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAH+etFH+etFABRRRQAVFPbQ3ShZ4lkAOQD2qW igCL7NCLY24jXySpUpjgg9RWXLpV3blhZyLNG67dszYMfXGD3Az35rXkJWNiOoBNZ0N/O90sbwMi l9u4yK24c84HI6d6abWwDrXSbe20hLG6KzpksxfjLE5JHpyeKlfSLCSEQvaxmMAADnjHTnOe5/Oo tSl8uVcorqIycHsc4zjvT7CVTcNAkjuqRq3z9QSTSAX+xtO/584seV5OOcBMY2gZ449KUaTYrK0g tUDuoVjk/MBjqM47DnrxWT/aM9lCt/PeSyxtPMj25VMbFLcpgA5AX1NI2vXc72Ttay2MBeRpTK23 eix7gQSudvPPA6cGgDcjs7eK6kuUiAnk4d8kk/4fhU9Ylt4lWaKZprSSCSFHZ0Zv7u08ZAPIYdqf Zaldzar9nZEaAmUly2GXawAGMc9aPIDYooooAKKKKACkPQ0tIeh+lAFaH70P+6f5VaqrD96H/dP8 qtUAFFFFABRRRQAUUUUAFFFFABVZ7mR5zBaQmaRfvHOFX6mrB6Gjw+AdKWXHzSu7MfU7iP6UAV/J 1f8A597X/v8AH/CjydY/597T/v8AH/Cq/wDwmlmvjdvDU8LxTFQY52YbHYjIXHY4z+VWtD8SQ60m puYvsyafdNbO0sgw23HzZ4wOaYDfJ1j/AJ97T/v8f8KPJ1j/AJ97T/v8f8K1kuYH8vZPC3m58vbI Dvx12+v4Ugu7YqpFzAQzbFIkXlv7o56+1FgMryNY/wCfe0/7/H/CjydY/wCfe0/7/H/Co/EXiaTQ 7/TbK202S/udQZ1iRJljwVAPJbjv+lP0DxRFrVzeWU9pNYajZEefbTkEqD0YMOCPegBfJ1j/AJ97 T/v8f8KPJ1j/AJ97T/v8f8K1EvLWWJ5YrqB4k+86yqVX6kHApDfWmHP2u2wgBY+cvyg9CeeM0WAz PJ1j/n3tP+/x/wAKPJ1j/n3tP+/x/wAK1Jb21gVGmureNXGULyqoYe2TzVSW/uo9eitfJtvsDW7S vcNOodWB6bM5K479KLAVvJ1j/n3tP+/x/wAKPJ1j/n3tP+/x/wAK1Y7u2mk2RXMEjkbtqSKxx64B 6e9S0WAxfJ1j/n3tP+/x/wAKPI1j/n3tP+/x/wAK2qKLAYvk6x/z72n/AH+P+FHk6x/z72n/AH+P +FbVFFgMXydY/wCfe0/7/H/CjydY/wCfe0/7/H/CtqiiwGL5Osf8+9p/3+P+FHk6x/z72n/f4/4V tUUWAxfJ1j/n3tP+/wAf8KPJ1j/n3tP+/wAf8K2qDRYDE8rVx/y72v8A3+P+FIU1Yf8ALva/9/j/ AIVtGmNRYDGJ1Uf8u9r/AN/T/hSFtUH/AC723/f0/wCFajCmMKQGbv1T/n3tv+/p/wAKPM1P/n3t v+/p/wAK0DTTSAo+ZqeM/Z7b/v6f8KaZ9SH/AC72/wD39P8AhWh/CaZjNJsRTE2pn/l3tv8Av6f8 KcH1Q/8ALvbf9/T/AIVaAwakU1POwKY/tU9Le1/7+n/ClCasf+Xe1/7/AB/wrRU1Ko9aXOwMsQ6w R/x72n/f4/4UeTrH/Pvaf9/j/hWyDS0c7C5iGLVx/wAu9r/3+P8AhUcjarGpLW9tgekp/wAK6Dbx UE0YINRKpJbBc577ffj/AJd4P+/h/wAKDqF+B/x7wf8Afw/4VZuIdjmoMetczxc0K5H/AGnfZ/49 4P8Av4f8KnjuNSlXckFsR/11P+FQOlOgmaFwR0qVjJdQuT+bqZBIt7b/AL+n/CmG51Fetvb/APfw /wCFWp7yC3t/tEsixoOpJ/lXH6x4mkvi0VoDFCerdGb/AArpp1KlT4Qual14oe0mMTxRO467HJA/ SoW8YspGbcZPYGuXTLNx+dTJGAcnknvXYo9xcx0n/CXPj/j3X86cPFcrdLZf++qwEhLGrcVvinyo XOzZTxHcP0tk/wC+jTjr1yP+XeL/AL7P+FZi/KwVBlj2FWVC2vLYeb07LT5ULnZebWLxFDNbwjPQ Fzn+VM/t26/594v++z/hWezl2LMSSaYWJOF60+RB7Rmn/b10TgW8RP8Avn/CpV1W9ZSwt4QB6yH/ AAqjbwjG5jhR1Pr7Cpi4bk/Ki+lHIg52WDq14FLGCAAdzIf8Kpt4qkVseQh+jGsvUtS8xvLjOEHp WenzHJo9mhe0Z0q+KJm6W6f99Gl/4SWf/n3j/wC+jWEp2jA61NCpdgBRyIPaM6C31a9us+VbRcer kf0q2smqN0t7b8ZT/hUdjCLeAE4Hck9B71kw+PrKSUutvdfYVbabzC7BzjJXO4LnvWtLC1a1/Zxv b+v6RXP3N0z3dsoe8t1Efd4m3Bfr7VbBDKCDkGpkIkXDAEHg9wazNIYtp6+isyj6AkVzNFpl6iii kMKKKKACiiigAooooAKhjs7eGUyRQRI56sqgGpqKAIbi1iuk2yrn0I4I/Go7WxS1lllDF5ZMBmPH A6CodRvJrZwIY1fPZphHj8T1qzZStPaiR+pLfkCQKAIYbbTre9kaGG3jupM7yqgM2euTT4dMsreP y4bSCNOTtVAByMH8xxWVNd7LGBGRQ0sqDcDnJyDn61f1S6uLa9sFtoWn82R1aJXVNwCk9W44NVJW Af8A2NpvkpF9gtvLRt6r5YwGxjP1qf7Jb+akvkR+ZGSyNtGVJ4JH1rGj16eK3zLAhMY3zCSYBwC5 UBAAdxGPbtSHXb2S6hjjt7eMi7MM0TyHeF2My/w4yduRgkds+kgdBRVLS786hatI6JHIrlHjVidh HY5A557ce5q7QAUUUUAFIehpaQ9DQBWh+9D/ALp/lVqqsP3of90/yq1QAUUUUAFFFFABRRRQAUUU UAI33TS+H/8AkCQfV/8A0I0jfdNL4e/5AkH1f/0I00Byt/4SuNZ8V+IXmikgjmht3sbvoFmTJDA+ x6/U1za+FvE114WuReWU6ztrP2q5t4wpadCoBZVb5W5ycHg17Mtq5GSQPal+yP8A3loA8o/4RfUN N8O2t/o1lqL31pqJuIrW6jjSQIy7XAVPlVTwcD61L4b8H6lpviaztLqB20u0f+0BOwyrTtEqlfqG yfwr1L7I/wDeWj7I/wDeWgDiPGNvfL4l8N6laabd30NjJK8y2ygsAQAOpH+RWJqnhnXvEMXiDV1t WsLm9ijt7a0eQCRokYFtxHALYHFepfZH/vLR9kf+8tAHlEvh29u21i40nw/PpFnJpDWv2N1VWuJu xCqSDj1NOtvAqrqun79EUW39iFJwUG03OON3q2f6eleq/ZH/ALy0fZH/ALy0AePXPh/XrnwvommX OiySLFZSKzrbxSTRS7jtUmQ/IpGORz6VctPD2s79OeewuC0fhye0kLDkSncFQ+/I/OvVfsj/AN5a Psj/AN5aHqC0PLtB8KXelaj4MuYNLe3kiglXUpFXBBI4DnvXpNWPsj/3lo+yP/eWncRXoqx9kf8A vLR9kf8AvLSAr0VY+yP/AHlo+yP/AHloAr0VY+yP/eWj7I/95aAK9FWPsj/3lo+yP/eWgCvQelWP sj/3lpPsj/3loArGmtVv7G/95aabFz/EtAFEimN1q+dPkP8AGv60w6ZIf40/WkBQNNNXzpUp/jT9 aP7Jl/vp+tJgUAPlNNArRGlS4Pzp+tJ/ZMv99P1qGmBR25FIAVODWgNKlH8afrTjpchAy6ZH1qHG QFOOpgamGmSj+NP1qRdPcdXWp5ZdgIUFSAVKLNx/EtOFq/8AeWjll2AiqNhVr7M3qKQ2rH+IUpQk +gjHu4cjNZrpg10slgzgjctU5NDlbpJH+tcFWhVbuoisYe3IxWZqeqQ6eMfflPRAen1rob7w5qck W2zuLaMnqz7sj6cVz7fDbVHYs17aFickndz+la4fBOT5qunkFmcrd3s97JumctzwvZfoKbHCW5bp XYR/Da+Tlru1J/4F/hUw+Ht93urb/wAe/wAK9eKjFWQrM5FY+wHFWYoc11C+Abxf+Xq3/wDHv8Kn TwRdr/y8W/8A49/hVXRPKzm44QBzUihpG2RDJ9a6FvBd83AubcD/AIF/hUp8IXiReXBcQKD95jnJ /SndC5Wc6WS1UrEd0nd/T6VCHz1PNdD/AMIRff8APzb/APj3+FNbwPfnpdW3/j3+FPmj3E4y7HPl yxwvWrNvANpZjhR1Pr7CtuLwTdoRvuIMdyM/4VO3hK8YgefAEHAAzx+lHNEXLLsYZbf/ALKL+lZe p6jtHlRnArrLjwffyJtjuLdVHru/wrJf4canJIWa8tOf97/CmpR7icZdjklJdsmrKfKK6ZPhzqC/ 8vdr/wCPf4U//hXuo/8AP3a/+Pf4U+aPcXJLscyvJrY0u33yAkcCtOPwBfIctdWx+m7/AArWsvC1 xar800JPtmk5R7lKEuxC1ss1q8TglHUowHXBGD/OvPIfh1qkayad9ogWxknEj3CudzIBwuzGc859 M160mkyr/Gn61J/Zsn95K6MLmFXCqSpta9/LZmjhfcp20QiiRAMKoCgHsAMD9BWVo3/HgP8Aff8A 9CNbMitFJscYIrG0b/kHj/ff/wBCNcMi4l+iiipKCij/AD0ooAKKKKACiiigAooooAZJDFMB5sSP jpvUHH504AKoCgADgAdBS0UAU10y3WdJcE+WcojHIU+oq05RR5km0BMnc38Pqc9qdWfrDBbeDdtK m4QMHPykc9fxxT3AsJFZ3Qinjjt5dnMUgVW2/Q9qVbG0VCi2sAQtvKiJcFvXGOvvVPQ5RcSX7KoV ftRUYHHCqD+uaz7fxFNdpBmK2xO0e0QzMTGTJtw/oe+O9DWtgN+GCK2jEdvFHFGOQsahR+QqSuXt 9d1GOJZbmSxkH2ZnKqxXL79vPpgckdhU1xrl80U0VvHaNLHHKzTROWQhQMFPU/N37ikB0VFclc3k 0TNFa33m2KsZPOlumBchFOwSDk8np+HtVy41m6uIVjgi8meOeFZQzkY3SLtU9/mU5NAHQ0h6H6Vg jxK6SRxSwRGUiTcI3OAUZgR06tt+Ud+fSobTVbq61iJJri38pyhP2aUsmTGxC5PckcjvigDbg+9D /un+VWqqw/eh/wB0/wAqtUAFFFFABRRRQAUUUUAFFFFACN9007w5/wAge3/3m/8AQzTW+6ad4c/5 A9t/vN/6GaaA6InAyaqWurWV9Dby21wsiXOfKIB+bHX6dO9WnG5GA6kYrlh4c1C1htxaG3OLVxLG 7HHneXsyOOh7/TPc0gOrorjIPCmoHThDNIQ0SS+Qv2k5jZnUqcqACQAeccZp954WvTd24gklFlFK zJFFc7DHlgQ2WVs9+Bj60AdhRRRQAVHOxWFypwQKkqK4/wCPd/pQAeQP78n/AH0aPIH9+T/vo1LR QBF5A/vyf99GjyB/fk/76NS0UAReQP78n/fRo8gf35P++jUtFAEXkD+/J/30aPIH9+T/AL6NS0UA ReQP78n/AH0aPIH9+T/vo1LRQBF5A/vyf99GjyB/fk/76NS0UAReQP78n/fRo8gf35P++jUtFAEX kD+/J/30aPIH9+T/AL6NS0UAReQP78n/AH0aPIH9+T/vo1LRQBF5A/vyf99GiLIeRckhSMZOe1S1 FH/rpfqP5UAS0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQA UUUUAFFFFABRRRQAUUUUAZOqNi7X/dH86wtG/wCQeP8Aff8A9CNbGsNi9T/cH8zWNov/ACDh/vt/ 6Eap7ImO7NCiiipKCiiigAooooAKKKKACiiigApCQBknAFLUc4LW8qqMkowA9eKAD7RD/wA9ov8A vsUrpHcRFXVJI2HIPINcDLHfLDJYtC32ySbzI2JIKpjp+NdGZZ9J0ewupEkBhBSWLk/eztyPZtv5 1TVgNuGGO2jWOGNY0X7qqMAVFZ2MFhbJBBGFRR1IGT7k9zWCk19pmoPG0ks/mRRQbSSwWZgSG9hn I/AVLoepTB7KxmlRw1vuDEl5ZCM5J5+UcdxznrUgb3kxZyIo88/wDv1/OlSNI1CxxoijgBVAAp1F AEYghEYQQxBFOQoQYB9QPWnFFJJKqSSDyO46U6igBnlRjpGnXP3R19frSLDFGhCRRqM5wqADPr9a kpD0P0oArQ/eh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt 9007w5/yB7b/AHm/9DNNAdHRSMcKSPSsG3v5LSG3la5a4M0O90kI+VuMYwOMk4xSA36Kw49dmd/N aKOOEQsxRnO4OH246etTNqb3GlRzqDbtJMIXJ58v5tpP6frQBrUVkvqDWVxFaQpNesRuZywJUZx2 HNQya1cvDmK3jRmKlC8mQV8wIc8cf/XoA3KZIyrGxcZUDmsoa5K5kKWLFFcRo5fAZt23BOOP1q80 rS2MrOqqw3KQr7hwcdaAGeZB/wA+5/If40eZB/z7n8h/jWTcTSJNdOtwymEoEjyNrZHTHvUcupTz wN5EDRKXVRIWx/Fg9utAG15kH/PufyH+NHmQf8+5/If41lxaqr3IiKfKWAWTdwQcjPI9qiXUZpZ4 2iCbJGQbWboDuzg49qANnzIP+fc/kP8AGjzIP+fc/kP8ajooAk8yD/n3P5D/ABo8yD/n3P5D/Go6 KAJPMg/59z+Q/wAaPMg/59z+Q/xqOigCTzIP+fc/kP8AGjzIP+fc/kP8ajooAk8yD/n3P5D/ABo8 yD/n3P5D/Go6KAJPMg/59z+Q/wAaPMg/59z+Q/xqOigCTzIP+fc/kP8AGkMtuBkwED6D/GmU1+i/ 7w/nQBJ9otv+eJ/IVZtpI5EPlLtAPIrHhk1E6i8csUAtV5WRSdze2M9av20nk2tzIBkplseuBQBf orCS5uFaaSR55EigViVdVGSuTxjNSDV5bdZjLGJFEkgQhxu+XsRjge9AGzRWfBqMs18bc2rAIdry BshWxn06c1C07vHdzm6ZHiLqsQIAGBxkdz3oA1qKxn1t7e3PmwAzIASgfO5duSelPfW2QSP9nzGN 4Q+ZyxXHbHHWgDWorKfWJUkEIs2eYMQyo+QAMdDj/aFOsLmWS5CyOWV1c4OPl2uR/Ij8qANOiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAMDWzi/Qf7A/maytF/ 5By/77f+hGtPXf8AkIpj+4P5mszRf+Qcv++3/oRq5fCjOPxM0KKKKg0Cij/PWigAooooAKKKKACi iigAooooATA9Bn6UtFFABSYGcgDPrilooAKKKKACiiigApD0NLSHoaAK0P3of90/yq1VWH70P+6f 5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8AIHtv95v/AEM00B0dQLZW qRuiW0KpJ99RGAG+o71DrLFdEv2UlWFvIQQcEfKa8SC679hW7Lal9mKh/N8x9u3pnOenvVRhzEyl Y9zNlalAhtoSqggDYMAHqKetvCsJhWJBEeNgUbfyr5//ALSvv+f67/7/ALf40f2lff8AP9d/9/2/ xq/Y+ZPtD337Da7Y1+zQ7YzlB5Y+U+3pTmtoGQq0MZUjBBUYxnOK8FW51V7Z7hbi+MEbBXkEr7VJ 6AnNJDeancTJDBdXskrnCoszksfQDNHsfMPaHvS2lusjOsEQdsZYIMnHTmleEGF40AXdnoO5rwa5 uNWs5fKuri+ikxna8zg/zqH+0r7/AJ/rv/v+3+NHsfMPaHuh0/M3mmOAyD+PHP54oGnYZmEUAZjk kDk/pXh8N3qlxIqQXN9I7MFASVzknoOvWkkvtRhleOW8vEkQlWVpnBUjqDzR7HzD2h7f/Za4YeTb Yblvl6/Xinf2d8oXy4Noxxjjjp2rwz+0r7/n+u/+/wC3+NTiTWGaBVl1Am5GYQJHzJzj5eeaPY+Y e0Pb/ss3rH+Zo+yzesf5mvD7ibV7SQpcT38TABiGlfgHoetQf2lff8/13/3/AG/xo9j5h7Q93+yz esf5mj7LN6x/ma8I/tK+/wCf67/7/t/jR/aV9/z/AF3/AN/2/wAaPY+Ye0Pd/ss3rH+Zo+yzesf5 mvE7VddvomltH1GaNeGZJXIH15qqdRvwSDfXYIOCPPb/ABo9l5h7TyPdvss3rH+Zo+yzesf5mvCP 7Svv+f67/wC/7f41Z067vLrU7WCS+vNksyo2J3zgkD1o9j5h7Q9u+yzesf5mj7LN6x/ma56bQPD1 vfJZzandR3LgFYm1CQMc9OM98VcPgrTFGWuNQA972T/GstC9TV+yzesf5mj7LN6x/may/wDhCdN/ 576j/wCBsn+NMfwdpMbIr3V+rOdqg30nzHrgc0aBqa/2Wb1j/M0jWkzDqg79TXPpoHh6S6lt01K7 aaJgsiC/kJUk4wefU1ai8HaTPEskVzfujDKst9IQf1osg1Nb7Jcf89E/z+FTW1uYo3VyG3Hmsb/h CdN/576j/wCBsn+NUhoHh0zywjU7oyRYEi/2g+VyQBnn1IFFkGp1EVrFDD5SoCuNp3c5HofWlFrA HZhBFucYY7BlvrWH/wAITpv/AD31H/wNk/xo/wCEJ03/AJ76j/4Gyf40aBqbvkRed53lJ5uMb9o3 Y+tI1tC0pkaGMyEbdxUZx6ZrCbwVpiqWa41EADJJvZOP1pI/Bmlyxq8dzqDIwyrC9kII/OjQNTee 1gl/1kET9/mQGlMERGDEmOeNo79awv8AhCdN/wCe+o/+Bsn+NH/CE6b/AM99R/8AA2T/ABo0DU3J LaCX/WQxvzu+ZQefX60JbRxzNKq4Yrt9gM56fU1h/wDCE6b/AM99R/8AA2T/ABpkvgzTYo2fztRO O322T/GiyDU6Siua/wCEKsv+et7/AOB0v+NcV45szoVzZrYXd6glVy4a6duQR0yfeqjFSdkxNtK5 61RXz5/aV9/z/Xf/AH/f/Gj+0r7/AJ/rv/v+/wDjV+x8yfaH0HRXz5/aV9/z/Xf/AH/f/Gj+0r7/ AJ/rv/v+/wDjR7HzD2h9B0V8+f2lff8AP9d/9/3/AMaP7Svv+f67/wC/7/40ex8w9ofQdFeA21xq t5OIbW4vppWBIRJXJOOvemzXmp20rRT3d5HIvVWmbI/Wj2XmHtPI+gKK+e/7Svv+f67/AO/7f40f 2lff8/13/wB/2/xo9j5h7Q+hKK+fTqGoBQxvLwKehMz4P611/gqwj1iwu7jUry9PlSBQRdOoAxns aUqfKrtjU7ux6nRXHy+HNJgz5t3epjHW+k7/AI1IPCmnsoKzX5B5BF7Jz+tZ6FanWUVyf/CJ6f8A 89tQ/wDA2T/Gj/hE9P8A+e2of+Bsn+NGganWUVyf/CJ6f/z21D/wNk/xo/4RPT/+e2of+Bsn+NGg anWUVyf/AAien/8APbUP/A2T/GkbwpY5AWW/yfW9l/xo0DU62iuRbwnZqpJkvsDn/j9l/wAas+C4 zAmrW/myyJDfMiGVy5A2Lxk/Wiy6Bcsasu7VUJ6LGP5msnRf+QeP99v/AEI10OoRq1wuRztH8657 Rv8AkHj/AH3/APQjVSeiJitWX6KKKgsP89aKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig AooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsP3of90/yq1QAUUUUAFFFFABRRRQAUUUUAI33TTvDn/IH tv8Aeb/0M01vumneHP8AkD23+83/AKGaaA1da/5AWof9e0n/AKCa8ssta0+1tbC4e43vb6W9o9sE bLuxPBOMbec9a9bu7cXllPbsxVZo2jJHUAjFcQPhNYAAf2ld/wDfKf4VcJJKzJkne6MT+0fCyzRt 5NsYxJmNRbHMaeURh+PmO/Hr60kl/wCGLqFEkENuwEDeZFachgD5nGMEdOOnpW5/wqew/wCgjd/9 8p/hR/wqew/6CN3/AN8p/hV80e5FpGXHqnhVZsOkHls0JZRAxUkKd/GB3x2FZK6npUXiTRL63CQx xIjXZjiKgOCcnaB6Y6V1X/Cp7D/oI3f/AHyn+FH/AAqew/6CN3/3yn+FClBdQcZM53TdR8PzTzPq EFrEft5lyYWbzIiDx3Oc4ODipIdR8Lm3jjubeJlSOEny4CHLhzv+buNuK3v+FT2H/QRu/wDvlP8A Cg/CjTwCTqV2AO5Cf4Uc0O4WkYWn65psEF5a3M1n5T3cUymK1Ko0YPIwFzuA9fzpJdQ8MvuIihMZ P3DCTJv83O7dj7uzt+GK2P8AhWekf9Bmb84/8KP+FZ6R/wBBmb84/wDCjmh3C0jmNcm0bUjax6f9 nt3N1IjusXlqsJYbCeOw/Gm3OtAeLEubC4jggs18i2eRCybFUgZA5w3P511P/Cs9I/6DM35x/wCF H/Cs9I/6DM35x/4U1OPcOWRkDWdCitdQtIQqwXHkO6LG212DZkVCRkD0z3z0pJdS8OJ57JDaSTeX P5JW1IQZI8pSpHLAZ5/Wtj/hWekf9Bmb84/8KP8AhWekf9Bmb84/8KXNDuO0jKguPDjw2sSolzcg sdwtDnJjONyKvIDdhn1qzfvoGl3JintLI3vkwl1eJhHyp38KCQ3IPQdulXV+GulI25NbnVh3VkBo Pw00kkk61OSepLISaOaPcLPscbouoWtpY63HO3ltd2pihXBOTnOM/wCNdVY2+kyDSw+nWyXN7bvN MkkPEXlxsM7fRmGffFT/APCs9I/6DM35x/4U4/DbSy2465cFsYzvTOKHOL6gotGOup+GVjYi2s5L oJEHeSFlhkOPnKgKSDn2GaxYZIZPFGnNbmAp5kI/cxGNQcjIIPU+p712H/Cs9I/6DM35x/4U6P4b 6VDKkketzq6EMrAx8Ed+lPniuorM39StL2W/1CGKwM0V7FEizM6COMjdksM7uMgjAP4Vmz6VrN3N qIuUvGtmKtHEtzjLLICNh35GVGedoz2q7/Y1z/0N17+cX+FH9jXP/Q3Xv5xf4VhZdzS5Po8GrJr9 5NeCZLSRSEjeUOoIb5SvzE8ryeBz69a0vJefWTLIpEVtHtjyPvO33iPoAB+JrG/sa5/6G69/OL/C j+xrn/obr384v8Kdl3C5PBpeoW5upEnmzNfeYId67BHvBJGBnkZyCaztO0jV0t47c/bbaIGNZg10 Duw+WMeD8q7eOME+meatf2Nc/wDQ3Xv5xf4Uf2Nc/wDQ23v5xf4UWXcCbQ7LVYdUvX1G4umRshAx UxY3fKV+YkHbjPyr+NQmwv5vDw0lrF4pITGPtCyJtkxIpLLznJAJOQOfWj+xrn/obb3/AMhf4U9P D99IuY/FOoMPVREf/ZaLLuBVnsdejubWO3NyYoLklZDcbt8XmDiTLDPy5wcMfpU0Ok6sqRRyXV4V lEbXDG5+YEOdwU5+UbfT+dTf8I3qP/Qzal/3zH/8TR/wjeo/9DNqX/fMf/xNFl3ApHTNeN/bLJc3 f2aNNoCOrA8sD5hLAnI284Y/Q102mwvb6Zawyja8cSqwznBArG/4RvUf+hm1L/vmP/4mj/hG9R/6 GbUv++Y//iaVl3C50VFc7/wjeo/9DNqX/fMf/wATR/wjeo/9DNqX/fMf/wATRZdwudFUN1/x7PWH /wAI3qP/AEM2pf8AfMf/AMTR/wAI3qP/AEM2pf8AfEf/AMTRZdwuMPhHSj4r/t3zp/OxnyPOPlb/ AO/tz19uneuS+KJzeaeR3WT/ANCFdh/wjOof9DJqH/fEX/xNUNR+Hw1Z0fUNavp2jGFLKnA/AVUL Rd7ildqx5JRXp/8Awqew/wCgjd/98p/hR/wqew/6CN3/AN8p/hW3tYmfIzzCivT/APhU9h/0Ebv/ AL5T/Cj/AIVPYf8AQRu/++U/wo9rEORnmFFen/8ACp7D/oI3f/fKf4Uf8KnsP+gjd/8AfKf4Ue1i HIzhvC9/b6Zri3N222IQyqeCckoQBxz1q/pt9oK6YPtdtbpJ++82IRMWckDy9jc7QDnPNdV/wqew /wCgjd/98p/hR/wqew/6CN3/AN8p/hSc4vqNRkjETV/DYv7nNvZ/Z99uIf8ARP4f+Wvb+f4UsWoe F5L2CeUW8aoHDxC1+SQeZ8uflPIT069M1tf8KnsP+gjd/wDfKf4Uf8KnsP8AoI3f/fKf4UuaHcLS MG+1nSrnS4bW3mtES2FwqJPaM/BbKFcdDjuT16itL4axiXRdQjf7rShT+K1c/wCFT2H/AEEbv/vl P8Kvaf4CbSkdLDXL6BXOWCqnJ/EVLceWyY0ne7LiR3cVszGIvcSNsYqQSigYyMnqf61fhQRwoiqV VVACntVD/hF7/wD6GTUP++I//iaP+EXv/wDoZNQ/74j/APiazsu5dzSorN/4Re//AOhk1D/viP8A +Jo/4Re//wChk1D/AL4j/wDiaLLuFzSorN/4Re//AOhk1D/viP8A+Jo/4Re//wChk1D/AL4j/wDi aLLuFzSpjY3jdjBBHP4VQ/4Re/8A+hk1D/viP/4mk/4Re+PXxHqH/fEf/wATRZdwuPtdLtrGS4li d2aQHAaTIQegpPCf+v1z/sIt/wCgJTf+EWvv+hj1D/v3F/8AE1oaFog0SC4T7VLcvcTGZ5JQASSA O30p6JAS3ozOv0Fc3o3/AB4f9tH/APQjXTXf+vX6VzOj/wDHh/20f/0I0PZCW7L9FFFSUFFFFABR RRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFIehpaQ9D9KAK0P3of90/yq1VWH70P+6f5V aoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8ge2/wB5v/QzTQHQu2xGbrgZ rITxHCttZ3F0vkxXFp9pZsltn3AFwBknL1sEAgg9DWIvhiNYHia8uJE8ryYRIsbCFMg4AK4PQctk 8CkBdXW9Pa9itPtIFxKoZI2Ug8jODkcHHODz7VUk8TWkV9PZvJCt1HcpbrCZgHcMEO4Dr/F+lJZ+ FrWy1GK8jkd5EVVbzUR2chdobeV3A4x0IHFXf7Jg3TtufM9wtw3ThlCgAe3yD9aAIv8AhI9L2zN9 qGISA2Eb5snA28fNzx8uaLXXrO7v57RDKskTqnzxMoYsu4YyPSqzeF4mt3ha8uHTZ5cIdY28hCQS FyvPQcnJGBzVqz0ZbKffFd3JUhNyMwO4qu3JOMnIxnntQBpVXv8A/jxl/wB2rFQXis9nKqgkleAK AKs2r6dDfGzaZBcj+AqcAkZAJxgZHT1rPPi3TkuRbvHKZPNiiJjhZkJkXcCDjkUPpq3OrXNxcT3I tpHR1gWM7WZVwCflyMHsDjinf2PahgY7m9TBhZQIs4aMYB5XuOD/AEoXmBa/4SDSdsx+0piHG47G wcttG3j5ueOM81Qm8UpbI081ogtpA/2Vt/zSFWC4ZSPlySMdfcdqki0e1R1MlzfSrER5CvGcRKHD YGFyeVA5ycClbSLR5ZWae8IJZoVMXFuzMGJX5fUA85oAns9btZ7AzStAJlwGSPcQGJIUfMoOcjHI 4NFl4i0u7s0nM0ceQNwZSApK7upHIwDz0OKhi0iySeGd5LuSZN5d2jP71m7sAuOMnGMAZpLjRbC6 gWGR7vyljjjK+WedmSh+71BOfqBQAt94osbW4WCBPtEmG3AArghcgZIwT0BGeM80kmsXkdrO50+2 M1pJsuEE5x0BGw7fmJDDggc0yTRLWeYtNeag8WXZYjHgKzjDMCFznnI54NK2kI0kMo1K/EqSGV3N up818ABmBTGQBgYxigCzZas11qCwyWqRwTCQ27h8s2wgNuXHHXI5P4VrbF/uj8qybSytrS9a4Et1 IBu8qN4jth3HLbcLnk+pOO1aP2yL0l/79N/hQBLsX+6Pyo2L/dH5VF9si9Jf+/Tf4UfbIvSX/v03 +FAEuxf7o/KjYv8AdH5VF9si9Jf+/Tf4UfbIvSX/AL9N/hQBLsX+6Pyo2L/dH5VF9si9Jf8Av03+ FH2yL0l/79N/hQBLsX+6PyqK4VREMADLAdPej7ZF6S/9+m/wqOa5R48KJcgg/wCqbsfpQBi2nijS r3xDNokVrfC+hyZA8DBUH94t0wex71s2X+tuP94fypn2lQ7OLdg7DBYRnJH1xUliDumYqwDMMZBH b3oAt0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFF FFABRRRQAUUUUAZuoSlLhQPQVz2jf8g8f77/APoRrd1Q/wClL/uj+dYWi/8AIPH++3/oRq5bIiPx Mv0UUVBYf56UUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0P0oArQ/eh /wB0/wAqtVVh+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33TTvDn /IHtv95v/QzTQHR0UVnXjyS6jFaLctboYjIWTG5yCBgEgjA7/Wkyox5nY0aKyJdbNuHZod8Ks0Yk Eg3M6rk/LjgcHnP4U281kw3I2jKRbgygj5zsUjnt96lzItUZt2sbNFZMGpTpa6hPcxgtbyYESODj 5QcBsD170h1qYFo/siidNxdTMAAqgHIOOT8w4x+NHMheyl0Neisy01OaaK9nkiURQndFhvmZdgbk Y4PNQDXZ3mEKWDl/LDOwbKxkglcnH5/1o5kP2M3obVFYkOsXEsMQngCNthaR45R1c8AAj86IPEfm J50tlNDbE/LK3RgQcdupxj8RS5kHsJ9jborLs9Yku9Qa2+wzIiZV5eqq4AJHTHfrn8KYt9N5O5pf +X8xEkD7m8gD+VO6F7KS0Zr0VgTaldpfqYpN8UcsxeMAfMihOB7jcT+GKs2OrNNLbxsoaOYMVnLY DfMcBRjk4HqKOZDdGSVzWorO1DVGsptiQCQLEZnJkC4UEA4Hc81Uj1yVbnyBby3DGV9zKMeWm7A7 dvfFHMgVGbV0blFYY8Rs3m4tPlRl/eFyE2HOHztzjj0/GrVjqzX17LEtpKsKFgs5+6xU4I6flyfw oUkwdGcVdo0qKxn1S6mW1ltoF2SykIplHzrtb73Hy9O2aYviUOxZbKcwJHvkkHIQ7d2OmPbOaXMg 9hN7I3KKxjqtxHqtvbyxKrzxZWNZMqPm5O7Hp271Yu9WFpO8TwsSo3DBHKYJLfhjH4indCdKWxo0 VlRa03nrDPbrHIefllDLjaWHOBzweKlsdTe+MhW32xoitnfksWUNgD8etF0DpSW5oUVh/wBuzkxS NbxrEEkaZPM+dCoBxggc8/8A16nGsSiVYZLVFl3YfE2UVdu7O7HoOmKXMhujNGrRWHD4ie4LRx2f 75WPDSbVKbd24Ej09qltde+1XSRraSiIorSTE/LGSu7BOMYweufwp8yB0Ki3Rr0UUUzIKKKKACii igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAytTG bpf90fzrB0b/AJB4/wB9/wD0I10GoDN2n+6P51z+jf8AHh/wN/8A0I1ctkRHdl+iiioLCiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tIehoArQ/eh/wB0/wAqtVVh+9D/ALp/ lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33TTvDn/IHtv95v/QzTQHR1BcW9 tdKEuYopVU5AdQcH8anqMnk0rXBNrVEf2S089pvIg81l2l9oyR6ZpsdjZRReXHb26R8/KqADng08 sQ6jsc5qC51CK1kCOrkld3ygYAzjuafKh80u5NFa2sMJhihhSI9UVQAfwpJrSzuMedBBJht3zKDz 0z9agh1OKeVI1SUF8gFgMdM9j7VO86RLLJM6pHGNzMxwFGMkk0WQc0r3uPENuHkcRx7pQA5wMsBx z6002tq06TGGEyxjCvtGVHoDTWu4EMYaaJTKCyAsPnAGSR64HNRDVtPZCwvrYqFRsiVcYc4U/Qng etFg5n3J47W1iTbHDCi5BwqgDg5H60i2VmnmbLeAeYwZ8IPmIOQT6nNRXWqWNkXF3eW8BQKWEkgX aCcAnPqRgVLBcw3UCTW0qSxOMq6MGU/Qiiwc0u44W1stybgRRCcjBkCjcR9aYbCxLSsba3LTf6w7 B8/19alzRmjlDml3EigtoVQRRxIIwQgVQNoPXFQvp9k4GIYkIXaGRQCBnOAewqfNGaOUOaXcgmsL W4vI7idUkaNdqBgCFOc5HvUj2lpI6O8ELMjFlYqCVJ5JHvT80Zo5Q55dyAabp4BAtLbBbeR5a8n1 +tSx21tFO80cUSSv951UAt9TTs0Zo5Qc5PdjI7W0ikZ44YVdm3FlUAk+v1pBZ2YlEoggEirsDbBk L6fSpCeKq3Op2enwRyX91BbrJwrSuFDHGcDNHKg5pdyVLGyjiEaW9uqDooQYHOf580C1jN89zJJv Yp5aqcYVe/1zUkciyxrJEyujgFWU5BB6EGnZ9qOUOeXcgGnWAg8kWtt5W7fs8tcZ9cetTJHDGpVF jUEYIAAzxj+VVf7VsDPNAL23M0GPNjEg3Jk4GR26irfPpRZA5Se7IVsbJERUt4AqElQEGFJGDinQ 2lpboEhhhjVSSAqgAZ60/NIGJkYdgBijlDmk+pSutHtJrbyYEgt0Lbm2woc8Y7jg+9Trp1iuzNvC zJGIwzKCduMYzUkNxFcpugkSVfVDketOeRY0LyEKg6sxwBS5UV7SdrXJNy+o/Ojcv94fnUMU8U6s 0MiSBWKMVOcMOCPqKV2wox6gfrTsQS7l/vD86Ny/3h+dVX1C0iExkuYU8ghZdzgeWT0B9M5GKsc+ lFgHbl/vD86Ny/3h+dQvPFFJGkkiI8pKxqxwXIGcD14FSc+lFgHbl/vD86Ny/wB4fnVC51nTrMsL q/tYSjbGEkoXDYzg++OfpVtJEkjV42VkYZVlOQR6g0WAk3L/AHh+dG5f7w/OombDoB0JOfyqvHq1 hLfNZR3ts92uQ0CyAuMdeOtFgLu5f7w/Ojcv94fnTefSo1njeZ4VdGljALoDyuemR2ziiwE25f7w /Ojcv94fnTGYIhZyFVRkk8ACoJ9QtLUKbi5hiDIzgu4XKgZJ57AdTRYC1uX+8Pzo3L/eH51VtL+1 1CEy2VxDcRg7S0ThgD6cVMzYRiOoBosIk3L/AHh+dG5f7w/Oql1qFrYW6zX1zDbxkgb5XCjPpk1Y B3AFeQeQR3osMfuX+8Pzo3L/AHh+dRJKkjOqMrNGdrgHO04zg/nSRXEU6loZEkVWKEoc4YHBH1FF gJty/wB4fnRuX+8PzpmaM0WAfuX+8Pzo3L/eH51E7FVyOuR/OnZosA/cv94fnRuX+8PzpmaM0WAf uX+8Pzo3L/eH50zNGaLAP3L/AHh+dG5f7w/OmZozRYB+5f7w/Ol6jio80sTFlbPZiKLAUr7/AI+l Poo/nXO6N/yDx/vv/wChGuh1I7Xz/s1z2i/8g4f77f8AoRpvZELdl+iiipLD/PWij/PWigAooooA KKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tIeh+lAFaH70P+6f5VaqrB96H/AHT/ACq1 QAUUUUAFFFFABRRRQAUUUUAI33TTvDn/ACB7b/eb/wBDNNb7pp3hz/kD23+83/oZpoDo6gJ+Y1PU Bjck9PzpoTGMf3ifjWbqTRi/j81gq+X378n/ABrUMLkg8ZHvS+XIPT86bsBmJNBJe2ogcMQTkD/d apNXtZL/AEfUrSHb5s9u8SbjgZKEDP51f8uT2/OkELgk8c+9LQLnMXWn6vcxabOLGBZrKOSEwm5B Dh4wu4NjjBHTFYd18PtRLWslrcQq9vFZwNluJo4zlwRjjBCkfjXonlP7fnR5T+3509NwOW1zQtQu vELalaRwyIv2UqjS7CxjkZmHQ44IxWroGnzafaXP2nastzdSXJiRtyxbznaD39fqTWp5T+350eU/ t+dCsgDNGaPKf2/Ojyn9vzo0AM0Zo8p/b86PKf2/OjQAzRmjyn9vzo8p/b86NADNGaPKf2/Ojyn9 vzo0AQng1i6xaXk11pF3YwR3Bs5Hd43l8vIaMrwcHua2/Kf2/OkELhQBjA460aAcNZeDNUgvLVJ7 2Q2iw7T5NxsEOd+9QNuWGWGOR09hW74asNW02LZqkkc8kuWlmEpOCuFRVUjoVGSfUmt3yn9vzo8p /b86NAONuvDurvf37WQhtredt4An3b2MqMWU7d0eVU7lyQTiqM3hTXJ0uyP3JnvGmaKK9wrR7SEC kqduDhjxySfQV6B5T+350eU/t+dGgXI7dGitoo3cu6IFZ2OSxA5JNV9RgkurC9ghKiSWEohbOASC BnFXPKf2/Ok8lwxPGT70OzBaHnDeD9TsDaWdvGkplW5LNC7wxxttVY2Y5OSMZA+tWLvwjrl7LqIu ninSeHaM3BCysHjZSVxlThWHXGTwK9A8p/b86PKf2/OjQDltC8OXWl6z9sISNJXuWmRJmYEOymMY 6HAB5rppD8o/3h/On+U/t+dIYXI5x1z1o0A5jW/DV5qPiKO5tZoorKaNTdhvvGSPcYWA78kZ/wB0 VmWnhDVjYQw3s7sQ7tKpvCQzGEqGG0DGXIOOemetd35T+350eU/t+dLQLnCf8Iprj3scrzxi4QMR em4ZmTMHlhQmOz5Oe+c1Z0zwxqCanZTXQWKxhkZzaC7eQIfLUA5/iywJwen412XlP7fnR5T+3509 AOTn0TU4PE02q28EFwhvPOWMzbCV8gR9SDzn9K2vD2nPo+hWtjK4eSJTuK9ASScD2GcD6VpeU/t+ dHlP7fnRoAxj86fU/wAq5O68IXVzFq8punFxLNcTWMauFRGkj2BywG4Hr3wK64wuSDxx70vlP7fn RoFzjpvDGqJfRfYpxCiLH5c/2hiYAEIddn8W5jnJ9fYVQtvCeuWrefBHFHJiNXQXjHzCInRmLEf3 mDYr0Dyn9vzo8p/b86NAucO3hTVZbOe3uhHcPLYiBZmvHAjIh2lNvRgX+bJ9c1Z1vw1fajp+nxW0 UETW9hPbtG0uQHdFVQDjkZHWuv8AKf2/Ojyn9vzo0C5jaLYXUOpahf3kaQNdiJBAkm/ARSNxOByc /kBWu5/dt9DTvKf2/OkMTkEHHPvRoBh+KNMutT0+AWMKPcxMXjczeWYyVIzyCGBzgqRyCawLnwnr 80mos17uknh2RyR3HlJyE+XYFyuCGIOfT1ruxE4GOPzpfKf2/OjQLnDax4W1mXz4tNdEtzcGSAfa mVov3aKvXIxkNx154PWum0Gwn02ynjuNm+S6mmG054dyR+PNaflP7fnR5T+350aAGaM0eU/t+dHl P7fnRoAyQ/J+I/nT80hhdhg4/Ol8p/b86NADNGaPKf2/Ojyn9vzo0AM0Zo8p/b86PKf2/OjQAzRm jyn9vzo8p/b86NADNOg+43+8ab5T+351JGmxMdycmkwRmaw2JFHqKwdF/wCQcv8Avt/6Ea2dZP8A pSD/AGR/OsbRf+Qcv++3/oRpy+FER+JmhRRRUGgUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUA FFFFABRRRQAUh6H6UtIehoArQ/eh/wB0/wAqtVVg+9D/ALp/lVqgAooooAKKKKACiiigAooooARv umneHP8AkD23+83/AKGaa33TTvDn/IHtv95v/QzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAw9Z/4/U/3B/M1j6L/AMg5f99v /QjWxrP/AB/J/uD+ZrH0X/kHL/vt/wChGtJfCjOPxM0KKKKzNAoo/wA9KKACiiigAooooAKKKKAC iiigAooooAKKKKACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsH3of90/yq1QAUUUUAFFFFABR RRQAUUUUAI33TTvDn/IHtv8Aeb/0M01vumneHP8AkD23+83/AKGaaA6OiiikAUUUUAFFFFABRRRQ AUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAB RRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAGHrP/AB+r/uD+ ZrH0X/kHL/vt/wChGtjWf+P1f9wfzNY+i/8AIOX/AH2/9CNaS+FGcfiZoUUUVmaBRRRQAUUUUAFF FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6H6UAVoPvQ/7p/lVqqsP3of90/yq1QAU UUUAFFFFABRRRQAUUUUAI33TTvDn/IHtv95v/QzTW+6ad4c/5A9t/vN/6GaaA6OiiikAUUUUAFFF FABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAGJrA/0 xf8AcH8zWNov/IOX/fb/ANCNbOs/8fa/7g/maxtF/wCQcv8Avt/6EauXwozj8TNCiiioNAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0NLSHofpQBWh+9D/ALp/lVqqdu4aSID+ FT/KrlABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/3m/wDQzTW+6ad4c/5A9t/vN/6GaaA6Oiii kAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF FFFAGJrP/H2v+4P5msbRf+Qcv++3/oRra1j/AI+1/wB0fzNYui/8g5f99v8A0I1cvhRnH4maFFFF QaBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6GgCjaf8AHwv0NX6p WybZY2z94H+VXaACiiigAooooAKKKKACiiigBG+6ad4c/wCQPbf7zf8AoZprfdNO8Of8ge2/3m/9 DNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigDG1j/j6X/dH8zWJov8AyDh/vt/6Ea29X/4+l/3R/M1iaL/yDh/vt/6EauXwozj8 TNCiiioNAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0P0paQ9D9KAK0H3of 90/yq1VWD70P+6f5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8ge2/3m /wDQzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAx9W/wCPpf8AdH8zWHov/IOH++3/AKEa3NW/4+l/3R/OsPRf+QcP99v/AEI1 cvhREfiZoUUUVBYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0P0oAr Q/eh/wB0/wAqtVVg+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33T TvDn/IHtv95v/QzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKA CiiigAooooAKKKKACiiigAooooAyNW/4+V/3R/OsLRf+QeP99v8A0I1vasP9IT/d/rWFov8AyDh/ vt/6EauXwoiPxMv0UUVBYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6H6Ut IehoArQfeh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt900 7w5/yB7b/eb/ANDNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigDK1b/AFyfSsDRf+QcP99v/QjXQaqP3i/SsDRf+QeP99v/AEI1 cvhRC+Jl+iiioLD/AD1ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0P0paQ9 D9KAK0P3of8AdP8AKrVVYPvQ/wC6f5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/AJA9t/vN/wCh mmt9007w5/yB7b/eb/0M00B0dFFFIAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKA CiiigAooooAKKKKACiiigAooooAKKKKAM3Ux84z6Vz2i/wDIPH++/wD6Ea6HUuXx/siue0X/AJBw /wB9v/QjVPZErdl+iiipKCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tI ehoArQ/eh/3T/KrVVYPvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt9007 w5/yB7b/AHm/9DNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA KKKKACiiigAooooAKKKKACiiigDM1H/Xr/u1z+jf8g8f77/+hGui1AZnX6Vzujf8g8f77/8AoRqn siVuy/RRRUlBRR/npRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6GgCt D96H/dP8qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TTvDn/IH tv8Aeb/0M00B0dFFFIAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA KKKKACiiigAooooAKKKKAKF9/r1+grm9G/48B/vv/wChGulvf9ev0Fc1o3/Hh/wN/wD0I1T2QluX 6KKKkYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0NAFaH78P8Aun+V Wqqw/eh/3T/KrVABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/3m/wDQzTW+6ad4c/5A9t/vN/6G aaA6OiiikAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU UUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR QAUUUUAFFFFAFG8/16/QVzWj/wDHh/wN/wD0I1013/r1+grmdH/48P8Ato//AKEab2Ei/RRRSGFF FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFIehpaQ9DQBWh+9D/un+VWqqw/fh/3 T/KrVABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/wB5v/QzTW+6ad4c/wCQPbf7zf8AoZpoDo6K KKQBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUU UAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ AUUUUAU7v/Xr9BXMaP8A8eH/AG0f/wBCNdPdf69foK5jR/8Ajw/7aP8A+hGmxF+iiikMKKKKACii igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD9+H/dP8qtVVh+9D/un+VWqA CiiigAooooAKKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TTvDn/IHtv8Aeb/0M00B0dFFFIAooooA KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAo oooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAKdz /r1+grmNH/48f+2j/wDoRrqLn/Xr9BXL6P8A8eP/AG0f/wBCNMC/RRRSAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD96H/dP8qtVVh+9D/un+VWqACiiigAooooA KKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TS+HjjRbc+jN/6EaaA6Siohcxkctg+lH2iL+/SAloqL 7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36P tEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foA loqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0R f36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF /foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaK i+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+ j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9/wDSgCG5/wBev0Fcvo//AB4/9tH/APQjXSyy CSYEdOlc1o//AB4/9tH/APQjTAv0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACkPQ0tIehoArQ/fh/wB0/wAqtVVh+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumjw/wD8 gSD6v/6EaG+6aXw//wAgWD6v/wChGmgNGiiigQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF FFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU UUAFFFFABRRRQAE4GabeXNlpdilzqExRXZUUDJLM3RVUckn0FOIyCKy9f0+bVYdNltJIxdaddJco kuQkmAQVJHTg8HHFDGWE17RHjif7S6mS6FlsZXV1mOcIykZU8d6kttQstStpbjTZXkiikaMuQ21m U4O0nqM9xxXK6l4MvNYe4nu54opb3U4bqeOGRh5UMaFAqvjJfBznArY0K2vdI0FNNv3gkW1zFbyR cb4h93cMDDY64zSBmyjB0DDoRmpY0i8iSad9saZJJOAAOpJqCEYgQH0FMvIFv9Ku9PdzH56Mm4ds 96bBC2uo6ZdRvIrzRRKobzJ1eJWU9CC2AR9PWrTGwQxhriMGXGwGbG/PTHPNYl7/AG3faYLd4rCN 0KZKy7vMAPzY3IQntkNWXB4UuFsljnNq8yWK26MTnYwmL5Bx0wQMgDp0pAdgRZLJJGZ0Eka73Uy8 qvqRnge9UjeWj3yWsDs7vAZ1dW3IVDBeueuTWLdaHdz2V/aBbUedK0yXQciSTLhwjDacDjGckcDi l0jSJ9Ov1uJSqr5MiMpl8whmcN1CqMcdgKEBvQvvJB6q2DWDo/8Ax4/9tH/9CNai3cdujyPzuf5R /erL0f8A48B7u5/8eNMC/RRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ 9DS0h6GgCtD96H/dP8qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigAqrbXD6Szo8TyWrsXUoMt GT1GO4q1RQA3/hIbL+7df9+Go/4SGy/u3X/fhqXiii4Cf8JDZf3br/vw1H/CQ2X926/78NS4oouA n/CQ2X926/78NR/wkNl/duv+/DUtGKLgJ/wkNl/duv8Avw1H/CQ2X926/wC/DUvFFFwE/wCEhsv7 t1/34aj/AISGy/u3X/fhqWincBP+Ehsv7t1/34aj/hIbL+7df9+GpaMUrgJ/wkNl/duv+/DUf8JD Zf3br/vw1LRRcBP+Ehsv7t1/34aj/hIbL+7df9+GpaKLgJ/wkNl/duv+/DUf8JDZf3br/vw1LRRc BP8AhIbL+7df9+Go/wCEhsv7t1/34aloouAn/CQ2X926/wC/DUf8JDZf3br/AL8NS0UXAT/hIbL+ 7df9+Go/4SGy/u3X/fhqWii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRii4Cf8JDZf3br/vw1H/C Q2X926/78NS0U7gJ/wAJDZf3br/vw1H/AAkNl/duv+/DUtFK4Cf8JDZf3br/AL8NR/wkNl/duv8A vw1Liii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRRcBP+Ehsv7t1/wB+Go/4SGy/u3X/AH4aloou An/CQ2X926/78NR/wkNl/duv+/DUtGKLgJ/wkNl/duv+/DUf8JDZf3br/vw1LRii4Cf8JDZf3br/ AL8NR/wkNl/duv8Avw1Liii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRii4Cf8JDZf3br/vw1H/C Q2X926/78NS8UYouAn/CQ2X926/78NR/wkNl/duv+/DUuKKLgJ/wkNl/duv+/DUf8JDZf3br/vw1 Liii4Cf8JDZf3br/AL8NUb61p8hy0d1n18hqlxRii4EH9rab/wA87r/vy9KNX00HPlXWR6wNU2KO KAE/4SGy/u3X/fhqa+uWEg+eO6OP+mDU+jFFwIP7W03/AJ53X/fl6P7W03/nndf9+XqfFGKAIP7W 03/nndf9+XpG1fTgpKw3ROOB5Lc1YoxQBjyTz3z/AOjo+88bmQqqD2z/AJ71p20C21ukS9FHX1qa igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD96H/dP8 qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigAooooAKKKKACiiigApD0NLSHoaAK0P3of8AdP8AKrVVYfvQ/wC6f5VaoAKK KKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooo oAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAC iiigAooooAKKKKACkPQ0tIehoArQ/eh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigA ooooAKKKKACiiigAooooAKKKKAEByBRRRQAUtFFABRRRQAUUUUAFJRRQAtFFFABRRRQAUUUUAFIe hoooArQfeh/3f6VaoooAKSiigBaKKKACiiigAooooASloooASloooAKKKKACiiigAooooAKKKKAC iiigAooooAKSiigBaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKSiigBaKKKAC iiigAooooAKKKKAEpaKKACiiigAooooASloooASiiigBaKKKACiiigApKKKAFooooASiiigBC2Dj FFFFAH//2Q== --_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_ Content-Type: image/png; name="blue.png" Content-Description: blue.png Content-Disposition: inline; filename="blue.png"; size=2834; creation-date="Thu, 23 Aug 2012 14:54:42 GMT"; modification-date="Thu, 23 Aug 2012 14:54:42 GMT" Content-ID: <dfb35e5d2df7434e8338047441010e53> Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAIwAAAAyCAYAAACOADM7AAAABmJLR0QA/gD+AP7rGNSCAAAACXBI WXMAAA3XAAAN1wFCKJt4AAAACXZwQWcAAACMAAAAMgCR0D3bAAAKaUlEQVR42u2ce5AUxRnAf313 3Al4eCAYFaIgyMNEUF6KlYoVIDBArDxqopWxQgViQlWsPHA0MUlZVoyKRsdSE4lGomjIaHS0UlHL wTIPpEgQFQUUjYIWdfIIScyBHi/Z6/zRM1xP3yzs7t3unOX8qra2H9M9vb3f9Pf19/WukFKSk1Mq dVkPIOejRS4wOWXR6wVGuP5I4foDsh5HjkL0VhtGuP5A4CFgNrAD+Lb0nKeyHtfHnd68wixGCQvA qcA9wvWPy3pQH3caan1D4fonAYeBDwEZjaFflAaok56zHRhsNG0B+gAHSrhHarn0nFp/3NLnxbKP B06I5kECO2UYZD2sLtRcYIBJwK+BoYBACU89cAjoAIRw/TuAJcClQGy//FJ6zvvH6ly4/qXAz4vU HQA2A4H0nIcz+OxH41eAHaU3AhdkPaA0MrFhhOuPB2YA5wBnA6ehni5dgKcBu4C5wLZS7Rfh+g8A 80u49HHgEuk5h2s+AeaYLbsO2AKMiIqWyzBYkPW40shihUF6zkbUUwSAcP0G4FHgS9pl10rPmQMs LbXfSBVNLPHyrwDfBO7JYg4MRqEempjnsh5QMXqL0Xsl8EUt3w5cXUE/w4AztfzzwGSUGrwoyuvM yfqDR5yLUssxL2U9oGJkssLoCNdfjLJXdBZIz9lQQXcTgSYt/4z0nHjy1wvX3wW8oNX3O8q4TgKm AGegjNB/As9JzzmYer1lTwKGoOyyV2UYtArLngLMQ9lh64EVRQxZ3V5pje4V9zsVGBRl22QYrDXu e0HUvwD+K8NgXbe/lKOQqcAI178MuM0ovk16zqMVdjnNyL9g5E2DrTVlTP1RRvM3gIFG9RvC9RdK z/lHoo2yQQJgeFR0hbDsT6FUns544Icp456qpV+RYaAL5RJgepR+FWXzxfcdA6zRrr0SqKrAZKaS hOt/DbjXKH5Geo7bjW71iT8AvGLUzzXyfzfGNBBlPyymq7AAjAWeFK5/slE+AvhklC4At6KEZb9x 3cJo+9x5T8s+ERinFa012uzU0vuMuu9r6W3AXd2Yu5LIRGCE618E/D6l6rpu9Hk8MEEr2iQ9p1Wr n4wShJgPgCeMbh6g02jeB9wILASe1q4ZBHzBaDeRThukHghRdskoQF+NmlH+JJ0JqB1ijCkw72np jiOfx7JPQrkdYm6QYXBMH1V3qYlKEq7fhNLvw1CTeztK55rcJlz/s8XshGPwaeBELd8sXP961Bd4 Bsqo1u2bm6Tn7NbGeCHKMI6ZLz3nsajuT6gtfjxfpxr31lXhThkG8470a9mrtPp2uq4652np94FN Rr0uMM1a+jI6fVTvAMsrmLOy6VGBEa5fB3wOpctHaK9TgVOAxmN0MRXlwPpWBbefYuTHAj8tcu39 0nNuMMq+qqXfjoUl4mSSq/HbRlv9S3/ZqBumpXcB/zPqz9fSm2UY/Nuo1wWmCUBYdiPwHa3ck2Hw YQVzVjbVWGFmkW7YmewDfga8CNwHnB6VXyZcf7X0nAfLvG8pntE3gSXSc5an1Olf+hDh+i+jVieJ UiOxwBSiMQMgLLsFOEtr+7xWB8rQjdkgw0BXK40o1RWTZrDu0dKx0X4xylMOynZZVuZcVUyPCoz0 nA7gR8L1N6FWmQIqZtRGpwoSwF7gRek5WwCE658P3A9Y0TV3C9ffUOrWOlrZdIfdXuBhlCqaqZU/ myYs0RZaNzybUV7oNFqBt7T8BJJ2iW6zDAPGFKkDGE1yBTLtF0gKTCF6/4FWtsTYVVWVqtgw0nNW lHn9LmCOcP2bgKuAvsAtqNWqFGLVF7NGes4i4fpjgNfpFNbzi7QfD/TX8vtQMa40VkvPKWh5fWfW DuhCfg5Ju8nc5k/RxpZYuTR0gWkTlj0D5YgEeJca2S4xvcXTC4D0nKvpdNWXc2hqEiqSHROrhR0k bYAzhesPTmmvG61tKAE6PXoNRRnTg6OX6VvRhfB1GQa7tbyu5v6D8qNQpH4bsDVlbLrADACu0fK/ qOXqAr1MYCLip7AcI+48I78WIIpuv6mVN5NUPWntN0nP2So9p016ThtwEKU6RpIMOyAsuw9JVWiu INO19AYZBma0fbKWXi/DoEBX9tBpu4wDLozS2+jqx6o6vVFgYt+JKKON/pTvJ6kWzKc6LTg5XEtv MeruAF5DqbZVgH6IayTJoOHf4oSw7LNICuKTeqfCsj9BUnhN+yamPXqZc3JrLfwuJpnHklKIBaa+ lIuF67eQ3KW8HtlEMabhPCmlG/3JnhX5ZHaifDeLtLqlxpmcySQfuvnCstdH6WXaZ9iPMsJ1xpOM ZaXZL6DsqfcB3UO8A7WzrDm9T2DqG7dTOHSIEgUGIc5GyhatZJ1Rv4HkmZ/xKb08o5UPRa0UkuQT vY6uQVJTFc5D7fQ6SNpUN8ow2GVcq7sB2ugq2DGHUYfLdG6SYbCPDMhcYIRlJwWjcGg/Z1/yATBE zJxXT0Pf4o0P7pWcO39W4nuVHS+JGfPq6dMXOjpgzNyt9En0MUF877fDee3x1iPlo2beTOPxnwGh qzahuhUAjwCLpOeYKkDfIT2BUl1XkxT2+2QYXJ8yen0H+JYMgz2kY9o126mh38UkITBRYGwp5e1Q usNjwL/Ql3VRX2D35mUI0UB90wyOZmc19i+wa+NB+vTrnMA9re00RO3q6iRbVtYxeOzt1NXHS3od e96dRkPT6CN9v/HUIRr738Dg0bMRDSdQVzeAjsJh+ra8SfMpf5S3XNzFoSYsewhJVbhKhoEnLDtE HV4vRGXPprQFFTdrRklk2u4opoVkyMOTYbCfjEgc0RSWPQhlQ/SruMfymCrD4IXud1N7In+ILgzT ZRj8tYfvcSLwOzoPer0DjKv1VlrHVEltqBhMafZD99mR1QfvAXT1tYfiNkhZCMvuD1yLCtbORsXg Yi7PUljAEJgoztFaYV8fN8yg4XsV95TkLJS32+QaGQZPl9tZT5O50ftRJLL1Pq8V9cjqEjHdyG8D rpdhkJmhq5MLTGX0QR2diLdnYQ/2vRq1wsRe6nUyDNq712XP0Wt/W53TO+mNoYGcXkwuMDll0eM2 TPRbnGnAvaaDSVj2bOA0GQY1j7Lm9AzVWGG+jIrwphlH3wXuzvpD51RONXZJ7aizLFcIyx4O3CXD IN527kUdJAJAWPbFqBXnVmHZV6FO3K+I6oahzgYPAX7T017UnMqoxgpTQAniONRJ/AeFZRc72+IA P47SPwEWAAjLbgL+jPJ1NAF/EZZd6o/sc6pINQSmARAyDL6OOm45mmSoX+cDVDiC6D0+azI0arcS FSkG9fcgORlTbcfdXtR5jqOdnpPGO3QK8nzU33KsoutvgXIyoBorjP7FN6OEsph3sE6rq9fS8RmQ RTIMTgP+QPJsbk5GVENgjgMQlv0QcDnwBp0nxgaQ/O+6dmCUsOxHUGdj459kbI/a3Sksew3qjE5L 1pOVUx2VtBJljxxAhf3v0v4TZRnKmI25ObruLdTZkvcAZBgcEpY9E3BRu6TrZBisznqycvJYUk6Z 5KGBnLLIBSanLHKBySmLXGByyiIXmJyy+D/P9uGVPOu6DAAAACh6VFh0U29mdHdhcmUAAHja801M LsrPTU3JTFRwyyxKLc8vyi5WsAAAYBUIJ4KDNosAAAAASUVORK5CYII= --_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_--
On 08/23/2012 05:54 PM, Scotto Alberto wrote: hi, can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don’t know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
------------------------------------------------------------------------
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Here you are Thread-47346::DEBUG::2007-06-30 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216] Thread-47346::INFO::2007-06-30 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getDeviceList, args: () Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece111%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) '/sbin/dmsetup status' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScsiSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-0' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScsiSerial) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::WARNING::2007-06-30 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Traceback (most recent call last): File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/device/scsi_disk/' Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList KeyError: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False) Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'hbtl'" - code 100 Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False Thread-47346::DEBUG::2007-06-30 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed Thread-47346::DEBUG::2007-06-30 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-47346::DEBUG::2007-06-30 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl' Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run File "/usr/share/vdsm/storage/task.py", line 1178, in prepare KeyError: 'hbtl' Alberto Scotto Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it -----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 17:00 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs On 08/23/2012 05:54 PM, Scotto Alberto wrote: hi, can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
---------------------------------------------------------------------- --
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
________________________________ -- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
Here you are
thanks, can you run the following? - ls -l /sys/block/ - dmsetup table - lsblk (if exists) it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216] Thread-47346::INFO::2007-06-30 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getDeviceList, args: () Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece111%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) '/sbin/dmsetup status' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScsiSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-0' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScsiSerial) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::WARNING::2007-06-30 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Traceback (most recent call last): File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/device/scsi_disk/' Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList KeyError: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False) Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'hbtl'" - code 100 Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False Thread-47346::DEBUG::2007-06-30 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed Thread-47346::DEBUG::2007-06-30 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-47346::DEBUG::2007-06-30 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl' Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run File "/usr/share/vdsm/storage/task.py", line 1178, in prepare KeyError: 'hbtl'
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 17:00 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
---------------------------------------------------------------------- --
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
I was going to reply.. :) It looks like it halts after an error due to cciss!c0d1, which btw is displayed by multipath -ll. That's just a local disk, isn't it? So it shouldn't even be listed. I may have attached it by mistake, playing with /sys/class/fc_* tools. So, if I remove that path, everything should go ok. Do you think so too?
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case More than that the fact is that the path scsi_disk/ doesn't exist in /sys/block/cciss!c0d1/device And this must be due to the fact that c0d1 is NOT a damn scsi disk
Anyway, here is your output [root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:37 cciss!c0d0 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/cciss!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/cciss!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 -> ../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 -> ../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30 01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 -> ../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 -> ../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 -> ../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 -> ../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 -> ../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 -> ../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx. 1 root root 0 2007-06-30 00:36 sda -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 -> ../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0 [root@pittor06vhxd020 ~]# dmsetup table 3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1 queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1 HostVG-Logging: 0 4194304 linear 104:4 24741888 HostVG-Swap: 0 24723456 linear 104:4 2048 3600508b1001035333920202020200005: 0 429925920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1 HostVG-Data: 0 40624128 linear 104:4 28936192 HostVG-Config: 0 16384 linear 104:4 24725504 live-rw: 0 2097152 snapshot 7:1 7:2 P 8 [root@pittor06vhxd020 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 99.3M 1 loop loop1 7:1 0 1G 1 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / loop2 7:2 0 512M 0 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / cciss!c0d0 104:0 0 33.9G 0 disk ââcciss!c0d0p1 104:1 0 243M 0 part ââcciss!c0d0p2 104:2 0 244M 0 part ââcciss!c0d0p3 104:3 0 244M 0 part ââcciss!c0d0p4 104:4 0 33.2G 0 part ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP] ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data cciss!c0d1 104:16 0 205G 0 disk ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath sr0 11:0 1 1024M 0 rom sdb 8:16 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sda 8:0 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdc 8:32 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdd 8:48 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sde 8:64 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdf 8:80 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdg 8:96 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdh 8:112 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath Alberto Scotto Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it -----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 18:01 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs On 08/23/2012 06:20 PM, Scotto Alberto wrote:
Here you are
thanks, can you run the following? - ls -l /sys/block/ - dmsetup table - lsblk (if exists) it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216] Thread-47346::INFO::2007-06-30 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getDeviceList, args: () Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece1 11%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_co unt,dev_size' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_ge MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_ge MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScs MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export MainProcess|--replace-whitespace --device=/dev/dm-0' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScs MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::WARNING::2007-06-30 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Traceback (most recent call last): File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/device/scsi_disk/' Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList KeyError: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False) Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'hbtl'" - code 100 Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False Thread-47346::DEBUG::2007-06-30 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed Thread-47346::DEBUG::2007-06-30 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll ) Owner.releaseAll requests {} resources {} Thread-47346::DEBUG::2007-06-30 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl' Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run File "/usr/share/vdsm/storage/task.py", line 1178, in prepare KeyError: 'hbtl'
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 17:00 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
--------------------------------------------------------------------- - --
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
________________________________ -- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
On 08/23/2012 07:13 PM, Scotto Alberto wrote:
I was going to reply.. :) It looks like it halts after an error due to cciss!c0d1, which btw is displayed by multipath -ll. That's just a local disk, isn't it? So it shouldn't even be listed. I may have attached it by mistake, playing with /sys/class/fc_* tools. So, if I remove that path, everything should go ok. Do you think so too?
Yes, please remove and check again (make sure to clean device-mapper table with dmsetup remove $dm). anyhow, vdsm should be more robust, so please file a bug for it(https://bugzilla.redhat.com/enter_bug.cgi?product=oVirt). Haim
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case More than that the fact is that the path scsi_disk/ doesn't exist in /sys/block/cciss!c0d1/device And this must be due to the fact that c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:37 cciss!c0d0 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/cciss!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/cciss!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 -> ../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 -> ../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30 01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 -> ../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 -> ../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 -> ../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 -> ../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 -> ../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 -> ../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx. 1 root root 0 2007-06-30 00:36 sda -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 -> ../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table 3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1 queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1 HostVG-Logging: 0 4194304 linear 104:4 24741888 HostVG-Swap: 0 24723456 linear 104:4 2048 3600508b1001035333920202020200005: 0 429925920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1 HostVG-Data: 0 40624128 linear 104:4 28936192 HostVG-Config: 0 16384 linear 104:4 24725504 live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 99.3M 1 loop loop1 7:1 0 1G 1 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / loop2 7:2 0 512M 0 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / cciss!c0d0 104:0 0 33.9G 0 disk ââcciss!c0d0p1 104:1 0 243M 0 part ââcciss!c0d0p2 104:2 0 244M 0 part ââcciss!c0d0p3 104:3 0 244M 0 part ââcciss!c0d0p4 104:4 0 33.2G 0 part ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP] ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data cciss!c0d1 104:16 0 205G 0 disk ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath sr0 11:0 1 1024M 0 rom sdb 8:16 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sda 8:0 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdc 8:32 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdd 8:48 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sde 8:64 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdf 8:80 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdg 8:96 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdh 8:112 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 18:01 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
Here you are
On 08/23/2012 06:20 PM, Scotto Alberto wrote: thanks, can you run the following?
- ls -l /sys/block/ - dmsetup table - lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216] Thread-47346::INFO::2007-06-30 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getDeviceList, args: () Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece1 11%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_co unt,dev_size' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_ge MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_ge MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScs MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export MainProcess|--replace-whitespace --device=/dev/dm-0' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScs MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::WARNING::2007-06-30 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Traceback (most recent call last): File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/device/scsi_disk/' Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList KeyError: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False) Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'hbtl'" - code 100 Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False Thread-47346::DEBUG::2007-06-30 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed Thread-47346::DEBUG::2007-06-30 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll ) Owner.releaseAll requests {} resources {} Thread-47346::DEBUG::2007-06-30 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl' Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run File "/usr/share/vdsm/storage/task.py", line 1178, in prepare KeyError: 'hbtl'
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 17:00 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
--------------------------------------------------------------------- - --
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
It worked! For the sake of knowledge, these are the steps I took (on RHEL6): 1) edited /etc/multipath.conf, adding --------------------- blacklist { wwid 3600508b1001035333920202020200005 } --------------------- 2) service multipathd reload 3) multipath -f 3600508b1001035333920202020200005 I also filed the bug as you suggested https://bugzilla.redhat.com/show_bug.cgi?id=851478 Thank you very much Haim! Best regards PS: actually adding a new domain still doesn't work.. What a mess... At least now it lists my LUNs correctly :) Alberto Scotto Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it -----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 18:22 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs On 08/23/2012 07:13 PM, Scotto Alberto wrote:
I was going to reply.. :) It looks like it halts after an error due to cciss!c0d1, which btw is displayed by multipath -ll. That's just a local disk, isn't it? So it shouldn't even be listed. I may have attached it by mistake, playing with /sys/class/fc_* tools. So, if I remove that path, everything should go ok. Do you think so too?
Yes, please remove and check again (make sure to clean device-mapper table with dmsetup remove $dm). anyhow, vdsm should be more robust, so please file a bug for it(https://bugzilla.redhat.com/enter_bug.cgi?product=oVirt). Haim
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case More than that the fact is that the path scsi_disk/ doesn't exist in /sys/block/cciss!c0d1/device And this must be due to the fact that c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:37 cciss!c0d0 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/ccis s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 -> ../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/ccis s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 -> ../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 -> ../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30 01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 -> ../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 -> ../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 -> ../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 -> ../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 -> ../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 -> ../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx. 1 root root 0 2007-06-30 00:36 sda -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh -> ../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0 0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 -> ../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table 3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1 queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1 HostVG-Logging: 0 4194304 linear 104:4 24741888 HostVG-Swap: 0 24723456 linear 104:4 2048 3600508b1001035333920202020200005: 0 429925920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1 HostVG-Data: 0 40624128 linear 104:4 28936192 HostVG-Config: 0 16384 linear 104:4 24725504 live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 99.3M 1 loop loop1 7:1 0 1G 1 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / loop2 7:2 0 512M 0 loop ââlive-rw (dm-1) 253:1 0 1G 0 dm / cciss!c0d0 104:0 0 33.9G 0 disk ââcciss!c0d0p1 104:1 0 243M 0 part ââcciss!c0d0p2 104:2 0 244M 0 part ââcciss!c0d0p3 104:3 0 244M 0 part ââcciss!c0d0p4 104:4 0 33.2G 0 part ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP] ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data cciss!c0d1 104:16 0 205G 0 disk ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath sr0 11:0 1 1024M 0 rom sdb 8:16 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sda 8:0 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdc 8:32 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdd 8:48 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sde 8:64 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdf 8:80 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdg 8:96 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath sdh 8:112 0 650G 0 disk ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 18:01 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
Here you are
On 08/23/2012 06:20 PM, Scotto Alberto wrote: thanks, can you run the following?
- ls -l /sys/block/ - dmsetup table - lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216] Thread-47346::INFO::2007-06-30 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getDeviceList, args: () Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to sampling method Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last result Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece 1 11%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_c o unt,dev_size' (cwd None) Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n /dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0 Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_g MainProcess|e MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_g MainProcess|e MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getSc MainProcess|s MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted MainProcess|--export --replace-whitespace --device=/dev/dm-0' (cwd MainProcess|None) MainProcess|Thread-47346::DEBUG::2007-06-30 MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getSc MainProcess|s MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0 Thread-47346::WARNING::2007-06-30 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Traceback (most recent call last): File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/device/scsi_disk/' Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList KeyError: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False) Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'hbtl'" - code 100 Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl' Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False Thread-47346::DEBUG::2007-06-30 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll ) Owner.cancelAll requests {} Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed Thread-47346::DEBUG::2007-06-30 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAl l ) Owner.releaseAll requests {} resources {} Thread-47346::DEBUG::2007-06-30 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll ) Owner.cancelAll requests {} Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl' Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run File "/usr/share/vdsm/storage/task.py", line 1178, in prepare KeyError: 'hbtl'
Alberto Scotto
Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-----Original Message----- From: Haim [mailto:hateya@redhat.com] Sent: giovedì 23 agosto 2012 17:00 To: Scotto Alberto Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null version, using default general for version 2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId = 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6 2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error message VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc lunList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 600 mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: () 2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6 2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceListQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS, error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 2:0:0:0 sda 8:0 active ready running | |- 2:0:1:0 sdd 8:48 active ready running | |- 3:0:0:0 sde 8:64 active ready running | `- 3:0:1:0 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 2:0:2:0 sdb 8:16 active ready running |- 2:0:3:0 sdc 8:32 active ready running |- 3:0:2:0 sdg 8:96 active ready running `- 3:0:3:0 sdh 8:112 active ready running 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 0:0:1:0 cciss!c0d1 104:16 active ready running ------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing> <maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it
-------------------------------------------------------------------- - - --
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
________________________________
-- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
________________________________ -- The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
--_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: multipart/related; boundary="_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_"; type="multipart/alternative" --_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: multipart/alternative; boundary="_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_" --_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable After two or three reinstallations of RHEV-H, surprisingly, now the behavio= r is as expected: i.e., it skips the cciss disk without breaking. Thread-11037::WARNING::2012-08-28 15:46:25,949::hsm::735::Storage.HSM::(_ge= tDeviceList) Ignoring partitioned device {'product': 'LOGICAL VOLUME', 'dm'= : 'dm-2', 'devtypes': ['FCP'], 'fwrev': '2.08', 'logicalblocksize': '512', = 'connections': [], 'devtype': 'FCP', 'physicalblocksize': '512', 'vendor': = 'HP', 'serial': 'SHP_LOGICAL_VOLUME_PH79MW7539', 'guid': '3600508b100103533= 3920202020200005', 'paths': [{'devnum': DeviceNumber(Major=3D104, Minor=3D1= 6), 'physdev': 'cciss!c0d1', 'type': 'FCP', 'state': 'active'}], 'capacity'= : '220122071040'} Attached the full vdsm log. I didn't upgrade vdsm, I just did a reinstall (two reinstalls, actually). N= o idea of what happened...! Btw, the procedure I posted to hide the cciss disk from multipath didn't wo= rk well: when rebooted, the hypervisor doesn't save the changes, due to the= live image on which it mounts the root file system. However now I'm experiencing high latency issues. The message from RHEV-M c= onsole is: "storage domain XYZ experienced a high latency of 14 seconds [..= .]". Any ideas? Should I detach cciss disk once again? I'm gonna try Alberto Scotto [Blue] Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it www.reply.it -----Original Message----- From: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of= Scotto Alberto Sent: venerd=EC 24 agosto 2012 11:44 To: Haim Cc: users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs It worked! For the sake of knowledge, these are the steps I took (on RHEL6): 1) edited /etc/multipath.conf, adding --------------------- blacklist { wwid 3600508b1001035333920202020200005 } --------------------- 2) service multipathd reload 3) multipath -f 3600508b1001035333920202020200005 I also filed the bug as you suggested https://bugzilla.redhat.com/show_bug.cgi?id=3D851478 Thank you very much Haim! Best regards PS: actually adding a new domain still doesn't work.. What a mess... At lea= st now it lists my LUNs correctly :) Alberto Scotto Blue Reply Via Cardinal Massaia, 83 10147 - Torino - ITALY phone: +39 011 29100 al.scotto@reply.it<mailto:al.scotto@reply.it> www.reply.it<http://www.reply.it> -----Original Message----- From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com]> Sent: gioved=EC 23 agosto 2012 18:22 To: Scotto Alberto Cc: users@ovirt.org<mailto:users@ovirt.org> Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs On 08/23/2012 07:13 PM, Scotto Alberto wrote:
I was going to reply.. :)
It looks like it halts after an error due to cciss!c0d1, which btw is dis= played by multipath -ll.
That's just a local disk, isn't it? So it shouldn't even be listed. I may= have attached it by mistake, playing with /sys/class/fc_* tools.
So, if I remove that path, everything should go ok. Do you think so too?
Yes, please remove and check again (make sure to clean device-mapper table = with dmsetup remove $dm). anyhow, vdsm should be more robust, so please file a bug for it(https://bug= zilla.redhat.com/enter_bug.cgi?product=3DoVirt). Haim
it appears that vdsm fails to handle a device with '!' in it
(cciss!c0d1), but let's make sure its indeed the case
More than that the fact is that the path scsi_disk/ doesn't exist in
/sys/block/cciss!c0d1/device And this must be due to the fact that
c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1 root
root 0 2007-06-30 00:37 cciss!c0d0 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/ccis
s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/ccis
s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 ->
../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30
00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root root 0
2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx. 1
root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 ->
../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30
01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root root 0
2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx. 1
root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 ->
../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 ->
../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 ->
../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 ->
../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 ->
../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root root 0
2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx. 1
root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 ->
../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root root 0
2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx. 1
root root 0 2007-06-30 00:36 sda ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 ->
../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table
3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1
queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80
1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1
HostVG-Logging: 0 4194304 linear 104:4 24741888
HostVG-Swap: 0 24723456 linear 104:4 2048
3600508b1001035333920202020200005: 0 429925920 multipath 1
queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1
HostVG-Data: 0 40624128 linear 104:4 28936192
HostVG-Config: 0 16384 linear 104:4 24725504
live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOU= NTPOINT
loop0 7:0 0 99.3M 1 loop
loop1 7:1 0 1G 1 loop
=E2=E2live-rw (dm-1) 253:1 0 1G 0 dm = /
loop2 7:2 0 512M 0 loop
=E2=E2live-rw (dm-1) 253:1 0 1G 0 dm = /
cciss!c0d0 104:0 0 33.9G 0 disk
=E2=E2cciss!c0d0p1 104:1 0 243M 0 part
=E2=E2cciss!c0d0p2 104:2 0 244M 0 part
=E2=E2cciss!c0d0p3 104:3 0 244M 0 part
=E2=E2cciss!c0d0p4 104:4 0 33.2G 0 part
=E2=E2HostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm = [SWAP]
=E2=E2HostVG-Config (dm-4) 253:4 0 8M 0 lvm = /config
=E2=E2HostVG-Logging (dm-5) 253:5 0 2G 0 lvm = /var/log
=E2=E2HostVG-Data (dm-6) 253:6 0 19.4G 0 lvm = /data
cciss!c0d1 104:16 0 205G 0 disk
=E2=E23600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sda 8:0 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdc 8:32 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdd 8:48 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sde 8:64 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdf 8:80 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdg 8:96 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdh 8:112 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto@reply.it<mailto:al.scotto@reply.it>
www.reply.it<http://www.reply.it>
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com]>
Sent: gioved=EC 23 agosto 2012 18:01
To: Scotto Alberto
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve
LUNs
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1)= , but let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30
00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper)
[10.16.250.216]
Thread-47346::INFO::2007-06-30
00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run
and protect: getDeviceList, args: ()
Thread-47346::DEBUG::2007-06-30
00:37:10,269::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state
preparing
Thread-47346::DEBUG::2007-06-30
00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter
sampling method (storage.sdc.refreshStorage)
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
sampling method
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter
sampling method (storage.iscsi.rescan)
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
sampling method
Thread-47346::DEBUG::2007-06-30
00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err>
=3D 'iscsiadm: No session found.\n'; <rc> =3D 21
Thread-47346::DEBUG::2007-06-30
00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last
result
Thread-47346::DEBUG::2007-06-30
00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/multipath' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS:
<err> =3D ''; <rc> =3D 0
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last
result
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm
reload operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n
/sbin/lvm pvs --config " devices { preferred_names =3D
[\\"^/dev/mapper/\\"] ignore_suspended_devices=3D1 write_cache_state=3D0
disable_after_error_count=3D3 filter =3D [
\\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece<fi= le:///\\%22a%253600508b1001035333920202020200005|3600601601cde1d0066b2fb054= dece>
1 11%\\", \\"r%.*%\\<file:///\\%22r%25.*%25\>" ] } global { locking_ty= pe=3D1
prioritise_write_locks=3D1 wait_for_locks=3D1 } backup { retain_min = =3D
50 retain_days =3D 0 } " --noheadings --units b --nosuffix --separator
| -o
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_c
o
unt,dev_size' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> =3D '
/dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n
/dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output
error\n /dev/sdh: read failed after 0 of 4096 at 697932177408:
Input/output error\n WARNING: Error counts reached a limit of 3.
Device /dev/sdh was disabled\n'; <rc> =3D 0
Thread-47346::DEBUG::2007-06-30
00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm
reload operation' released the operation mutex
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_g
MainProcess|e
MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None)
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_g
MainProcess|e
MainProcess|tPathsStatus) SUCCESS: <err> =3D ''; <rc> =3D 0
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getSc
MainProcess|s
MainProcess|iSerial) '/sbin/scsi_id --page=3D0x80 --whitelisted
MainProcess|--export --replace-whitespace --device=3D/dev/dm-0' (cwd
MainProcess|None)
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getSc
MainProcess|s
MainProcess|iSerial) SUCCESS: <err> =3D ''; <rc> =3D 0
Thread-47346::WARNING::2007-06-30
00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem = getting hbtl from device `cciss!c0d1` Traceback (most recent call last):
File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIt=
er
File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
OSError: [Errno 2] No such file or directory: '/sys/block/cciss!c0d1/dev= ice/scsi_disk/'
Thread-47346::ERROR::2007-06-30
00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error = Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDevice=
List
File "/usr/share/vdsm/storage/hsm.py", line 759, in
_getDeviceList
KeyError: 'hbtl'
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._run:
0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing
(force False)
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
Thread-47346::INFO::2007-06-30
00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task
is aborted: "'hbtl'" - code 100
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Tas= k::(_debug) Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'h= btl'
Thread-47346::DEBUG::2007-06-30
00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
Thread-47346::DEBUG::2007-06-30
00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
Thread-47346::DEBUG::2007-06-30
00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll
)
Owner.cancelAll requests {}
Thread-47346::DEBUG::2007-06-30
00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing ->
state aborting
Thread-47346::DEBUG::2007-06-30
00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
Thread-47346::DEBUG::2007-06-30
00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting ->
state failed
Thread-47346::DEBUG::2007-06-30
00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAl
l
) Owner.releaseAll requests {} resources {}
Thread-47346::DEBUG::2007-06-30
00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll
)
Owner.cancelAll requests {}
Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::106::Storage.D= ispatcher.Protect::(run) 'hbtl'
Thread-47346::ERROR::2007-06-30 00:37:11,034::dispatcher::107::Storage.D= ispatcher.Protect::(run) Traceback (most recent call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
KeyError: 'hbtl'
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto@reply.it<mailto:al.scotto@reply.it>
www.reply.it<http://www.reply.it>
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com]>
Sent: gioved=EC 23 agosto 2012 17:00
To: Scotto Alberto
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [Users] [rhev 3] add new domain fails: Could not
retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList comma= nd?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any
LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN
[org.ovirt.engine.core.bll.GetConfigurationValueQuery]
(http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
version, using default general for version
2007-06-29 21:50:07,911 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =3D
7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), log id:
60bdafe6
2007-06-29 21:50:08,726 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
message VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error =3D Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Command
org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
return value
Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
lunList Null
mStatus Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
mCode 600
mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Vds: pittor06vhxd020
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.VDSCommandBase]
(http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Except= ion:
VDSErrorException: VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error =3D Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
60bdafe6
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
(http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
message is VdcBLLException:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
error =3D Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll
3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=3D650G
features=3D'1 queue_if_no_path' hwhandler=3D'1 emc' wp=3Drw
|-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive
| |- 2:0:0:0 sda 8:0 active ready running
| |- 2:0:1:0 sdd 8:48 active ready running
| |- 3:0:0:0 sde 8:64 active ready running
| `- 3:0:1:0 sdf 8:80 active ready running
`-+- policy=3D'round-robin 0' prio=3D0 status=3Denabled
|- 2:0:2:0 sdb 8:16 active ready running
|- 2:0:3:0 sdc 8:32 active ready running
|- 3:0:2:0 sdg 8:96 active ready running
`- 3:0:3:0 sdh 8:112 active ready running
3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=3D205G
features=3D'1 queue_if_no_path' hwhandler=3D'0' wp=3Drw
`-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive
`- 0:0:1:0 cciss!c0d1 104:16 active ready running
------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what
paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing>
<maxHostID> <version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto@reply.it<mailto:al.scotto@reply.it>
www.reply.it<http://www.reply.it>
--------------------------------------------------------------------
-
-
--
--
The information transmitted is intended for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of,
or taking of any action in reliance upon, this information by
persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete
the material from any computer.
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
________________________________
--
The information transmitted is intended for the person or entity to whic= h it is addressed and may contain confidential and/or privileged material. = Any review, retransmission, dissemination or other use of, or taking of any= action in reliance upon, this information by persons or entities other tha= n the intended recipient is prohibited. If you received this in error, plea= se contact the sender and delete the material from any computer.
________________________________
--
The information transmitted is intended for the person or entity to which= it is addressed and may contain confidential and/or privileged material. A= ny review, retransmission, dissemination or other use of, or taking of any = action in reliance upon, this information by persons or entities other than= the intended recipient is prohibited. If you received this in error, pleas= e contact the sender and delete the material from any computer.
________________________________ -- The information transmitted is intended for the person or entity to which i= t is addressed and may contain confidential and/or privileged material. Any= review, retransmission, dissemination or other use of, or taking of any ac= tion in reliance upon, this information by persons or entities other than t= he intended recipient is prohibited. If you received this in error, please = contact the sender and delete the material from any computer. _______________________________________________ Users mailing list Users@ovirt.org<mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users ________________________________ -- The information transmitted is intended for the person or entity to which i= t is addressed and may contain confidential and/or privileged material. Any= review, retransmission, dissemination or other use of, or taking of any ac= tion in reliance upon, this information by persons or entities other than t= he intended recipient is prohibited. If you received this in error, please = contact the sender and delete the material from any computer. --_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <style> <!-- @font-face {font-family:Calibri} @font-face {font-family:Tahoma} p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"} a:link, span.MsoHyperlink {color:blue; text-decoration:underline} a:visited, span.MsoHyperlinkFollowed {color:purple; text-decoration:underline} p.MsoPlainText, li.MsoPlainText, div.MsoPlainText {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {margin:0cm; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif"} span.PlainTextChar {font-family:"Calibri","sans-serif"} span.BalloonTextChar {font-family:"Tahoma","sans-serif"} .MsoChpDefault {font-family:"Calibri","sans-serif"} @page WordSection1 {margin:70.85pt 2.0cm 2.0cm 2.0cm} div.WordSection1 {} --> </style> </head> <body lang=3D"IT" link=3D"blue" vlink=3D"purple"> <div class=3D"WordSection1"> <p class=3D"MsoPlainText"><span lang=3D"EN-US">After two or three reinstall= ations of RHEV-H, surprisingly, now the behavior is as expected: i.e., it s= kips the cciss disk without breaking.</span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US" style=3D"background:yellow">= Thread-11037::WARNING::2012-08-28 15:46:25,949::hsm::735::Storage.HSM::(_ge= tDeviceList) Ignoring partitioned device {'product': 'LOGICAL VOLUME', 'dm'= : 'dm-2', 'devtypes': ['FCP'], 'fwrev': '2.08', 'logicalblocksize': '512', 'connections': [], 'devtype': 'FCP', 'p= hysicalblocksize': '512', 'vendor': 'HP', 'serial': 'SHP_LOGICAL_VOLUME_PH7= 9MW7539', 'guid': '3600508b1001035333920202020200005', 'paths': [{'devnum':= DeviceNumber(Major=3D104, Minor=3D16), 'physdev': 'cciss!c0d1', 'type': 'FCP', 'state': 'active'}], 'capacity': '= 220122071040'}</span><span lang=3D"EN-US"></span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US">Attached the full vdsm log.<= /span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US">I didn’t upgrade vdsm,= I just did a reinstall (<i>two</i> reinstalls, actually). No idea of what = happened…!</span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US">Btw, the procedure I posted = to hide the cciss disk from multipath didn’t work well: when rebooted= , the hypervisor doesn’t save the changes, due to the live image on w= hich it mounts the root file system. </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US">However now I’m experi= encing high latency issues. The message from RHEV-M console is: “stor= age domain XYZ experienced a high latency of 14 seconds […]”.</= span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US">Any ideas? Should I detach c= ciss disk once again? I’m gonna try</span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US" style=3D""><br> <br> </p> <div align=3D"left"> <p style=3D"font-family:Calibri,Sans-Serif; font-size:10pt"><span style=3D"= color:#000000; font-weight:bold">Alberto Scotto</span> <span style=3D"color:#808080"></span><br> <br> <span style=3D"color:#000000"><img border=3D"0" alt=3D"Blue" src=3D"cid:e6e= 5a80a848243228ec57fb928078181" style=3D"margin:0px"> </span><br> <span style=3D"color:#808080">Via Cardinal Massaia, 83<br> 10147 - Torino - ITALY <br> phone: +39 011 29100 <br> <a href=3D"al.scotto@reply.it" target=3D"" style=3D"color:blue; text-decora= tion:underline">al.scotto@reply.it</a> <br> <a title=3D"" href=3D"www.reply.it" target=3D"" style=3D"color:blue; text-d= ecoration:underline">www.reply.it</a> </span><br> </p> </div> -----Original Message-----<br> From: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of= Scotto Alberto<br> Sent: venerd=EC 24 agosto 2012 11:44<br> To: Haim<br> Cc: users@ovirt.org<br> Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs= </span><span lang=3D"EN-US"></span> <p></p> <p class=3D"MsoPlainText"><span lang=3D"EN-US"> </span></p> <p class=3D"MsoPlainText">It worked!</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">For the sake of knowledge, these are the steps I = took (on RHEL6):</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">1) edited /etc/multipath.conf, adding</p> <p class=3D"MsoPlainText">---------------------</p> <p class=3D"MsoPlainText">blacklist {</p> <p class=3D"MsoPlainText"> wwid 3600508b10010353339202020= 20200005 }</p> <p class=3D"MsoPlainText">---------------------</p> <p class=3D"MsoPlainText">2) service multipathd reload</p> <p class=3D"MsoPlainText">3) multipath -f 3600508b1001035333920202020200005= </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">I also filed the bug as you suggested</p> <p class=3D"MsoPlainText"><a href=3D"https://bugzilla.redhat.com/show_bug.c= gi?id=3D851478"><span style=3D"color:windowtext; text-decoration:none">http= s://bugzilla.redhat.com/show_bug.cgi?id=3D851478</span></a></p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Thank you very much Haim!</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Best regards</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">PS: actually adding a new domain still doesn't wo= rk.. What a mess... At least now it lists my LUNs correctly :)</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Alberto Scotto</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Blue Reply</p> <p class=3D"MsoPlainText">Via Cardinal Massaia, 83</p> <p class=3D"MsoPlainText">10147 - Torino - ITALY</p> <p class=3D"MsoPlainText">phone: +39 011 29100</p> <p class=3D"MsoPlainText"><a href=3D"mailto:al.scotto@reply.it"><span style= =3D"color:windowtext; text-decoration:none">al.scotto@reply.it</span></a></= p> <p class=3D"MsoPlainText"><a href=3D"http://www.reply.it"><span style=3D"co= lor:windowtext; text-decoration:none">www.reply.it</span></a></p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">-----Original Message-----</p> <p class=3D"MsoPlainText">From: Haim <a href=3D"mailto:[mailto:hateya@redha= t.com]"><span style=3D"color:windowtext; text-decoration:none">[mailto:hate= ya@redhat.com]</span></a></p> <p class=3D"MsoPlainText">Sent: gioved=EC 23 agosto 2012 18:22</p> <p class=3D"MsoPlainText">To: Scotto Alberto</p> <p class=3D"MsoPlainText">Cc: <a href=3D"mailto:users@ovirt.org"><span styl= e=3D"color:windowtext; text-decoration:none">users@ovirt.org</span></a></p> <p class=3D"MsoPlainText">Subject: Re: [Users] [rhev 3] add new domain fail= s: Could not retrieve LUNs</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">On 08/23/2012 07:13 PM, Scotto Alberto wrote:</p> <p class=3D"MsoPlainText">> I was going to reply.. :)</p> <p class=3D"MsoPlainText">> It looks like it halts after an error due to= cciss!c0d1, which btw is displayed by multipath -ll.</p> <p class=3D"MsoPlainText">> That's just a local disk, isn't it? So it sh= ouldn't even be listed. I may have attached it by mistake, playing with /sy= s/class/fc_* tools.</p> <p class=3D"MsoPlainText">> So, if I remove that path, everything should= go ok. Do you think so too?</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Yes, please remove and check again (make sure to = clean device-mapper table with dmsetup remove $dm).</p> <p class=3D"MsoPlainText">anyhow, vdsm should be more robust, so please fil= e a bug for it(<a href=3D"https://bugzilla.redhat.com/enter_bug.cgi?product= =3DoVirt"><span style=3D"color:windowtext; text-decoration:none">https://bu= gzilla.redhat.com/enter_bug.cgi?product=3DoVirt</span></a>).</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">Haim</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">>> it appears that vdsm fails to handle a d= evice with '!' in it </p> <p class=3D"MsoPlainText">>> (cciss!c0d1), but let's make sure its in= deed the case</p> <p class=3D"MsoPlainText">> More than that the fact is that the path scs= i_disk/ doesn't exist in </p> <p class=3D"MsoPlainText">> /sys/block/cciss!c0d1/device And this must b= e due to the fact that</p> <p class=3D"MsoPlainText">> c0d1 is NOT a damn scsi disk</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> Anyway, here is your output</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# ls -l /sys/block/ = total 0 lrwxrwxrwx. 1 root </p> <p class=3D"MsoPlainText">> root 0 2007-06-30 00:37 cciss!c0d0 -> </p=
<p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:03.0/0000:06:0= 0.0/cciss0/c0d0/block/ccis</p> <p class=3D"MsoPlainText">> s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 = 00:32 cciss!c0d1 -> </p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:03.0/0000:06:0= 0.0/cciss0/c0d1/block/ccis</p> <p class=3D"MsoPlainText">> s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 = 01:17 dm-0 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/dm-0 lrwxrwxrwx. 1 = root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:40 dm-1 -> ../devices/virtual/block/dm= -1 lrwxrwxrwx. 1 root root 0</p> <p class=3D"MsoPlainText">> 2007-06-30 01:17 dm-2 -> ../devices/virtu= al/block/dm-2 lrwxrwxrwx. 1 </p> <p class=3D"MsoPlainText">> root root 0 2007-06-30 00:49 dm-3 -> ../d= evices/virtual/block/dm-3 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 d= m-4 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/dm-4 lrwxrwxrwx. 1 = root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 01:00 dm-5 -> ../devices/virtual/block/dm= -5 lrwxrwxrwx. 1 root root 0</p> <p class=3D"MsoPlainText">> 2007-06-30 00:49 dm-6 -> ../devices/virtu= al/block/dm-6 lrwxrwxrwx. 1 </p> <p class=3D"MsoPlainText">> root root 0 2007-06-30 00:10 loop0 -> ../= devices/virtual/block/loop0 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 l= oop1 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/loop1 lrwxrwxrwx. 1= root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 loop2 -> ../devices/virtual/block/l= oop2 lrwxrwxrwx. 1 root root</p> <p class=3D"MsoPlainText">> 0 2007-06-30 00:10 loop3 -> ../devices/vi= rtual/block/loop3 lrwxrwxrwx.</p> <p class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 loop4 -> .= ./devices/virtual/block/loop4 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 l= oop5 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/loop5 lrwxrwxrwx. 1= root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 loop6 -> ../devices/virtual/block/l= oop6 lrwxrwxrwx. 1 root root</p> <p class=3D"MsoPlainText">> 0 2007-06-30 00:10 loop7 -> ../devices/vi= rtual/block/loop7 lrwxrwxrwx.</p> <p class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram0 -> ..= /devices/virtual/block/ram0 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 r= am1 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/ram1 lrwxrwxrwx. 1 = root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 ram10 -> ../devices/virtual/block/r= am10 lrwxrwxrwx. 1 root root</p> <p class=3D"MsoPlainText">> 0 2007-06-30 00:10 ram11 -> ../devices/vi= rtual/block/ram11 lrwxrwxrwx.</p> <p class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram12 -> .= ./devices/virtual/block/ram12 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 r= am13 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/ram13 lrwxrwxrwx. 1= root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 ram14 -> ../devices/virtual/block/r= am14 lrwxrwxrwx. 1 root root</p> <p class=3D"MsoPlainText">> 0 2007-06-30 00:10 ram15 -> ../devices/vi= rtual/block/ram15 lrwxrwxrwx.</p> <p class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram2 -> ..= /devices/virtual/block/ram2 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 r= am3 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/ram3 lrwxrwxrwx. 1 = root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 ram4 -> ../devices/virtual/block/ra= m4 lrwxrwxrwx. 1 root root 0</p> <p class=3D"MsoPlainText">> 2007-06-30 00:10 ram5 -> ../devices/virtu= al/block/ram5 lrwxrwxrwx. 1 </p> <p class=3D"MsoPlainText">> root root 0 2007-06-30 00:10 ram6 -> ../d= evices/virtual/block/ram6 </p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 r= am7 -></p> <p class=3D"MsoPlainText">> ../devices/virtual/block/ram7 lrwxrwxrwx. 1 = root root 0 2007-06-30</p> <p class=3D"MsoPlainText">> 00:10 ram8 -> ../devices/virtual/block/ra= m8 lrwxrwxrwx. 1 root root 0</p> <p class=3D"MsoPlainText">> 2007-06-30 00:10 ram9 -> ../devices/virtu= al/block/ram9 lrwxrwxrwx. 1 </p> <p class=3D"MsoPlainText">> root root 0 2007-06-30 00:36 sda -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/bl= ock/sda</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= db -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/bl= ock/sdb</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= dc -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/bl= ock/sdc</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= dd -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/bl= ock/sdd</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= de -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/bl= ock/sde</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= df -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/bl= ock/sdf</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= dg -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/bl= ock/sdg</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 s= dh -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:02.0/0000:09:0= 0.0/0000:0a:00.0/0000:0b:0</p> <p class=3D"MsoPlainText">> 0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/bl= ock/sdh</p> <p class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 s= r0 -></p> <p class=3D"MsoPlainText">> ../devices/pci0000:00/0000:00:1f.1/host0/tar= get0:0:0/0:0:0:0/block/sr0</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# dmsetup table</p> <p class=3D"MsoPlainText">> 3600601601cde1d0066b2fb054dece111: 0 1363148= 800 multipath 1 </p> <p class=3D"MsoPlainText">> queue_if_no_path 1 emc 2 1 round-robin 0 4 1= 8:0 1 8:48 1 8:64 1 8:80</p> <p class=3D"MsoPlainText">> 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:1= 12 1</p> <p class=3D"MsoPlainText">> HostVG-Logging: 0 4194304 linear 104:4 24741= 888</p> <p class=3D"MsoPlainText">> HostVG-Swap: 0 24723456 linear 104:4 2048</p=
</p> <p class=3D"MsoPlainText">> Subject: Re: [Users] [rhev 3] add new domain= fails: Could not retrieve </p> <p class=3D"MsoPlainText">> LUNs</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> On 08/23/2012 06:20 PM, Scotto Alberto wrote= :</p> <p class=3D"MsoPlainText">>> Here you are</p> <p class=3D"MsoPlainText">> thanks, can you run the following?</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> - ls -l /sys/block/</p> <p class=3D"MsoPlainText">> - dmsetup table</p> <p class=3D"MsoPlainText">> - lsblk (if exists)</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> it appears that vdsm fails to handle a devic= e with '!' in it (cciss!c0d1), but let's make sure its indeed the case.</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,268::clientIF::239::Storage.Dis=
</a></p> <p class=3D"MsoPlainText">>> Subject: Re: [Users] [rhev 3] add new do=
<p class=3D"MsoPlainText">> 3600508b1001035333920202020200005: 0 4299259= 20 multipath 1 </p> <p class=3D"MsoPlainText">> queue_if_no_path 0 1 1 round-robin 0 1 1 104= :16 1</p> <p class=3D"MsoPlainText">> HostVG-Data: 0 40624128 linear 104:4 2893619= 2</p> <p class=3D"MsoPlainText">> HostVG-Config: 0 16384 linear 104:4 24725504= </p> <p class=3D"MsoPlainText">> live-rw: 0 2097152 snapshot 7:1 7:2 P 8</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# lsblk</p> <p class=3D"MsoPlainText">> NAME &nbs= p; &= nbsp; &nbs= p; MAJ:MIN RM SIZE RO TYPE&= nbsp; MOUNTPOINT</p> <p class=3D"MsoPlainText">> loop0 &nb= sp; = &nb= sp; 7:0 0 = 99.3M 1 loop</p> <p class=3D"MsoPlainText">> loop1 &nb= sp; = &nb= sp; 7:1 0 = 1G 1 loop</p> <p class=3D"MsoPlainText">> =E2=E2live-rw (dm-1) = &nb= sp; 253:1 = 0 1G 0 dm /</p> <p class=3D"MsoPlainText">> loop2 &nb= sp; = &nb= sp; 7:2 0 = 512M 0 loop</p> <p class=3D"MsoPlainText">> =E2=E2live-rw (dm-1) = &nb= sp; 253:1 = 0 1G 0 dm /</p> <p class=3D"MsoPlainText">> cciss!c0d0 &nbs= p; &= nbsp; &nbs= p; 104:0 0 33.9G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E2cciss!c0d0p1 &n= bsp; = ; 104:1&n= bsp; 0 243M 0 part</p> <p class=3D"MsoPlainText">> =E2=E2cciss!c0d0p2 &n= bsp; = ; 104:2&n= bsp; 0 244M 0 part</p> <p class=3D"MsoPlainText">> =E2=E2cciss!c0d0p3 &n= bsp; = ; 104:3&n= bsp; 0 244M 0 part</p> <p class=3D"MsoPlainText">> =E2=E2cciss!c0d0p4 &n= bsp; = ; 104:4&n= bsp; 0 33.2G 0 part</p> <p class=3D"MsoPlainText">> =E2=E2HostVG-Swap (dm-3)&n= bsp; = ; 253:3 0 = 11.8G 0 lvm [SWAP]</p> <p class=3D"MsoPlainText">> =E2=E2HostVG-Config (dm-4)= &nb= sp; 253:4 0 &nbs= p; 8M 0 lvm /config</p> <p class=3D"MsoPlainText">> =E2=E2HostVG-Logging (dm-5= ) &n= bsp; 253:5 0 &nb= sp; 2G 0 lvm /var/log</p> <p class=3D"MsoPlainText">> =E2=E2HostVG-Data (dm-6)&n= bsp; = ; 253:6 0 = 19.4G 0 lvm /data</p> <p class=3D"MsoPlainText">> cciss!c0d1 &nbs= p; &= nbsp; &nbs= p; 104:16 0 205G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600508b1001035333920202020200005 (dm-= 0) 253:0 0 205G 0 mpath</p> <p class=3D"MsoPlainText">> sr0 = ; &n= bsp; = ; 11:0 1&= nbsp; 1024M 0 rom</p> <p class=3D"MsoPlainText">> sdb = ; &n= bsp; = ; 8:16 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sda = ; &n= bsp; = ; 8:0 &nbs= p; 0 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sdc = ; &n= bsp; = ; 8:32 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sdd = ; &n= bsp; = ; 8:48 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sde = ; &n= bsp; = ; 8:64 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sdf = ; &n= bsp; = ; 8:80 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sdg = ; &n= bsp; = ; 8:96 0&= nbsp; 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> sdh &nbs= p; &= nbsp; &nbs= p; 8:112 0 = 650G 0 disk</p> <p class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111 (dm-= 2) 253:2 0 650G 0 mpath</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> Alberto Scotto</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> Blue Reply</p> <p class=3D"MsoPlainText">> Via Cardinal Massaia, 83</p> <p class=3D"MsoPlainText">> 10147 - Torino - ITALY</p> <p class=3D"MsoPlainText">> phone: +39 011 29100</p> <p class=3D"MsoPlainText">> <a href=3D"mailto:al.scotto@reply.it"><span = style=3D"color:windowtext; text-decoration:none">al.scotto@reply.it</span><= /a></p> <p class=3D"MsoPlainText">> <a href=3D"http://www.reply.it"><span style= =3D"color:windowtext; text-decoration:none">www.reply.it</span></a></p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> -----Original Message-----</p> <p class=3D"MsoPlainText">> From: Haim <a href=3D"mailto:[mailto:hateya@= redhat.com]"> <span style=3D"color:windowtext; text-decoration:none">[mailto:hateya@redha= t.com]</span></a></p> <p class=3D"MsoPlainText">> Sent: gioved=EC 23 agosto 2012 18:01</p> <p class=3D"MsoPlainText">> To: Scotto Alberto</p> <p class=3D"MsoPlainText">> Cc: <a href=3D"mailto:users@ovirt.org"><span= style=3D"color:windowtext; text-decoration:none">users@ovirt.org</span></a= patcher.Protect::(wrapper)</p> <p class=3D"MsoPlainText">>> [10.16.250.216]</p> <p class=3D"MsoPlainText">>> Thread-47346::INFO::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,269::dispatcher::94::Storage.Di= spatcher.Protect::(run) Run </p> <p class=3D"MsoPlainText">>> and protect: getDeviceList, args: ()</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,269::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: mo= ving from state init -> state </p> <p class=3D"MsoPlainText">>> preparing</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,269::misc::1010::SamplingMethod= ::(__call__) Trying to enter </p> <p class=3D"MsoPlainText">>> sampling method (storage.sdc.refreshStor= age)</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,270::misc::1012::SamplingMethod= ::(__call__) Got in to </p> <p class=3D"MsoPlainText">>> sampling method</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,270::misc::1010::SamplingMethod= ::(__call__) Trying to enter </p> <p class=3D"MsoPlainText">>> sampling method (storage.iscsi.rescan)</= p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,270::misc::1012::SamplingMethod= ::(__call__) Got in to </p> <p class=3D"MsoPlainText">>> sampling method</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,271::iscsi::699::Storage.Misc.e= xcCmd::(rescan)</p> <p class=3D"MsoPlainText">>> '/usr/bin/sudo -n /sbin/iscsiadm -m sess= ion -R' (cwd None)</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,300::iscsi::699::Storage.Misc.e= xcCmd::(rescan) FAILED: <err> </p> <p class=3D"MsoPlainText">>> =3D 'iscsiadm: No session found.\n'; <= ;rc> =3D 21</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,301::misc::1020::SamplingMethod= ::(__call__) Returning last </p> <p class=3D"MsoPlainText">>> result</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,661::multipath::61::Storage.Mis= c.excCmd::(rescan)</p> <p class=3D"MsoPlainText">>> '/usr/bin/sudo -n /sbin/multipath' (cwd = None)</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,785::multipath::61::Storage.Mis= c.excCmd::(rescan) SUCCESS:</p> <p class=3D"MsoPlainText">>> <err> =3D ''; <rc> =3D 0</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,786::lvm::547::OperationMutex::= (_invalidateAllPvs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' got the opera= tion mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,786::lvm::549::OperationMutex::= (_invalidateAllPvs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' released the = operation mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,786::lvm::559::OperationMutex::= (_invalidateAllVgs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' got the opera= tion mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,787::lvm::561::OperationMutex::= (_invalidateAllVgs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' released the = operation mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,787::lvm::580::OperationMutex::= (_invalidateAllLvs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' got the opera= tion mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,788::lvm::582::OperationMutex::= (_invalidateAllLvs) Operation </p> <p class=3D"MsoPlainText">>> 'lvm invalidate operation' released the = operation mutex</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,788::misc::1020::SamplingMethod= ::(__call__) Returning last </p> <p class=3D"MsoPlainText">>> result</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,788::lvm::406::OperationMutex::= (_reloadpvs) Operation 'lvm </p> <p class=3D"MsoPlainText">>> reload operation' got the operation mute= x</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,791::lvm::374::Storage.Misc.exc= Cmd::(cmd) '/usr/bin/sudo -n </p> <p class=3D"MsoPlainText">>> /sbin/lvm pvs --config " devices { = preferred_names =3D </p> <p class=3D"MsoPlainText">>> [\\"^/dev/mapper/\\"] ignore_s= uspended_devices=3D1 write_cache_state=3D0</p> <p class=3D"MsoPlainText">>> disable_after_error_count=3D3 filter =3D= [ </p> <p class=3D"MsoPlainText">>> <a href=3D"file:///\\%22a%253600508b1001= 035333920202020200005|3600601601cde1d0066b2fb054dece"> <span style=3D"color:windowtext; text-decoration:none">\\"a%3600508b10= 01035333920202020200005|3600601601cde1d0066b2fb054dece</span></a></p> <p class=3D"MsoPlainText">>> 1 11%\\", <a href=3D"file:///\\%22r= %25.*%25\"><span style=3D"color:windowtext; text-decoration:none">\\"r= %.*%\\</span></a>" ] } global { locking_type=3D1</p> <p class=3D"MsoPlainText">>> prioritise_write_locks=3D1 wait_fo= r_locks=3D1 } backup { retain_min =3D</p> <p class=3D"MsoPlainText">>> 50 retain_days =3D 0 } " --no= headings --units b --nosuffix --separator</p> <p class=3D"MsoPlainText">>> | -o</p> <p class=3D"MsoPlainText">>> uuid,name,size,vg_name,vg_uuid,pe_start,= pe_count,pe_alloc_count,mda_c</p> <p class=3D"MsoPlainText">>> o</p> <p class=3D"MsoPlainText">>> unt,dev_size' (cwd None)</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,997::lvm::374::Storage.Misc.exc= Cmd::(cmd) SUCCESS: <err> =3D '</p> <p class=3D"MsoPlainText">>> /dev/sdh: read failed after 0 of 4096 at= 0: Input/output error\n</p> <p class=3D"MsoPlainText">>> /dev/sdh: read failed after 0 of 4096 at= 697932120064: Input/output </p> <p class=3D"MsoPlainText">>> error\n /dev/sdh: read failed afte= r 0 of 4096 at 697932177408:</p> <p class=3D"MsoPlainText">>> Input/output error\n WARNING: Erro= r counts reached a limit of 3.</p> <p class=3D"MsoPlainText">>> Device /dev/sdh was disabled\n'; <rc&= gt; =3D 0</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:10,998::lvm::429::OperationMutex::= (_reloadpvs) Operation 'lvm </p> <p class=3D"MsoPlainText">>> reload operation' released the operation= mutex</p> <p class=3D"MsoPlainText">>> MainProcess|Thread-47346::DEBUG::2007-06= -30</p> <p class=3D"MsoPlainText">>> MainProcess|00:37:11,005::devicemapper::= 144::Storage.Misc.excCmd::(_g</p> <p class=3D"MsoPlainText">>> MainProcess|e</p> <p class=3D"MsoPlainText">>> MainProcess|tPathsStatus) '/sbin/dmsetup= status' (cwd None)</p> <p class=3D"MsoPlainText">>> MainProcess|Thread-47346::DEBUG::2007-06= -30</p> <p class=3D"MsoPlainText">>> MainProcess|00:37:11,014::devicemapper::= 144::Storage.Misc.excCmd::(_g</p> <p class=3D"MsoPlainText">>> MainProcess|e</p> <p class=3D"MsoPlainText">>> MainProcess|tPathsStatus) SUCCESS: <e= rr> =3D ''; <rc> =3D 0</p> <p class=3D"MsoPlainText">>> MainProcess|Thread-47346::DEBUG::2007-06= -30</p> <p class=3D"MsoPlainText">>> MainProcess|00:37:11,019::multipath::159= ::Storage.Misc.excCmd::(getSc</p> <p class=3D"MsoPlainText">>> MainProcess|s</p> <p class=3D"MsoPlainText">>> MainProcess|iSerial) '/sbin/scsi_id --pa= ge=3D0x80 --whitelisted </p> <p class=3D"MsoPlainText">>> MainProcess|--export --replace-whitespac= e --device=3D/dev/dm-0' (cwd</p> <p class=3D"MsoPlainText">>> MainProcess|None)</p> <p class=3D"MsoPlainText">>> MainProcess|Thread-47346::DEBUG::2007-06= -30</p> <p class=3D"MsoPlainText">>> MainProcess|00:37:11,026::multipath::159= ::Storage.Misc.excCmd::(getSc</p> <p class=3D"MsoPlainText">>> MainProcess|s</p> <p class=3D"MsoPlainText">>> MainProcess|iSerial) SUCCESS: <err>= ; =3D ''; <rc> =3D 0</p> <p class=3D"MsoPlainText">>> Thread-47346::WARNING::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,027::multipath::261::Storage.Mu= ltipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Trace= back (most recent call last):</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/multipath.py", line 259, in pathListIter</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/multipath.py", line 182, in getHBTL</p> <p class=3D"MsoPlainText">>> OSError: [Errno 2] No such file or direc= tory: '/sys/block/cciss!c0d1/device/scsi_disk/'</p> <p class=3D"MsoPlainText">>> Thread-47346::ERROR::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,029::task::868::TaskManager.Tas= k::(_setError) Unexpected error Traceback (most recent call last):</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/task.py", line 876, in _run</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/hsm.py", line 696, in public_getDeviceList</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/hsm.py", line 759, in </p> <p class=3D"MsoPlainText">>> _getDeviceList</p> <p class=3D"MsoPlainText">>> KeyError: 'hbtl'</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,030::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: Ta= sk._run:</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408 () = {} failed - stopping task</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,030::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: st= opping in state preparing </p> <p class=3D"MsoPlainText">>> (force False)</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,030::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: re= f 1 aborting True</p> <p class=3D"MsoPlainText">>> Thread-47346::INFO::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,031::task::1171::TaskManager.Ta= sk::(prepare) aborting: Task </p> <p class=3D"MsoPlainText">>> is aborted: "'hbtl'" - code 10= 0</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30 00:37:11= ,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27= fde8124408: Prepare: aborted: 'hbtl'</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,031::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: re= f 0 aborting True</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,032::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: Ta= sk._doAbort: force False</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,032::resourceManager::821::Reso= urceManager.Owner::(cancelAll</p> <p class=3D"MsoPlainText">>> )</p> <p class=3D"MsoPlainText">>> Owner.cancelAll requests {}</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,032::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: mo= ving from state preparing -> </p> <p class=3D"MsoPlainText">>> state aborting</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,033::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: _a= borting: recover policy none</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,033::task::495::TaskManager.Tas= k::(_debug) Task</p> <p class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408: mo= ving from state aborting -> </p> <p class=3D"MsoPlainText">>> state failed</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,033::resourceManager::786::Reso= urceManager.Owner::(releaseAl</p> <p class=3D"MsoPlainText">>> l</p> <p class=3D"MsoPlainText">>> ) Owner.releaseAll requests {} resources= {}</p> <p class=3D"MsoPlainText">>> Thread-47346::DEBUG::2007-06-30</p> <p class=3D"MsoPlainText">>> 00:37:11,034::resourceManager::821::Reso= urceManager.Owner::(cancelAll</p> <p class=3D"MsoPlainText">>> )</p> <p class=3D"MsoPlainText">>> Owner.cancelAll requests {}</p> <p class=3D"MsoPlainText">>> Thread-47346::ERROR::2007-06-30 00:37:11= ,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'</p> <p class=3D"MsoPlainText">>> Thread-47346::ERROR::2007-06-30 00:37:11= ,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most re= cent call last):</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/dispatcher.py", line 96, in run</p> <p class=3D"MsoPlainText">>> File "/usr/= share/vdsm/storage/task.py", line 1178, in prepare</p> <p class=3D"MsoPlainText">>> KeyError: 'hbtl'</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> Alberto Scotto</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> Blue Reply</p> <p class=3D"MsoPlainText">>> Via Cardinal Massaia, 83</p> <p class=3D"MsoPlainText">>> 10147 - Torino - ITALY</p> <p class=3D"MsoPlainText">>> phone: +39 011 29100</p> <p class=3D"MsoPlainText">>> <a href=3D"mailto:al.scotto@reply.it"><s= pan style=3D"color:windowtext; text-decoration:none">al.scotto@reply.it</sp= an></a></p> <p class=3D"MsoPlainText">>> <a href=3D"http://www.reply.it"><span st= yle=3D"color:windowtext; text-decoration:none">www.reply.it</span></a></p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> -----Original Message-----</p> <p class=3D"MsoPlainText">>> From: Haim <a href=3D"mailto:[mailto:hat= eya@redhat.com]"> <span style=3D"color:windowtext; text-decoration:none">[mailto:hateya@redha= t.com]</span></a></p> <p class=3D"MsoPlainText">>> Sent: gioved=EC 23 agosto 2012 17:00</p> <p class=3D"MsoPlainText">>> To: Scotto Alberto</p> <p class=3D"MsoPlainText">>> Cc: <a href=3D"mailto:users@ovirt.org"><= span style=3D"color:windowtext; text-decoration:none">users@ovirt.org</span= main fails: Could not </p> <p class=3D"MsoPlainText">>> retrieve LUNs</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> On 08/23/2012 05:54 PM, Scotto Alberto w= rote:</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> hi,</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> can you attach full vdsm log during the = execution of getDeviceList command?</p> <p class=3D"MsoPlainText">>>> Hi all,</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> I'm trying to configure a FCP storag= e domain on RHEV 3.</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> I try to add a new domain from the c= onsole, but it can't find any</p> <p class=3D"MsoPlainText">>>> LUNs: "Could not retrieve LUNs,= please check your storage"</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Here is the output from /var/log/rhe= vm/rhevm.log:</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> ------------------------------------= </p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:07,811 WARN</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.bll.GetConfig= urationValueQuery]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) calling GetCon= figurationValueQuery with null </p> <p class=3D"MsoPlainText">>>> version, using default general for v= ersion</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:07,911 INFO</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.GetDeviceListVDSCommand]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) START, GetDevi= ceListVDSCommand(vdsId =3D </p> <p class=3D"MsoPlainText">>>> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a= , storageType=3DFCP), log id:</p> <p class=3D"MsoPlainText">>>> 60bdafe6</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,726 ERROR</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.BrokerCommandBase]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Failed in GetD= eviceListVDS method</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 ERROR</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.BrokerCommandBase]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Error code Blo= ckDeviceActionError and error </p> <p class=3D"MsoPlainText">>>> message VDSGenericException: VDSErro= rException: Failed to </p> <p class=3D"MsoPlainText">>>> GetDeviceListVDS, error =3D Error bl= ock device action: ()</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 INFO</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.BrokerCommandBase]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Command</p> <p class=3D"MsoPlainText">>>> org.ovirt.engine.core.vdsbroker.vdsb= roker.GetDeviceListVDSCommand</p> <p class=3D"MsoPlainText">>>> return value</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Class Name:</p> <p class=3D"MsoPlainText">>>> org.ovirt.engine.core.vdsbroker.vdsb= roker.LUNListReturnForXmlRpc</p> <p class=3D"MsoPlainText">>>> lunList Null</p> <p class=3D"MsoPlainText">>>> mStatus Class Name:</p> <p class=3D"MsoPlainText">>>> org.ovirt.engine.core.vdsbroker.vdsb= roker.StatusForXmlRpc</p> <p class=3D"MsoPlainText">>>> mCode 600</p> <p class=3D"MsoPlainText">>>> mMessage Error block device action: = ()</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 INFO</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.BrokerCommandBase]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Vds: pittor06v= hxd020</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 ERROR</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.VDS= CommandBase]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Command GetDev= iceListVDS execution failed. Exception:</p> <p class=3D"MsoPlainText">>>> VDSErrorException: VDSGenericExcepti= on: VDSErrorException: Failed to </p> <p class=3D"MsoPlainText">>>> GetDeviceListVDS, error =3D Error bl= ock device action: ()</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 INFO</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.vdsbroker.vds= broker.GetDeviceListVDSCommand]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) FINISH, GetDev= iceListVDSCommand, log id:</p> <p class=3D"MsoPlainText">>>> 60bdafe6</p> <p class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727 ERROR</p> <p class=3D"MsoPlainText">>>> [org.ovirt.engine.core.bll.storage.G= etDeviceListQuery]</p> <p class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Query GetDevic= eListQuery failed. Exception </p> <p class=3D"MsoPlainText">>>> message is VdcBLLException:</p> <p class=3D"MsoPlainText">>>> org.ovirt.engine.core.vdsbroker.vdsb= roker.VDSErrorException:</p> <p class=3D"MsoPlainText">>>> VDSGenericException: VDSErrorExcepti= on: Failed to GetDeviceListVDS, </p> <p class=3D"MsoPlainText">>>> error =3D Error block device action:= ()</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> ------------------------------------= ----------</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> First question: do LUNs have to be v= isible from RHEV-H or RHEV-M?</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Currently they are visible only from= the hypervisor.</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> ------------------------------------= ----</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> [root@pittor06vhxd020 log]# multipat= h -ll</p> <p class=3D"MsoPlainText">>>> 3600601601cde1d0066b2fb054dece111 dm= -2 DGC,RAID 5 size=3D650G</p> <p class=3D"MsoPlainText">>>> features=3D'1 queue_if_no_path' hwha= ndler=3D'1 emc' wp=3Drw</p> <p class=3D"MsoPlainText">>>> |-+- policy=3D'round-robin 0' pr= io=3D1 status=3Dactive</p> <p class=3D"MsoPlainText">>>> | |- 2:0:0:0 sda 8:0 active ready ru= nning</p> <p class=3D"MsoPlainText">>>> | |- 2:0:1:0 sdd 8:48 active ready r= unning</p> <p class=3D"MsoPlainText">>>> | |- 3:0:0:0 sde 8:64 active ready r= unning</p> <p class=3D"MsoPlainText">>>> | `- 3:0:1:0 sdf 8:80 active ready r= unning</p> <p class=3D"MsoPlainText">>>> `-+- policy=3D'round-robin 0' pr= io=3D0 status=3Denabled</p> <p class=3D"MsoPlainText">>>> |- 2:0:2:0 sdb 8:16 active ready run= ning</p> <p class=3D"MsoPlainText">>>> |- 2:0:3:0 sdc 8:32 active ready run= ning</p> <p class=3D"MsoPlainText">>>> |- 3:0:2:0 sdg 8:96 active ready run= ning</p> <p class=3D"MsoPlainText">>>> `- 3:0:3:0 sdh 8:112 active ready ru= nning</p> <p class=3D"MsoPlainText">>>> 3600508b1001035333920202020200005 dm= -0 HP,LOGICAL VOLUME size=3D205G</p> <p class=3D"MsoPlainText">>>> features=3D'1 queue_if_no_path' hwha= ndler=3D'0' wp=3Drw</p> <p class=3D"MsoPlainText">>>> `-+- policy=3D'round-robin 0' pr= io=3D1 status=3Dactive</p> <p class=3D"MsoPlainText">>>> `- 0:0:1:0 cciss!c0d1 104:16 active = ready running</p> <p class=3D"MsoPlainText">>>> ------------------------------------= ------------------</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Our SAN device is Clariion AX150. Is= it compatible with ovirt?</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> vdsClient -s 0 getDeviceListgives me= :</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Error block device action: ()</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Could it be due to SPM turned off? (= I have only one host)</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> [root@pittor06vhxd020 log]# ps axu |= grep -i spm</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> root 16068 0.0 0.0 7888 868 pts/1 R&= #43; 00:04 0:00 grep -i spm</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> How can I turn it on? I know the com= mand but I don't know what </p> <p class=3D"MsoPlainText">>>> paramaters append</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> spmStart</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> <spUUID> <prevID> <pr= evLVER> <recoveryMode> <scsiFencing> </p> <p class=3D"MsoPlainText">>>> <maxHostID> <version></p=
</a></p> <p class=3D"MsoPlainText">>>> <a href=3D"http://lists.ovirt.org/ma= ilman/listinfo/users"> <span style=3D"color:windowtext; text-decoration:none">http://lists.ovirt.o= rg/mailman/listinfo/users</span></a></p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> ________________________________</p> <p class=3D"MsoPlainText">>> </p> <p class=3D"MsoPlainText">>> --</p> <p class=3D"MsoPlainText">>> The information transmitted is intended = for the person or entity to which it is addressed and may contain confident= ial and/or privileged material. Any review, retransmission, dissemination o= r other use of, or taking of any action in reliance upon, this information by persons or entities other than the i= ntended recipient is prohibited. If you received this in error, please cont= act the sender and delete the material from any computer.</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> ________________________________</p> <p class=3D"MsoPlainText">> </p> <p class=3D"MsoPlainText">> --</p> <p class=3D"MsoPlainText">> The information transmitted is intended for =
<p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Thank you very much for any hints.</= p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> AS</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Alberto Scotto</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> Blue</p> <p class=3D"MsoPlainText">>>> Via Cardinal Massaia, 83</p> <p class=3D"MsoPlainText">>>> 10147 - Torino - ITALY</p> <p class=3D"MsoPlainText">>>> phone: +39 011 29100</p> <p class=3D"MsoPlainText">>>> <a href=3D"mailto:al.scotto@reply.it= "><span style=3D"color:windowtext; text-decoration:none">al.scotto@reply.it= </span></a></p> <p class=3D"MsoPlainText">>>> <a href=3D"http://www.reply.it"><spa= n style=3D"color:windowtext; text-decoration:none">www.reply.it</span></a><= /p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> ------------------------------------= --------------------------------</p> <p class=3D"MsoPlainText">>>> -</p> <p class=3D"MsoPlainText">>>> -</p> <p class=3D"MsoPlainText">>>> --</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> --</p> <p class=3D"MsoPlainText">>>> The information transmitted is inten= ded for the person or entity to </p> <p class=3D"MsoPlainText">>>> which it is addressed and may contai= n confidential and/or privileged </p> <p class=3D"MsoPlainText">>>> material. Any review, retransmission= , dissemination or other use of, </p> <p class=3D"MsoPlainText">>>> or taking of any action in reliance = upon, this information by </p> <p class=3D"MsoPlainText">>>> persons or entities other than the i= ntended recipient is prohibited.</p> <p class=3D"MsoPlainText">>>> If you received this in error, pleas= e contact the sender and delete </p> <p class=3D"MsoPlainText">>>> the material from any computer.</p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> </p> <p class=3D"MsoPlainText">>>> ____________________________________= ___________</p> <p class=3D"MsoPlainText">>>> Users mailing list</p> <p class=3D"MsoPlainText">>>> <a href=3D"mailto:Users@ovirt.org"><= span style=3D"color:windowtext; text-decoration:none">Users@ovirt.org</span= the person or entity to which it is addressed and may contain confidential = and/or privileged material. Any review, retransmission, dissemination or ot= her use of, or taking of any action in reliance upon, this information by persons or entities other than the inte= nded recipient is prohibited. If you received this in error, please contact= the sender and delete the material from any computer.</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">________________________________</p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText">--</p> <p class=3D"MsoPlainText">The information transmitted is intended for the p= erson or entity to which it is addressed and may contain confidential and/o= r privileged material. Any review, retransmission, dissemination or other u= se of, or taking of any action in reliance upon, this information by persons or entities other than the inte= nded recipient is prohibited. If you received this in error, please contact= the sender and delete the material from any computer.</p> <p class=3D"MsoPlainText">_______________________________________________</= p> <p class=3D"MsoPlainText">Users mailing list</p> <p class=3D"MsoPlainText"><a href=3D"mailto:Users@ovirt.org"><span style=3D= "color:windowtext; text-decoration:none">Users@ovirt.org</span></a></p> <p class=3D"MsoPlainText"><a href=3D"http://lists.ovirt.org/mailman/listinf= o/users"><span style=3D"color:windowtext; text-decoration:none">http://list= s.ovirt.org/mailman/listinfo/users</span></a></p> <p class=3D"MsoPlainText"> </p> <p class=3D"MsoPlainText"> </p> </div> <br> <hr> <font face=3D"Arial" color=3D"Gray" size=3D"1"><br> --<br> The information transmitted is intended for the person or entity to which i= t is addressed and may contain confidential and/or privileged material. Any= review, retransmission, dissemination or other use of, or taking of any ac= tion in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If= you received this in error, please contact the sender and delete the mater= ial from any computer.<br> </font> </body> </html> --_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_-- --_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: image/png; name="blue.png" Content-Description: blue.png Content-Disposition: inline; filename="blue.png"; size=2834; creation-date="Tue, 28 Aug 2012 16:11:45 GMT"; modification-date="Tue, 28 Aug 2012 16:11:45 GMT" Content-ID: <e6e5a80a848243228ec57fb928078181> Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAIwAAAAyCAYAAACOADM7AAAABmJLR0QA/gD+AP7rGNSCAAAACXBI WXMAAA3XAAAN1wFCKJt4AAAACXZwQWcAAACMAAAAMgCR0D3bAAAKaUlEQVR42u2ce5AUxRnAf313 3Al4eCAYFaIgyMNEUF6KlYoVIDBArDxqopWxQgViQlWsPHA0MUlZVoyKRsdSE4lGomjIaHS0UlHL wTIPpEgQFQUUjYIWdfIIScyBHi/Z6/zRM1xP3yzs7t3unOX8qra2H9M9vb3f9Pf19/WukFKSk1Mq dVkPIOejRS4wOWXR6wVGuP5I4foDsh5HjkL0VhtGuP5A4CFgNrAD+Lb0nKeyHtfHnd68wixGCQvA qcA9wvWPy3pQH3caan1D4fonAYeBDwEZjaFflAaok56zHRhsNG0B+gAHSrhHarn0nFp/3NLnxbKP B06I5kECO2UYZD2sLtRcYIBJwK+BoYBACU89cAjoAIRw/TuAJcClQGy//FJ6zvvH6ly4/qXAz4vU HQA2A4H0nIcz+OxH41eAHaU3AhdkPaA0MrFhhOuPB2YA5wBnA6ehni5dgKcBu4C5wLZS7Rfh+g8A 80u49HHgEuk5h2s+AeaYLbsO2AKMiIqWyzBYkPW40shihUF6zkbUUwSAcP0G4FHgS9pl10rPmQMs LbXfSBVNLPHyrwDfBO7JYg4MRqEempjnsh5QMXqL0Xsl8EUt3w5cXUE/w4AztfzzwGSUGrwoyuvM yfqDR5yLUssxL2U9oGJkssLoCNdfjLJXdBZIz9lQQXcTgSYt/4z0nHjy1wvX3wW8oNX3O8q4TgKm AGegjNB/As9JzzmYer1lTwKGoOyyV2UYtArLngLMQ9lh64EVRQxZ3V5pje4V9zsVGBRl22QYrDXu e0HUvwD+K8NgXbe/lKOQqcAI178MuM0ovk16zqMVdjnNyL9g5E2DrTVlTP1RRvM3gIFG9RvC9RdK z/lHoo2yQQJgeFR0hbDsT6FUns544Icp456qpV+RYaAL5RJgepR+FWXzxfcdA6zRrr0SqKrAZKaS hOt/DbjXKH5Geo7bjW71iT8AvGLUzzXyfzfGNBBlPyymq7AAjAWeFK5/slE+AvhklC4At6KEZb9x 3cJo+9x5T8s+ERinFa012uzU0vuMuu9r6W3AXd2Yu5LIRGCE618E/D6l6rpu9Hk8MEEr2iQ9p1Wr n4wShJgPgCeMbh6g02jeB9wILASe1q4ZBHzBaDeRThukHghRdskoQF+NmlH+JJ0JqB1ijCkw72np jiOfx7JPQrkdYm6QYXBMH1V3qYlKEq7fhNLvw1CTeztK55rcJlz/s8XshGPwaeBELd8sXP961Bd4 Bsqo1u2bm6Tn7NbGeCHKMI6ZLz3nsajuT6gtfjxfpxr31lXhThkG8470a9mrtPp2uq4652np94FN Rr0uMM1a+jI6fVTvAMsrmLOy6VGBEa5fB3wOpctHaK9TgVOAxmN0MRXlwPpWBbefYuTHAj8tcu39 0nNuMMq+qqXfjoUl4mSSq/HbRlv9S3/ZqBumpXcB/zPqz9fSm2UY/Nuo1wWmCUBYdiPwHa3ck2Hw YQVzVjbVWGFmkW7YmewDfga8CNwHnB6VXyZcf7X0nAfLvG8pntE3gSXSc5an1Olf+hDh+i+jVieJ UiOxwBSiMQMgLLsFOEtr+7xWB8rQjdkgw0BXK40o1RWTZrDu0dKx0X4xylMOynZZVuZcVUyPCoz0 nA7gR8L1N6FWmQIqZtRGpwoSwF7gRek5WwCE658P3A9Y0TV3C9ffUOrWOlrZdIfdXuBhlCqaqZU/ myYs0RZaNzybUV7oNFqBt7T8BJJ2iW6zDAPGFKkDGE1yBTLtF0gKTCF6/4FWtsTYVVWVqtgw0nNW lHn9LmCOcP2bgKuAvsAtqNWqFGLVF7NGes4i4fpjgNfpFNbzi7QfD/TX8vtQMa40VkvPKWh5fWfW DuhCfg5Ju8nc5k/RxpZYuTR0gWkTlj0D5YgEeJca2S4xvcXTC4D0nKvpdNWXc2hqEiqSHROrhR0k bYAzhesPTmmvG61tKAE6PXoNRRnTg6OX6VvRhfB1GQa7tbyu5v6D8qNQpH4bsDVlbLrADACu0fK/ qOXqAr1MYCLip7AcI+48I78WIIpuv6mVN5NUPWntN0nP2So9p016ThtwEKU6RpIMOyAsuw9JVWiu INO19AYZBma0fbKWXi/DoEBX9tBpu4wDLozS2+jqx6o6vVFgYt+JKKON/pTvJ6kWzKc6LTg5XEtv MeruAF5DqbZVgH6IayTJoOHf4oSw7LNICuKTeqfCsj9BUnhN+yamPXqZc3JrLfwuJpnHklKIBaa+ lIuF67eQ3KW8HtlEMabhPCmlG/3JnhX5ZHaifDeLtLqlxpmcySQfuvnCstdH6WXaZ9iPMsJ1xpOM ZaXZL6DsqfcB3UO8A7WzrDm9T2DqG7dTOHSIEgUGIc5GyhatZJ1Rv4HkmZ/xKb08o5UPRa0UkuQT vY6uQVJTFc5D7fQ6SNpUN8ow2GVcq7sB2ugq2DGHUYfLdG6SYbCPDMhcYIRlJwWjcGg/Z1/yATBE zJxXT0Pf4o0P7pWcO39W4nuVHS+JGfPq6dMXOjpgzNyt9En0MUF877fDee3x1iPlo2beTOPxnwGh qzahuhUAjwCLpOeYKkDfIT2BUl1XkxT2+2QYXJ8yen0H+JYMgz2kY9o126mh38UkITBRYGwp5e1Q usNjwL/Ql3VRX2D35mUI0UB90wyOZmc19i+wa+NB+vTrnMA9re00RO3q6iRbVtYxeOzt1NXHS3od e96dRkPT6CN9v/HUIRr738Dg0bMRDSdQVzeAjsJh+ra8SfMpf5S3XNzFoSYsewhJVbhKhoEnLDtE HV4vRGXPprQFFTdrRklk2u4opoVkyMOTYbCfjEgc0RSWPQhlQ/SruMfymCrD4IXud1N7In+ILgzT ZRj8tYfvcSLwOzoPer0DjKv1VlrHVEltqBhMafZD99mR1QfvAXT1tYfiNkhZCMvuD1yLCtbORsXg Yi7PUljAEJgoztFaYV8fN8yg4XsV95TkLJS32+QaGQZPl9tZT5O50ftRJLL1Pq8V9cjqEjHdyG8D rpdhkJmhq5MLTGX0QR2diLdnYQ/2vRq1wsRe6nUyDNq712XP0Wt/W53TO+mNoYGcXkwuMDll0eM2 TPRbnGnAvaaDSVj2bOA0GQY1j7Lm9AzVWGG+jIrwphlH3wXuzvpD51RONXZJ7aizLFcIyx4O3CXD IN527kUdJAJAWPbFqBXnVmHZV6FO3K+I6oahzgYPAX7T017UnMqoxgpTQAniONRJ/AeFZRc72+IA P47SPwEWAAjLbgL+jPJ1NAF/EZZd6o/sc6pINQSmARAyDL6OOm45mmSoX+cDVDiC6D0+azI0arcS FSkG9fcgORlTbcfdXtR5jqOdnpPGO3QK8nzU33KsoutvgXIyoBorjP7FN6OEsph3sE6rq9fS8RmQ RTIMTgP+QPJsbk5GVENgjgMQlv0QcDnwBp0nxgaQ/O+6dmCUsOxHUGdj459kbI/a3Sksew3qjE5L 1pOVUx2VtBJljxxAhf3v0v4TZRnKmI25ObruLdTZkvcAZBgcEpY9E3BRu6TrZBisznqycvJYUk6Z 5KGBnLLIBSanLHKBySmLXGByyiIXmJyy+D/P9uGVPOu6DAAAACh6VFh0U29mdHdhcmUAAHja801M LsrPTU3JTFRwyyxKLc8vyi5WsAAAYBUIJ4KDNosAAAAASUVORK5CYII= --_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_-- --_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_ Content-Type: text/plain; name="add storage domain (fcp) SUCCEEDS - vdsm log.txt" Content-Description: add storage domain (fcp) SUCCEEDS - vdsm log.txt Content-Disposition: attachment; filename="add storage domain (fcp) SUCCEEDS - vdsm log.txt"; size=8263; creation-date="Tue, 28 Aug 2012 16:02:28 GMT"; modification-date="Tue, 28 Aug 2012 16:02:28 GMT" Content-Transfer-Encoding: base64 VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNDQ6OmNsaWVudElGOjoy Mzk6OlN0b3JhZ2UuRGlzcGF0Y2hlci5Qcm90ZWN0Ojood3JhcHBlcikgWzEwLjE2LjI1MC4yMTZd DQpUaHJlYWQtMTEwMzc6OklORk86OjIwMTItMDgtMjggMTU6NDY6MjUsMTQ2OjpkaXNwYXRjaGVy Ojo5NDo6U3RvcmFnZS5EaXNwYXRjaGVyLlByb3RlY3Q6OihydW4pIFJ1biBhbmQgcHJvdGVjdDog Z2V0RGV2aWNlTGlzdCwgYXJnczogKCkNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg MTU6NDY6MjUsMTQ2Ojp0YXNrOjo0OTU6OlRhc2tNYW5hZ2VyLlRhc2s6OihfZGVidWcpIFRhc2sg MDcxNWI4OGUtMWZjZi00ZGRhLTkzMjgtMDQ2NTNlYTgyNTZlOiBtb3ZpbmcgZnJvbSBzdGF0ZSBp bml0IC0+IHN0YXRlIHByZXBhcmluZw0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAx NTo0NjoyNSwxNDc6Om1pc2M6OjEwMTA6OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18pIFRyeWlu ZyB0byBlbnRlciBzYW1wbGluZyBtZXRob2QgKHN0b3JhZ2Uuc2RjLnJlZnJlc2hTdG9yYWdlKQ0K VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNDg6Om1pc2M6OjEwMTI6 OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18pIEdvdCBpbiB0byBzYW1wbGluZyBtZXRob2QNClRo cmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsMTQ5OjptaXNjOjoxMDEwOjpT YW1wbGluZ01ldGhvZDo6KF9fY2FsbF9fKSBUcnlpbmcgdG8gZW50ZXIgc2FtcGxpbmcgbWV0aG9k IChzdG9yYWdlLmlzY3NpLnJlc2NhbikNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg MTU6NDY6MjUsMTQ5OjptaXNjOjoxMDEyOjpTYW1wbGluZ01ldGhvZDo6KF9fY2FsbF9fKSBHb3Qg aW4gdG8gc2FtcGxpbmcgbWV0aG9kDQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1 OjQ2OjI1LDE1MDo6aXNjc2k6OjY5OTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KHJlc2NhbikgJy91 c3IvYmluL3N1ZG8gLW4gL3NiaW4vaXNjc2lhZG0gLW0gc2Vzc2lvbiAtUicgKGN3ZCBOb25lKQ0K VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNzY6OmlzY3NpOjo2OTk6 OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihyZXNjYW4pIEZBSUxFRDogPGVycj4gPSAnaXNjc2lhZG06 IE5vIHNlc3Npb24gZm91bmQuXG4nOyA8cmM+ID0gMjENClRocmVhZC0xMTAzNzo6REVCVUc6OjIw MTItMDgtMjggMTU6NDY6MjUsMTc3OjptaXNjOjoxMDIwOjpTYW1wbGluZ01ldGhvZDo6KF9fY2Fs bF9fKSBSZXR1cm5pbmcgbGFzdCByZXN1bHQNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgt MjggMTU6NDY6MjUsNTQyOjptdWx0aXBhdGg6OjYxOjpTdG9yYWdlLk1pc2MuZXhjQ21kOjoocmVz Y2FuKSAnL3Vzci9iaW4vc3VkbyAtbiAvc2Jpbi9tdWx0aXBhdGgnIChjd2QgTm9uZSkNClRocmVh ZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsNjU3OjptdWx0aXBhdGg6OjYxOjpT dG9yYWdlLk1pc2MuZXhjQ21kOjoocmVzY2FuKSBTVUNDRVNTOiA8ZXJyPiA9ICcnOyA8cmM+ID0g MA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw2NTg6Omx2bTo6NTQ3 OjpPcGVyYXRpb25NdXRleDo6KF9pbnZhbGlkYXRlQWxsUHZzKSBPcGVyYXRpb24gJ2x2bSBpbnZh bGlkYXRlIG9wZXJhdGlvbicgZ290IHRoZSBvcGVyYXRpb24gbXV0ZXgNClRocmVhZC0xMTAzNzo6 REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsNjU5Ojpsdm06OjU0OTo6T3BlcmF0aW9uTXV0ZXg6 OihfaW52YWxpZGF0ZUFsbFB2cykgT3BlcmF0aW9uICdsdm0gaW52YWxpZGF0ZSBvcGVyYXRpb24n IHJlbGVhc2VkIHRoZSBvcGVyYXRpb24gbXV0ZXgNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTIt MDgtMjggMTU6NDY6MjUsNjU5Ojpsdm06OjU1OTo6T3BlcmF0aW9uTXV0ZXg6OihfaW52YWxpZGF0 ZUFsbFZncykgT3BlcmF0aW9uICdsdm0gaW52YWxpZGF0ZSBvcGVyYXRpb24nIGdvdCB0aGUgb3Bl cmF0aW9uIG11dGV4DQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2 MDo6bHZtOjo1NjE6Ok9wZXJhdGlvbk11dGV4OjooX2ludmFsaWRhdGVBbGxWZ3MpIE9wZXJhdGlv biAnbHZtIGludmFsaWRhdGUgb3BlcmF0aW9uJyByZWxlYXNlZCB0aGUgb3BlcmF0aW9uIG11dGV4 DQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2MDo6bHZtOjo1ODA6 Ok9wZXJhdGlvbk11dGV4OjooX2ludmFsaWRhdGVBbGxMdnMpIE9wZXJhdGlvbiAnbHZtIGludmFs aWRhdGUgb3BlcmF0aW9uJyBnb3QgdGhlIG9wZXJhdGlvbiBtdXRleA0KVGhyZWFkLTExMDM3OjpE RUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw2NjA6Omx2bTo6NTgyOjpPcGVyYXRpb25NdXRleDo6 KF9pbnZhbGlkYXRlQWxsTHZzKSBPcGVyYXRpb24gJ2x2bSBpbnZhbGlkYXRlIG9wZXJhdGlvbicg cmVsZWFzZWQgdGhlIG9wZXJhdGlvbiBtdXRleA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0w OC0yOCAxNTo0NjoyNSw2NjE6Om1pc2M6OjEwMjA6OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18p IFJldHVybmluZyBsYXN0IHJlc3VsdA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAx NTo0NjoyNSw2NjE6Omx2bTo6NDA2OjpPcGVyYXRpb25NdXRleDo6KF9yZWxvYWRwdnMpIE9wZXJh dGlvbiAnbHZtIHJlbG9hZCBvcGVyYXRpb24nIGdvdCB0aGUgb3BlcmF0aW9uIG11dGV4DQpUaHJl YWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2NDo6bHZtOjozNzQ6OlN0b3Jh Z2UuTWlzYy5leGNDbWQ6OihjbWQpICcvdXNyL2Jpbi9zdWRvIC1uIC9zYmluL2x2bSBwdnMgLS1j b25maWcgIiBkZXZpY2VzIHsgcHJlZmVycmVkX25hbWVzID0gW1xcIl4vZGV2L21hcHBlci9cXCJd IGlnbm9yZV9zdXNwZW5kZWRfZGV2aWNlcz0xIHdyaXRlX2NhY2hlX3N0YXRlPTAgZGlzYWJsZV9h ZnRlcl9lcnJvcl9jb3VudD0zIGZpbHRlciA9IFsgXFwiYSUzNjAwNTA4YjEwMDEwMzUzMzM5MjAy MDIwMjAyMDAwMDV8MzYwMDYwMTYwMWNkZTFkMDBkNGNiMzczNGQyZWRlMTExJVxcIiwgXFwiciUu KiVcXCIgXSB9ICBnbG9iYWwgeyAgbG9ja2luZ190eXBlPTEgIHByaW9yaXRpc2Vfd3JpdGVfbG9j a3M9MSAgd2FpdF9mb3JfbG9ja3M9MSB9ICBiYWNrdXAgeyAgcmV0YWluX21pbiA9IDUwICByZXRh aW5fZGF5cyA9IDAgfSAiIC0tbm9oZWFkaW5ncyAtLXVuaXRzIGIgLS1ub3N1ZmZpeCAtLXNlcGFy YXRvciB8IC1vIHV1aWQsbmFtZSxzaXplLHZnX25hbWUsdmdfdXVpZCxwZV9zdGFydCxwZV9jb3Vu dCxwZV9hbGxvY19jb3VudCxtZGFfY291bnQsZGV2X3NpemUnIChjd2QgTm9uZSkNClRocmVhZC0x MTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsODkxOjpsdm06OjM3NDo6U3RvcmFnZS5N aXNjLmV4Y0NtZDo6KGNtZCkgU1VDQ0VTUzogPGVycj4gPSAiICBDb3VsZG4ndCBmaW5kIGRldmlj ZSB3aXRoIHV1aWQgNEU3RDhJLTY3cFItc0IxWC1mam8wLVFvUTItejJBeS00MVBIZzQuXG4iOyA8 cmM+ID0gMA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw4OTI6Omx2 bTo6NDI5OjpPcGVyYXRpb25NdXRleDo6KF9yZWxvYWRwdnMpIE9wZXJhdGlvbiAnbHZtIHJlbG9h ZCBvcGVyYXRpb24nIHJlbGVhc2VkIHRoZSBvcGVyYXRpb24gbXV0ZXgNCk1haW5Qcm9jZXNzfFRo cmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsODkyOjpkZXZpY2VtYXBwZXI6 OjE0NDo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KF9nZXRQYXRoc1N0YXR1cykgJy9zYmluL2Rtc2V0 dXAgc3RhdHVzJyAoY3dkIE5vbmUpDQpNYWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoy MDEyLTA4LTI4IDE1OjQ2OjI1LDkwOTo6ZGV2aWNlbWFwcGVyOjoxNDQ6OlN0b3JhZ2UuTWlzYy5l eGNDbWQ6OihfZ2V0UGF0aHNTdGF0dXMpIFNVQ0NFU1M6IDxlcnI+ID0gJyc7IDxyYz4gPSAwDQpN YWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDkxNTo6 bXVsdGlwYXRoOjoxNTk6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihnZXRTY3NpU2VyaWFsKSAnL3Ni aW4vc2NzaV9pZCAtLXBhZ2U9MHg4MCAtLXdoaXRlbGlzdGVkIC0tZXhwb3J0IC0tcmVwbGFjZS13 aGl0ZXNwYWNlIC0tZGV2aWNlPS9kZXYvZG0tMCcgKGN3ZCBOb25lKQ0KTWFpblByb2Nlc3N8VGhy ZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5MjU6Om11bHRpcGF0aDo6MTU5 OjpTdG9yYWdlLk1pc2MuZXhjQ21kOjooZ2V0U2NzaVNlcmlhbCkgU1VDQ0VTUzogPGVycj4gPSAn JzsgPHJjPiA9IDANCk1haW5Qcm9jZXNzfFRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg MTU6NDY6MjUsOTM2OjptdWx0aXBhdGg6OjE1OTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KGdldFNj c2lTZXJpYWwpICcvc2Jpbi9zY3NpX2lkIC0tcGFnZT0weDgwIC0td2hpdGVsaXN0ZWQgLS1leHBv cnQgLS1yZXBsYWNlLXdoaXRlc3BhY2UgLS1kZXZpY2U9L2Rldi9kbS0yJyAoY3dkIE5vbmUpDQpN YWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDk0Njo6 bXVsdGlwYXRoOjoxNTk6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihnZXRTY3NpU2VyaWFsKSBTVUND RVNTOiA8ZXJyPiA9ICcnOyA8cmM+ID0gMA0KVGhyZWFkLTExMDM3OjpXQVJOSU5HOjoyMDEyLTA4 LTI4IDE1OjQ2OjI1LDk0Nzo6bXVsdGlwYXRoOjoyNjE6OlN0b3JhZ2UuTXVsdGlwYXRoOjoocGF0 aExpc3RJdGVyKSBQcm9ibGVtIGdldHRpbmcgaGJ0bCBmcm9tIGRldmljZSBgY2Npc3MhYzBkMWAN ClRocmVhZC0xMTAzNzo6V0FSTklORzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5NDk6OmhzbTo6NzM1 OjpTdG9yYWdlLkhTTTo6KF9nZXREZXZpY2VMaXN0KSBJZ25vcmluZyBwYXJ0aXRpb25lZCBkZXZp Y2Ugeydwcm9kdWN0JzogJ0xPR0lDQUwgVk9MVU1FJywgJ2RtJzogJ2RtLTInLCAnZGV2dHlwZXMn OiBbJ0ZDUCddLCAnZndyZXYnOiAnMi4wOCcsICdsb2dpY2FsYmxvY2tzaXplJzogJzUxMicsICdj b25uZWN0aW9ucyc6IFtdLCAnZGV2dHlwZSc6ICdGQ1AnLCAncGh5c2ljYWxibG9ja3NpemUnOiAn NTEyJywgJ3ZlbmRvcic6ICdIUCcsICdzZXJpYWwnOiAnU0hQX0xPR0lDQUxfVk9MVU1FX1BINzlN Vzc1MzknLCAnZ3VpZCc6ICczNjAwNTA4YjEwMDEwMzUzMzM5MjAyMDIwMjAyMDAwMDUnLCAncGF0 aHMnOiBbeydkZXZudW0nOiBEZXZpY2VOdW1iZXIoTWFqb3I9MTA0LCBNaW5vcj0xNiksICdwaHlz ZGV2JzogJ2NjaXNzIWMwZDEnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJ31dLCAn Y2FwYWNpdHknOiAnMjIwMTIyMDcxMDQwJ30NClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgt MjggMTU6NDY6MjUsOTUwOjp0YXNrOjo0OTU6OlRhc2tNYW5hZ2VyLlRhc2s6OihfZGVidWcpIFRh c2sgMDcxNWI4OGUtMWZjZi00ZGRhLTkzMjgtMDQ2NTNlYTgyNTZlOiBmaW5pc2hlZDogeydkZXZM aXN0JzogW3snZndyZXYnOiAnMDIyMCcsICd2Z1VVSUQnOiAnYUJ6SHNsLXhKcGUtNGRBRC05VVNB LUdoY0stcnNpMi1EWlpHb1UnLCAncGF0aGxpc3QnOiBbXSwgJ2xvZ2ljYWxibG9ja3NpemUnOiAn NTEyJywgJ2RldnR5cGUnOiAnRkNQJywgJ3BoeXNpY2FsYmxvY2tzaXplJzogJzUxMicsICdzZXJp YWwnOiAnU0RHQ19SQUlEXzVfRkNOSFcwNzMzMDAwMDknLCAnR1VJRCc6ICczNjAwNjAxNjAxY2Rl MWQwMGQ0Y2IzNzM0ZDJlZGUxMTEnLCAncHJvZHVjdElEJzogJ1JBSUQgNScsICd2ZW5kb3JJRCc6 ICdER0MnLCAnY2FwYWNpdHknOiAnNjk3OTMyMTg1NjAwJywgJ3BhcnRpdGlvbmVkJzogRmFsc2Us ICdwYXRoc3RhdHVzJzogW3sncGh5c2Rldic6ICdzZGEnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5 cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGInLCAnc3RhdGUnOiAnYWN0 aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGUnLCAnc3Rh dGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdz ZGYnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5 c2Rldic6ICdzZGMnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcw J30sIHsncGh5c2Rldic6ICdzZGQnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywg J2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGcnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUn OiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGgnLCAnc3RhdGUnOiAnYWN0aXZl JywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ31dLCAncHZVVUlEJzogJ01vSE0xdS1keGVoLUdP MVYtYXhDby1MZlAyLTMzdVUtRjRpZmNEJ31dfQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0w OC0yOCAxNTo0NjoyNSw5NTE6OnRhc2s6OjQ5NTo6VGFza01hbmFnZXIuVGFzazo6KF9kZWJ1Zykg VGFzayAwNzE1Yjg4ZS0xZmNmLTRkZGEtOTMyOC0wNDY1M2VhODI1NmU6IG1vdmluZyBmcm9tIHN0 YXRlIHByZXBhcmluZyAtPiBzdGF0ZSBmaW5pc2hlZA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAx Mi0wOC0yOCAxNTo0NjoyNSw5NTE6OnJlc291cmNlTWFuYWdlcjo6Nzg2OjpSZXNvdXJjZU1hbmFn ZXIuT3duZXI6OihyZWxlYXNlQWxsKSBPd25lci5yZWxlYXNlQWxsIHJlcXVlc3RzIHt9IHJlc291 cmNlcyB7fQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5NTE6OnJl c291cmNlTWFuYWdlcjo6ODIxOjpSZXNvdXJjZU1hbmFnZXIuT3duZXI6OihjYW5jZWxBbGwpIE93 bmVyLmNhbmNlbEFsbCByZXF1ZXN0cyB7fQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0y OCAxNTo0NjoyNSw5NTI6OnRhc2s6OjQ5NTo6VGFza01hbmFnZXIuVGFzazo6KF9kZWJ1ZykgVGFz ayAwNzE1Yjg4ZS0xZmNmLTRkZGEtOTMyOC0wNDY1M2VhODI1NmU6IHJlZiAwIGFib3J0aW5nIEZh bHNlDQpUaHJlYWQtMTEwMzc6OklORk86OjIwMTItMDgtMjggMTU6NDY6MjUsOTUyOjpkaXNwYXRj aGVyOjoxMDA6OlN0b3JhZ2UuRGlzcGF0Y2hlci5Qcm90ZWN0OjoocnVuKSBSdW4gYW5kIHByb3Rl Y3Q6IGdldERldmljZUxpc3QsIFJldHVybiByZXNwb25zZTogeydzdGF0dXMnOiB7J21lc3NhZ2Un OiAnT0snLCAnY29kZSc6IDB9LCAnZGV2TGlzdCc6IFt7J3ZlbmRvcklEJzogJ0RHQycsICdjYXBh Y2l0eSc6ICc2OTc5MzIxODU2MDAnLCAnZndyZXYnOiAnMDIyMCcsICdwYXJ0aXRpb25lZCc6IEZh bHNlLCAndmdVVUlEJzogJ2FCekhzbC14SnBlLTRkQUQtOVVTQS1HaGNLLXJzaTItRFpaR29VJywg J3BhdGhsaXN0JzogW10sICdsb2dpY2FsYmxvY2tzaXplJzogJzUxMicsICdwYXRoc3RhdHVzJzog W3sncGh5c2Rldic6ICdzZGEnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1 bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGInLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0 aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGUnLCAndHlwZSc6ICdGQ1AnLCAnc3Rh dGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGYnLCAndHlwZSc6ICdG Q1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGMnLCAn dHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6 ICdzZGQnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsn cGh5c2Rldic6ICdzZGcnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6 ICcwJ30sIHsncGh5c2Rldic6ICdzZGgnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZl JywgJ2x1bic6ICcwJ31dLCAnZGV2dHlwZSc6ICdGQ1AnLCAncGh5c2ljYWxibG9ja3NpemUnOiAn NTEyJywgJ3B2VVVJRCc6ICdNb0hNMXUtZHhlaC1HTzFWLWF4Q28tTGZQMi0zM3VVLUY0aWZjRCcs ICdzZXJpYWwnOiAnU0RHQ19SQUlEXzVfRkNOSFcwNzMzMDAwMDknLCAnR1VJRCc6ICczNjAwNjAx NjAxY2RlMWQwMGQ0Y2IzNzM0ZDJlZGUxMTEnLCAncHJvZHVjdElEJzogJ1JBSUQgNSd9XX0NCg== --_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_--
participants (2)
-
Haim -
Scotto Alberto