5:54 p.m.
--_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_
Content-Type: multipart/alternative;
boundary="_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_"
--_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any LUNs: "Co=
uld not retrieve LUNs, please check your storage"
[cid:image002.jpg@01CD8150.7C76A710]
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN [org.ovirt.engine.core.bll.GetConfigurationVa=
lueQuery] (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with nul=
l version, using default general for version
2007-06-29 21:50:07,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand=
(vdsId =3D 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), log id=
: 60bdafe6
2007-06-29 21:50:08,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br=
okerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.Br=
okerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError an=
d error message VDSGenericException: VDSErrorException: Failed to GetDevice=
ListVDS, error =3D Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Br=
okerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.vdsbro=
ker.vdsbroker.GetDeviceListVDSCommand return value
Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRp=
c
lunList Null
mStatus Class Name: org.ovirt.engine.core.vdsbroker.v=
dsbroker.StatusForXmlRpc
mCode 600
mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.B=
rokerCommandBase] (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBa=
se] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Except=
ion: VDSErrorException: VDSGenericException: VDSErrorException: Failed to G=
etDeviceListVDS, error =3D Error block device action: ()
2007-06-29 21:50:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSComman=
d, log id: 60bdafe6
2007-06-29 21:50:08,727 ERROR [org.ovirt.engine.core.bll.storage.GetDeviceL=
istQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception =
message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSEr=
rorException: VDSGenericException: VDSErrorException: Failed to GetDeviceLi=
stVDS, error =3D Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll
3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5
size=3D650G features=3D'1 queue_if_no_path' hwhandler=3D'1 emc' wp=3Drw
|-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive
| |- 2:0:0:0 sda 8:0 active ready running
| |- 2:0:1:0 sdd 8:48 active ready running
| |- 3:0:0:0 sde 8:64 active ready running
| `- 3:0:1:0 sdf 8:80 active ready running
`-+- policy=3D'round-robin 0' prio=3D0 status=3Denabled
|- 2:0:2:0 sdb 8:16 active ready running
|- 2:0:3:0 sdc 8:32 active ready running
|- 3:0:2:0 sdg 8:96 active ready running
`- 3:0:3:0 sdh 8:112 active ready running
3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME
size=3D205G features=3D'1 queue_if_no_path' hwhandler=3D'0' wp=3Drw
`-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive
`- 0:0:1:0 cciss!c0d1 104:16 active ready running
------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceList gives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i sp=
m
How can I turn it on? I know the command but I don't know what paramaters a=
ppend
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing> <maxHostI=
D> <version
Thank you very much for any hints.
AS
Alberto Scotto
[Blue]
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
________________________________
--
The information transmitted is intended for the person or entity to which i=
t is addressed and may contain confidential and/or privileged material. Any=
review, retransmission, dissemination or other use of, or taking of any ac=
tion in reliance upon, this information by persons or entities other than t=
he intended recipient is prohibited. If you received this in error, please =
contact the sender and delete the material from any computer.
--_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Dus-ascii"=
<style <!--
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p
{margin-right:0cm;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif"}
span.EmailStyle17
{font-family:"Calibri","sans-serif";
color:windowtext}
span.BalloonTextChar
{font-family:"Tahoma","sans-serif"}
.MsoChpDefault
{font-family:"Calibri","sans-serif"}
@page WordSection1
{margin:70.85pt 2.0cm 2.0cm 2.0cm}
div.WordSection1
{}
-- </style
</head <body lang=3D"IT"
link=3D"blue" vlink=3D"purple" <div
class=3D"WordSection1" <p><span
lang=3D"EN-US">Hi all,</span></p
<p><span lang=3D"EN-US">I'm trying to configure a FCP
storage domain on RHE=
V 3.</span></p <p><span
lang=3D"EN-US">I try to add a new domain from the console, but it =
can't find any LUNs: "Could not retrieve LUNs, please check your stora=
ge"</span></p <p><img
width=3D"755" height=3D"565" id=3D"Picture_x0020_2"
src=3D"cid:imag=
e002.jpg(a)01CD8150.7C76A710"></p
<p><span lang=3D"EN-US"> </span></p <p><span lang=3D"EN-US">Here is the
output from /var/log/rhevm/rhevm.log:</=
span></p <p><span
lang=3D"EN-US"> ------------------------------------</span></=
p <p><span lang=3D"EN-US">2007-06-29
21:50:07,811 WARN [org.ovirt.engin=
e.core.bll.GetConfigurationValueQuery] (http-0.0.0.0-8443-1) calling GetCon=
figurationValueQuery with null version, using default general for version<b=
r 2007-06-29 21:50:07,911 INFO
[org.ovirt.engine.core.vdsbroker.vdsbrok=
er.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) START, GetDeviceListVDSCo=
mmand(vdsId =3D 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), l=
og id: 60bdafe6<br 2007-06-29 21:50:08,726 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.Br=
okerCommandBase] (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method<br=
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.Br=
okerCommandBase] (http-0.0.0.0-8443-1) Error code BlockDeviceActionError an=
d error message VDSGenericException: VDSErrorException: Failed to GetDevice=
ListVDS, error =3D Error block device
action: ()<br 2007-06-29 21:50:08,727
INFO [org.ovirt.engine.core.vdsbroker.vdsbrok=
er.BrokerCommandBase] (http-0.0.0.0-8443-1) Command org.ovirt.engine.core.v=
dsbroker.vdsbroker.GetDeviceListVDSCommand return value</span></p <p><span lang=3D"EN-US">Class Name:
org.ovirt.engine.core.vdsbroker.vdsbrok=
er.LUNListReturnForXmlRpc<br
lunList &n=
bsp;
Null<br
mStatus &n=
bsp;
Class Name=
: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc<br
mCode &nbs=
p;
=
600<br
mMessage &=
nbsp;
Error block dev=
ice action: ()</span></p <p><span
lang=3D"EN-US"> 2007-06-29 21:50:08,727 INFO [org.ovirt=
.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8443-1) V=
ds: pittor06vhxd020<br 2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.VDSCommandBa=
se] (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Except=
ion: VDSErrorException: VDSGenericException: VDSErrorException: Failed to G=
etDeviceListVDS, error =3D Error block
device action: ()<br 2007-06-29 21:50:08,727
INFO [org.ovirt.engine.core.vdsbroker.vdsbrok=
er.GetDeviceListVDSCommand] (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSC=
ommand, log id: 60bdafe6<br 2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.bll.storage.GetDeviceL=
istQuery] (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception =
message is VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSEr=
rorException: VDSGenericException:
VDSErrorException: Failed to GetDeviceListVDS, error =3D Error block devic=
e action: ()</span></p <p><span
lang=3D"EN-US">----------------------------------------------</spa=
n></p <p><span
lang=3D"EN-US"> </span></p
<p><span lang=3D"EN-US">First question: do LUNs have to be
visible from RHE=
V-H or RHEV-M?</span></p <p><span
lang=3D"EN-US">Currently they are visible only from the hypervisor=
.</span></p <p><span
lang=3D"EN-US">----------------------------------------</span></p <p><span
lang=3D"EN-US">[root@pittor06vhxd020 log]# multipath -ll<br 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5<br size=3D650G features=3D'1 queue_if_no_path'
hwhandler=3D'1 emc' wp=3Drw<br
|-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive<br | |- 2:0:0:0
sda
8:0 &=
nbsp; active ready running<br | |- 2:0:1:0
sdd
8:48 =
active ready running<br | |- 3:0:0:0
sde
8:64 =
active ready running<br | `- 3:0:1:0
sdf
8:80 =
active ready running<br `-+-
policy=3D'round-robin 0' prio=3D0 status=3Denabled<br |- 2:0:2:0
sdb
8:16 &=
nbsp; active ready running<br |- 2:0:3:0
sdc
8:32 &=
nbsp; active ready running<br |- 3:0:2:0
sdg
8:96 &=
nbsp; active ready running<br `- 3:0:3:0
sdh
8:112 =
active ready running<br 3600508b1001035333920202020200005
dm-0 HP,LOGICAL VOLUME<br size=3D205G features=3D'1
queue_if_no_path' hwhandler=3D'0' wp=3Drw<br
`-+- policy=3D'round-robin 0' prio=3D1 status=3Dactive<br `- 0:0:1:0 cciss!c0d1 104:16 active ready
running<br
------------------------------------------------------</span></p <p><span
lang=3D"EN-US"> </span></p
<p><span lang=3D"EN-US">Our SAN device is Clariion AX150. Is
it compatible =
with ovirt?</span></p <p><span
lang=3D"EN-US" style=3D"font-family:"Courier
New"">vdsCl=
ient -s 0 getDeviceList</span><span lang=3D"EN-US"> gives
me:</span></p <p><span
lang=3D"EN-US" style=3D"font-family:"Courier
New"">Error=
block device action: ()</span></p
<p><span lang=3D"EN-US"> </span></p <p><span lang=3D"EN-US">Could it be due
to SPM turned off? (I have only one=
host)</span></p <p><span
lang=3D"EN-US" style=3D"font-family:"Courier
New"">[root=
@pittor06vhxd020 log]# ps axu | grep -i spm</span></p <p><span lang=3D"EN-US"
style=3D"font-family:"Courier New"">root&=
nbsp; 16068 0.0 0.0
7888 &nb=
sp; 868 pts/1 R+
00:04 0:00 g=
rep -i spm</span></p <p><span
lang=3D"EN-US" style=3D"font-family:"Courier
New""> =
;</span></p <p><span
lang=3D"EN-US">How can I turn it on? I know the command but I don&=
#8217;t know what paramaters append</span></p
<p><span lang=3D"EN-US"
style=3D"font-family:"Courier New"">spmSt=
art</span></p <p><span
lang=3D"EN-US" style=3D"font-family:"Courier
New""> =
; <spUUID>
<prevID> <pre=
vLVER> <recoveryMode> <scsiFencing>
<maxHostID> <ve=
rsion></span></p <p><span
lang=3D"EN-US"> </span></p
<p><span lang=3D"EN-US">Thank you very much for any
hints.</span></p <p><span
lang=3D"EN-US"> </span></p
<p><span lang=3D"EN-US">AS</span></p </div
<br <br <div
align=3D"left" <p
style=3D"font-family:Calibri,Sans-Serif; font-size:10pt"><span
style=3D"=
color:#000000; font-weight:bold">Alberto Scotto</span <span
style=3D"color:#808080"></span><br
<br <span
style=3D"color:#000000"><img border=3D"0" alt=3D"Blue"
src=3D"cid:dfb=
35e5d2df7434e8338047441010e53" style=3D"margin:0px" </span><br <span
style=3D"color:#808080">Via Cardinal Massaia, 83<br 10147 - Torino - ITALY <br phone:
+39 011 29100 <br <a
href=3D"al.scotto(a)reply.it" target=3D"" style=3D"color:blue;
text-decora=
tion:underline">al.scotto@reply.it</a
<br <a title=3D""
href=3D"www.reply.it" target=3D"" style=3D"color:blue; text-d=
ecoration:underline">www.reply.it</a
</span><br </p </div
<br <hr <font
face=3D"Arial" color=3D"Gray" size=3D"1"><br --<br The
information transmitted is intended for the person or entity to which i=
t is addressed and may contain confidential and/or privileged material. Any=
review, retransmission, dissemination or other use of, or taking of any ac=
tion in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If=
you received this in error, please contact the sender and delete the mater=
ial from any computer.<br </font </body
</html
--_000_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_--
--_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_
Content-Type: image/jpeg; name="image002.jpg"
Content-Description: image002.jpg
Content-Disposition: inline; filename="image002.jpg"; size=28792;
creation-date="Thu, 23 Aug 2012 14:54:39 GMT";
modification-date="Thu, 23 Aug 2012 14:54:39 GMT"
Content-ID: <image002.jpg(a)01CD8150.7C76A710
Content-Transfer-Encoding: base64
/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAoHBwkHBgoJCAkLCwoMDxkQDw4ODx4WFxIZJCAmJSMg
IyIoLTkwKCo2KyIjMkQyNjs9QEBAJjBGS0U+Sjk/QD3/2wBDAQsLCw8NDx0QEB09KSMpPT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT3/wAARCAI1AvMDASIA
AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA
AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3
ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm
p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEA
AwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSEx
BhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElK
U1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3
uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD0FgGU
huQetVjp6Ek+dcj2Eppby9iskDzMFT1IJ/lRb3sdzbyTR/Mqc5HfjP8AWgBv9np/z3uf+/po/s6P
/nvc/wDf01QS4e5MZa9e3mcbgpOFHoMd6nuNUlg0hLoRoXMgjcsSET5tpc4BO3im7rcSdyx/Z0f/
AD3uf+/po/s6P/nvc/8Af01H/a9tDb77qaNWXAkMeWUEruyDjkY5zUba/ZxzbZRNFH5Hn+ZJEygD
dtA5HUmi4y8lvGkHlDdt65J5JznOah/s6P8A573P/f01WHiPS2SNluciToRGxx823njj5uOe9XrW
6ivbdZoGLIxI5UqQQcEEHkEEHikBF/Z0f/Pe5/7+mj+zk/573P8A39NW6KLgVP7Oj/573P8A39NH
9nR/897n/v6at0UXArxWUcUok3yuy9N75AqxRRQAVBNaJM+4vKh77H25qeigCp/Z0f8Az2uf+/pq
yqLHEEQAKq4AHYU6kPQ/SgCrEA3lKehGfyFWHcqVCgEnPXgVBB96H/dP8qfOcPHj/a/lQA7e56eX
z/tH/CjdJ/dj/wC+j/hXKvourW8MaWE6IqRbQAxDKSwLKDxweoPbmp47LXvtcBkvSbcBBJ8/zHj5
+AMcFRj/AHmp2A6Lzjv2Zh3/AN3fz+VLuk9I/wDvo/4VhXOn3MuoTyQ2sEbvKsiXhILrhQNuMZ/X
FQxafq7qvmXdxEFwQv2ksc5Xdlsc5G8j0yKLAdGXkAyRGB6lj/hTZZ2hjLyKu0YzgnPJx6Vzb2Gu
G4UeeWgEXlkPLncfUj16c/rV2O3ure2n+0zyyiQIfncttbdzj0GMUWA2ndg5VQuQMkscUbpPSP8A
76P+FVtRjaaO4hTG6SEqM+pzWPJYaosjzJeT8AlYxKdv3hgbcf3c8euKEgOh3Sekf/fR/wAKTfIO
0f8A30f8K5iK01xreFhLOpODh7o5Rvl+ZuPmXr8narltp93F9oM0sszz2vllpJiwD88AdhzRYDaS
VpBlDCwHGVfNLuk9I/8Avo/4VzH9magih7W1SzGxUeG2uBGZCA3zbguByRx1OKsLp+rCQSNfSs+Q
xHm4TIYcbcdNu7j1xTsBv75M4xHn/eP+FIJiJkicDLgkFTnpjP8AOubs9O1toQl7dupMgJZZsuFy
uQG/BumOvataxWaOSyjuXLyrHIGYnJPTv3pWA0mbaBgZJpvmE9l/Olk6r9f6VUWGRUwBg55wQMj6
4pAWvMb0X86VnwBgcn1qrFC8ZBboFI4NWG6J9P6UAL5hPQL+dG9vRfzqr9nkQDYRnABxxS+VP3kO
frQBZ3t6L+dHmEckDH1qsI5/7/Puc59/anIkq7/MbIPSgC3RRRQAUUUUAFFFFABRRRQAUUUUAFFF
FABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU
AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAGcd/wBaKKKAIbi3FwoHmSRkfxRnBot7dbeMqGZyxyzO
clvrU1IThSfQZoAxbzw0t5eJMb2dI0wBEoHAHYHtWm9oDarBBLLbhcbXiI3D8wR+YquurRtcCEB9
+4KcxsByQOGPB61k+MtQvLNLKO0ZlWV28wocNgYxgj60272TFZLUvwaKIrtjgLBHb+REdwZmJzli
CMA8kd+p7cUreHLR7dYGknKBChyy/NlgwJ4xkMARjA9sVjeHdQvP7Ygt5Wu2inVy32gcZUZG05Nb
H9p3cc8ssi232KO6Nu2AwkUZwGz0PJHGBQ1Z2GTpo0CIFLyMQqrn5VyA24cKAOvtVq2tktYmSMsQ
0jyHce7MWP6mskeJ7a8aBNLPntJcLC5KEhVKlt3HqFOP/rVbtdfsLzf5Mj5jLBlZORtGT+hpAaNF
ZUOtiXUUthBIyyOypIqkgAKGy3p1rVoAKKKKACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lUsqRzJ
slj3r6GoofvQ/wC6f5VLmmgIPsFn/wA+ifl/9ej7BZ/8+ifl/wDXqYH97/wH+tZFn4ntblUeeKW0
jlhaaN5iuHRTg9CcEZHB9aYGj9gs/wDn0T8v/r0fYLP/AJ9E/L/69VX8R6Sixs1/CBIMr15GcdMc
c8c96eNd01hBsvIm+0AmLGfnx/LkY5oAn+wWf/Pon5f/AF6ctlaIwZbVAwOQcdKqQa/p1xLDCt3G
J5gpSPOScjIGRx0B79qrzeKLKHVpdOdZftEUsUZGBg+Z0I56DvQI15oYrgATQhwOmah+wWf/AD6J
+X/16qnxFpIjZzfw7VbYTz1wTxxzwCePSgeI9JMPnDUIfL3bd3PXG706Y5z0xSGWvsFn/wA+ifl/
9ej7BZ/8+ifl/wDXqcMGAIIIIyCO9LmnYRX+wWf/AD6J+X/16PsFn/z6J+X/ANerGaM0AV/7Ps/+
fRPy/wDr1JDbwW7EwwBCeCQKkzSZosA4kEYKk0mE/wCedJmjNKwwwn/POnEgjBUmm5ozRYAwn/PO
jCf886M0E8UWAMJ/zzpcIP4KoXmp/YPscS28txLcsUjSMqOQuTksQOgplv4h024jtiLlUe4XKRvw
w6gg9gcgj3xxRYDU3/7JpN/+yazIPEek3MyRQahC7vwoGfm4z1xjoD+VH/CQ6aVDrdxtF8+584C7
Rk5B56c8UWA1N/8Asmjf/smqC6rBPYy3ViftaxEhljIByOo+bGDUFl4gtLu1SaU/ZS6owjlYbtr/
AHDwT1wfyosBrb/Y0b/9k1lr4i0l7d51v4TEmNzDPfkcYyRweR6GtBJFkjV42DIwBVlOQR60WAk3
/wCyaN/+yaZmmyH5PxH86LAS7/8AZNG//ZNZt7rMVnqVvZGGaWadS42beFBAJ5IJ65wMnANRW/iX
S7iON/tSx+a7IgkBBJDbc+wzjk+tFgNff/smjf8A7JrMHiHSiAft0WC/lg4OC2cYzj14z0pTr+li
SVDfQ74QS454wQD255IHHeiwGlv/ANk0b/8AZNZl9rUNnpX9oxRvd2w5LQsvA6Z+YjvxjrQ2u2EM
Je7uI7ZlZlkSRgSjKASDjI4BB49aLAae/wD2TRv/ANk1nf27pnlxyfbYgkrFUJyMkdR046jr61fz
RYQ7f/smjf8A7JpuaTNFhj9/+yaN/wDsmmZpc0WAdv8A9k0b/wDZNRjmZR6j+orJsvE9ndxTzMPI
ghYqzySITkNtxtUlhk9MjmiwG1v/ANk0b/8AZNZb+ItJQKWv4QGQSDqflJwCeOOQRz3pY/EWlSo7
R30TKg3NjPAzt6Y554470WA09/8Asmjf/sms601uyvr021rL5r+SswZVO0qSQOfwq/miwh2//ZNG
/wD2TTM0ZosMfv8A9k0b/wDZNVp7mKzguLid9kUS73b0AHNU11yGOES38E9gjFRGZwD5hYZAG0nn
jp1FFgNXf/smjf7Gs06/parMxvoQICBIcn5cnHpzzxxnmnxazp89zHbxXkTzSoHRATllOcH9D+VF
hF/f/smjf/smseTxBDDfSQSW8yxRzpbvcZXYHYZUYznHIGcUsniXS0aIC6VzJOIPkUnaxGeeOBjv
RYZrb/8AZNLv/wBk1Sh1ayuLWe5guUeG3z5rDPyYGefw5qCLWozbme6t57OHarJJMBhw3TG0k59u
vNFgNTf/ALJo3/7JrM/4SHSvMVBfws7LvULk5GNwxgegJx7Uy38S6Vc20MwvEQSxecqyAqQuMnP4
A8e1FgNbf/smjf8A7JrMu9ctLbSRqKsZrdmVUKcbixwOWwAPc8VGPElgJpIrh5LaSKJJZFlQ/KGP
A4zk/SiwjX3/AOyaN/8AsmsqLxHpk161ql0vmAJgkHa+8EqAe/ANSwa3p11cRwQXkUk0qCREUnLK
QSD+QNFgNDf/ALJo3/7JphPBrN1PWotIS0EsbO1ySifvEQAhcnJYgCiwzV3/AOyaN/8Asmsxtf02
MSebdxxtEwSRTklGIzt44Jx6Z9elTRarYz3f2aG6iefbuCKc5GAevToQfxosIu7/APZNG/8A2TWQ
fENql9eW8qyRx2YzPcNjYnAPIzuA54OMEg1L/b2meXG/22LbI/lqTnlsgY6cdR19RRYZpb/9k0b/
APZNZza7pq+fuvYv3B2yYydpzjHTnnjjNXIpo54klidXjcblZTkEetFgJd/+yaN/+yaikP7s/h/O
n55osIdu/wBk/pRTc0UWGSUUUUgKq6fEs4k3zNg5CNISoP0qh4g0E64tvtnWJoST8yFgQcehHpWz
RQJq+hytvpNn4avYL+/vLeJV3RqVjYbiw6HJPapP7U8NfanlfVfMDyGUwvIxjDngnbioPiJ/yB7T
/r4/9lNc7DZQLoVvLJbRMZot244yTz+P5VrGCkrsm7vZHUQ6h4ZhgSEapvijbciO7EJ8pXA46YYj
8qgSbwskRRNWkUnq4lYNjbt2529McY9hXEaXBBcsVuGO7y8xoHCeY3HG48DjP1q1PopSKORJ40Dq
7usx2mIK2Pm4+g471TpxI9pI7dNW8NxSxyRamEaN942s3PyhcHjkYA4q7/wleh/9BKH8m/wrzg6H
cgoGeAGRyqfMTuwAcjAPGCDmlfQLuN3SRoEdS2FMnL7Rk7eOaPZx7h7SXY9HXxVojMFXUYSScAYb
k/lWt3ryGfSLrTZ7V7lUAeVRgNkg5Bwfz7V6+33j9aznFR2LhJy3EoooqCwooooAKQ9DS0h6GgCr
D96H/dP8qkzUcP3of90/yqby1zjfyOcZFNMQwH97/wAB/rWEvhO3+ypBNdzyrDH5duSqgw/MGyMD
5jkDr6V0PlDruNHlD+8ad0BgHwrbO8sklzO8szb5Xwo3t5gkzgDA5UD6Ux/CNs9zDKbu4xFJ5gQh
SM7y/GRxycHHUAV0XlD+8aPKGcbjn0oug1MO18M21oYtk8zeW8LjOOTGpVe3+0c0l54Wsr7UVvpX
lFwtwtwrLgYKrt2/Q4B+ord8of3jSKiOu5JNw9VIIougOdtvB9tbSFzd3EjZBywUHhWXkgcnDHk8
02TwXYyCMmeXfGqIGZEbhUCdCCOQPwPSul8of3jQYgOrEflRdBqRqAihVGAowB7Uuaf5Q/vGjyh/
eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0AzNGaf5Q/vGjyh/
eNF0AzNGeKf5Q/vGjyh/eNF0Bm3unG+NnLHcyW01qxeN0VW5K7TkMMdDVIeE7NAqRTTpCRH5sYwf
OZCWViSMg5JJx1rf8kf3jR5Q/vGi6AwI/CtrGqBbm4+URAEEA/u1ZQenB+Y1BF4KtI4JIjdXDeYG
DNtUE7k2E8DrjnPrXTeUP7xo8of3jRdAULfTo7ZbwI7n7W+984+X5QvH4Cs0+G/9P051kHkWFqYU
yfmkbGFLDGMDk/jXQ+UP7xo8of3jRdAcpb+CYjY2qXt00l1BGkayCNWVAob5QpGCPmPXnpXSW8K2
1tFBH9yNQg4A4H0qfyh/eNHlD+8ad0AzNNkPy/iP51L5Q/vGjyR/eNK6Ay9U0dNWkhM1xKsUTK/l
qqkEqwIIJGVPGMg9Kp/8IrB8wN3cbJMrIuFG9C/mbCcdAxPPXBxXQeUP7xo8of3jRdBqc7P4Rtp7
S2tjd3CxW5YhcKQxZ9+cEcEHgEc4NUo/B0087pf3ERtU3/Z1jXJTMokGQRjjHQ5zmuv8of3jR5Q/
vGi6DUy30aF9COlGRxEV2l1VVY85JwBgc+1ZuqeGZdRu9VuBJFG11AsMIySFORucjHUgKO/Sum8o
f3jR5Q/vGi6A52TwhaTXK3Ms8j3BdnkkaND5mSpxgjC/dAyOetb+af5Q/vGjyh/eNF0AzNGaf5Q/
vGjyh/eNF0AzNGaf5Q/vGjyh/eNF0BGGxMp9B/UVmroMEemx2kUsiGKf7RHMAu5X3FgemDyT1rW8
kepo8of3jRdAYEfhW1jjlUXFwxmRFkY4yxWQyFunUsT7elJJ4Ut5FO27uEYlzkbT96TzCCMcjPGD
1FdB5Q/vGjyh/eNF0Gpj6ToMWkSI0NxK4WAQFWVQGAZmB4HB+YjjitXNP8of3jR5Q/vGi6AZmjNP
8of3jR5Q/vGi6Aq3NvFe29xbTruimTY4zjIIqmNGZ/sxub+5nNtMk0e5VUAqCAMAe/J61reSP7xo
8of3jRdAc83hK1eF4nuJnj8tookZUIiRn3kdPm59aW18J2lpe2Nyk87PZR7E3YywwRyepHPTpwK6
Dyh/eNHlD+8aLoDGl0CKe8mlkuZjDNMtw9vtUKXUAKd2M4GAcetVbfwhBbDKX10ZDKshkwoYlVK8
4HJIY5PXPNdH5Q/vGjyh/eNF0Bg2HhWzs4J4pZJrkTEFix2Z+Xbztxkkdc555p0/hwXditrdX9xM
sTI0JdExHs6cYw3HBzW55Q/vGjyh/eNF0BiReGrWJlKyygCSOTACgZRSo4A4+8aqJ4Ls1GGuZXzC
IWLRpuOFKAg4yCAe3XFdN5Q/vGjyh/eNF0BRnsfO01LNJ3jCKqbwqtuA4wQQQQR1FZR8IW4t0hiv
blI1jRCuFIfaWIyCOnzHjp09K6Pyh/eNHlD+8aLoDnU8JQRJEsV7cL5Qi2napwYwQD07hiCKfaeF
LSzvrG6SeYvZxCJMhRuGMfMQMn6dK3/KH940eUP7xougGE8Gqk9hFdz2U8hO61LMi4BDbl2nNXvK
H940eSP7xougMG88L2t4koaWQM9010GKK2xmTYQARgjA71JYeHLTTtTe9tyQWB+QonBICk7sZ6L0
zjk1teUP7xo8of3jRdAc7rnhj+1vtMoun894XihDqAse4AHJA3EexzyabceD4LpzJNe3DSPJ5kh2
ph2+XBxjAICAZHOCa6Tyh/eNHlD+8aLoDn5fCsE1sLdrucwxy+bBGyIyxNuLHgj5vvEc9q2LS3Sz
s4beP7kSBRwBnHsOPyqx5Q/vGjyh/eNF0BFIfkP4fzp2af5IP8Ro8of3jRdAM/z0op/kj+8aKLoC
SiiipGFFFFAHO+M9LvNW023isYvNdJtzDcBgbSO9c+NI8SDTIrL+zYykS7FfzwDj3Gea9BPQ/Sq4
I3R4ckFcn56uM2lYnl1uec2vhTxBaNmKxQ8Yw7RsPyJqz/Ynig532qSZ3Z8wxtkMckHPbPOK7PVZ
bqDTHmsrlYnjbq0YkDAsBjk1DL4itrK7lspvPnlt0PmSxxYBYLuIx2OPw5xmn7Rk+yRyX9ieKfMD
m3ViN3DNGR8wAIxnpgAYq1FpWviKVp7FpZ2DBPmiCpuULkHORx2FdJ/wkcK+Z51ndxCNXLFlU/Mq
7ioweTt59O2c0Sa6VuYkaCSEL5hmSXaGwI96kHOOf84o9o+w1TXc45/DXiC5uopLiyj3K6szgxhm
wRySDzXpJOWNYS+KoWVZxbTC0EMkkshxlChxtxnJ/D1qa08RQXk1tGltcI05YAyBVC7TjqT8x74X
JqZSchxiomvRRRUlBRRRQAUh6GlpD0NAFaD70P8Aun+VZWqWlyNQvL21id3W1WPaBnzlO/co9xkE
f/XrVh+9D/un+VWqAMK0k1BL+CJjcBV2qYjABEItg534zu3ds/h3puqSauguHtJJFTzxGuId2yPb
94AAlvm46Hit+igDnY7vVo5pDIbidVSJ8x25VAMqHAUruLYyeCfoDUFyl/ekXOL2DDLhkiw+wTA4
2kH+HnGK6migDHtJLy4g1OKZrjcNwilMewcg8ICoPHHJznsapWaajaC2tYjcAoEVY/s6iJlK5dmY
Dhsk8cdBwea6WigDn4rnVb4wov2q2UiNZXa3AIbDb8bhjqBzyKhZtSubi0W5+15WSFvLWAeWyj7z
M2OGz2z+FdNRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR
QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA
BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRVS9u5LbHlwyTE/wAMe3P15NAF
uqcctjd3LLHIjzRnGAxBH09fwqC4uXm0K+kIKOscgHYjC+3eq+23uDFDOghWJQyBOOeeQw5HSqSA
15oI54jFKgaNuqnv3qGTTbWWeWZ42LzDEgEjBX4xyoOCcd8ZrOt764lsreOOch57qSBZ3UMwVS2O
OhbA71YS9lttVSyndp9yqqvtCncQxJOP93tUgS3mk291azxhFR5Q/wA+M4Zl2k4+lIND0/7OIWtw
688s7FskYJ3E56D147VRn8Rt9nd7ayYv9nE8fmyKokz1C+uO5HSnyeJUjlnjNhdsYAQ7KuV3jGU3
dM84znmgC4NFsAip9nyoLHl2Od33sknnPvSjSbTdAzJI5t23R+ZM77T68k5I7Z6UWmom5vJbaS3a
CSNQ2HcEt64A6gHjd0NXaACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsP3of90/yq1QAUUUUA
FFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAU
UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRR
RQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAH+etFH+etFABRRRQAVFPbQ3ShZ4lkAOQD2qW
igCL7NCLY24jXySpUpjgg9RWXLpV3blhZyLNG67dszYMfXGD3Az35rXkJWNiOoBNZ0N/O90sbwMi
l9u4yK24c84HI6d6abWwDrXSbe20hLG6KzpksxfjLE5JHpyeKlfSLCSEQvaxmMAADnjHTnOe5/Oo
tSl8uVcorqIycHsc4zjvT7CVTcNAkjuqRq3z9QSTSAX+xtO/584seV5OOcBMY2gZ449KUaTYrK0g
tUDuoVjk/MBjqM47DnrxWT/aM9lCt/PeSyxtPMj25VMbFLcpgA5AX1NI2vXc72Ttay2MBeRpTK23
eix7gQSudvPPA6cGgDcjs7eK6kuUiAnk4d8kk/4fhU9Ylt4lWaKZprSSCSFHZ0Zv7u08ZAPIYdqf
Zaldzar9nZEaAmUly2GXawAGMc9aPIDYooooAKKKKACkPQ0tIeh+lAFaH70P+6f5VaqrD96H/dP8
qtUAFFFFABRRRQAUUUUAFFFFABVZ7mR5zBaQmaRfvHOFX6mrB6Gjw+AdKWXHzSu7MfU7iP6UAV/J
1f8A597X/v8AH/CjydY/597T/v8AH/Cq/wDwmlmvjdvDU8LxTFQY52YbHYjIXHY4z+VWtD8SQ60m
puYvsyafdNbO0sgw23HzZ4wOaYDfJ1j/AJ97T/v8f8KPJ1j/AJ97T/v8f8K1kuYH8vZPC3m58vbI
Dvx12+v4Ugu7YqpFzAQzbFIkXlv7o56+1FgMryNY/wCfe0/7/H/CjydY/wCfe0/7/H/Co/EXiaTQ
7/TbK202S/udQZ1iRJljwVAPJbjv+lP0DxRFrVzeWU9pNYajZEefbTkEqD0YMOCPegBfJ1j/AJ97
T/v8f8KPJ1j/AJ97T/v8f8K1EvLWWJ5YrqB4k+86yqVX6kHApDfWmHP2u2wgBY+cvyg9CeeM0WAz
PJ1j/n3tP+/x/wAKPJ1j/n3tP+/x/wAK1Jb21gVGmureNXGULyqoYe2TzVSW/uo9eitfJtvsDW7S
vcNOodWB6bM5K479KLAVvJ1j/n3tP+/x/wAKPJ1j/n3tP+/x/wAK1Y7u2mk2RXMEjkbtqSKxx64B
6e9S0WAxfJ1j/n3tP+/x/wAKPI1j/n3tP+/x/wAK2qKLAYvk6x/z72n/AH+P+FHk6x/z72n/AH+P
+FbVFFgMXydY/wCfe0/7/H/CjydY/wCfe0/7/H/CtqiiwGL5Osf8+9p/3+P+FHk6x/z72n/f4/4V
tUUWAxfJ1j/n3tP+/wAf8KPJ1j/n3tP+/wAf8K2qDRYDE8rVx/y72v8A3+P+FIU1Yf8ALva/9/j/
AIVtGmNRYDGJ1Uf8u9r/AN/T/hSFtUH/AC723/f0/wCFajCmMKQGbv1T/n3tv+/p/wAKPM1P/n3t
v+/p/wAK0DTTSAo+ZqeM/Z7b/v6f8KaZ9SH/AC72/wD39P8AhWh/CaZjNJsRTE2pn/l3tv8Av6f8
KcH1Q/8ALvbf9/T/AIVaAwakU1POwKY/tU9Le1/7+n/ClCasf+Xe1/7/AB/wrRU1Ko9aXOwMsQ6w
R/x72n/f4/4UeTrH/Pvaf9/j/hWyDS0c7C5iGLVx/wAu9r/3+P8AhUcjarGpLW9tgekp/wAK6Dbx
UE0YINRKpJbBc577ffj/AJd4P+/h/wAKDqF+B/x7wf8Afw/4VZuIdjmoMetczxc0K5H/AGnfZ/49
4P8Av4f8KnjuNSlXckFsR/11P+FQOlOgmaFwR0qVjJdQuT+bqZBIt7b/AL+n/CmG51Fetvb/APfw
/wCFWp7yC3t/tEsixoOpJ/lXH6x4mkvi0VoDFCerdGb/AArpp1KlT4Qual14oe0mMTxRO467HJA/
SoW8YspGbcZPYGuXTLNx+dTJGAcnknvXYo9xcx0n/CXPj/j3X86cPFcrdLZf++qwEhLGrcVvinyo
XOzZTxHcP0tk/wC+jTjr1yP+XeL/AL7P+FZi/KwVBlj2FWVC2vLYeb07LT5ULnZebWLxFDNbwjPQ
Fzn+VM/t26/594v++z/hWezl2LMSSaYWJOF60+RB7Rmn/b10TgW8RP8Avn/CpV1W9ZSwt4QB6yH/
AAqjbwjG5jhR1Pr7Cpi4bk/Ki+lHIg52WDq14FLGCAAdzIf8Kpt4qkVseQh+jGsvUtS8xvLjOEHp
WenzHJo9mhe0Z0q+KJm6W6f99Gl/4SWf/n3j/wC+jWEp2jA61NCpdgBRyIPaM6C31a9us+VbRcer
kf0q2smqN0t7b8ZT/hUdjCLeAE4Hck9B71kw+PrKSUutvdfYVbabzC7BzjJXO4LnvWtLC1a1/Zxv
b+v6RXP3N0z3dsoe8t1Efd4m3Bfr7VbBDKCDkGpkIkXDAEHg9wazNIYtp6+isyj6AkVzNFpl6iii
kMKKKKACiiigAooooAKhjs7eGUyRQRI56sqgGpqKAIbi1iuk2yrn0I4I/Go7WxS1lllDF5ZMBmPH
A6CodRvJrZwIY1fPZphHj8T1qzZStPaiR+pLfkCQKAIYbbTre9kaGG3jupM7yqgM2euTT4dMsreP
y4bSCNOTtVAByMH8xxWVNd7LGBGRQ0sqDcDnJyDn61f1S6uLa9sFtoWn82R1aJXVNwCk9W44NVJW
Af8A2NpvkpF9gtvLRt6r5YwGxjP1qf7Jb+akvkR+ZGSyNtGVJ4JH1rGj16eK3zLAhMY3zCSYBwC5
UBAAdxGPbtSHXb2S6hjjt7eMi7MM0TyHeF2My/w4yduRgkds+kgdBRVLS786hatI6JHIrlHjVidh
HY5A557ce5q7QAUUUUAFIehpaQ9DQBWh+9D/ALp/lVqqsP3of90/yq1QAUUUUAFFFFABRRRQAUUU
UAI33TS+H/8AkCQfV/8A0I0jfdNL4e/5AkH1f/0I00Byt/4SuNZ8V+IXmikgjmht3sbvoFmTJDA+
x6/U1za+FvE114WuReWU6ztrP2q5t4wpadCoBZVb5W5ycHg17Mtq5GSQPal+yP8A3loA8o/4RfUN
N8O2t/o1lqL31pqJuIrW6jjSQIy7XAVPlVTwcD61L4b8H6lpviaztLqB20u0f+0BOwyrTtEqlfqG
yfwr1L7I/wDeWj7I/wDeWgDiPGNvfL4l8N6laabd30NjJK8y2ygsAQAOpH+RWJqnhnXvEMXiDV1t
WsLm9ijt7a0eQCRokYFtxHALYHFepfZH/vLR9kf+8tAHlEvh29u21i40nw/PpFnJpDWv2N1VWuJu
xCqSDj1NOtvAqrqun79EUW39iFJwUG03OON3q2f6eleq/ZH/ALy0fZH/ALy0AePXPh/XrnwvommX
OiySLFZSKzrbxSTRS7jtUmQ/IpGORz6VctPD2s79OeewuC0fhye0kLDkSncFQ+/I/OvVfsj/AN5a
Psj/AN5aHqC0PLtB8KXelaj4MuYNLe3kiglXUpFXBBI4DnvXpNWPsj/3lo+yP/eWncRXoqx9kf8A
vLR9kf8AvLSAr0VY+yP/AHlo+yP/AHloAr0VY+yP/eWj7I/95aAK9FWPsj/3lo+yP/eWgCvQelWP
sj/3lpPsj/3loArGmtVv7G/95aabFz/EtAFEimN1q+dPkP8AGv60w6ZIf40/WkBQNNNXzpUp/jT9
aP7Jl/vp+tJgUAPlNNArRGlS4Pzp+tJ/ZMv99P1qGmBR25FIAVODWgNKlH8afrTjpchAy6ZH1qHG
QFOOpgamGmSj+NP1qRdPcdXWp5ZdgIUFSAVKLNx/EtOFq/8AeWjll2AiqNhVr7M3qKQ2rH+IUpQk
+gjHu4cjNZrpg10slgzgjctU5NDlbpJH+tcFWhVbuoisYe3IxWZqeqQ6eMfflPRAen1rob7w5qck
W2zuLaMnqz7sj6cVz7fDbVHYs17aFickndz+la4fBOT5qunkFmcrd3s97JumctzwvZfoKbHCW5bp
XYR/Da+Tlru1J/4F/hUw+Ht93urb/wAe/wAK9eKjFWQrM5FY+wHFWYoc11C+Abxf+Xq3/wDHv8Kn
TwRdr/y8W/8A49/hVXRPKzm44QBzUihpG2RDJ9a6FvBd83AubcD/AIF/hUp8IXiReXBcQKD95jnJ
/SndC5Wc6WS1UrEd0nd/T6VCHz1PNdD/AMIRff8APzb/APj3+FNbwPfnpdW3/j3+FPmj3E4y7HPl
yxwvWrNvANpZjhR1Pr7CtuLwTdoRvuIMdyM/4VO3hK8YgefAEHAAzx+lHNEXLLsYZbf/ALKL+lZe
p6jtHlRnArrLjwffyJtjuLdVHru/wrJf4canJIWa8tOf97/CmpR7icZdjklJdsmrKfKK6ZPhzqC/
8vdr/wCPf4U//hXuo/8AP3a/+Pf4U+aPcXJLscyvJrY0u33yAkcCtOPwBfIctdWx+m7/AArWsvC1
xar800JPtmk5R7lKEuxC1ss1q8TglHUowHXBGD/OvPIfh1qkayad9ogWxknEj3CudzIBwuzGc859
M160mkyr/Gn61J/Zsn95K6MLmFXCqSpta9/LZmjhfcp20QiiRAMKoCgHsAMD9BWVo3/HgP8Aff8A
9CNbMitFJscYIrG0b/kHj/ff/wBCNcMi4l+iiipKCij/AD0ooAKKKKACiiigAooooAZJDFMB5sSP
jpvUHH504AKoCgADgAdBS0UAU10y3WdJcE+WcojHIU+oq05RR5km0BMnc38Pqc9qdWfrDBbeDdtK
m4QMHPykc9fxxT3AsJFZ3Qinjjt5dnMUgVW2/Q9qVbG0VCi2sAQtvKiJcFvXGOvvVPQ5RcSX7KoV
ftRUYHHCqD+uaz7fxFNdpBmK2xO0e0QzMTGTJtw/oe+O9DWtgN+GCK2jEdvFHFGOQsahR+QqSuXt
9d1GOJZbmSxkH2ZnKqxXL79vPpgckdhU1xrl80U0VvHaNLHHKzTROWQhQMFPU/N37ikB0VFclc3k
0TNFa33m2KsZPOlumBchFOwSDk8np+HtVy41m6uIVjgi8meOeFZQzkY3SLtU9/mU5NAHQ0h6H6Vg
jxK6SRxSwRGUiTcI3OAUZgR06tt+Ud+fSobTVbq61iJJri38pyhP2aUsmTGxC5PckcjvigDbg+9D
/un+VWqqw/eh/wB0/wAqtUAFFFFABRRRQAUUUUAFFFFACN9007w5/wAge3/3m/8AQzTW+6ad4c/5
A9t/vN/6GaaA6InAyaqWurWV9Dby21wsiXOfKIB+bHX6dO9WnG5GA6kYrlh4c1C1htxaG3OLVxLG
7HHneXsyOOh7/TPc0gOrorjIPCmoHThDNIQ0SS+Qv2k5jZnUqcqACQAeccZp954WvTd24gklFlFK
zJFFc7DHlgQ2WVs9+Bj60AdhRRRQAVHOxWFypwQKkqK4/wCPd/pQAeQP78n/AH0aPIH9+T/vo1LR
QBF5A/vyf99GjyB/fk/76NS0UAReQP78n/fRo8gf35P++jUtFAEXkD+/J/30aPIH9+T/AL6NS0UA
ReQP78n/AH0aPIH9+T/vo1LRQBF5A/vyf99GjyB/fk/76NS0UAReQP78n/fRo8gf35P++jUtFAEX
kD+/J/30aPIH9+T/AL6NS0UAReQP78n/AH0aPIH9+T/vo1LRQBF5A/vyf99GiLIeRckhSMZOe1S1
FH/rpfqP5UAS0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQA
UUUUAFFFFABRRRQAUUUUAZOqNi7X/dH86wtG/wCQeP8Aff8A9CNbGsNi9T/cH8zWNov/ACDh/vt/
6Eap7ImO7NCiiipKCiiigAooooAKKKKACiiigApCQBknAFLUc4LW8qqMkowA9eKAD7RD/wA9ov8A
vsUrpHcRFXVJI2HIPINcDLHfLDJYtC32ySbzI2JIKpjp+NdGZZ9J0ewupEkBhBSWLk/eztyPZtv5
1TVgNuGGO2jWOGNY0X7qqMAVFZ2MFhbJBBGFRR1IGT7k9zWCk19pmoPG0ks/mRRQbSSwWZgSG9hn
I/AVLoepTB7KxmlRw1vuDEl5ZCM5J5+UcdxznrUgb3kxZyIo88/wDv1/OlSNI1CxxoijgBVAAp1F
AEYghEYQQxBFOQoQYB9QPWnFFJJKqSSDyO46U6igBnlRjpGnXP3R19frSLDFGhCRRqM5wqADPr9a
kpD0P0oArQ/eh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt
9007w5/yB7b/AHm/9DNNAdHRSMcKSPSsG3v5LSG3la5a4M0O90kI+VuMYwOMk4xSA36Kw49dmd/N
aKOOEQsxRnO4OH246etTNqb3GlRzqDbtJMIXJ58v5tpP6frQBrUVkvqDWVxFaQpNesRuZywJUZx2
HNQya1cvDmK3jRmKlC8mQV8wIc8cf/XoA3KZIyrGxcZUDmsoa5K5kKWLFFcRo5fAZt23BOOP1q80
rS2MrOqqw3KQr7hwcdaAGeZB/wA+5/If40eZB/z7n8h/jWTcTSJNdOtwymEoEjyNrZHTHvUcupTz
wN5EDRKXVRIWx/Fg9utAG15kH/PufyH+NHmQf8+5/If41lxaqr3IiKfKWAWTdwQcjPI9qiXUZpZ4
2iCbJGQbWboDuzg49qANnzIP+fc/kP8AGjzIP+fc/kP8ajooAk8yD/n3P5D/ABo8yD/n3P5D/Go6
KAJPMg/59z+Q/wAaPMg/59z+Q/xqOigCTzIP+fc/kP8AGjzIP+fc/kP8ajooAk8yD/n3P5D/ABo8
yD/n3P5D/Go6KAJPMg/59z+Q/wAaPMg/59z+Q/xqOigCTzIP+fc/kP8AGkMtuBkwED6D/GmU1+i/
7w/nQBJ9otv+eJ/IVZtpI5EPlLtAPIrHhk1E6i8csUAtV5WRSdze2M9av20nk2tzIBkplseuBQBf
orCS5uFaaSR55EigViVdVGSuTxjNSDV5bdZjLGJFEkgQhxu+XsRjge9AGzRWfBqMs18bc2rAIdry
BshWxn06c1C07vHdzm6ZHiLqsQIAGBxkdz3oA1qKxn1t7e3PmwAzIASgfO5duSelPfW2QSP9nzGN
4Q+ZyxXHbHHWgDWorKfWJUkEIs2eYMQyo+QAMdDj/aFOsLmWS5CyOWV1c4OPl2uR/Ij8qANOiiig
AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAMDWzi/Qf7A/maytF/
5By/77f+hGtPXf8AkIpj+4P5mszRf+Qcv++3/oRq5fCjOPxM0KKKKg0Cij/PWigAooooAKKKKACi
iigAooooATA9Bn6UtFFABSYGcgDPrilooAKKKKACiiigApD0NLSHoaAK0P3of90/yq1VWH70P+6f
5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8AIHtv95v/AEM00B0dQLZW
qRuiW0KpJ99RGAG+o71DrLFdEv2UlWFvIQQcEfKa8SC679hW7Lal9mKh/N8x9u3pnOenvVRhzEyl
Y9zNlalAhtoSqggDYMAHqKetvCsJhWJBEeNgUbfyr5//ALSvv+f67/7/ALf40f2lff8AP9d/9/2/
xq/Y+ZPtD337Da7Y1+zQ7YzlB5Y+U+3pTmtoGQq0MZUjBBUYxnOK8FW51V7Z7hbi+MEbBXkEr7VJ
6AnNJDeancTJDBdXskrnCoszksfQDNHsfMPaHvS2lusjOsEQdsZYIMnHTmleEGF40AXdnoO5rwa5
uNWs5fKuri+ikxna8zg/zqH+0r7/AJ/rv/v+3+NHsfMPaHuh0/M3mmOAyD+PHP54oGnYZmEUAZjk
kDk/pXh8N3qlxIqQXN9I7MFASVzknoOvWkkvtRhleOW8vEkQlWVpnBUjqDzR7HzD2h7f/Za4YeTb
Yblvl6/Xinf2d8oXy4Noxxjjjp2rwz+0r7/n+u/+/wC3+NTiTWGaBVl1Am5GYQJHzJzj5eeaPY+Y
e0Pb/ss3rH+Zo+yzesf5mvD7ibV7SQpcT38TABiGlfgHoetQf2lff8/13/3/AG/xo9j5h7Q93+yz
esf5mj7LN6x/ma8I/tK+/wCf67/7/t/jR/aV9/z/AF3/AN/2/wAaPY+Ye0Pd/ss3rH+Zo+yzesf5
mvE7VddvomltH1GaNeGZJXIH15qqdRvwSDfXYIOCPPb/ABo9l5h7TyPdvss3rH+Zo+yzesf5mvCP
7Svv+f67/wC/7f41Z067vLrU7WCS+vNksyo2J3zgkD1o9j5h7Q9u+yzesf5mj7LN6x/ma56bQPD1
vfJZzandR3LgFYm1CQMc9OM98VcPgrTFGWuNQA972T/GstC9TV+yzesf5mj7LN6x/may/wDhCdN/
576j/wCBsn+NMfwdpMbIr3V+rOdqg30nzHrgc0aBqa/2Wb1j/M0jWkzDqg79TXPpoHh6S6lt01K7
aaJgsiC/kJUk4wefU1ai8HaTPEskVzfujDKst9IQf1osg1Nb7Jcf89E/z+FTW1uYo3VyG3Hmsb/h
CdN/576j/wCBsn+NUhoHh0zywjU7oyRYEi/2g+VyQBnn1IFFkGp1EVrFDD5SoCuNp3c5HofWlFrA
HZhBFucYY7BlvrWH/wAITpv/AD31H/wNk/xo/wCEJ03/AJ76j/4Gyf40aBqbvkRed53lJ5uMb9o3
Y+tI1tC0pkaGMyEbdxUZx6ZrCbwVpiqWa41EADJJvZOP1pI/Bmlyxq8dzqDIwyrC9kII/OjQNTee
1gl/1kET9/mQGlMERGDEmOeNo79awv8AhCdN/wCe+o/+Bsn+NH/CE6b/AM99R/8AA2T/ABo0DU3J
LaCX/WQxvzu+ZQefX60JbRxzNKq4Yrt9gM56fU1h/wDCE6b/AM99R/8AA2T/ABpkvgzTYo2fztRO
O322T/GiyDU6Siua/wCEKsv+et7/AOB0v+NcV45szoVzZrYXd6glVy4a6duQR0yfeqjFSdkxNtK5
61RXz5/aV9/z/Xf/AH/f/Gj+0r7/AJ/rv/v+/wDjV+x8yfaH0HRXz5/aV9/z/Xf/AH/f/Gj+0r7/
AJ/rv/v+/wDjR7HzD2h9B0V8+f2lff8AP9d/9/3/AMaP7Svv+f67/wC/7/40ex8w9ofQdFeA21xq
t5OIbW4vppWBIRJXJOOvemzXmp20rRT3d5HIvVWmbI/Wj2XmHtPI+gKK+e/7Svv+f67/AO/7f40f
2lff8/13/wB/2/xo9j5h7Q+hKK+fTqGoBQxvLwKehMz4P611/gqwj1iwu7jUry9PlSBQRdOoAxns
aUqfKrtjU7ux6nRXHy+HNJgz5t3epjHW+k7/AI1IPCmnsoKzX5B5BF7Jz+tZ6FanWUVyf/CJ6f8A
89tQ/wDA2T/Gj/hE9P8A+e2of+Bsn+NGganWUVyf/CJ6f/z21D/wNk/xo/4RPT/+e2of+Bsn+NGg
anWUVyf/AAien/8APbUP/A2T/GkbwpY5AWW/yfW9l/xo0DU62iuRbwnZqpJkvsDn/j9l/wAas+C4
zAmrW/myyJDfMiGVy5A2Lxk/Wiy6Bcsasu7VUJ6LGP5msnRf+QeP99v/AEI10OoRq1wuRztH8657
Rv8AkHj/AH3/APQjVSeiJitWX6KKKgsP89aKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig
AooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsP3of90/yq1QAUUUUAFFFFABRRRQAUUUUAI33TTvDn/IH
tv8Aeb/0M01vumneHP8AkD23+83/AKGaaA1da/5AWof9e0n/AKCa8ssta0+1tbC4e43vb6W9o9sE
bLuxPBOMbec9a9bu7cXllPbsxVZo2jJHUAjFcQPhNYAAf2ld/wDfKf4VcJJKzJkne6MT+0fCyzRt
5NsYxJmNRbHMaeURh+PmO/Hr60kl/wCGLqFEkENuwEDeZFachgD5nGMEdOOnpW5/wqew/wCgjd/9
8p/hR/wqew/6CN3/AN8p/hV80e5FpGXHqnhVZsOkHls0JZRAxUkKd/GB3x2FZK6npUXiTRL63CQx
xIjXZjiKgOCcnaB6Y6V1X/Cp7D/oI3f/AHyn+FH/AAqew/6CN3/3yn+FClBdQcZM53TdR8PzTzPq
EFrEft5lyYWbzIiDx3Oc4ODipIdR8Lm3jjubeJlSOEny4CHLhzv+buNuK3v+FT2H/QRu/wDvlP8A
Cg/CjTwCTqV2AO5Cf4Uc0O4WkYWn65psEF5a3M1n5T3cUymK1Ko0YPIwFzuA9fzpJdQ8MvuIihMZ
P3DCTJv83O7dj7uzt+GK2P8AhWekf9Bmb84/8KP+FZ6R/wBBmb84/wDCjmh3C0jmNcm0bUjax6f9
nt3N1IjusXlqsJYbCeOw/Gm3OtAeLEubC4jggs18i2eRCybFUgZA5w3P511P/Cs9I/6DM35x/wCF
H/Cs9I/6DM35x/4U1OPcOWRkDWdCitdQtIQqwXHkO6LG212DZkVCRkD0z3z0pJdS8OJ57JDaSTeX
P5JW1IQZI8pSpHLAZ5/Wtj/hWekf9Bmb84/8KP8AhWekf9Bmb84/8KXNDuO0jKguPDjw2sSolzcg
sdwtDnJjONyKvIDdhn1qzfvoGl3JintLI3vkwl1eJhHyp38KCQ3IPQdulXV+GulI25NbnVh3VkBo
Pw00kkk61OSepLISaOaPcLPscbouoWtpY63HO3ltd2pihXBOTnOM/wCNdVY2+kyDSw+nWyXN7bvN
MkkPEXlxsM7fRmGffFT/APCs9I/6DM35x/4U4/DbSy2465cFsYzvTOKHOL6gotGOup+GVjYi2s5L
oJEHeSFlhkOPnKgKSDn2GaxYZIZPFGnNbmAp5kI/cxGNQcjIIPU+p712H/Cs9I/6DM35x/4U6P4b
6VDKkketzq6EMrAx8Ed+lPniuorM39StL2W/1CGKwM0V7FEizM6COMjdksM7uMgjAP4Vmz6VrN3N
qIuUvGtmKtHEtzjLLICNh35GVGedoz2q7/Y1z/0N17+cX+FH9jXP/Q3Xv5xf4VhZdzS5Po8GrJr9
5NeCZLSRSEjeUOoIb5SvzE8ryeBz69a0vJefWTLIpEVtHtjyPvO33iPoAB+JrG/sa5/6G69/OL/C
j+xrn/obr384v8Kdl3C5PBpeoW5upEnmzNfeYId67BHvBJGBnkZyCaztO0jV0t47c/bbaIGNZg10
Duw+WMeD8q7eOME+meatf2Nc/wDQ3Xv5xf4Uf2Nc/wDQ23v5xf4UWXcCbQ7LVYdUvX1G4umRshAx
UxY3fKV+YkHbjPyr+NQmwv5vDw0lrF4pITGPtCyJtkxIpLLznJAJOQOfWj+xrn/obb3/AMhf4U9P
D99IuY/FOoMPVREf/ZaLLuBVnsdejubWO3NyYoLklZDcbt8XmDiTLDPy5wcMfpU0Ok6sqRRyXV4V
lEbXDG5+YEOdwU5+UbfT+dTf8I3qP/Qzal/3zH/8TR/wjeo/9DNqX/fMf/xNFl3ApHTNeN/bLJc3
f2aNNoCOrA8sD5hLAnI284Y/Q102mwvb6Zawyja8cSqwznBArG/4RvUf+hm1L/vmP/4mj/hG9R/6
GbUv++Y//iaVl3C50VFc7/wjeo/9DNqX/fMf/wATR/wjeo/9DNqX/fMf/wATRZdwudFUN1/x7PWH
/wAI3qP/AEM2pf8AfMf/AMTR/wAI3qP/AEM2pf8AfEf/AMTRZdwuMPhHSj4r/t3zp/OxnyPOPlb/
AO/tz19uneuS+KJzeaeR3WT/ANCFdh/wjOof9DJqH/fEX/xNUNR+Hw1Z0fUNavp2jGFLKnA/AVUL
Rd7ildqx5JRXp/8Awqew/wCgjd/98p/hR/wqew/6CN3/AN8p/hW3tYmfIzzCivT/APhU9h/0Ebv/
AL5T/Cj/AIVPYf8AQRu/++U/wo9rEORnmFFen/8ACp7D/oI3f/fKf4Uf8KnsP+gjd/8AfKf4Ue1i
HIzhvC9/b6Zri3N222IQyqeCckoQBxz1q/pt9oK6YPtdtbpJ++82IRMWckDy9jc7QDnPNdV/wqew
/wCgjd/98p/hR/wqew/6CN3/AN8p/hSc4vqNRkjETV/DYv7nNvZ/Z99uIf8ARP4f+Wvb+f4UsWoe
F5L2CeUW8aoHDxC1+SQeZ8uflPIT069M1tf8KnsP+gjd/wDfKf4Uf8KnsP8AoI3f/fKf4UuaHcLS
MG+1nSrnS4bW3mtES2FwqJPaM/BbKFcdDjuT16itL4axiXRdQjf7rShT+K1c/wCFT2H/AEEbv/vl
P8Kvaf4CbSkdLDXL6BXOWCqnJ/EVLceWyY0ne7LiR3cVszGIvcSNsYqQSigYyMnqf61fhQRwoiqV
VVACntVD/hF7/wD6GTUP++I//iaP+EXv/wDoZNQ/74j/APiazsu5dzSorN/4Re//AOhk1D/viP8A
+Jo/4Re//wChk1D/AL4j/wDiaLLuFzSorN/4Re//AOhk1D/viP8A+Jo/4Re//wChk1D/AL4j/wDi
aLLuFzSpjY3jdjBBHP4VQ/4Re/8A+hk1D/viP/4mk/4Re+PXxHqH/fEf/wATRZdwuPtdLtrGS4li
d2aQHAaTIQegpPCf+v1z/sIt/wCgJTf+EWvv+hj1D/v3F/8AE1oaFog0SC4T7VLcvcTGZ5JQASSA
O30p6JAS3ozOv0Fc3o3/AB4f9tH/APQjXTXf+vX6VzOj/wDHh/20f/0I0PZCW7L9FFFSUFFFFABR
RRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFIehpaQ9D9KAK0P3of90/yq1VWH70P+6f5V
aoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8ge2/wB5v/QzTQHQu2xGbrgZ
rITxHCttZ3F0vkxXFp9pZsltn3AFwBknL1sEAgg9DWIvhiNYHia8uJE8ryYRIsbCFMg4AK4PQctk
8CkBdXW9Pa9itPtIFxKoZI2Ug8jODkcHHODz7VUk8TWkV9PZvJCt1HcpbrCZgHcMEO4Dr/F+lJZ+
FrWy1GK8jkd5EVVbzUR2chdobeV3A4x0IHFXf7Jg3TtufM9wtw3ThlCgAe3yD9aAIv8AhI9L2zN9
qGISA2Eb5snA28fNzx8uaLXXrO7v57RDKskTqnzxMoYsu4YyPSqzeF4mt3ha8uHTZ5cIdY28hCQS
FyvPQcnJGBzVqz0ZbKffFd3JUhNyMwO4qu3JOMnIxnntQBpVXv8A/jxl/wB2rFQXis9nKqgkleAK
AKs2r6dDfGzaZBcj+AqcAkZAJxgZHT1rPPi3TkuRbvHKZPNiiJjhZkJkXcCDjkUPpq3OrXNxcT3I
tpHR1gWM7WZVwCflyMHsDjinf2PahgY7m9TBhZQIs4aMYB5XuOD/AEoXmBa/4SDSdsx+0piHG47G
wcttG3j5ueOM81Qm8UpbI081ogtpA/2Vt/zSFWC4ZSPlySMdfcdqki0e1R1MlzfSrER5CvGcRKHD
YGFyeVA5ycClbSLR5ZWae8IJZoVMXFuzMGJX5fUA85oAns9btZ7AzStAJlwGSPcQGJIUfMoOcjHI
4NFl4i0u7s0nM0ceQNwZSApK7upHIwDz0OKhi0iySeGd5LuSZN5d2jP71m7sAuOMnGMAZpLjRbC6
gWGR7vyljjjK+WedmSh+71BOfqBQAt94osbW4WCBPtEmG3AArghcgZIwT0BGeM80kmsXkdrO50+2
M1pJsuEE5x0BGw7fmJDDggc0yTRLWeYtNeag8WXZYjHgKzjDMCFznnI54NK2kI0kMo1K/EqSGV3N
up818ABmBTGQBgYxigCzZas11qCwyWqRwTCQ27h8s2wgNuXHHXI5P4VrbF/uj8qybSytrS9a4Et1
IBu8qN4jth3HLbcLnk+pOO1aP2yL0l/79N/hQBLsX+6Pyo2L/dH5VF9si9Jf+/Tf4UfbIvSX/v03
+FAEuxf7o/KjYv8AdH5VF9si9Jf+/Tf4UfbIvSX/AL9N/hQBLsX+6Pyo2L/dH5VF9si9Jf8Av03+
FH2yL0l/79N/hQBLsX+6PyqK4VREMADLAdPej7ZF6S/9+m/wqOa5R48KJcgg/wCqbsfpQBi2nijS
r3xDNokVrfC+hyZA8DBUH94t0wex71s2X+tuP94fypn2lQ7OLdg7DBYRnJH1xUliDumYqwDMMZBH
b3oAt0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFF
FFABRRRQAUUUUAZuoSlLhQPQVz2jf8g8f77/APoRrd1Q/wClL/uj+dYWi/8AIPH++3/oRq5bIiPx
Mv0UUVBYf56UUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0P0oArQ/eh
/wB0/wAqtVVh+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33TTvDn
/IHtv95v/QzTQHR0UVnXjyS6jFaLctboYjIWTG5yCBgEgjA7/Wkyox5nY0aKyJdbNuHZod8Ks0Yk
Eg3M6rk/LjgcHnP4U281kw3I2jKRbgygj5zsUjnt96lzItUZt2sbNFZMGpTpa6hPcxgtbyYESODj
5QcBsD170h1qYFo/siidNxdTMAAqgHIOOT8w4x+NHMheyl0Neisy01OaaK9nkiURQndFhvmZdgbk
Y4PNQDXZ3mEKWDl/LDOwbKxkglcnH5/1o5kP2M3obVFYkOsXEsMQngCNthaR45R1c8AAj86IPEfm
J50tlNDbE/LK3RgQcdupxj8RS5kHsJ9jborLs9Yku9Qa2+wzIiZV5eqq4AJHTHfrn8KYt9N5O5pf
+X8xEkD7m8gD+VO6F7KS0Zr0VgTaldpfqYpN8UcsxeMAfMihOB7jcT+GKs2OrNNLbxsoaOYMVnLY
DfMcBRjk4HqKOZDdGSVzWorO1DVGsptiQCQLEZnJkC4UEA4Hc81Uj1yVbnyBby3DGV9zKMeWm7A7
dvfFHMgVGbV0blFYY8Rs3m4tPlRl/eFyE2HOHztzjj0/GrVjqzX17LEtpKsKFgs5+6xU4I6flyfw
oUkwdGcVdo0qKxn1S6mW1ltoF2SykIplHzrtb73Hy9O2aYviUOxZbKcwJHvkkHIQ7d2OmPbOaXMg
9hN7I3KKxjqtxHqtvbyxKrzxZWNZMqPm5O7Hp271Yu9WFpO8TwsSo3DBHKYJLfhjH4indCdKWxo0
VlRa03nrDPbrHIefllDLjaWHOBzweKlsdTe+MhW32xoitnfksWUNgD8etF0DpSW5oUVh/wBuzkxS
NbxrEEkaZPM+dCoBxggc8/8A16nGsSiVYZLVFl3YfE2UVdu7O7HoOmKXMhujNGrRWHD4ie4LRx2f
75WPDSbVKbd24Ej09qltde+1XSRraSiIorSTE/LGSu7BOMYweufwp8yB0Ki3Rr0UUUzIKKKKACii
igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAytTG
bpf90fzrB0b/AJB4/wB9/wD0I10GoDN2n+6P51z+jf8AHh/wN/8A0I1ctkRHdl+iiioLCiiigAoo
ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tIehoArQ/eh/wB0/wAqtVVh+9D/ALp/
lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33TTvDn/IHtv95v/QzTQHR1BcW9
tdKEuYopVU5AdQcH8anqMnk0rXBNrVEf2S089pvIg81l2l9oyR6ZpsdjZRReXHb26R8/KqADng08
sQ6jsc5qC51CK1kCOrkld3ygYAzjuafKh80u5NFa2sMJhihhSI9UVQAfwpJrSzuMedBBJht3zKDz
0z9agh1OKeVI1SUF8gFgMdM9j7VO86RLLJM6pHGNzMxwFGMkk0WQc0r3uPENuHkcRx7pQA5wMsBx
z6002tq06TGGEyxjCvtGVHoDTWu4EMYaaJTKCyAsPnAGSR64HNRDVtPZCwvrYqFRsiVcYc4U/Qng
etFg5n3J47W1iTbHDCi5BwqgDg5H60i2VmnmbLeAeYwZ8IPmIOQT6nNRXWqWNkXF3eW8BQKWEkgX
aCcAnPqRgVLBcw3UCTW0qSxOMq6MGU/Qiiwc0u44W1stybgRRCcjBkCjcR9aYbCxLSsba3LTf6w7
B8/19alzRmjlDml3EigtoVQRRxIIwQgVQNoPXFQvp9k4GIYkIXaGRQCBnOAewqfNGaOUOaXcgmsL
W4vI7idUkaNdqBgCFOc5HvUj2lpI6O8ELMjFlYqCVJ5JHvT80Zo5Q55dyAabp4BAtLbBbeR5a8n1
+tSx21tFO80cUSSv951UAt9TTs0Zo5Qc5PdjI7W0ikZ44YVdm3FlUAk+v1pBZ2YlEoggEirsDbBk
L6fSpCeKq3Op2enwRyX91BbrJwrSuFDHGcDNHKg5pdyVLGyjiEaW9uqDooQYHOf580C1jN89zJJv
Yp5aqcYVe/1zUkciyxrJEyujgFWU5BB6EGnZ9qOUOeXcgGnWAg8kWtt5W7fs8tcZ9cetTJHDGpVF
jUEYIAAzxj+VVf7VsDPNAL23M0GPNjEg3Jk4GR26irfPpRZA5Se7IVsbJERUt4AqElQEGFJGDinQ
2lpboEhhhjVSSAqgAZ60/NIGJkYdgBijlDmk+pSutHtJrbyYEgt0Lbm2woc8Y7jg+9Trp1iuzNvC
zJGIwzKCduMYzUkNxFcpugkSVfVDketOeRY0LyEKg6sxwBS5UV7SdrXJNy+o/Ojcv94fnUMU8U6s
0MiSBWKMVOcMOCPqKV2wox6gfrTsQS7l/vD86Ny/3h+dVX1C0iExkuYU8ghZdzgeWT0B9M5GKsc+
lFgHbl/vD86Ny/3h+dQvPFFJGkkiI8pKxqxwXIGcD14FSc+lFgHbl/vD86Ny/wB4fnVC51nTrMsL
q/tYSjbGEkoXDYzg++OfpVtJEkjV42VkYZVlOQR6g0WAk3L/AHh+dG5f7w/OombDoB0JOfyqvHq1
hLfNZR3ts92uQ0CyAuMdeOtFgLu5f7w/Ojcv94fnTefSo1njeZ4VdGljALoDyuemR2ziiwE25f7w
/Ojcv94fnTGYIhZyFVRkk8ACoJ9QtLUKbi5hiDIzgu4XKgZJ57AdTRYC1uX+8Pzo3L/eH51VtL+1
1CEy2VxDcRg7S0ThgD6cVMzYRiOoBosIk3L/AHh+dG5f7w/Oql1qFrYW6zX1zDbxkgb5XCjPpk1Y
B3AFeQeQR3osMfuX+8Pzo3L/AHh+dRJKkjOqMrNGdrgHO04zg/nSRXEU6loZEkVWKEoc4YHBH1FF
gJty/wB4fnRuX+8PzpmaM0WAfuX+8Pzo3L/eH51E7FVyOuR/OnZosA/cv94fnRuX+8PzpmaM0WAf
uX+8Pzo3L/eH50zNGaLAP3L/AHh+dG5f7w/OmZozRYB+5f7w/Ol6jio80sTFlbPZiKLAUr7/AI+l
Poo/nXO6N/yDx/vv/wChGuh1I7Xz/s1z2i/8g4f77f8AoRpvZELdl+iiipLD/PWij/PWigAooooA
KKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tIeh+lAFaH70P+6f5VaqrB96H/AHT/ACq1
QAUUUUAFFFFABRRRQAUUUUAI33TTvDn/ACB7b/eb/wBDNNb7pp3hz/kD23+83/oZpoDo6gJ+Y1PU
Bjck9PzpoTGMf3ifjWbqTRi/j81gq+X378n/ABrUMLkg8ZHvS+XIPT86bsBmJNBJe2ogcMQTkD/d
apNXtZL/AEfUrSHb5s9u8SbjgZKEDP51f8uT2/OkELgk8c+9LQLnMXWn6vcxabOLGBZrKOSEwm5B
Dh4wu4NjjBHTFYd18PtRLWslrcQq9vFZwNluJo4zlwRjjBCkfjXonlP7fnR5T+3509NwOW1zQtQu
vELalaRwyIv2UqjS7CxjkZmHQ44IxWroGnzafaXP2nastzdSXJiRtyxbznaD39fqTWp5T+350eU/
t+dCsgDNGaPKf2/Ojyn9vzo0AM0Zo8p/b86PKf2/OjQAzRmjyn9vzo8p/b86NADNGaPKf2/Ojyn9
vzo0AQng1i6xaXk11pF3YwR3Bs5Hd43l8vIaMrwcHua2/Kf2/OkELhQBjA460aAcNZeDNUgvLVJ7
2Q2iw7T5NxsEOd+9QNuWGWGOR09hW74asNW02LZqkkc8kuWlmEpOCuFRVUjoVGSfUmt3yn9vzo8p
/b86NAONuvDurvf37WQhtredt4An3b2MqMWU7d0eVU7lyQTiqM3hTXJ0uyP3JnvGmaKK9wrR7SEC
kqduDhjxySfQV6B5T+350eU/t+dGgXI7dGitoo3cu6IFZ2OSxA5JNV9RgkurC9ghKiSWEohbOASC
BnFXPKf2/Ok8lwxPGT70OzBaHnDeD9TsDaWdvGkplW5LNC7wxxttVY2Y5OSMZA+tWLvwjrl7LqIu
ninSeHaM3BCysHjZSVxlThWHXGTwK9A8p/b86PKf2/OjQDltC8OXWl6z9sISNJXuWmRJmYEOymMY
6HAB5rppD8o/3h/On+U/t+dIYXI5x1z1o0A5jW/DV5qPiKO5tZoorKaNTdhvvGSPcYWA78kZ/wB0
VmWnhDVjYQw3s7sQ7tKpvCQzGEqGG0DGXIOOemetd35T+350eU/t+dLQLnCf8Iprj3scrzxi4QMR
em4ZmTMHlhQmOz5Oe+c1Z0zwxqCanZTXQWKxhkZzaC7eQIfLUA5/iywJwen412XlP7fnR5T+3509
AOTn0TU4PE02q28EFwhvPOWMzbCV8gR9SDzn9K2vD2nPo+hWtjK4eSJTuK9ASScD2GcD6VpeU/t+
dHlP7fnRoAxj86fU/wAq5O68IXVzFq8punFxLNcTWMauFRGkj2BywG4Hr3wK64wuSDxx70vlP7fn
RoFzjpvDGqJfRfYpxCiLH5c/2hiYAEIddn8W5jnJ9fYVQtvCeuWrefBHFHJiNXQXjHzCInRmLEf3
mDYr0Dyn9vzo8p/b86NAucO3hTVZbOe3uhHcPLYiBZmvHAjIh2lNvRgX+bJ9c1Z1vw1fajp+nxW0
UETW9hPbtG0uQHdFVQDjkZHWuv8AKf2/Ojyn9vzo0C5jaLYXUOpahf3kaQNdiJBAkm/ARSNxOByc
/kBWu5/dt9DTvKf2/OkMTkEHHPvRoBh+KNMutT0+AWMKPcxMXjczeWYyVIzyCGBzgqRyCawLnwnr
80mos17uknh2RyR3HlJyE+XYFyuCGIOfT1ruxE4GOPzpfKf2/OjQLnDax4W1mXz4tNdEtzcGSAfa
mVov3aKvXIxkNx154PWum0Gwn02ynjuNm+S6mmG054dyR+PNaflP7fnR5T+350aAGaM0eU/t+dHl
P7fnRoAyQ/J+I/nT80hhdhg4/Ol8p/b86NADNGaPKf2/Ojyn9vzo0AM0Zo8p/b86PKf2/OjQAzRm
jyn9vzo8p/b86NADNOg+43+8ab5T+351JGmxMdycmkwRmaw2JFHqKwdF/wCQcv8Avt/6Ea2dZP8A
pSD/AGR/OsbRf+Qcv++3/oRpy+FER+JmhRRRUGgUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUA
FFFFABRRRQAUh6H6UtIehoArQ/eh/wB0/wAqtVVg+9D/ALp/lVqgAooooAKKKKACiiigAooooARv
umneHP8AkD23+83/AKGaa33TTvDn/IHtv95v/QzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAK
KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo
ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAw9Z/4/U/3B/M1j6L/AMg5f99v
/QjWxrP/AB/J/uD+ZrH0X/kHL/vt/wChGtJfCjOPxM0KKKKzNAoo/wA9KKACiiigAooooAKKKKAC
iiigAooooAKKKKACiiigAooooAKQ9D9KWkPQ0AVofvQ/7p/lVqqsH3of90/yq1QAUUUUAFFFFABR
RRQAUUUUAI33TTvDn/IHtv8Aeb/0M01vumneHP8AkD23+83/AKGaaA6OiiikAUUUUAFFFFABRRRQ
AUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAB
RRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAGHrP/AB+r/uD+
ZrH0X/kHL/vt/wChGtjWf+P1f9wfzNY+i/8AIOX/AH2/9CNaS+FGcfiZoUUUVmaBRRRQAUUUUAFF
FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6H6UAVoPvQ/7p/lVqqsP3of90/yq1QAU
UUUAFFFFABRRRQAUUUUAI33TTvDn/IHtv95v/QzTW+6ad4c/5A9t/vN/6GaaA6OiiikAUUUUAFFF
FABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU
AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAGJrA/0
xf8AcH8zWNov/IOX/fb/ANCNbOs/8fa/7g/maxtF/wCQcv8Avt/6EauXwozj8TNCiiioNAooooAK
KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0NLSHofpQBWh+9D/ALp/lVqqdu4aSID+
FT/KrlABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/3m/wDQzTW+6ad4c/5A9t/vN/6GaaA6Oiii
kAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA
BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF
FFFAGJrP/H2v+4P5msbRf+Qcv++3/oRra1j/AI+1/wB0fzNYui/8g5f99v8A0I1cvhRnH4maFFFF
QaBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6GgCjaf8AHwv0NX6p
WybZY2z94H+VXaACiiigAooooAKKKKACiiigBG+6ad4c/wCQPbf7zf8AoZprfdNO8Of8ge2/3m/9
DNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK
KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo
ooAKKKKACiiigDG1j/j6X/dH8zWJov8AyDh/vt/6Ea29X/4+l/3R/M1iaL/yDh/vt/6EauXwozj8
TNCiiioNAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0P0paQ9D9KAK0H3of
90/yq1VWD70P+6f5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/kD23+83/oZprfdNO8Of8ge2/3m
/wDQzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK
KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo
AKKKKACiiigAooooAx9W/wCPpf8AdH8zWHov/IOH++3/AKEa3NW/4+l/3R/OsPRf+QcP99v/AEI1
cvhREfiZoUUUVBYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0P0oAr
Q/eh/wB0/wAqtVVg+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumneHP8AkD23+83/AKGaa33T
TvDn/IHtv95v/QzTQHR0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii
gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKA
CiiigAooooAKKKKACiiigAooooAyNW/4+V/3R/OsLRf+QeP99v8A0I1vasP9IT/d/rWFov8AyDh/
vt/6EauXwoiPxMv0UUVBYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6H6Ut
IehoArQfeh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt900
7w5/yB7b/eb/ANDNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK
KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo
AKKKKACiiigAooooAKKKKACiiigDK1b/AFyfSsDRf+QcP99v/QjXQaqP3i/SsDRf+QeP99v/AEI1
cvhRC+Jl+iiioLD/AD1ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApD0P0paQ9
D9KAK0P3of8AdP8AKrVVYPvQ/wC6f5VaoAKKKKACiiigAooooAKKKKAEb7pp3hz/AJA9t/vN/wCh
mmt9007w5/yB7b/eb/0M00B0dFFFIAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii
gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKA
CiiigAooooAKKKKACiiigAooooAKKKKAM3Ux84z6Vz2i/wDIPH++/wD6Ea6HUuXx/siue0X/AJBw
/wB9v/QjVPZErdl+iiipKCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkPQ0tI
ehoArQ/eh/3T/KrVVYPvQ/7p/lVqgAooooAKKKKACiiigAooooARvumneHP+QPbf7zf+hmmt9007
w5/yB7b/AHm/9DNNAdHRRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA
KKKKACiiigAooooAKKKKACiiigDM1H/Xr/u1z+jf8g8f77/+hGui1AZnX6Vzujf8g8f77/8AoRqn
siVuy/RRRUlBRR/npRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSHofpS0h6GgCt
D96H/dP8qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TTvDn/IH
tv8Aeb/0M00B0dFFFIAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA
KKKKACiiigAooooAKKKKAKF9/r1+grm9G/48B/vv/wChGulvf9ev0Fc1o3/Hh/wN/wD0I1T2QluX
6KKKkYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUh6GlpD0NAFaH78P8Aun+V
Wqqw/eh/3T/KrVABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/3m/wDQzTW+6ad4c/5A9t/vN/6G
aaA6OiiikAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU
UUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR
QAUUUUAFFFFAFG8/16/QVzWj/wDHh/wN/wD0I1013/r1+grmdH/48P8Ato//AKEab2Ei/RRRSGFF
FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFIehpaQ9DQBWh+9D/un+VWqqw/fh/3
T/KrVABRRRQAUUUUAFFFFABRRRQAjfdNO8Of8ge2/wB5v/QzTW+6ad4c/wCQPbf7zf8AoZpoDo6K
KKQBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUU
UAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ
AUUUUAU7v/Xr9BXMaP8A8eH/AG0f/wBCNdPdf69foK5jR/8Ajw/7aP8A+hGmxF+iiikMKKKKACii
igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD9+H/dP8qtVVh+9D/un+VWqA
CiiigAooooAKKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TTvDn/IHtv8Aeb/0M00B0dFFFIAooooA
KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAo
oooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAKdz
/r1+grmNH/48f+2j/wDoRrqLn/Xr9BXL6P8A8eP/AG0f/wBCNMC/RRRSAKKKKACiiigAooooAKKK
KACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD96H/dP8qtVVh+9D/un+VWqACiiigAooooA
KKKKACiiigBG+6ad4c/5A9t/vN/6Gaa33TS+HjjRbc+jN/6EaaA6Siohcxkctg+lH2iL+/SAloqL
7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36P
tEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foA
loqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0R
f36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF
/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaK
i+0Rf36PtEX9+gCWiovtEX9+j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9+gCWiovtEX9+
j7RF/foAloqL7RF/fo+0Rf36AJaKi+0Rf36PtEX9/wDSgCG5/wBev0Fcvo//AB4/9tH/APQjXSyy
CSYEdOlc1o//AB4/9tH/APQjTAv0UUUgCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK
KKKACkPQ0tIehoArQ/fh/wB0/wAqtVVh+9D/ALp/lVqgAooooAKKKKACiiigAooooARvumjw/wD8
gSD6v/6EaG+6aXw//wAgWD6v/wChGmgNGiiigQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF
FFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU
UUAFFFFABRRRQAE4GabeXNlpdilzqExRXZUUDJLM3RVUckn0FOIyCKy9f0+bVYdNltJIxdaddJco
kuQkmAQVJHTg8HHFDGWE17RHjif7S6mS6FlsZXV1mOcIykZU8d6kttQstStpbjTZXkiikaMuQ21m
U4O0nqM9xxXK6l4MvNYe4nu54opb3U4bqeOGRh5UMaFAqvjJfBznArY0K2vdI0FNNv3gkW1zFbyR
cb4h93cMDDY64zSBmyjB0DDoRmpY0i8iSad9saZJJOAAOpJqCEYgQH0FMvIFv9Ku9PdzH56Mm4ds
96bBC2uo6ZdRvIrzRRKobzJ1eJWU9CC2AR9PWrTGwQxhriMGXGwGbG/PTHPNYl7/AG3faYLd4rCN
0KZKy7vMAPzY3IQntkNWXB4UuFsljnNq8yWK26MTnYwmL5Bx0wQMgDp0pAdgRZLJJGZ0Eka73Uy8
qvqRnge9UjeWj3yWsDs7vAZ1dW3IVDBeueuTWLdaHdz2V/aBbUedK0yXQciSTLhwjDacDjGckcDi
l0jSJ9Ov1uJSqr5MiMpl8whmcN1CqMcdgKEBvQvvJB6q2DWDo/8Ax4/9tH/9CNai3cdujyPzuf5R
/erL0f8A48B7u5/8eNMC/RRRSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ
9DS0h6GgCtD96H/dP8qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigAqrbXD6Szo8TyWrsXUoMt
GT1GO4q1RQA3/hIbL+7df9+Go/4SGy/u3X/fhqXiii4Cf8JDZf3br/vw1H/CQ2X926/78NS4oouA
n/CQ2X926/78NR/wkNl/duv+/DUtGKLgJ/wkNl/duv8Avw1H/CQ2X926/wC/DUvFFFwE/wCEhsv7
t1/34aj/AISGy/u3X/fhqWincBP+Ehsv7t1/34aj/hIbL+7df9+GpaMUrgJ/wkNl/duv+/DUf8JD
Zf3br/vw1LRRcBP+Ehsv7t1/34aj/hIbL+7df9+GpaKLgJ/wkNl/duv+/DUf8JDZf3br/vw1LRRc
BP8AhIbL+7df9+Go/wCEhsv7t1/34aloouAn/CQ2X926/wC/DUf8JDZf3br/AL8NS0UXAT/hIbL+
7df9+Go/4SGy/u3X/fhqWii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRii4Cf8JDZf3br/vw1H/C
Q2X926/78NS0U7gJ/wAJDZf3br/vw1H/AAkNl/duv+/DUtFK4Cf8JDZf3br/AL8NR/wkNl/duv8A
vw1Liii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRRcBP+Ehsv7t1/wB+Go/4SGy/u3X/AH4aloou
An/CQ2X926/78NR/wkNl/duv+/DUtGKLgJ/wkNl/duv+/DUf8JDZf3br/vw1LRii4Cf8JDZf3br/
AL8NR/wkNl/duv8Avw1Liii4Cf8ACQ2X926/78NR/wAJDZf3br/vw1LRii4Cf8JDZf3br/vw1H/C
Q2X926/78NS8UYouAn/CQ2X926/78NR/wkNl/duv+/DUuKKLgJ/wkNl/duv+/DUf8JDZf3br/vw1
Liii4Cf8JDZf3br/AL8NUb61p8hy0d1n18hqlxRii4EH9rab/wA87r/vy9KNX00HPlXWR6wNU2KO
KAE/4SGy/u3X/fhqa+uWEg+eO6OP+mDU+jFFwIP7W03/AJ53X/fl6P7W03/nndf9+XqfFGKAIP7W
03/nndf9+XpG1fTgpKw3ROOB5Lc1YoxQBjyTz3z/AOjo+88bmQqqD2z/AJ71p20C21ukS9FHX1qa
igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKQ9DS0h6GgCtD96H/dP8
qtVVh+9D/un+VWqACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK
KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo
ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii
gAooooAKKKKACiiigAooooAKKKKACiiigApD0NLSHoaAK0P3of8AdP8AKrVVYfvQ/wC6f5VaoAKK
KKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooo
oAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig
AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAC
iiigAooooAKKKKACkPQ0tIehoArQ/eh/3T/KrVVYfvQ/7p/lVqgAooooAKKKKACiiigAooooAKKK
KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo
AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigA
ooooAKKKKACiiigAooooAKKKKAEByBRRRQAUtFFABRRRQAUUUUAFJRRQAtFFFABRRRQAUUUUAFIe
hoooArQfeh/3f6VaoooAKSiigBaKKKACiiigAooooASloooASloooAKKKKACiiigAooooAKKKKAC
iiigAooooAKSiigBaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKSiigBaKKKAC
iiigAooooAKKKKAEpaKKACiiigAooooASloooASiiigBaKKKACiiigApKKKAFooooASiiigBC2Dj
FFFFAH//2Q==
--_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_
Content-Type: image/png; name="blue.png"
Content-Description: blue.png
Content-Disposition: inline; filename="blue.png"; size=2834;
creation-date="Thu, 23 Aug 2012 14:54:42 GMT";
modification-date="Thu, 23 Aug 2012 14:54:42 GMT"
Content-ID: <dfb35e5d2df7434e8338047441010e53
Content-Transfer-Encoding: base64
iVBORw0KGgoAAAANSUhEUgAAAIwAAAAyCAYAAACOADM7AAAABmJLR0QA/gD+AP7rGNSCAAAACXBI
WXMAAA3XAAAN1wFCKJt4AAAACXZwQWcAAACMAAAAMgCR0D3bAAAKaUlEQVR42u2ce5AUxRnAf313
3Al4eCAYFaIgyMNEUF6KlYoVIDBArDxqopWxQgViQlWsPHA0MUlZVoyKRsdSE4lGomjIaHS0UlHL
wTIPpEgQFQUUjYIWdfIIScyBHi/Z6/zRM1xP3yzs7t3unOX8qra2H9M9vb3f9Pf19/WukFKSk1Mq
dVkPIOejRS4wOWXR6wVGuP5I4foDsh5HjkL0VhtGuP5A4CFgNrAD+Lb0nKeyHtfHnd68wixGCQvA
qcA9wvWPy3pQH3caan1D4fonAYeBDwEZjaFflAaok56zHRhsNG0B+gAHSrhHarn0nFp/3NLnxbKP
B06I5kECO2UYZD2sLtRcYIBJwK+BoYBACU89cAjoAIRw/TuAJcClQGy//FJ6zvvH6ly4/qXAz4vU
HQA2A4H0nIcz+OxH41eAHaU3AhdkPaA0MrFhhOuPB2YA5wBnA6ehni5dgKcBu4C5wLZS7Rfh+g8A
80u49HHgEuk5h2s+AeaYLbsO2AKMiIqWyzBYkPW40shihUF6zkbUUwSAcP0G4FHgS9pl10rPmQMs
LbXfSBVNLPHyrwDfBO7JYg4MRqEempjnsh5QMXqL0Xsl8EUt3w5cXUE/w4AztfzzwGSUGrwoyuvM
yfqDR5yLUssxL2U9oGJkssLoCNdfjLJXdBZIz9lQQXcTgSYt/4z0nHjy1wvX3wW8oNX3O8q4TgKm
AGegjNB/As9JzzmYer1lTwKGoOyyV2UYtArLngLMQ9lh64EVRQxZ3V5pje4V9zsVGBRl22QYrDXu
e0HUvwD+K8NgXbe/lKOQqcAI178MuM0ovk16zqMVdjnNyL9g5E2DrTVlTP1RRvM3gIFG9RvC9RdK
z/lHoo2yQQJgeFR0hbDsT6FUns544Icp456qpV+RYaAL5RJgepR+FWXzxfcdA6zRrr0SqKrAZKaS
hOt/DbjXKH5Geo7bjW71iT8AvGLUzzXyfzfGNBBlPyymq7AAjAWeFK5/slE+AvhklC4At6KEZb9x
3cJo+9x5T8s+ERinFa012uzU0vuMuu9r6W3AXd2Yu5LIRGCE618E/D6l6rpu9Hk8MEEr2iQ9p1Wr
n4wShJgPgCeMbh6g02jeB9wILASe1q4ZBHzBaDeRThukHghRdskoQF+NmlH+JJ0JqB1ijCkw72np
jiOfx7JPQrkdYm6QYXBMH1V3qYlKEq7fhNLvw1CTeztK55rcJlz/s8XshGPwaeBELd8sXP961Bd4
Bsqo1u2bm6Tn7NbGeCHKMI6ZLz3nsajuT6gtfjxfpxr31lXhThkG8470a9mrtPp2uq4652np94FN
Rr0uMM1a+jI6fVTvAMsrmLOy6VGBEa5fB3wOpctHaK9TgVOAxmN0MRXlwPpWBbefYuTHAj8tcu39
0nNuMMq+qqXfjoUl4mSSq/HbRlv9S3/ZqBumpXcB/zPqz9fSm2UY/Nuo1wWmCUBYdiPwHa3ck2Hw
YQVzVjbVWGFmkW7YmewDfga8CNwHnB6VXyZcf7X0nAfLvG8pntE3gSXSc5an1Olf+hDh+i+jVieJ
UiOxwBSiMQMgLLsFOEtr+7xWB8rQjdkgw0BXK40o1RWTZrDu0dKx0X4xylMOynZZVuZcVUyPCoz0
nA7gR8L1N6FWmQIqZtRGpwoSwF7gRek5WwCE658P3A9Y0TV3C9ffUOrWOlrZdIfdXuBhlCqaqZU/
myYs0RZaNzybUV7oNFqBt7T8BJJ2iW6zDAPGFKkDGE1yBTLtF0gKTCF6/4FWtsTYVVWVqtgw0nNW
lHn9LmCOcP2bgKuAvsAtqNWqFGLVF7NGes4i4fpjgNfpFNbzi7QfD/TX8vtQMa40VkvPKWh5fWfW
DuhCfg5Ju8nc5k/RxpZYuTR0gWkTlj0D5YgEeJca2S4xvcXTC4D0nKvpdNWXc2hqEiqSHROrhR0k
bYAzhesPTmmvG61tKAE6PXoNRRnTg6OX6VvRhfB1GQa7tbyu5v6D8qNQpH4bsDVlbLrADACu0fK/
qOXqAr1MYCLip7AcI+48I78WIIpuv6mVN5NUPWntN0nP2So9p016ThtwEKU6RpIMOyAsuw9JVWiu
INO19AYZBma0fbKWXi/DoEBX9tBpu4wDLozS2+jqx6o6vVFgYt+JKKON/pTvJ6kWzKc6LTg5XEtv
MeruAF5DqbZVgH6IayTJoOHf4oSw7LNICuKTeqfCsj9BUnhN+yamPXqZc3JrLfwuJpnHklKIBaa+
lIuF67eQ3KW8HtlEMabhPCmlG/3JnhX5ZHaifDeLtLqlxpmcySQfuvnCstdH6WXaZ9iPMsJ1xpOM
ZaXZL6DsqfcB3UO8A7WzrDm9T2DqG7dTOHSIEgUGIc5GyhatZJ1Rv4HkmZ/xKb08o5UPRa0UkuQT
vY6uQVJTFc5D7fQ6SNpUN8ow2GVcq7sB2ugq2DGHUYfLdG6SYbCPDMhcYIRlJwWjcGg/Z1/yATBE
zJxXT0Pf4o0P7pWcO39W4nuVHS+JGfPq6dMXOjpgzNyt9En0MUF877fDee3x1iPlo2beTOPxnwGh
qzahuhUAjwCLpOeYKkDfIT2BUl1XkxT2+2QYXJ8yen0H+JYMgz2kY9o126mh38UkITBRYGwp5e1Q
usNjwL/Ql3VRX2D35mUI0UB90wyOZmc19i+wa+NB+vTrnMA9re00RO3q6iRbVtYxeOzt1NXHS3od
e96dRkPT6CN9v/HUIRr738Dg0bMRDSdQVzeAjsJh+ra8SfMpf5S3XNzFoSYsewhJVbhKhoEnLDtE
HV4vRGXPprQFFTdrRklk2u4opoVkyMOTYbCfjEgc0RSWPQhlQ/SruMfymCrD4IXud1N7In+ILgzT
ZRj8tYfvcSLwOzoPer0DjKv1VlrHVEltqBhMafZD99mR1QfvAXT1tYfiNkhZCMvuD1yLCtbORsXg
Yi7PUljAEJgoztFaYV8fN8yg4XsV95TkLJS32+QaGQZPl9tZT5O50ftRJLL1Pq8V9cjqEjHdyG8D
rpdhkJmhq5MLTGX0QR2diLdnYQ/2vRq1wsRe6nUyDNq712XP0Wt/W53TO+mNoYGcXkwuMDll0eM2
TPRbnGnAvaaDSVj2bOA0GQY1j7Lm9AzVWGG+jIrwphlH3wXuzvpD51RONXZJ7aizLFcIyx4O3CXD
IN527kUdJAJAWPbFqBXnVmHZV6FO3K+I6oahzgYPAX7T017UnMqoxgpTQAniONRJ/AeFZRc72+IA
P47SPwEWAAjLbgL+jPJ1NAF/EZZd6o/sc6pINQSmARAyDL6OOm45mmSoX+cDVDiC6D0+azI0arcS
FSkG9fcgORlTbcfdXtR5jqOdnpPGO3QK8nzU33KsoutvgXIyoBorjP7FN6OEsph3sE6rq9fS8RmQ
RTIMTgP+QPJsbk5GVENgjgMQlv0QcDnwBp0nxgaQ/O+6dmCUsOxHUGdj459kbI/a3Sksew3qjE5L
1pOVUx2VtBJljxxAhf3v0v4TZRnKmI25ObruLdTZkvcAZBgcEpY9E3BRu6TrZBisznqycvJYUk6Z
5KGBnLLIBSanLHKBySmLXGByyiIXmJyy+D/P9uGVPOu6DAAAACh6VFh0U29mdHdhcmUAAHja801M
LsrPTU3JTFRwyyxKLc8vyi5WsAAAYBUIJ4KDNosAAAAASUVORK5CYII=
--_005_C8B8517ADA90DB40A482797D59EB838964184566CED01MBXS08repl_--
6 p.m.
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any
LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN
[org.ovirt.engine.core.bll.GetConfigurationValueQuery]
(http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
version, using default general for version
2007-06-29 21:50:07,911 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id: 60bdafe6
2007-06-29 21:50:08,726 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
message VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error = Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Command
org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
return value
Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
lunList Null
mStatus Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
mCode 600
mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Vds: pittor06vhxd020
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1)
Command GetDeviceListVDS execution failed. Exception:
VDSErrorException: VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error = Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id: 60bdafe6
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
(http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
message is VdcBLLException:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll
3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5
size=650G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| |- 2:0:0:0 sda 8:0 active ready running
| |- 2:0:1:0 sdd 8:48 active ready running
| |- 3:0:0:0 sde 8:64 active ready running
| `- 3:0:1:0 sdf 8:80 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
|- 2:0:2:0 sdb 8:16 active ready running
|- 2:0:3:0 sdc 8:32 active ready running
|- 3:0:2:0 sdg 8:96 active ready running
`- 3:0:3:0 sdh 8:112 active ready running
3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME
size=205G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 0:0:1:0 cciss!c0d1 104:16 active ready running
------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don’t know what
paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing>
<maxHostID>
<version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
------------------------------------------------------------------------
--
The information transmitted is intended for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of,
or taking of any action in reliance upon, this information by persons
or entities other than the intended recipient is prohibited. If you
received this in error, please contact the sender and delete the
material from any computer.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:20 p.m.
Here you are
Thread-47346::DEBUG::2007-06-30
00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216]
Thread-47346::INFO::2007-06-30
00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect:
getDeviceList, args: ()
Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing
Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__)
Trying to enter sampling method (storage.sdc.refreshStorage)
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got
in to sampling method
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__)
Trying to enter sampling method (storage.iscsi.rescan)
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got
in to sampling method
Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan)
FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__)
Returning last result
Thread-47346::DEBUG::2007-06-30 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/multipath' (cwd None)
Thread-47346::DEBUG::2007-06-30 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan)
SUCCESS: <err> = ''; <rc> = 0
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__)
Returning last result
Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs)
Operation 'lvm reload operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd)
'/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names =
[\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
disable_after_error_count=3 filter = [
\\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece111%\\",
\\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1
wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings
--units b --nosuffix --separator | -o
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size'
(cwd None)
Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd)
SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output
error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n
/dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING:
Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0
Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs)
Operation 'lvm reload operation' released the operation mutex
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) '/sbin/dmsetup
status' (cwd None)
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS:
<err> = ''; <rc> = 0
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScsiSerial) '/sbin/scsi_id
--page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-0' (cwd
None)
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScsiSerial) SUCCESS: <err> =
''; <rc> = 0
Thread-47346::WARNING::2007-06-30
00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from
device `cciss!c0d1`
Traceback (most recent call last):
File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter
File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/device/scsi_disk/'
Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError)
Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList
File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList
KeyError: 'hbtl'
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 ()
{} failed - stopping task
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False)
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare)
aborting: Task is aborted: "'hbtl'" - code 100
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl'
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
Thread-47346::DEBUG::2007-06-30
00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll
requests {}
Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting
Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed
Thread-47346::DEBUG::2007-06-30
00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll
requests {} resources {}
Thread-47346::DEBUG::2007-06-30
00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll
requests {}
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent
call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
KeyError: 'hbtl'
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 17:00
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
Hi all,
I'm trying to configure a FCP storage domain on RHEV 3.
I try to add a new domain from the console, but it can't find any
LUNs: "Could not retrieve LUNs, please check your storage"
Here is the output from /var/log/rhevm/rhevm.log:
------------------------------------
2007-06-29 21:50:07,811 WARN
[org.ovirt.engine.core.bll.GetConfigurationValueQuery]
(http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
version, using default general for version
2007-06-29 21:50:07,911 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id:
60bdafe6
2007-06-29 21:50:08,726 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
message VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error = Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Command
org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
return value
Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
lunList Null
mStatus Class Name:
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
mCode 600
mMessage Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(http-0.0.0.0-8443-1) Vds: pittor06vhxd020
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1)
Command GetDeviceListVDS execution failed. Exception:
VDSErrorException: VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS, error = Error block device action: ()
2007-06-29 21:50:08,727 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
(http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
60bdafe6
2007-06-29 21:50:08,727 ERROR
[org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
(http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
message is VdcBLLException:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
error = Error block device action: ()
----------------------------------------------
First question: do LUNs have to be visible from RHEV-H or RHEV-M?
Currently they are visible only from the hypervisor.
----------------------------------------
[root@pittor06vhxd020 log]# multipath -ll
3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G
features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| |- 2:0:0:0 sda 8:0 active ready running
| |- 2:0:1:0 sdd 8:48 active ready running
| |- 3:0:0:0 sde 8:64 active ready running
| `- 3:0:1:0 sdf 8:80 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
|- 2:0:2:0 sdb 8:16 active ready running
|- 2:0:3:0 sdc 8:32 active ready running
|- 3:0:2:0 sdg 8:96 active ready running
`- 3:0:3:0 sdh 8:112 active ready running
3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G
features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 0:0:1:0 cciss!c0d1 104:16 active ready running
------------------------------------------------------
Our SAN device is Clariion AX150. Is it compatible with ovirt?
vdsClient -s 0 getDeviceListgives me:
Error block device action: ()
Could it be due to SPM turned off? (I have only one host)
[root@pittor06vhxd020 log]# ps axu | grep -i spm
root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
How can I turn it on? I know the command but I don't know what
paramaters append
spmStart
<spUUID> <prevID> <prevLVER> <recoveryMode> <scsiFencing>
<maxHostID>
<version>
Thank you very much for any hints.
AS
Alberto Scotto
Blue
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
----------------------------------------------------------------------
--
--
The information transmitted is intended for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of,
or taking of any action in reliance upon, this information by persons
or entities other than the intended recipient is prohibited. If you
received this in error, please contact the sender and delete the
material from any computer.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
7 p.m.
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but
let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30
00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.16.250.216]
Thread-47346::INFO::2007-06-30
00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect:
getDeviceList, args: ()
Thread-47346::DEBUG::2007-06-30 00:37:10,269::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state preparing
Thread-47346::DEBUG::2007-06-30 00:37:10,269::misc::1010::SamplingMethod::(__call__)
Trying to enter sampling method (storage.sdc.refreshStorage)
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got
in to sampling method
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1010::SamplingMethod::(__call__)
Trying to enter sampling method (storage.iscsi.rescan)
Thread-47346::DEBUG::2007-06-30 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got
in to sampling method
Thread-47346::DEBUG::2007-06-30 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-47346::DEBUG::2007-06-30 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan)
FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
Thread-47346::DEBUG::2007-06-30 00:37:10,301::misc::1020::SamplingMethod::(__call__)
Returning last result
Thread-47346::DEBUG::2007-06-30
00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n
/sbin/multipath' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS: <err> =
''; <rc> = 0
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate
operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate
operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30 00:37:10,788::misc::1020::SamplingMethod::(__call__)
Returning last result
Thread-47346::DEBUG::2007-06-30 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs)
Operation 'lvm reload operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd)
'/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names =
[\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
disable_after_error_count=3 filter = [
\\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece111%\\",
\\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1
wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings
--units b --nosuffix --separator | -o
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size'
(cwd None)
Thread-47346::DEBUG::2007-06-30 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd)
SUCCESS: <err> = ' /dev/sdh: read failed after 0 of 4096 at 0: Input/output
error\n /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output error\n
/dev/sdh: read failed after 0 of 4096 at 697932177408: Input/output error\n WARNING:
Error counts reached a limit of 3. Device /dev/sdh was disabled\n'; <rc> = 0
Thread-47346::DEBUG::2007-06-30 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs)
Operation 'lvm reload operation' released the operation mutex
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) '/sbin/dmsetup
status' (cwd None)
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS:
<err> = ''; <rc> = 0
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScsiSerial) '/sbin/scsi_id
--page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-0' (cwd
None)
MainProcess|Thread-47346::DEBUG::2007-06-30
00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScsiSerial) SUCCESS: <err> =
''; <rc> = 0
Thread-47346::WARNING::2007-06-30
00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from
device `cciss!c0d1`
Traceback (most recent call last):
File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter
File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/device/scsi_disk/'
Thread-47346::ERROR::2007-06-30 00:37:11,029::task::868::TaskManager.Task::(_setError)
Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList
File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList
KeyError: 'hbtl'
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._run: 0be1d461-f8fa-4c20-861d-27fde8124408 ()
{} failed - stopping task
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing (force False)
Thread-47346::DEBUG::2007-06-30 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
Thread-47346::INFO::2007-06-30 00:37:11,031::task::1171::TaskManager.Task::(prepare)
aborting: Task is aborted: "'hbtl'" - code 100
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl'
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
Thread-47346::DEBUG::2007-06-30
00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll
requests {}
Thread-47346::DEBUG::2007-06-30 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting
Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
Thread-47346::DEBUG::2007-06-30 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed
Thread-47346::DEBUG::2007-06-30
00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll
requests {} resources {}
Thread-47346::DEBUG::2007-06-30
00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll
requests {}
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent
call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
KeyError: 'hbtl'
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 17:00
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
> Hi all,
>
> I'm trying to configure a FCP storage domain on RHEV 3.
>
> I try to add a new domain from the console, but it can't find any
> LUNs: "Could not retrieve LUNs, please check your storage"
>
> Here is the output from /var/log/rhevm/rhevm.log:
>
> ------------------------------------
>
> 2007-06-29 21:50:07,811 WARN
> [org.ovirt.engine.core.bll.GetConfigurationValueQuery]
> (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
> version, using default general for version
> 2007-06-29 21:50:07,911 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
> (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id:
> 60bdafe6
> 2007-06-29 21:50:08,726 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
> message VDSGenericException: VDSErrorException: Failed to
> GetDeviceListVDS, error = Error block device action: ()
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Command
> org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
> return value
>
> Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
> lunList Null
> mStatus Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
> mCode 600
> mMessage Error block device action: ()
>
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (http-0.0.0.0-8443-1)
> Command GetDeviceListVDS execution failed. Exception:
> VDSErrorException: VDSGenericException: VDSErrorException: Failed to
> GetDeviceListVDS, error = Error block device action: ()
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
> (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
> 60bdafe6
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
> (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
> message is VdcBLLException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
> error = Error block device action: ()
>
> ----------------------------------------------
>
> First question: do LUNs have to be visible from RHEV-H or RHEV-M?
>
> Currently they are visible only from the hypervisor.
>
> ----------------------------------------
>
> [root@pittor06vhxd020 log]# multipath -ll
> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G
> features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
> |-+- policy='round-robin 0' prio=1 status=active
> | |- 2:0:0:0 sda 8:0 active ready running
> | |- 2:0:1:0 sdd 8:48 active ready running
> | |- 3:0:0:0 sde 8:64 active ready running
> | `- 3:0:1:0 sdf 8:80 active ready running
> `-+- policy='round-robin 0' prio=0 status=enabled
> |- 2:0:2:0 sdb 8:16 active ready running
> |- 2:0:3:0 sdc 8:32 active ready running
> |- 3:0:2:0 sdg 8:96 active ready running
> `- 3:0:3:0 sdh 8:112 active ready running
> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G
> features='1 queue_if_no_path' hwhandler='0' wp=rw
> `-+- policy='round-robin 0' prio=1 status=active
> `- 0:0:1:0 cciss!c0d1 104:16 active ready running
> ------------------------------------------------------
>
> Our SAN device is Clariion AX150. Is it compatible with ovirt?
>
> vdsClient -s 0 getDeviceListgives me:
>
> Error block device action: ()
>
> Could it be due to SPM turned off? (I have only one host)
>
> [root@pittor06vhxd020 log]# ps axu | grep -i spm
>
> root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
>
> How can I turn it on? I know the command but I don't know what
> paramaters append
>
> spmStart
>
> <spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing> <maxHostID>
> <version>
>
> Thank you very much for any hints.
>
> AS
>
>
>
> Alberto Scotto
>
> Blue
> Via Cardinal Massaia, 83
> 10147 - Torino - ITALY
> phone: +39 011 29100
> al.scotto(a)reply.it
> www.reply.it
>
>
> ----------------------------------------------------------------------
> --
>
> --
> The information transmitted is intended for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of,
> or taking of any action in reliance upon, this information by persons
> or entities other than the intended recipient is prohibited. If you
> received this in error, please contact the sender and delete the
> material from any computer.
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
7:13 p.m.
I was going to reply.. :)
It looks like it halts after an error due to cciss!c0d1, which btw is displayed by
multipath -ll.
That's just a local disk, isn't it? So it shouldn't even be listed. I may have
attached it by mistake, playing with /sys/class/fc_* tools.
So, if I remove that path, everything should go ok. Do you think so too?
it appears that vdsm fails to handle a device with '!' in it
(cciss!c0d1), but let's make sure its indeed the case
More than that the fact is
that the path scsi_disk/ doesn't exist in /sys/block/cciss!c0d1/device
And this must be due to the fact that c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/
total 0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:37 cciss!c0d0 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/cciss!c0d0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/cciss!c0d1
lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 -> ../devices/virtual/block/dm-0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:40 dm-1 -> ../devices/virtual/block/dm-1
lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 -> ../devices/virtual/block/dm-4
lrwxrwxrwx. 1 root root 0 2007-06-30 01:00 dm-5 -> ../devices/virtual/block/dm-5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 -> ../devices/virtual/block/loop1
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop2 -> ../devices/virtual/block/loop2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 -> ../devices/virtual/block/loop5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop6 -> ../devices/virtual/block/loop6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 -> ../devices/virtual/block/ram1
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram10 -> ../devices/virtual/block/ram10
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 -> ../devices/virtual/block/ram13
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram14 -> ../devices/virtual/block/ram14
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 -> ../devices/virtual/block/ram3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram4 -> ../devices/virtual/block/ram4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 -> ../devices/virtual/block/ram7
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram8 -> ../devices/virtual/block/ram8
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9
lrwxrwxrwx. 1 root root 0 2007-06-30 00:36 sda ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 ->
../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table
3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1 queue_if_no_path 1 emc 2 1
round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112
1
HostVG-Logging: 0 4194304 linear 104:4 24741888
HostVG-Swap: 0 24723456 linear 104:4 2048
3600508b1001035333920202020200005: 0 429925920 multipath 1 queue_if_no_path 0 1 1
round-robin 0 1 1 104:16 1
HostVG-Data: 0 40624128 linear 104:4 28936192
HostVG-Config: 0 16384 linear 104:4 24725504
live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 99.3M 1 loop
loop1 7:1 0 1G 1 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
loop2 7:2 0 512M 0 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
cciss!c0d0 104:0 0 33.9G 0 disk
ââcciss!c0d0p1 104:1 0 243M 0 part
ââcciss!c0d0p2 104:2 0 244M 0 part
ââcciss!c0d0p3 104:3 0 244M 0 part
ââcciss!c0d0p4 104:4 0 33.2G 0 part
ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP]
ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config
ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log
ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data
cciss!c0d1 104:16 0 205G 0 disk
ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sda 8:0 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdc 8:32 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdd 8:48 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sde 8:64 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdf 8:80 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdg 8:96 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdh 8:112 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 18:01
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but
let's make sure its indeed the case.
Thread-47346::DEBUG::2007-06-30
00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper)
[10.16.250.216]
Thread-47346::INFO::2007-06-30
00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run
and protect: getDeviceList, args: ()
Thread-47346::DEBUG::2007-06-30
00:37:10,269::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state
preparing
Thread-47346::DEBUG::2007-06-30
00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter
sampling method (storage.sdc.refreshStorage)
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
sampling method
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter
sampling method (storage.iscsi.rescan)
Thread-47346::DEBUG::2007-06-30
00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
sampling method
Thread-47346::DEBUG::2007-06-30
00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo
-n /sbin/iscsiadm -m session -R' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err>
= 'iscsiadm: No session found.\n'; <rc> = 21
Thread-47346::DEBUG::2007-06-30
00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last
result
Thread-47346::DEBUG::2007-06-30
00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/multipath' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS:
<err> = ''; <rc> = 0
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last
result
Thread-47346::DEBUG::2007-06-30
00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm
reload operation' got the operation mutex
Thread-47346::DEBUG::2007-06-30
00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n
/sbin/lvm pvs --config " devices { preferred_names =
[\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
disable_after_error_count=3 filter = [
\\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece1
11%\\", \\"r%.*%\\" ] } global { locking_type=1
prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min =
50 retain_days = 0 } " --noheadings --units b --nosuffix --separator
| -o
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_co
unt,dev_size' (cwd None)
Thread-47346::DEBUG::2007-06-30
00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '
/dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n
/dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output
error\n /dev/sdh: read failed after 0 of 4096 at 697932177408:
Input/output error\n WARNING: Error counts reached a limit of 3.
Device /dev/sdh was disabled\n'; <rc> = 0
Thread-47346::DEBUG::2007-06-30
00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm
reload operation' released the operation mutex
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_ge
MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None)
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_ge
MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScs
MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export
MainProcess|--replace-whitespace --device=/dev/dm-0' (cwd None)
MainProcess|Thread-47346::DEBUG::2007-06-30
MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScs
MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0
Thread-47346::WARNING::2007-06-30
00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl from
device `cciss!c0d1` Traceback (most recent call last):
File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter
File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/device/scsi_disk/'
Thread-47346::ERROR::2007-06-30
00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most
recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
File "/usr/share/vdsm/storage/hsm.py", line 696, in public_getDeviceList
File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList
KeyError: 'hbtl'
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._run:
0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing
(force False)
Thread-47346::DEBUG::2007-06-30
00:37:11,030::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
Thread-47346::INFO::2007-06-30
00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task
is aborted: "'hbtl'" - code 100
Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl'
Thread-47346::DEBUG::2007-06-30
00:37:11,031::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
Thread-47346::DEBUG::2007-06-30
00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
Thread-47346::DEBUG::2007-06-30
00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-47346::DEBUG::2007-06-30
00:37:11,032::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing ->
state aborting
Thread-47346::DEBUG::2007-06-30
00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
Thread-47346::DEBUG::2007-06-30
00:37:11,033::task::495::TaskManager.Task::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting ->
state failed
Thread-47346::DEBUG::2007-06-30
00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll
) Owner.releaseAll requests {} resources {}
Thread-47346::DEBUG::2007-06-30
00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'
Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent
call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
KeyError: 'hbtl'
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 17:00
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve
LUNs
On 08/23/2012 05:54 PM, Scotto Alberto wrote:
hi,
can you attach full vdsm log during the execution of getDeviceList command?
> Hi all,
>
> I'm trying to configure a FCP storage domain on RHEV 3.
>
> I try to add a new domain from the console, but it can't find any
> LUNs: "Could not retrieve LUNs, please check your storage"
>
> Here is the output from /var/log/rhevm/rhevm.log:
>
> ------------------------------------
>
> 2007-06-29 21:50:07,811 WARN
> [org.ovirt.engine.core.bll.GetConfigurationValueQuery]
> (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
> version, using default general for version
> 2007-06-29 21:50:07,911 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
> (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id:
> 60bdafe6
> 2007-06-29 21:50:08,726 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
> message VDSGenericException: VDSErrorException: Failed to
> GetDeviceListVDS, error = Error block device action: ()
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Command
> org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
> return value
>
> Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
> lunList Null
> mStatus Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
> mCode 600
> mMessage Error block device action: ()
>
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
> (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception:
> VDSErrorException: VDSGenericException: VDSErrorException: Failed to
> GetDeviceListVDS, error = Error block device action: ()
> 2007-06-29 21:50:08,727 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
> (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
> 60bdafe6
> 2007-06-29 21:50:08,727 ERROR
> [org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
> (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
> message is VdcBLLException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
> error = Error block device action: ()
>
> ----------------------------------------------
>
> First question: do LUNs have to be visible from RHEV-H or RHEV-M?
>
> Currently they are visible only from the hypervisor.
>
> ----------------------------------------
>
> [root@pittor06vhxd020 log]# multipath -ll
> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G
> features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
> |-+- policy='round-robin 0' prio=1 status=active
> | |- 2:0:0:0 sda 8:0 active ready running
> | |- 2:0:1:0 sdd 8:48 active ready running
> | |- 3:0:0:0 sde 8:64 active ready running
> | `- 3:0:1:0 sdf 8:80 active ready running
> `-+- policy='round-robin 0' prio=0 status=enabled
> |- 2:0:2:0 sdb 8:16 active ready running
> |- 2:0:3:0 sdc 8:32 active ready running
> |- 3:0:2:0 sdg 8:96 active ready running
> `- 3:0:3:0 sdh 8:112 active ready running
> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G
> features='1 queue_if_no_path' hwhandler='0' wp=rw
> `-+- policy='round-robin 0' prio=1 status=active
> `- 0:0:1:0 cciss!c0d1 104:16 active ready running
> ------------------------------------------------------
>
> Our SAN device is Clariion AX150. Is it compatible with ovirt?
>
> vdsClient -s 0 getDeviceListgives me:
>
> Error block device action: ()
>
> Could it be due to SPM turned off? (I have only one host)
>
> [root@pittor06vhxd020 log]# ps axu | grep -i spm
>
> root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
>
> How can I turn it on? I know the command but I don't know what
> paramaters append
>
> spmStart
>
> <spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing> <maxHostID>
> <version>
>
> Thank you very much for any hints.
>
> AS
>
>
>
> Alberto Scotto
>
> Blue
> Via Cardinal Massaia, 83
> 10147 - Torino - ITALY
> phone: +39 011 29100
> al.scotto(a)reply.it
> www.reply.it
>
>
> ---------------------------------------------------------------------
> -
> --
>
> --
> The information transmitted is intended for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of,
> or taking of any action in reliance upon, this information by persons
> or entities other than the intended recipient is prohibited. If you
> received this in error, please contact the sender and delete the
> material from any computer.
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
7:22 p.m.
On 08/23/2012 07:13 PM, Scotto Alberto wrote:
I was going to reply.. :)
It looks like it halts after an error due to cciss!c0d1, which btw is displayed by
multipath -ll.
That's just a local disk, isn't it? So it shouldn't even be listed. I may
have attached it by mistake, playing with /sys/class/fc_* tools.
So, if I remove that path, everything should go ok. Do you think so too?
Yes, please remove and check again (make sure to clean device-mapper table with dmsetup
remove $dm).
anyhow, vdsm should be more robust, so please file a bug for
it(https://bugzilla.redhat.com/enter_bug.cgi?product=oVirt).
Haim
> it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1),
but let's make sure its indeed the case
More than that the fact is that the path scsi_disk/ doesn't exist in
/sys/block/cciss!c0d1/device
And this must be due to the fact that c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/
total 0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:37 cciss!c0d0 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/cciss!c0d0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/cciss!c0d1
lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 -> ../devices/virtual/block/dm-0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:40 dm-1 -> ../devices/virtual/block/dm-1
lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 -> ../devices/virtual/block/dm-4
lrwxrwxrwx. 1 root root 0 2007-06-30 01:00 dm-5 -> ../devices/virtual/block/dm-5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 -> ../devices/virtual/block/loop1
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop2 -> ../devices/virtual/block/loop2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 -> ../devices/virtual/block/loop5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop6 -> ../devices/virtual/block/loop6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 -> ../devices/virtual/block/ram1
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram10 -> ../devices/virtual/block/ram10
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 -> ../devices/virtual/block/ram13
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram14 -> ../devices/virtual/block/ram14
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 -> ../devices/virtual/block/ram3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram4 -> ../devices/virtual/block/ram4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 -> ../devices/virtual/block/ram7
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram8 -> ../devices/virtual/block/ram8
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9
lrwxrwxrwx. 1 root root 0 2007-06-30 00:36 sda ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:00.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 ->
../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table
3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1 queue_if_no_path 1 emc 2 1
round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112
1
HostVG-Logging: 0 4194304 linear 104:4 24741888
HostVG-Swap: 0 24723456 linear 104:4 2048
3600508b1001035333920202020200005: 0 429925920 multipath 1 queue_if_no_path 0 1 1
round-robin 0 1 1 104:16 1
HostVG-Data: 0 40624128 linear 104:4 28936192
HostVG-Config: 0 16384 linear 104:4 24725504
live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 99.3M 1 loop
loop1 7:1 0 1G 1 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
loop2 7:2 0 512M 0 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
cciss!c0d0 104:0 0 33.9G 0 disk
ââcciss!c0d0p1 104:1 0 243M 0 part
ââcciss!c0d0p2 104:2 0 244M 0 part
ââcciss!c0d0p3 104:3 0 244M 0 part
ââcciss!c0d0p4 104:4 0 33.2G 0 part
ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP]
ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config
ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log
ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data
cciss!c0d1 104:16 0 205G 0 disk
ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sda 8:0 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdc 8:32 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdd 8:48 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sde 8:64 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdf 8:80 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdg 8:96 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdh 8:112 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 18:01
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
> Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but
let's make sure its indeed the case.
>
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper)
> [10.16.250.216]
> Thread-47346::INFO::2007-06-30
> 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run
> and protect: getDeviceList, args: ()
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state
> preparing
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter
> sampling method (storage.sdc.refreshStorage)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter
> sampling method (storage.iscsi.rescan)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo
> -n /sbin/iscsiadm -m session -R' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err>
> = 'iscsiadm: No session found.\n'; <rc> = 21
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
> '/usr/bin/sudo -n /sbin/multipath' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS:
> <err> = ''; <rc> = 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm
> reload operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n
> /sbin/lvm pvs --config " devices { preferred_names =
> [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
> disable_after_error_count=3 filter = [
> \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece1
> 11%\\", \\"r%.*%\\" ] } global { locking_type=1
> prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min =
> 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator
> | -o
> uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_co
> unt,dev_size' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '
> /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n
> /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output
> error\n /dev/sdh: read failed after 0 of 4096 at 697932177408:
> Input/output error\n WARNING: Error counts reached a limit of 3.
> Device /dev/sdh was disabled\n'; <rc> = 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm
> reload operation' released the operation mutex
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_ge
> MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_ge
> MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getScs
> MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted --export
> MainProcess|--replace-whitespace --device=/dev/dm-0' (cwd None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getScs
> MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0
> Thread-47346::WARNING::2007-06-30
> 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl
from device `cciss!c0d1` Traceback (most recent call last):
> File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter
> File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
> OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/device/scsi_disk/'
> Thread-47346::ERROR::2007-06-30
> 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback
(most recent call last):
> File "/usr/share/vdsm/storage/task.py", line 876, in _run
> File "/usr/share/vdsm/storage/hsm.py", line 696, in
public_getDeviceList
> File "/usr/share/vdsm/storage/hsm.py", line 759, in _getDeviceList
> KeyError: 'hbtl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run:
> 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing
> (force False)
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
> Thread-47346::INFO::2007-06-30
> 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task
> is aborted: "'hbtl'" - code 100
> Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug)
Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing ->
> state aborting
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting ->
> state failed
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAll
> ) Owner.releaseAll requests {} resources {}
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent
call last):
> File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
> File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
> KeyError: 'hbtl'
>
>
>
>
>
>
>
> Alberto Scotto
>
> Blue Reply
> Via Cardinal Massaia, 83
> 10147 - Torino - ITALY
> phone: +39 011 29100
> al.scotto(a)reply.it
> www.reply.it
>
> -----Original Message-----
> From: Haim [mailto:hateya@redhat.com]
> Sent: giovedì 23 agosto 2012 17:00
> To: Scotto Alberto
> Cc: users(a)ovirt.org
> Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve
> LUNs
>
> On 08/23/2012 05:54 PM, Scotto Alberto wrote:
>
> hi,
>
> can you attach full vdsm log during the execution of getDeviceList command?
>> Hi all,
>>
>> I'm trying to configure a FCP storage domain on RHEV 3.
>>
>> I try to add a new domain from the console, but it can't find any
>> LUNs: "Could not retrieve LUNs, please check your storage"
>>
>> Here is the output from /var/log/rhevm/rhevm.log:
>>
>> ------------------------------------
>>
>> 2007-06-29 21:50:07,811 WARN
>> [org.ovirt.engine.core.bll.GetConfigurationValueQuery]
>> (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
>> version, using default general for version
>> 2007-06-29 21:50:07,911 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
>> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id:
>> 60bdafe6
>> 2007-06-29 21:50:08,726 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
>> message VDSGenericException: VDSErrorException: Failed to
>> GetDeviceListVDS, error = Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Command
>> org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
>> return value
>>
>> Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
>> lunList Null
>> mStatus Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>> mCode 600
>> mMessage Error block device action: ()
>>
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
>> (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception:
>> VDSErrorException: VDSGenericException: VDSErrorException: Failed to
>> GetDeviceListVDS, error = Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
>> 60bdafe6
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
>> (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
>> message is VdcBLLException:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
>> VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
>> error = Error block device action: ()
>>
>> ----------------------------------------------
>>
>> First question: do LUNs have to be visible from RHEV-H or RHEV-M?
>>
>> Currently they are visible only from the hypervisor.
>>
>> ----------------------------------------
>>
>> [root@pittor06vhxd020 log]# multipath -ll
>> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G
>> features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
>> |-+- policy='round-robin 0' prio=1 status=active
>> | |- 2:0:0:0 sda 8:0 active ready running
>> | |- 2:0:1:0 sdd 8:48 active ready running
>> | |- 3:0:0:0 sde 8:64 active ready running
>> | `- 3:0:1:0 sdf 8:80 active ready running
>> `-+- policy='round-robin 0' prio=0 status=enabled
>> |- 2:0:2:0 sdb 8:16 active ready running
>> |- 2:0:3:0 sdc 8:32 active ready running
>> |- 3:0:2:0 sdg 8:96 active ready running
>> `- 3:0:3:0 sdh 8:112 active ready running
>> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G
>> features='1 queue_if_no_path' hwhandler='0' wp=rw
>> `-+- policy='round-robin 0' prio=1 status=active
>> `- 0:0:1:0 cciss!c0d1 104:16 active ready running
>> ------------------------------------------------------
>>
>> Our SAN device is Clariion AX150. Is it compatible with ovirt?
>>
>> vdsClient -s 0 getDeviceListgives me:
>>
>> Error block device action: ()
>>
>> Could it be due to SPM turned off? (I have only one host)
>>
>> [root@pittor06vhxd020 log]# ps axu | grep -i spm
>>
>> root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
>>
>> How can I turn it on? I know the command but I don't know what
>> paramaters append
>>
>> spmStart
>>
>> <spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing> <maxHostID>
>> <version>
>>
>> Thank you very much for any hints.
>>
>> AS
>>
>>
>>
>> Alberto Scotto
>>
>> Blue
>> Via Cardinal Massaia, 83
>> 10147 - Torino - ITALY
>> phone: +39 011 29100
>> al.scotto(a)reply.it
>> www.reply.it
>>
>>
>> ---------------------------------------------------------------------
>> -
>> --
>>
>> --
>> The information transmitted is intended for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of,
>> or taking of any action in reliance upon, this information by persons
>> or entities other than the intended recipient is prohibited. If you
>> received this in error, please contact the sender and delete the
>> material from any computer.
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
> ________________________________
>
> --
> The information transmitted is intended for the person or entity to which it is
addressed and may contain confidential and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer.
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
12:44 p.m.
It worked!
For the sake of knowledge, these are the steps I took (on RHEL6):
1) edited /etc/multipath.conf, adding
---------------------
blacklist {
wwid 3600508b1001035333920202020200005
}
---------------------
2) service multipathd reload
3) multipath -f 3600508b1001035333920202020200005
I also filed the bug as you suggested
https://bugzilla.redhat.com/show_bug.cgi?id=851478
Thank you very much Haim!
Best regards
PS: actually adding a new domain still doesn't work.. What a mess... At least now it
lists my LUNs correctly :)
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 18:22
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 07:13 PM, Scotto Alberto wrote:
I was going to reply.. :)
It looks like it halts after an error due to cciss!c0d1, which btw is displayed by
multipath -ll.
That's just a local disk, isn't it? So it shouldn't even be listed. I may
have attached it by mistake, playing with /sys/class/fc_* tools.
So, if I remove that path, everything should go ok. Do you think so too?
Yes, please remove and check again (make sure to clean device-mapper table with dmsetup
remove $dm).
anyhow, vdsm should be more robust, so please file a bug for
it(https://bugzilla.redhat.com/enter_bug.cgi?product=oVirt).
Haim
> it appears that vdsm fails to handle a device with '!' in it
> (cciss!c0d1), but let's make sure its indeed the case
More than that the fact is that the path scsi_disk/ doesn't exist in
/sys/block/cciss!c0d1/device And this must be due to the fact that
c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1 root
root 0 2007-06-30 00:37 cciss!c0d0 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/ccis
s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 ->
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/ccis
s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 ->
../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30
00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root root 0
2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx. 1
root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 ->
../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30
01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root root 0
2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx. 1
root root 0 2007-06-30 00:10 loop0 -> ../devices/virtual/block/loop0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 ->
../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 loop4 -> ../devices/virtual/block/loop4
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 ->
../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram0 -> ../devices/virtual/block/ram0
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 ->
../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram12 -> ../devices/virtual/block/ram12
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 ->
../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root root
0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15 lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram2 -> ../devices/virtual/block/ram2
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 ->
../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root root 0
2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx. 1
root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6
lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 ->
../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root root 0
2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx. 1
root root 0 2007-06-30 00:36 sda ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg
lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh ->
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh
lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 ->
../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table
3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1
queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80
1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1
HostVG-Logging: 0 4194304 linear 104:4 24741888
HostVG-Swap: 0 24723456 linear 104:4 2048
3600508b1001035333920202020200005: 0 429925920 multipath 1
queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1
HostVG-Data: 0 40624128 linear 104:4 28936192
HostVG-Config: 0 16384 linear 104:4 24725504
live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 99.3M 1 loop
loop1 7:1 0 1G 1 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
loop2 7:2 0 512M 0 loop
ââlive-rw (dm-1) 253:1 0 1G 0 dm /
cciss!c0d0 104:0 0 33.9G 0 disk
ââcciss!c0d0p1 104:1 0 243M 0 part
ââcciss!c0d0p2 104:2 0 244M 0 part
ââcciss!c0d0p3 104:3 0 244M 0 part
ââcciss!c0d0p4 104:4 0 33.2G 0 part
ââHostVG-Swap (dm-3) 253:3 0 11.8G 0 lvm [SWAP]
ââHostVG-Config (dm-4) 253:4 0 8M 0 lvm /config
ââHostVG-Logging (dm-5) 253:5 0 2G 0 lvm /var/log
ââHostVG-Data (dm-6) 253:6 0 19.4G 0 lvm /data
cciss!c0d1 104:16 0 205G 0 disk
ââ3600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0 mpath
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sda 8:0 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdc 8:32 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdd 8:48 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sde 8:64 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdf 8:80 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdg 8:96 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
sdh 8:112 0 650G 0 disk
ââ3600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0 mpath
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]
Sent: giovedì 23 agosto 2012 18:01
To: Scotto Alberto
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve
LUNs
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
> Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it (cciss!c0d1), but
let's make sure its indeed the case.
>
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper)
> [10.16.250.216]
> Thread-47346::INFO::2007-06-30
> 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run
> and protect: getDeviceList, args: ()
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init -> state
> preparing
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to enter
> sampling method (storage.sdc.refreshStorage)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to enter
> sampling method (storage.iscsi.rescan)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan)
> '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err>
> = 'iscsiadm: No session found.\n'; <rc> = 21
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
> '/usr/bin/sudo -n /sbin/multipath' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan) SUCCESS:
> <err> = ''; <rc> = 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs) Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs) Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation 'lvm
> reload operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n
> /sbin/lvm pvs --config " devices { preferred_names =
> [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
> disable_after_error_count=3 filter = [
> \\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece
> 1 11%\\", \\"r%.*%\\" ] } global { locking_type=1
> prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min =
> 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator
> | -o
> uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_c
> o
> unt,dev_size' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '
> /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n
> /dev/sdh: read failed after 0 of 4096 at 697932120064: Input/output
> error\n /dev/sdh: read failed after 0 of 4096 at 697932177408:
> Input/output error\n WARNING: Error counts reached a limit of 3.
> Device /dev/sdh was disabled\n'; <rc> = 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation 'lvm
> reload operation' released the operation mutex
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_g
> MainProcess|e
> MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_g
> MainProcess|e
> MainProcess|tPathsStatus) SUCCESS: <err> = ''; <rc> = 0
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getSc
> MainProcess|s
> MainProcess|iSerial) '/sbin/scsi_id --page=0x80 --whitelisted
> MainProcess|--export --replace-whitespace --device=/dev/dm-0' (cwd
> MainProcess|None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
> MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getSc
> MainProcess|s
> MainProcess|iSerial) SUCCESS: <err> = ''; <rc> = 0
> Thread-47346::WARNING::2007-06-30
> 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter) Problem getting hbtl
from device `cciss!c0d1` Traceback (most recent call last):
> File "/usr/share/vdsm/storage/multipath.py", line 259, in pathListIter
> File "/usr/share/vdsm/storage/multipath.py", line 182, in getHBTL
> OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/device/scsi_disk/'
> Thread-47346::ERROR::2007-06-30
> 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected error Traceback
(most recent call last):
> File "/usr/share/vdsm/storage/task.py", line 876, in _run
> File "/usr/share/vdsm/storage/hsm.py", line 696, in
public_getDeviceList
> File "/usr/share/vdsm/storage/hsm.py", line 759, in
> _getDeviceList
> KeyError: 'hbtl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run:
> 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing
> (force False)
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
> Thread-47346::INFO::2007-06-30
> 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting: Task
> is aborted: "'hbtl'" - code 100
> Thread-47346::DEBUG::2007-06-30 00:37:11,031::task::495::TaskManager.Task::(_debug)
Task 0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'hbtl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll
> )
> Owner.cancelAll requests {}
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing ->
> state aborting
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy none
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting ->
> state failed
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAl
> l
> ) Owner.releaseAll requests {} resources {}
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll
> )
> Owner.cancelAll requests {}
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent
call last):
> File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
> File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
> KeyError: 'hbtl'
>
>
>
>
>
>
>
> Alberto Scotto
>
> Blue Reply
> Via Cardinal Massaia, 83
> 10147 - Torino - ITALY
> phone: +39 011 29100
> al.scotto(a)reply.it
> www.reply.it
>
> -----Original Message-----
> From: Haim [mailto:hateya@redhat.com]
> Sent: giovedì 23 agosto 2012 17:00
> To: Scotto Alberto
> Cc: users(a)ovirt.org
> Subject: Re: [Users] [rhev 3] add new domain fails: Could not
> retrieve LUNs
>
> On 08/23/2012 05:54 PM, Scotto Alberto wrote:
>
> hi,
>
> can you attach full vdsm log during the execution of getDeviceList command?
>> Hi all,
>>
>> I'm trying to configure a FCP storage domain on RHEV 3.
>>
>> I try to add a new domain from the console, but it can't find any
>> LUNs: "Could not retrieve LUNs, please check your storage"
>>
>> Here is the output from /var/log/rhevm/rhevm.log:
>>
>> ------------------------------------
>>
>> 2007-06-29 21:50:07,811 WARN
>> [org.ovirt.engine.core.bll.GetConfigurationValueQuery]
>> (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with null
>> version, using default general for version
>> 2007-06-29 21:50:07,911 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId =
>> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=FCP), log id:
>> 60bdafe6
>> 2007-06-29 21:50:08,726 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and error
>> message VDSGenericException: VDSErrorException: Failed to
>> GetDeviceListVDS, error = Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Command
>> org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
>> return value
>>
>> Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
>> lunList Null
>> mStatus Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>> mCode 600
>> mMessage Error block device action: ()
>>
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
>> (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution failed. Exception:
>> VDSErrorException: VDSGenericException: VDSErrorException: Failed to
>> GetDeviceListVDS, error = Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log id:
>> 60bdafe6
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
>> (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed. Exception
>> message is VdcBLLException:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
>> VDSGenericException: VDSErrorException: Failed to GetDeviceListVDS,
>> error = Error block device action: ()
>>
>> ----------------------------------------------
>>
>> First question: do LUNs have to be visible from RHEV-H or RHEV-M?
>>
>> Currently they are visible only from the hypervisor.
>>
>> ----------------------------------------
>>
>> [root@pittor06vhxd020 log]# multipath -ll
>> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=650G
>> features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
>> |-+- policy='round-robin 0' prio=1 status=active
>> | |- 2:0:0:0 sda 8:0 active ready running
>> | |- 2:0:1:0 sdd 8:48 active ready running
>> | |- 3:0:0:0 sde 8:64 active ready running
>> | `- 3:0:1:0 sdf 8:80 active ready running
>> `-+- policy='round-robin 0' prio=0 status=enabled
>> |- 2:0:2:0 sdb 8:16 active ready running
>> |- 2:0:3:0 sdc 8:32 active ready running
>> |- 3:0:2:0 sdg 8:96 active ready running
>> `- 3:0:3:0 sdh 8:112 active ready running
>> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME size=205G
>> features='1 queue_if_no_path' hwhandler='0' wp=rw
>> `-+- policy='round-robin 0' prio=1 status=active
>> `- 0:0:1:0 cciss!c0d1 104:16 active ready running
>> ------------------------------------------------------
>>
>> Our SAN device is Clariion AX150. Is it compatible with ovirt?
>>
>> vdsClient -s 0 getDeviceListgives me:
>>
>> Error block device action: ()
>>
>> Could it be due to SPM turned off? (I have only one host)
>>
>> [root@pittor06vhxd020 log]# ps axu | grep -i spm
>>
>> root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
>>
>> How can I turn it on? I know the command but I don't know what
>> paramaters append
>>
>> spmStart
>>
>> <spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing>
>> <maxHostID> <version>
>>
>> Thank you very much for any hints.
>>
>> AS
>>
>>
>>
>> Alberto Scotto
>>
>> Blue
>> Via Cardinal Massaia, 83
>> 10147 - Torino - ITALY
>> phone: +39 011 29100
>> al.scotto(a)reply.it
>> www.reply.it
>>
>>
>> --------------------------------------------------------------------
>> -
>> -
>> --
>>
>> --
>> The information transmitted is intended for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of,
>> or taking of any action in reliance upon, this information by
>> persons or entities other than the intended recipient is prohibited.
>> If you received this in error, please contact the sender and delete
>> the material from any computer.
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
> ________________________________
>
> --
> The information transmitted is intended for the person or entity to which it is
addressed and may contain confidential and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer.
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
________________________________
--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any computer.
7:11 p.m.
--_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: multipart/related;
boundary="_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_";
type="multipart/alternative"
--_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: multipart/alternative;
boundary="_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_"
--_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
After two or three reinstallations of RHEV-H, surprisingly, now the behavio=
r is as expected: i.e., it skips the cciss disk without breaking.
Thread-11037::WARNING::2012-08-28 15:46:25,949::hsm::735::Storage.HSM::(_ge=
tDeviceList) Ignoring partitioned device {'product': 'LOGICAL VOLUME',
'dm'=
: 'dm-2', 'devtypes': ['FCP'], 'fwrev': '2.08',
'logicalblocksize': '512', =
'connections': [], 'devtype': 'FCP', 'physicalblocksize':
'512', 'vendor': =
'HP', 'serial': 'SHP_LOGICAL_VOLUME_PH79MW7539', 'guid':
'3600508b100103533=
3920202020200005', 'paths': [{'devnum': DeviceNumber(Major=3D104,
Minor=3D1=
6), 'physdev': 'cciss!c0d1', 'type': 'FCP',
'state': 'active'}], 'capacity'=
: '220122071040'}
Attached the full vdsm log.
I didn't upgrade vdsm, I just did a reinstall (two reinstalls, actually). N=
o idea of what happened...!
Btw, the procedure I posted to hide the cciss disk from multipath didn't wo=
rk well: when rebooted, the hypervisor doesn't save the changes, due to the=
live image on which it mounts the root file system.
However now I'm experiencing high latency issues. The message from RHEV-M c=
onsole is: "storage domain XYZ experienced a high latency of 14 seconds [..=
.]".
Any ideas? Should I detach cciss disk once again? I'm gonna try
Alberto Scotto
[Blue]
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto(a)reply.it
www.reply.it
-----Original Message-----
From: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of=
Scotto Alberto
Sent: venerd=EC 24 agosto 2012 11:44
To: Haim
Cc: users(a)ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
It worked!
For the sake of knowledge, these are the steps I took (on RHEL6):
1) edited /etc/multipath.conf, adding
---------------------
blacklist {
wwid 3600508b1001035333920202020200005 }
---------------------
2) service multipathd reload
3) multipath -f 3600508b1001035333920202020200005
I also filed the bug as you suggested
https://bugzilla.redhat.com/show_bug.cgi?id=3D851478
Thank you very much Haim!
Best regards
PS: actually adding a new domain still doesn't work.. What a mess... At lea=
st now it lists my LUNs correctly :)
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
al.scotto@reply.it<mailto:al.scotto@reply.it
www.reply.it<http://www.reply.it
-----Original Message-----
From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com]
Sent: gioved=EC 23 agosto 2012 18:22
To: Scotto Alberto
Cc: users@ovirt.org<mailto:users@ovirt.org
Subject: Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs
On 08/23/2012 07:13 PM, Scotto Alberto wrote:
> www.reply.it<http://www.reply.it
>
>
>
>
>
>
>
>
>
>> www.reply.it<http://www.reply.it
>
>
>>> <spUUID> <prevID> <prevLVER> <recoveryMode>
<scsiFencing
>>> <maxHostID> <version
>>
>>
>>
>>
>>> www.reply.it<http://www.reply.it
>>
>>
>>
>
>
http://lists.ovirt.org/mailman/listinfo/users
________________________________
--
The information transmitted is intended for the person or entity to which i=
t is addressed and may contain confidential and/or privileged material. Any=
review, retransmission, dissemination or other use of, or taking of any ac=
tion in reliance upon, this information by persons or entities other than t=
he intended recipient is prohibited. If you received this in error, please =
contact the sender and delete the material from any computer.
--_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Diso-8859-=
1" <style <!--
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif"}
span.PlainTextChar
{font-family:"Calibri","sans-serif"}
span.BalloonTextChar
{font-family:"Tahoma","sans-serif"}
.MsoChpDefault
{font-family:"Calibri","sans-serif"}
@page WordSection1
{margin:70.85pt 2.0cm 2.0cm 2.0cm}
div.WordSection1
{}
-- </style
</head <body lang=3D"IT"
link=3D"blue" vlink=3D"purple" <div
class=3D"WordSection1" <p
class=3D"MsoPlainText"><span lang=3D"EN-US">After two or
three reinstall=
ations of RHEV-H, surprisingly, now the behavior is as expected: i.e., it s=
kips the cciss disk without breaking.</span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US"
style=3D"background:yellow">=
Thread-11037::WARNING::2012-08-28 15:46:25,949::hsm::735::Storage.HSM::(_ge=
tDeviceList) Ignoring partitioned device {'product': 'LOGICAL VOLUME',
'dm'=
: 'dm-2', 'devtypes': ['FCP'], 'fwrev':
'2.08', 'logicalblocksize': '512', 'connections': [],
'devtype': 'FCP', 'p=
hysicalblocksize': '512', 'vendor': 'HP', 'serial':
'SHP_LOGICAL_VOLUME_PH7=
9MW7539', 'guid': '3600508b1001035333920202020200005',
'paths': [{'devnum':=
DeviceNumber(Major=3D104, Minor=3D16),
'physdev': 'cciss!c0d1', 'type': 'FCP', 'state':
'active'}], 'capacity': '=
220122071040'}</span><span
lang=3D"EN-US"></span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US">Attached the full
vdsm log.<=
/span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US">I
didn’t upgrade vdsm,=
I just did a reinstall (<i>two</i> reinstalls, actually). No idea of what =
happened…!</span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US">Btw, the
procedure I posted =
to hide the cciss disk from multipath didn’t work well: when rebooted=
, the hypervisor doesn’t save the changes, due to the live image on w=
hich it mounts the root file system.
</span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US">However now
I’m experi=
encing high latency issues. The message from RHEV-M console is: “stor=
age domain XYZ experienced a high latency of 14 seconds […]”.</=
span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US">Any ideas? Should
I detach c=
ciss disk once again? I’m gonna try</span></p <p class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText"><span lang=3D"EN-US"
style=3D""><br <br
</p <div align=3D"left" <p style=3D"font-family:Calibri,Sans-Serif;
font-size:10pt"><span style=3D"=
color:#000000; font-weight:bold">Alberto Scotto</span <span
style=3D"color:#808080"></span><br
<br <span
style=3D"color:#000000"><img border=3D"0" alt=3D"Blue"
src=3D"cid:e6e=
5a80a848243228ec57fb928078181" style=3D"margin:0px" </span><br <span
style=3D"color:#808080">Via Cardinal Massaia, 83<br 10147 - Torino - ITALY <br phone:
+39 011 29100 <br <a
href=3D"al.scotto(a)reply.it" target=3D"" style=3D"color:blue;
text-decora=
tion:underline">al.scotto@reply.it</a
<br <a title=3D""
href=3D"www.reply.it" target=3D"" style=3D"color:blue; text-d=
ecoration:underline">www.reply.it</a
</span><br </p </div
-----Original Message-----<br From:
users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of=
Scotto Alberto<br Sent: venerd=EC 24 agosto 2012
11:44<br To: Haim<br Cc: users(a)ovirt.org<br Subject:
Re: [Users] [rhev 3] add new domain fails: Could not retrieve LUNs=
</span><span lang=3D"EN-US"></span
<p></p <p
class=3D"MsoPlainText"><span
lang=3D"EN-US"> </span></p <p
class=3D"MsoPlainText">It worked!</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">For the sake of knowledge, these are the steps I =
took (on RHEL6):</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">1) edited /etc/multipath.conf, adding</p <p
class=3D"MsoPlainText">---------------------</p <p class=3D"MsoPlainText">blacklist {</p <p
class=3D"MsoPlainText"> wwid
3600508b10010353339202020=
20200005 }</p <p
class=3D"MsoPlainText">---------------------</p <p class=3D"MsoPlainText">2) service multipathd
reload</p <p
class=3D"MsoPlainText">3) multipath -f 3600508b1001035333920202020200005=
</p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">I also filed the bug
as you suggested</p <p
class=3D"MsoPlainText"><a
href=3D"https://bugzilla.redhat.com/show_bug.c=
gi?id=3D851478"><span style=3D"color:windowtext;
text-decoration:none">http=
s://bugzilla.redhat.com/show_bug.cgi?id=3D851478</span></a>&l... <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">Thank you very much
Haim!</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">Best regards</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">PS: actually adding a new domain still doesn't
wo=
rk.. What a mess... At least now it lists my LUNs correctly :)</p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">Alberto
Scotto</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">Blue Reply</p <p
class=3D"MsoPlainText">Via Cardinal Massaia, 83</p <p class=3D"MsoPlainText">10147 - Torino -
ITALY</p <p
class=3D"MsoPlainText">phone: +39 011 29100</p <p class=3D"MsoPlainText"><a
href=3D"mailto:al.scotto@reply.it"><span style=
=3D"color:windowtext;
text-decoration:none">al.scotto@reply.it</span></a></=
p <p class=3D"MsoPlainText"><a
href=3D"http://www.reply.it"><span style=3D"co=
lor:windowtext;
text-decoration:none">www.reply.it</span></a></p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">-----Original
Message-----</p <p
class=3D"MsoPlainText">From: Haim <a
href=3D"mailto:[mailto:hateya@redha=
t.com]"><span style=3D"color:windowtext;
text-decoration:none">[mailto:hate=
ya(a)redhat.com]</span></a></p <p
class=3D"MsoPlainText">Sent: gioved=EC 23 agosto 2012 18:22</p <p class=3D"MsoPlainText">To: Scotto
Alberto</p <p
class=3D"MsoPlainText">Cc: <a
href=3D"mailto:users@ovirt.org"><span styl=
e=3D"color:windowtext;
text-decoration:none">users@ovirt.org</span></a></p <p class=3D"MsoPlainText">Subject: Re: [Users]
[rhev 3] add new domain fail=
s: Could not retrieve LUNs</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">On 08/23/2012 07:13 PM, Scotto Alberto
wrote:</p <p
class=3D"MsoPlainText">> I was going to reply.. :)</p <p class=3D"MsoPlainText">> It looks
like it halts after an error due to=
cciss!c0d1, which btw is displayed by multipath -ll.</p <p
class=3D"MsoPlainText">> That's just a local disk, isn't it?
So it sh=
ouldn't even be listed. I may have attached it by mistake, playing with /sy=
s/class/fc_* tools.</p <p
class=3D"MsoPlainText">> So, if I remove that path, everything
should=
go ok. Do you think so too?</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">Yes, please remove and check again (make sure to =
clean device-mapper table with dmsetup remove $dm).</p <p
class=3D"MsoPlainText">anyhow, vdsm should be more robust, so please fil=
e a bug for it(<a href=3D"https://bugzilla.redhat.com/enter_bug.cgi?product=
=3DoVirt"><span style=3D"color:windowtext;
text-decoration:none">https://bu=
gzilla.redhat.com/enter_bug.cgi?product=3DoVirt</span></a>).&... <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">Haim</p <p class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">>> it appears that vdsm fails to handle
a d=
evice with '!' in it
</p <p class=3D"MsoPlainText">>>
(cciss!c0d1), but let's make sure its in=
deed the case</p <p
class=3D"MsoPlainText">> More than that the fact is that the path
scs=
i_disk/ doesn't exist in
</p <p class=3D"MsoPlainText">>
/sys/block/cciss!c0d1/device And this must b=
e due to the fact that</p <p
class=3D"MsoPlainText">> c0d1 is NOT a damn scsi disk</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> Anyway, here is your output</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# ls -l /sys/block/
=
total 0 lrwxrwxrwx. 1 root
</p <p class=3D"MsoPlainText">> root 0
2007-06-30 00:37 cciss!c0d0 -> </p=
<p class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:03.0/0000:06:0=
0.0/cciss0/c0d0/block/ccis</p <p
class=3D"MsoPlainText">> s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30
=
00:32 cciss!c0d1 ->
</p <p class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:03.0/0000:06:0=
0.0/cciss0/c0d1/block/ccis</p <p
class=3D"MsoPlainText">> s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30
=
01:17 dm-0 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/dm-0 lrwxrwxrwx. 1
=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:40 dm-1 ->
../devices/virtual/block/dm=
-1 lrwxrwxrwx. 1 root root 0</p <p
class=3D"MsoPlainText">> 2007-06-30 01:17 dm-2 ->
../devices/virtu=
al/block/dm-2 lrwxrwxrwx. 1
</p <p class=3D"MsoPlainText">> root root 0
2007-06-30 00:49 dm-3 -> ../d=
evices/virtual/block/dm-3
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:49 d=
m-4 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/dm-4 lrwxrwxrwx. 1
=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 01:00 dm-5 ->
../devices/virtual/block/dm=
-5 lrwxrwxrwx. 1 root root 0</p <p
class=3D"MsoPlainText">> 2007-06-30 00:49 dm-6 ->
../devices/virtu=
al/block/dm-6 lrwxrwxrwx. 1
</p <p class=3D"MsoPlainText">> root root 0
2007-06-30 00:10 loop0 -> ../=
devices/virtual/block/loop0
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 l=
oop1 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/loop1 lrwxrwxrwx.
1=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 loop2 ->
../devices/virtual/block/l=
oop2 lrwxrwxrwx. 1 root root</p <p
class=3D"MsoPlainText">> 0 2007-06-30 00:10 loop3 ->
../devices/vi=
rtual/block/loop3 lrwxrwxrwx.</p <p
class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 loop4
-> .=
./devices/virtual/block/loop4
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 l=
oop5 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/loop5 lrwxrwxrwx.
1=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 loop6 ->
../devices/virtual/block/l=
oop6 lrwxrwxrwx. 1 root root</p <p
class=3D"MsoPlainText">> 0 2007-06-30 00:10 loop7 ->
../devices/vi=
rtual/block/loop7 lrwxrwxrwx.</p <p
class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram0 ->
..=
/devices/virtual/block/ram0
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 r=
am1 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/ram1 lrwxrwxrwx. 1
=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 ram10 ->
../devices/virtual/block/r=
am10 lrwxrwxrwx. 1 root root</p <p
class=3D"MsoPlainText">> 0 2007-06-30 00:10 ram11 ->
../devices/vi=
rtual/block/ram11 lrwxrwxrwx.</p <p
class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram12
-> .=
./devices/virtual/block/ram12
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 r=
am13 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/ram13 lrwxrwxrwx.
1=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 ram14 ->
../devices/virtual/block/r=
am14 lrwxrwxrwx. 1 root root</p <p
class=3D"MsoPlainText">> 0 2007-06-30 00:10 ram15 ->
../devices/vi=
rtual/block/ram15 lrwxrwxrwx.</p <p
class=3D"MsoPlainText">> 1 root root 0 2007-06-30 00:10 ram2 ->
..=
/devices/virtual/block/ram2
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 r=
am3 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/ram3 lrwxrwxrwx. 1
=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 ram4 ->
../devices/virtual/block/ra=
m4 lrwxrwxrwx. 1 root root 0</p <p
class=3D"MsoPlainText">> 2007-06-30 00:10 ram5 ->
../devices/virtu=
al/block/ram5 lrwxrwxrwx. 1
</p <p class=3D"MsoPlainText">> root root 0
2007-06-30 00:10 ram6 -> ../d=
evices/virtual/block/ram6
</p <p class=3D"MsoPlainText">> lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 r=
am7 -></p <p
class=3D"MsoPlainText">> ../devices/virtual/block/ram7 lrwxrwxrwx. 1
=
root root 0 2007-06-30</p <p
class=3D"MsoPlainText">> 00:10 ram8 ->
../devices/virtual/block/ra=
m8 lrwxrwxrwx. 1 root root 0</p <p
class=3D"MsoPlainText">> 2007-06-30 00:10 ram9 ->
../devices/virtu=
al/block/ram9 lrwxrwxrwx. 1
</p <p class=3D"MsoPlainText">> root root 0
2007-06-30 00:36 sda -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/bl=
ock/sda</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
db -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/bl=
ock/sdb</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
dc -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/bl=
ock/sdc</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
dd -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/bl=
ock/sdd</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
de -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/bl=
ock/sde</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
df -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/bl=
ock/sdf</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
dg -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/bl=
ock/sdg</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59
s=
dh -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:02.0/0000:09:0=
0.0/0000:0a:00.0/0000:0b:0</p <p
class=3D"MsoPlainText">>
0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/bl=
ock/sdh</p <p
class=3D"MsoPlainText">> lrwxrwxrwx. 1 root root 0 2007-06-30 00:49
s=
r0 -></p <p
class=3D"MsoPlainText">>
../devices/pci0000:00/0000:00:1f.1/host0/tar=
get0:0:0/0:0:0:0/block/sr0</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# dmsetup
table</p <p
class=3D"MsoPlainText">> 3600601601cde1d0066b2fb054dece111: 0
1363148=
800 multipath 1
</p <p class=3D"MsoPlainText">>
queue_if_no_path 1 emc 2 1 round-robin 0 4 1=
8:0 1 8:48 1 8:64 1 8:80</p <p
class=3D"MsoPlainText">> 1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1
8:1=
12 1</p <p
class=3D"MsoPlainText">> HostVG-Logging: 0 4194304 linear 104:4
24741=
888</p <p
class=3D"MsoPlainText">> HostVG-Swap: 0 24723456 linear 104:4
2048</p=
<p class=3D"MsoPlainText">>
3600508b1001035333920202020200005: 0 4299259=
20 multipath 1
</p <p class=3D"MsoPlainText">>
queue_if_no_path 0 1 1 round-robin 0 1 1 104=
:16 1</p <p
class=3D"MsoPlainText">> HostVG-Data: 0 40624128 linear 104:4
2893619=
2</p <p class=3D"MsoPlainText">>
HostVG-Config: 0 16384 linear 104:4 24725504=
</p <p class=3D"MsoPlainText">> live-rw: 0
2097152 snapshot 7:1 7:2 P 8</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> [root@pittor06vhxd020 ~]# lsblk</p <p class=3D"MsoPlainText">>
NAME &nbs=
p; &=
nbsp; &nbs=
p; MAJ:MIN
RM SIZE RO TYPE&=
nbsp; MOUNTPOINT</p <p
class=3D"MsoPlainText">>
loop0 &nb=
sp; =
&nb=
sp;
7:0 0 =
99.3M 1 loop</p <p
class=3D"MsoPlainText">>
loop1 &nb=
sp; =
&nb=
sp;
7:1 0 =
1G 1 loop</p <p
class=3D"MsoPlainText">> =E2=E2live-rw
(dm-1) =
&nb=
sp;
253:1 =
0 1G 0
dm /</p <p
class=3D"MsoPlainText">>
loop2 &nb=
sp; =
&nb=
sp;
7:2 0 =
512M 0 loop</p <p
class=3D"MsoPlainText">> =E2=E2live-rw
(dm-1) =
&nb=
sp;
253:1 =
0 1G 0
dm /</p <p
class=3D"MsoPlainText">>
cciss!c0d0 &nbs=
p; &=
nbsp; &nbs=
p; 104:0 0 33.9G 0
disk</p <p
class=3D"MsoPlainText">>
=E2=E2cciss!c0d0p1 &n=
bsp;  =
;
104:1&n=
bsp; 0 243M 0 part</p <p class=3D"MsoPlainText">>
=E2=E2cciss!c0d0p2 &n=
bsp;  =
;
104:2&n=
bsp; 0 244M 0 part</p <p class=3D"MsoPlainText">>
=E2=E2cciss!c0d0p3 &n=
bsp;  =
;
104:3&n=
bsp; 0 244M 0 part</p <p class=3D"MsoPlainText">>
=E2=E2cciss!c0d0p4 &n=
bsp;  =
;
104:4&n=
bsp; 0 33.2G 0 part</p <p
class=3D"MsoPlainText">>
=E2=E2HostVG-Swap (dm-3)&n=
bsp;  =
;
253:3 0 =
11.8G 0 lvm [SWAP]</p <p
class=3D"MsoPlainText">>
=E2=E2HostVG-Config (dm-4)=
&nb=
sp; 253:4
0 &nbs=
p; 8M 0 lvm /config</p <p
class=3D"MsoPlainText">>
=E2=E2HostVG-Logging (dm-5=
) &n=
bsp; 253:5
0 &nb=
sp; 2G 0 lvm /var/log</p <p
class=3D"MsoPlainText">>
=E2=E2HostVG-Data (dm-6)&n=
bsp;  =
;
253:6 0 =
19.4G 0 lvm /data</p <p
class=3D"MsoPlainText">>
cciss!c0d1 &nbs=
p; &=
nbsp; &nbs=
p; 104:16 0 205G 0
disk</p <p
class=3D"MsoPlainText">> =E2=E23600508b1001035333920202020200005
(dm-=
0) 253:0 0 205G 0
mpath</p <p
class=3D"MsoPlainText">>
sr0  =
; &n=
bsp;  =
;
11:0 1&=
nbsp; 1024M 0 rom</p <p
class=3D"MsoPlainText">>
sdb  =
; &n=
bsp;  =
;
8:16 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sda  =
; &n=
bsp;  =
;
8:0 &nbs=
p; 0 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sdc  =
; &n=
bsp;  =
;
8:32 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sdd  =
; &n=
bsp;  =
;
8:48 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sde  =
; &n=
bsp;  =
;
8:64 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sdf  =
; &n=
bsp;  =
;
8:80 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sdg  =
; &n=
bsp;  =
;
8:96 0&=
nbsp; 650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">>
sdh &nbs=
p; &=
nbsp; &nbs=
p; 8:112
0 =
650G 0 disk</p <p
class=3D"MsoPlainText">> =E2=E23600601601cde1d0066b2fb054dece111
(dm-=
2) 253:2 0 650G 0
mpath</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> Alberto Scotto</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> Blue Reply</p <p
class=3D"MsoPlainText">> Via Cardinal Massaia, 83</p <p class=3D"MsoPlainText">> 10147 -
Torino - ITALY</p <p
class=3D"MsoPlainText">> phone: +39 011 29100</p <p class=3D"MsoPlainText">> <a
href=3D"mailto:al.scotto@reply.it"><span =
style=3D"color:windowtext;
text-decoration:none">al.scotto@reply.it</span><=
/a></p <p
class=3D"MsoPlainText">> <a
href=3D"http://www.reply.it"><span style=
=3D"color:windowtext;
text-decoration:none">www.reply.it</span></a></p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> -----Original Message-----</p <p class=3D"MsoPlainText">> From: Haim
<a href=3D"mailto:[mailto:hateya@=
redhat.com]" <span
style=3D"color:windowtext; text-decoration:none">[mailto:hateya@redha=
t.com]</span></a></p <p
class=3D"MsoPlainText">> Sent: gioved=EC 23 agosto 2012
18:01</p <p
class=3D"MsoPlainText">> To: Scotto Alberto</p <p class=3D"MsoPlainText">> Cc: <a
href=3D"mailto:users@ovirt.org"><span=
style=3D"color:windowtext;
text-decoration:none">users@ovirt.org</span></a=
></p <p
class=3D"MsoPlainText">> Subject: Re: [Users] [rhev 3] add new
domain=
fails: Could not retrieve
</p <p class=3D"MsoPlainText">>
LUNs</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> On 08/23/2012 06:20 PM, Scotto Alberto
wrote=
:</p <p class=3D"MsoPlainText">>> Here
you are</p <p
class=3D"MsoPlainText">> thanks, can you run the following?</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> - ls -l /sys/block/</p <p class=3D"MsoPlainText">> - dmsetup
table</p <p
class=3D"MsoPlainText">> - lsblk (if exists)</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> it appears that vdsm fails to handle a
devic=
e with '!' in it (cciss!c0d1), but let's make sure its indeed the
case.</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,268::clientIF::239::Storage.Dis=
patcher.Protect::(wrapper)</p <p
class=3D"MsoPlainText">>> [10.16.250.216]</p <p class=3D"MsoPlainText">>>
Thread-47346::INFO::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,269::dispatcher::94::Storage.Di=
spatcher.Protect::(run) Run
</p <p class=3D"MsoPlainText">>> and
protect: getDeviceList, args: ()</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,269::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
mo=
ving from state init -> state
</p <p class=3D"MsoPlainText">>>
preparing</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,269::misc::1010::SamplingMethod=
::(__call__) Trying to enter
</p <p class=3D"MsoPlainText">>>
sampling method (storage.sdc.refreshStor=
age)</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,270::misc::1012::SamplingMethod=
::(__call__) Got in to
</p <p class=3D"MsoPlainText">>>
sampling method</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,270::misc::1010::SamplingMethod=
::(__call__) Trying to enter
</p <p class=3D"MsoPlainText">>>
sampling method (storage.iscsi.rescan)</=
p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,270::misc::1012::SamplingMethod=
::(__call__) Got in to
</p <p class=3D"MsoPlainText">>>
sampling method</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,271::iscsi::699::Storage.Misc.e=
xcCmd::(rescan)</p <p
class=3D"MsoPlainText">>> '/usr/bin/sudo -n /sbin/iscsiadm
-m sess=
ion -R' (cwd None)</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,300::iscsi::699::Storage.Misc.e=
xcCmd::(rescan) FAILED: <err>
</p <p class=3D"MsoPlainText">>> =3D
'iscsiadm: No session found.\n'; <=
;rc> =3D 21</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,301::misc::1020::SamplingMethod=
::(__call__) Returning last
</p <p class=3D"MsoPlainText">>>
result</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,661::multipath::61::Storage.Mis=
c.excCmd::(rescan)</p <p
class=3D"MsoPlainText">>> '/usr/bin/sudo -n
/sbin/multipath' (cwd =
None)</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,785::multipath::61::Storage.Mis=
c.excCmd::(rescan) SUCCESS:</p <p
class=3D"MsoPlainText">>> <err> =3D '';
<rc> =3D 0</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,786::lvm::547::OperationMutex::=
(_invalidateAllPvs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' got the opera=
tion mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,786::lvm::549::OperationMutex::=
(_invalidateAllPvs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' released the =
operation mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,786::lvm::559::OperationMutex::=
(_invalidateAllVgs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' got the opera=
tion mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,787::lvm::561::OperationMutex::=
(_invalidateAllVgs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' released the =
operation mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,787::lvm::580::OperationMutex::=
(_invalidateAllLvs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' got the opera=
tion mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,788::lvm::582::OperationMutex::=
(_invalidateAllLvs) Operation
</p <p class=3D"MsoPlainText">>>
'lvm invalidate operation' released the =
operation mutex</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,788::misc::1020::SamplingMethod=
::(__call__) Returning last
</p <p class=3D"MsoPlainText">>>
result</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,788::lvm::406::OperationMutex::=
(_reloadpvs) Operation 'lvm
</p <p class=3D"MsoPlainText">>>
reload operation' got the operation mute=
x</p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,791::lvm::374::Storage.Misc.exc=
Cmd::(cmd) '/usr/bin/sudo -n
</p <p class=3D"MsoPlainText">>>
/sbin/lvm pvs --config " devices { =
preferred_names =3D </p <p
class=3D"MsoPlainText">>>
[\\"^/dev/mapper/\\"] ignore_s=
uspended_devices=3D1 write_cache_state=3D0</p <p
class=3D"MsoPlainText">>> disable_after_error_count=3D3 filter
=3D=
[ </p <p
class=3D"MsoPlainText">>> <a
href=3D"file:///\\%22a%253600508b1001=
035333920202020200005|3600601601cde1d0066b2fb054dece" <span
style=3D"color:windowtext; text-decoration:none">\\"a%3600508b10=
01035333920202020200005|3600601601cde1d0066b2fb054dece</span></a></p <p class=3D"MsoPlainText">>> 1
11%\\", <a href=3D"file:///\\%22r=
%25.*%25\"><span style=3D"color:windowtext;
text-decoration:none">\\"r=
%.*%\\</span></a>" ] } global {
locking_type=3D1</p <p
class=3D"MsoPlainText">>> prioritise_write_locks=3D1
wait_fo=
r_locks=3D1 } backup { retain_min =3D</p <p class=3D"MsoPlainText">>>
50 retain_days =3D 0 } " --no=
headings --units b --nosuffix --separator</p <p
class=3D"MsoPlainText">>> | -o</p <p class=3D"MsoPlainText">>>
uuid,name,size,vg_name,vg_uuid,pe_start,=
pe_count,pe_alloc_count,mda_c</p <p
class=3D"MsoPlainText">>> o</p <p
class=3D"MsoPlainText">>> unt,dev_size' (cwd
None)</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,997::lvm::374::Storage.Misc.exc=
Cmd::(cmd) SUCCESS: <err> =3D '</p <p
class=3D"MsoPlainText">>> /dev/sdh: read failed after 0 of 4096
at=
0: Input/output error\n</p <p
class=3D"MsoPlainText">>> /dev/sdh: read failed after 0 of 4096
at=
697932120064: Input/output
</p <p class=3D"MsoPlainText">>>
error\n /dev/sdh: read failed afte=
r 0 of 4096 at 697932177408:</p <p
class=3D"MsoPlainText">>> Input/output error\n
WARNING: Erro=
r counts reached a limit of 3.</p <p
class=3D"MsoPlainText">>> Device /dev/sdh was disabled\n';
<rc&=
gt; =3D 0</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:10,998::lvm::429::OperationMutex::=
(_reloadpvs) Operation 'lvm
</p <p class=3D"MsoPlainText">>>
reload operation' released the operation=
mutex</p <p
class=3D"MsoPlainText">>>
MainProcess|Thread-47346::DEBUG::2007-06=
-30</p <p
class=3D"MsoPlainText">>>
MainProcess|00:37:11,005::devicemapper::=
144::Storage.Misc.excCmd::(_g</p <p
class=3D"MsoPlainText">>> MainProcess|e</p <p class=3D"MsoPlainText">>>
MainProcess|tPathsStatus) '/sbin/dmsetup=
status' (cwd None)</p <p
class=3D"MsoPlainText">>>
MainProcess|Thread-47346::DEBUG::2007-06=
-30</p <p
class=3D"MsoPlainText">>>
MainProcess|00:37:11,014::devicemapper::=
144::Storage.Misc.excCmd::(_g</p <p
class=3D"MsoPlainText">>> MainProcess|e</p <p class=3D"MsoPlainText">>>
MainProcess|tPathsStatus) SUCCESS: <e=
rr> =3D ''; <rc> =3D 0</p <p
class=3D"MsoPlainText">>>
MainProcess|Thread-47346::DEBUG::2007-06=
-30</p <p
class=3D"MsoPlainText">>>
MainProcess|00:37:11,019::multipath::159=
::Storage.Misc.excCmd::(getSc</p <p
class=3D"MsoPlainText">>> MainProcess|s</p <p class=3D"MsoPlainText">>>
MainProcess|iSerial) '/sbin/scsi_id --pa=
ge=3D0x80 --whitelisted
</p <p class=3D"MsoPlainText">>>
MainProcess|--export --replace-whitespac=
e --device=3D/dev/dm-0' (cwd</p <p
class=3D"MsoPlainText">>> MainProcess|None)</p <p class=3D"MsoPlainText">>>
MainProcess|Thread-47346::DEBUG::2007-06=
-30</p <p
class=3D"MsoPlainText">>>
MainProcess|00:37:11,026::multipath::159=
::Storage.Misc.excCmd::(getSc</p <p
class=3D"MsoPlainText">>> MainProcess|s</p <p class=3D"MsoPlainText">>>
MainProcess|iSerial) SUCCESS: <err>=
; =3D ''; <rc> =3D 0</p <p
class=3D"MsoPlainText">>>
Thread-47346::WARNING::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,027::multipath::261::Storage.Mu=
ltipath::(pathListIter) Problem getting hbtl from device `cciss!c0d1` Trace=
back (most recent call last):</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/multipath.py", line 259, in pathListIter</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/multipath.py", line 182, in getHBTL</p <p class=3D"MsoPlainText">>>
OSError: [Errno 2] No such file or direc=
tory: '/sys/block/cciss!c0d1/device/scsi_disk/'</p <p
class=3D"MsoPlainText">>>
Thread-47346::ERROR::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,029::task::868::TaskManager.Tas=
k::(_setError) Unexpected error Traceback (most recent call last):</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/task.py", line 876, in _run</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/hsm.py", line 696, in public_getDeviceList</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/hsm.py", line 759, in
</p <p class=3D"MsoPlainText">>>
_getDeviceList</p <p
class=3D"MsoPlainText">>> KeyError: 'hbtl'</p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,030::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
Ta=
sk._run:</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408
() =
{} failed - stopping task</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,030::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
st=
opping in state preparing
</p <p class=3D"MsoPlainText">>>
(force False)</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,030::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
re=
f 1 aborting True</p <p
class=3D"MsoPlainText">>>
Thread-47346::INFO::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,031::task::1171::TaskManager.Ta=
sk::(prepare) aborting: Task
</p <p class=3D"MsoPlainText">>> is
aborted: "'hbtl'" - code 10=
0</p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30 00:37:11=
,031::task::495::TaskManager.Task::(_debug) Task 0be1d461-f8fa-4c20-861d-27=
fde8124408: Prepare: aborted: 'hbtl'</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,031::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
re=
f 0 aborting True</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,032::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
Ta=
sk._doAbort: force False</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,032::resourceManager::821::Reso=
urceManager.Owner::(cancelAll</p <p
class=3D"MsoPlainText">>> )</p <p
class=3D"MsoPlainText">>> Owner.cancelAll requests {}</p <p class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,032::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
mo=
ving from state preparing ->
</p <p class=3D"MsoPlainText">>>
state aborting</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,033::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
_a=
borting: recover policy none</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,033::task::495::TaskManager.Tas=
k::(_debug) Task</p <p
class=3D"MsoPlainText">>> 0be1d461-f8fa-4c20-861d-27fde8124408:
mo=
ving from state aborting ->
</p <p class=3D"MsoPlainText">>>
state failed</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,033::resourceManager::786::Reso=
urceManager.Owner::(releaseAl</p <p
class=3D"MsoPlainText">>> l</p <p
class=3D"MsoPlainText">>> ) Owner.releaseAll requests {}
resources=
{}</p <p
class=3D"MsoPlainText">>>
Thread-47346::DEBUG::2007-06-30</p <p
class=3D"MsoPlainText">>>
00:37:11,034::resourceManager::821::Reso=
urceManager.Owner::(cancelAll</p <p
class=3D"MsoPlainText">>> )</p <p
class=3D"MsoPlainText">>> Owner.cancelAll requests {}</p <p class=3D"MsoPlainText">>>
Thread-47346::ERROR::2007-06-30 00:37:11=
,034::dispatcher::106::Storage.Dispatcher.Protect::(run) 'hbtl'</p <p class=3D"MsoPlainText">>>
Thread-47346::ERROR::2007-06-30 00:37:11=
,034::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most re=
cent call last):</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/dispatcher.py", line 96, in run</p <p
class=3D"MsoPlainText">>>
File "/usr/=
share/vdsm/storage/task.py", line 1178, in prepare</p <p class=3D"MsoPlainText">>>
KeyError: 'hbtl'</p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
Alberto Scotto</p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>> Blue
Reply</p <p
class=3D"MsoPlainText">>> Via Cardinal Massaia, 83</p <p class=3D"MsoPlainText">>>
10147 - Torino - ITALY</p <p
class=3D"MsoPlainText">>> phone: +39 011
29100</p <p
class=3D"MsoPlainText">>> <a
href=3D"mailto:al.scotto@reply.it"><s=
pan style=3D"color:windowtext;
text-decoration:none">al.scotto@reply.it</sp=
an></a></p <p
class=3D"MsoPlainText">>> <a
href=3D"http://www.reply.it"><span st=
yle=3D"color:windowtext;
text-decoration:none">www.reply.it</span></a></p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
-----Original Message-----</p <p
class=3D"MsoPlainText">>> From: Haim <a
href=3D"mailto:[mailto:hat=
eya(a)redhat.com]" <span
style=3D"color:windowtext; text-decoration:none">[mailto:hateya@redha=
t.com]</span></a></p <p
class=3D"MsoPlainText">>> Sent: gioved=EC 23 agosto 2012
17:00</p <p
class=3D"MsoPlainText">>> To: Scotto Alberto</p <p class=3D"MsoPlainText">>> Cc:
<a href=3D"mailto:users@ovirt.org"><=
span style=3D"color:windowtext;
text-decoration:none">users@ovirt.org</span=
></a></p <p
class=3D"MsoPlainText">>> Subject: Re: [Users] [rhev 3] add new
do=
main fails: Could not
</p <p class=3D"MsoPlainText">>>
retrieve LUNs</p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>> On
08/23/2012 05:54 PM, Scotto Alberto w=
rote:</p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
hi,</p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>> can
you attach full vdsm log during the =
execution of getDeviceList command?</p <p
class=3D"MsoPlainText">>>> Hi all,</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> I'm trying to configure a
FCP storag=
e domain on RHEV 3.</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> I try to add a new domain
from the c=
onsole, but it can't find any</p <p
class=3D"MsoPlainText">>>> LUNs: "Could not
retrieve LUNs,=
please check your storage"</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Here is the output from
/var/log/rhe=
vm/rhevm.log:</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>>
------------------------------------=
</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:07,811
WARN</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.bll.GetConfig=
urationValueQuery]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) calling
GetCon=
figurationValueQuery with null
</p <p
class=3D"MsoPlainText">>>> version, using default
general for v=
ersion</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:07,911
INFO</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.GetDeviceListVDSCommand]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) START,
GetDevi=
ceListVDSCommand(vdsId =3D
</p <p
class=3D"MsoPlainText">>>>
7e077f4c-25d8-11dc-bbcb-001cc4c2469a=
, storageType=3DFCP), log id:</p <p
class=3D"MsoPlainText">>>> 60bdafe6</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,726
ERROR</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.BrokerCommandBase]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Failed
in GetD=
eviceListVDS method</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
ERROR</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.BrokerCommandBase]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Error
code Blo=
ckDeviceActionError and error
</p <p
class=3D"MsoPlainText">>>> message VDSGenericException:
VDSErro=
rException: Failed to
</p <p
class=3D"MsoPlainText">>>> GetDeviceListVDS, error =3D
Error bl=
ock device action: ()</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
INFO</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.BrokerCommandBase]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1)
Command</p <p
class=3D"MsoPlainText">>>>
org.ovirt.engine.core.vdsbroker.vdsb=
roker.GetDeviceListVDSCommand</p <p
class=3D"MsoPlainText">>>> return value</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Class Name:</p <p
class=3D"MsoPlainText">>>>
org.ovirt.engine.core.vdsbroker.vdsb=
roker.LUNListReturnForXmlRpc</p <p
class=3D"MsoPlainText">>>> lunList Null</p <p
class=3D"MsoPlainText">>>> mStatus Class Name:</p <p
class=3D"MsoPlainText">>>>
org.ovirt.engine.core.vdsbroker.vdsb=
roker.StatusForXmlRpc</p <p
class=3D"MsoPlainText">>>> mCode 600</p <p
class=3D"MsoPlainText">>>> mMessage Error block device
action: =
()</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
INFO</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.BrokerCommandBase]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Vds:
pittor06v=
hxd020</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
ERROR</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.VDS=
CommandBase]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Command
GetDev=
iceListVDS execution failed. Exception:</p <p
class=3D"MsoPlainText">>>> VDSErrorException:
VDSGenericExcepti=
on: VDSErrorException: Failed to
</p <p
class=3D"MsoPlainText">>>> GetDeviceListVDS, error =3D
Error bl=
ock device action: ()</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
INFO</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.vdsbroker.vds=
broker.GetDeviceListVDSCommand]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) FINISH,
GetDev=
iceListVDSCommand, log id:</p <p
class=3D"MsoPlainText">>>> 60bdafe6</p <p
class=3D"MsoPlainText">>>> 2007-06-29 21:50:08,727
ERROR</p <p
class=3D"MsoPlainText">>>>
[org.ovirt.engine.core.bll.storage.G=
etDeviceListQuery]</p <p
class=3D"MsoPlainText">>>> (http-0.0.0.0-8443-1) Query
GetDevic=
eListQuery failed. Exception
</p <p
class=3D"MsoPlainText">>>> message is
VdcBLLException:</p <p
class=3D"MsoPlainText">>>>
org.ovirt.engine.core.vdsbroker.vdsb=
roker.VDSErrorException:</p <p
class=3D"MsoPlainText">>>> VDSGenericException:
VDSErrorExcepti=
on: Failed to GetDeviceListVDS,
</p <p
class=3D"MsoPlainText">>>> error =3D Error block device
action:=
()</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>>
------------------------------------=
----------</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> First question: do LUNs have
to be v=
isible from RHEV-H or RHEV-M?</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Currently they are visible
only from=
the hypervisor.</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>>
------------------------------------=
----</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> [root@pittor06vhxd020 log]#
multipat=
h -ll</p <p
class=3D"MsoPlainText">>>>
3600601601cde1d0066b2fb054dece111 dm=
-2 DGC,RAID 5 size=3D650G</p <p
class=3D"MsoPlainText">>>> features=3D'1
queue_if_no_path' hwha=
ndler=3D'1 emc' wp=3Drw</p <p
class=3D"MsoPlainText">>>> |-+-
policy=3D'round-robin 0' pr=
io=3D1 status=3Dactive</p <p
class=3D"MsoPlainText">>>> | |- 2:0:0:0 sda 8:0 active
ready ru=
nning</p <p
class=3D"MsoPlainText">>>> | |- 2:0:1:0 sdd 8:48 active
ready r=
unning</p <p
class=3D"MsoPlainText">>>> | |- 3:0:0:0 sde 8:64 active
ready r=
unning</p <p
class=3D"MsoPlainText">>>> | `- 3:0:1:0 sdf 8:80 active
ready r=
unning</p <p
class=3D"MsoPlainText">>>> `-+-
policy=3D'round-robin 0' pr=
io=3D0 status=3Denabled</p <p
class=3D"MsoPlainText">>>> |- 2:0:2:0 sdb 8:16 active
ready run=
ning</p <p
class=3D"MsoPlainText">>>> |- 2:0:3:0 sdc 8:32 active
ready run=
ning</p <p
class=3D"MsoPlainText">>>> |- 3:0:2:0 sdg 8:96 active
ready run=
ning</p <p
class=3D"MsoPlainText">>>> `- 3:0:3:0 sdh 8:112 active
ready ru=
nning</p <p
class=3D"MsoPlainText">>>>
3600508b1001035333920202020200005 dm=
-0 HP,LOGICAL VOLUME size=3D205G</p <p
class=3D"MsoPlainText">>>> features=3D'1
queue_if_no_path' hwha=
ndler=3D'0' wp=3Drw</p <p
class=3D"MsoPlainText">>>> `-+-
policy=3D'round-robin 0' pr=
io=3D1 status=3Dactive</p <p
class=3D"MsoPlainText">>>> `- 0:0:1:0 cciss!c0d1 104:16
active =
ready running</p <p
class=3D"MsoPlainText">>>>
------------------------------------=
------------------</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Our SAN device is Clariion
AX150. Is=
it compatible with ovirt?</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> vdsClient -s 0
getDeviceListgives me=
:</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Error block device action:
()</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Could it be due to SPM turned
off? (=
I have only one host)</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> [root@pittor06vhxd020 log]#
ps axu |=
grep -i spm</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> root 16068 0.0 0.0 7888 868
pts/1 R&=
#43; 00:04 0:00 grep -i spm</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> How can I turn it on? I know
the com=
mand but I don't know what
</p <p
class=3D"MsoPlainText">>>> paramaters append</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> spmStart</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> <spUUID>
<prevID> <pr=
evLVER> <recoveryMode> <scsiFencing>
</p <p
class=3D"MsoPlainText">>>> <maxHostID>
<version></p=
<p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Thank you very much for any
hints.</=
p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> AS</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Alberto Scotto</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> Blue</p <p
class=3D"MsoPlainText">>>> Via Cardinal Massaia,
83</p <p
class=3D"MsoPlainText">>>> 10147 - Torino -
ITALY</p <p
class=3D"MsoPlainText">>>> phone: +39 011
29100</p <p
class=3D"MsoPlainText">>>> <a
href=3D"mailto:al.scotto@reply.it=
"><span style=3D"color:windowtext;
text-decoration:none">al.scotto@reply.it=
</span></a></p <p
class=3D"MsoPlainText">>>> <a
href=3D"http://www.reply.it"><spa=
n style=3D"color:windowtext;
text-decoration:none">www.reply.it</span></a><=
/p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>>
------------------------------------=
--------------------------------</p <p
class=3D"MsoPlainText">>>> -</p <p
class=3D"MsoPlainText">>>> -</p <p
class=3D"MsoPlainText">>>> --</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> --</p <p
class=3D"MsoPlainText">>>> The information transmitted
is inten=
ded for the person or entity to
</p <p
class=3D"MsoPlainText">>>> which it is addressed and may
contai=
n confidential and/or privileged
</p <p
class=3D"MsoPlainText">>>> material. Any review,
retransmission=
, dissemination or other use of,
</p <p
class=3D"MsoPlainText">>>> or taking of any action in
reliance =
upon, this information by
</p <p
class=3D"MsoPlainText">>>> persons or entities other
than the i=
ntended recipient is prohibited.</p <p
class=3D"MsoPlainText">>>> If you received this in
error, pleas=
e contact the sender and delete
</p <p
class=3D"MsoPlainText">>>> the material from any
computer.</p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>> </p <p
class=3D"MsoPlainText">>>>
____________________________________=
___________</p <p
class=3D"MsoPlainText">>>> Users mailing list</p <p
class=3D"MsoPlainText">>>> <a
href=3D"mailto:Users@ovirt.org"><=
span style=3D"color:windowtext;
text-decoration:none">Users@ovirt.org</span=
></a></p <p
class=3D"MsoPlainText">>>> <a
href=3D"http://lists.ovirt.org/ma=
ilman/listinfo/users" <span
style=3D"color:windowtext; text-decoration:none">http://lists.ovirt.o=
rg/mailman/listinfo/users</span></a></p <p
class=3D"MsoPlainText">>> </p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
________________________________</p <p
class=3D"MsoPlainText">>> </p <p class=3D"MsoPlainText">>>
--</p <p
class=3D"MsoPlainText">>> The information transmitted is
intended =
for the person or entity to which it is addressed and may contain confident=
ial and/or privileged material. Any review, retransmission, dissemination o=
r other use of, or taking of any action
in reliance upon, this information by persons or entities other than the i=
ntended recipient is prohibited. If you received this in error, please cont=
act the sender and delete the material from any computer.</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> ________________________________</p <p
class=3D"MsoPlainText">> </p <p
class=3D"MsoPlainText">> --</p <p
class=3D"MsoPlainText">> The information transmitted is intended for
=
the person or entity to which it is addressed and may contain confidential =
and/or privileged material. Any review, retransmission, dissemination or ot=
her use of, or taking of any action in
reliance upon, this information by persons or entities other than the inte=
nded recipient is prohibited. If you received this in error, please contact=
the sender and delete the material from any computer.</p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText"> </p <p
class=3D"MsoPlainText">________________________________</p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText">--</p <p class=3D"MsoPlainText">The information
transmitted is intended for the p=
erson or entity to which it is addressed and may contain confidential and/o=
r privileged material. Any review, retransmission, dissemination or other u=
se of, or taking of any action in
reliance upon, this information by persons or entities other than the inte=
nded recipient is prohibited. If you received this in error, please contact=
the sender and delete the material from any computer.</p <p
class=3D"MsoPlainText">_______________________________________________</=
p <p class=3D"MsoPlainText">Users mailing
list</p <p
class=3D"MsoPlainText"><a
href=3D"mailto:Users@ovirt.org"><span style=3D=
"color:windowtext;
text-decoration:none">Users@ovirt.org</span></a></p <p class=3D"MsoPlainText"><a
href=3D"http://lists.ovirt.org/mailman/listinf=
o/users"><span style=3D"color:windowtext;
text-decoration:none">http://list=
s.ovirt.org/mailman/listinfo/users</span></a></p <p class=3D"MsoPlainText"> </p <p class=3D"MsoPlainText"> </p </div
<br <hr <font
face=3D"Arial" color=3D"Gray" size=3D"1"><br --<br The
information transmitted is intended for the person or entity to which i=
t is addressed and may contain confidential and/or privileged material. Any=
review, retransmission, dissemination or other use of, or taking of any ac=
tion in reliance upon, this information
by persons or entities other than the intended recipient is prohibited. If=
you received this in error, please contact the sender and delete the mater=
ial from any computer.<br </font </body
</html
--_000_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_--
--_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: image/png; name="blue.png"
Content-Description: blue.png
Content-Disposition: inline; filename="blue.png"; size=2834;
creation-date="Tue, 28 Aug 2012 16:11:45 GMT";
modification-date="Tue, 28 Aug 2012 16:11:45 GMT"
Content-ID: <e6e5a80a848243228ec57fb928078181
Content-Transfer-Encoding: base64
iVBORw0KGgoAAAANSUhEUgAAAIwAAAAyCAYAAACOADM7AAAABmJLR0QA/gD+AP7rGNSCAAAACXBI
WXMAAA3XAAAN1wFCKJt4AAAACXZwQWcAAACMAAAAMgCR0D3bAAAKaUlEQVR42u2ce5AUxRnAf313
3Al4eCAYFaIgyMNEUF6KlYoVIDBArDxqopWxQgViQlWsPHA0MUlZVoyKRsdSE4lGomjIaHS0UlHL
wTIPpEgQFQUUjYIWdfIIScyBHi/Z6/zRM1xP3yzs7t3unOX8qra2H9M9vb3f9Pf19/WukFKSk1Mq
dVkPIOejRS4wOWXR6wVGuP5I4foDsh5HjkL0VhtGuP5A4CFgNrAD+Lb0nKeyHtfHnd68wixGCQvA
qcA9wvWPy3pQH3caan1D4fonAYeBDwEZjaFflAaok56zHRhsNG0B+gAHSrhHarn0nFp/3NLnxbKP
B06I5kECO2UYZD2sLtRcYIBJwK+BoYBACU89cAjoAIRw/TuAJcClQGy//FJ6zvvH6ly4/qXAz4vU
HQA2A4H0nIcz+OxH41eAHaU3AhdkPaA0MrFhhOuPB2YA5wBnA6ehni5dgKcBu4C5wLZS7Rfh+g8A
80u49HHgEuk5h2s+AeaYLbsO2AKMiIqWyzBYkPW40shihUF6zkbUUwSAcP0G4FHgS9pl10rPmQMs
LbXfSBVNLPHyrwDfBO7JYg4MRqEempjnsh5QMXqL0Xsl8EUt3w5cXUE/w4AztfzzwGSUGrwoyuvM
yfqDR5yLUssxL2U9oGJkssLoCNdfjLJXdBZIz9lQQXcTgSYt/4z0nHjy1wvX3wW8oNX3O8q4TgKm
AGegjNB/As9JzzmYer1lTwKGoOyyV2UYtArLngLMQ9lh64EVRQxZ3V5pje4V9zsVGBRl22QYrDXu
e0HUvwD+K8NgXbe/lKOQqcAI178MuM0ovk16zqMVdjnNyL9g5E2DrTVlTP1RRvM3gIFG9RvC9RdK
z/lHoo2yQQJgeFR0hbDsT6FUns544Icp456qpV+RYaAL5RJgepR+FWXzxfcdA6zRrr0SqKrAZKaS
hOt/DbjXKH5Geo7bjW71iT8AvGLUzzXyfzfGNBBlPyymq7AAjAWeFK5/slE+AvhklC4At6KEZb9x
3cJo+9x5T8s+ERinFa012uzU0vuMuu9r6W3AXd2Yu5LIRGCE618E/D6l6rpu9Hk8MEEr2iQ9p1Wr
n4wShJgPgCeMbh6g02jeB9wILASe1q4ZBHzBaDeRThukHghRdskoQF+NmlH+JJ0JqB1ijCkw72np
jiOfx7JPQrkdYm6QYXBMH1V3qYlKEq7fhNLvw1CTeztK55rcJlz/s8XshGPwaeBELd8sXP961Bd4
Bsqo1u2bm6Tn7NbGeCHKMI6ZLz3nsajuT6gtfjxfpxr31lXhThkG8470a9mrtPp2uq4652np94FN
Rr0uMM1a+jI6fVTvAMsrmLOy6VGBEa5fB3wOpctHaK9TgVOAxmN0MRXlwPpWBbefYuTHAj8tcu39
0nNuMMq+qqXfjoUl4mSSq/HbRlv9S3/ZqBumpXcB/zPqz9fSm2UY/Nuo1wWmCUBYdiPwHa3ck2Hw
YQVzVjbVWGFmkW7YmewDfga8CNwHnB6VXyZcf7X0nAfLvG8pntE3gSXSc5an1Olf+hDh+i+jVieJ
UiOxwBSiMQMgLLsFOEtr+7xWB8rQjdkgw0BXK40o1RWTZrDu0dKx0X4xylMOynZZVuZcVUyPCoz0
nA7gR8L1N6FWmQIqZtRGpwoSwF7gRek5WwCE658P3A9Y0TV3C9ffUOrWOlrZdIfdXuBhlCqaqZU/
myYs0RZaNzybUV7oNFqBt7T8BJJ2iW6zDAPGFKkDGE1yBTLtF0gKTCF6/4FWtsTYVVWVqtgw0nNW
lHn9LmCOcP2bgKuAvsAtqNWqFGLVF7NGes4i4fpjgNfpFNbzi7QfD/TX8vtQMa40VkvPKWh5fWfW
DuhCfg5Ju8nc5k/RxpZYuTR0gWkTlj0D5YgEeJca2S4xvcXTC4D0nKvpdNWXc2hqEiqSHROrhR0k
bYAzhesPTmmvG61tKAE6PXoNRRnTg6OX6VvRhfB1GQa7tbyu5v6D8qNQpH4bsDVlbLrADACu0fK/
qOXqAr1MYCLip7AcI+48I78WIIpuv6mVN5NUPWntN0nP2So9p016ThtwEKU6RpIMOyAsuw9JVWiu
INO19AYZBma0fbKWXi/DoEBX9tBpu4wDLozS2+jqx6o6vVFgYt+JKKON/pTvJ6kWzKc6LTg5XEtv
MeruAF5DqbZVgH6IayTJoOHf4oSw7LNICuKTeqfCsj9BUnhN+yamPXqZc3JrLfwuJpnHklKIBaa+
lIuF67eQ3KW8HtlEMabhPCmlG/3JnhX5ZHaifDeLtLqlxpmcySQfuvnCstdH6WXaZ9iPMsJ1xpOM
ZaXZL6DsqfcB3UO8A7WzrDm9T2DqG7dTOHSIEgUGIc5GyhatZJ1Rv4HkmZ/xKb08o5UPRa0UkuQT
vY6uQVJTFc5D7fQ6SNpUN8ow2GVcq7sB2ugq2DGHUYfLdG6SYbCPDMhcYIRlJwWjcGg/Z1/yATBE
zJxXT0Pf4o0P7pWcO39W4nuVHS+JGfPq6dMXOjpgzNyt9En0MUF877fDee3x1iPlo2beTOPxnwGh
qzahuhUAjwCLpOeYKkDfIT2BUl1XkxT2+2QYXJ8yen0H+JYMgz2kY9o126mh38UkITBRYGwp5e1Q
usNjwL/Ql3VRX2D35mUI0UB90wyOZmc19i+wa+NB+vTrnMA9re00RO3q6iRbVtYxeOzt1NXHS3od
e96dRkPT6CN9v/HUIRr738Dg0bMRDSdQVzeAjsJh+ra8SfMpf5S3XNzFoSYsewhJVbhKhoEnLDtE
HV4vRGXPprQFFTdrRklk2u4opoVkyMOTYbCfjEgc0RSWPQhlQ/SruMfymCrD4IXud1N7In+ILgzT
ZRj8tYfvcSLwOzoPer0DjKv1VlrHVEltqBhMafZD99mR1QfvAXT1tYfiNkhZCMvuD1yLCtbORsXg
Yi7PUljAEJgoztFaYV8fN8yg4XsV95TkLJS32+QaGQZPl9tZT5O50ftRJLL1Pq8V9cjqEjHdyG8D
rpdhkJmhq5MLTGX0QR2diLdnYQ/2vRq1wsRe6nUyDNq712XP0Wt/W53TO+mNoYGcXkwuMDll0eM2
TPRbnGnAvaaDSVj2bOA0GQY1j7Lm9AzVWGG+jIrwphlH3wXuzvpD51RONXZJ7aizLFcIyx4O3CXD
IN527kUdJAJAWPbFqBXnVmHZV6FO3K+I6oahzgYPAX7T017UnMqoxgpTQAniONRJ/AeFZRc72+IA
P47SPwEWAAjLbgL+jPJ1NAF/EZZd6o/sc6pINQSmARAyDL6OOm45mmSoX+cDVDiC6D0+azI0arcS
FSkG9fcgORlTbcfdXtR5jqOdnpPGO3QK8nzU33KsoutvgXIyoBorjP7FN6OEsph3sE6rq9fS8RmQ
RTIMTgP+QPJsbk5GVENgjgMQlv0QcDnwBp0nxgaQ/O+6dmCUsOxHUGdj459kbI/a3Sksew3qjE5L
1pOVUx2VtBJljxxAhf3v0v4TZRnKmI25ObruLdTZkvcAZBgcEpY9E3BRu6TrZBisznqycvJYUk6Z
5KGBnLLIBSanLHKBySmLXGByyiIXmJyy+D/P9uGVPOu6DAAAACh6VFh0U29mdHdhcmUAAHja801M
LsrPTU3JTFRwyyxKLc8vyi5WsAAAYBUIJ4KDNosAAAAASUVORK5CYII=
--_005_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_--
--_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_
Content-Type: text/plain;
name="add storage domain (fcp) SUCCEEDS - vdsm log.txt"
Content-Description: add storage domain (fcp) SUCCEEDS - vdsm log.txt
Content-Disposition: attachment;
filename="add storage domain (fcp) SUCCEEDS - vdsm log.txt"; size=8263;
creation-date="Tue, 28 Aug 2012 16:02:28 GMT";
modification-date="Tue, 28 Aug 2012 16:02:28 GMT"
Content-Transfer-Encoding: base64
VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNDQ6OmNsaWVudElGOjoy
Mzk6OlN0b3JhZ2UuRGlzcGF0Y2hlci5Qcm90ZWN0Ojood3JhcHBlcikgWzEwLjE2LjI1MC4yMTZd
DQpUaHJlYWQtMTEwMzc6OklORk86OjIwMTItMDgtMjggMTU6NDY6MjUsMTQ2OjpkaXNwYXRjaGVy
Ojo5NDo6U3RvcmFnZS5EaXNwYXRjaGVyLlByb3RlY3Q6OihydW4pIFJ1biBhbmQgcHJvdGVjdDog
Z2V0RGV2aWNlTGlzdCwgYXJnczogKCkNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg
MTU6NDY6MjUsMTQ2Ojp0YXNrOjo0OTU6OlRhc2tNYW5hZ2VyLlRhc2s6OihfZGVidWcpIFRhc2sg
MDcxNWI4OGUtMWZjZi00ZGRhLTkzMjgtMDQ2NTNlYTgyNTZlOiBtb3ZpbmcgZnJvbSBzdGF0ZSBp
bml0IC0+IHN0YXRlIHByZXBhcmluZw0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAx
NTo0NjoyNSwxNDc6Om1pc2M6OjEwMTA6OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18pIFRyeWlu
ZyB0byBlbnRlciBzYW1wbGluZyBtZXRob2QgKHN0b3JhZ2Uuc2RjLnJlZnJlc2hTdG9yYWdlKQ0K
VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNDg6Om1pc2M6OjEwMTI6
OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18pIEdvdCBpbiB0byBzYW1wbGluZyBtZXRob2QNClRo
cmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsMTQ5OjptaXNjOjoxMDEwOjpT
YW1wbGluZ01ldGhvZDo6KF9fY2FsbF9fKSBUcnlpbmcgdG8gZW50ZXIgc2FtcGxpbmcgbWV0aG9k
IChzdG9yYWdlLmlzY3NpLnJlc2NhbikNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg
MTU6NDY6MjUsMTQ5OjptaXNjOjoxMDEyOjpTYW1wbGluZ01ldGhvZDo6KF9fY2FsbF9fKSBHb3Qg
aW4gdG8gc2FtcGxpbmcgbWV0aG9kDQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1
OjQ2OjI1LDE1MDo6aXNjc2k6OjY5OTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KHJlc2NhbikgJy91
c3IvYmluL3N1ZG8gLW4gL3NiaW4vaXNjc2lhZG0gLW0gc2Vzc2lvbiAtUicgKGN3ZCBOb25lKQ0K
VGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSwxNzY6OmlzY3NpOjo2OTk6
OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihyZXNjYW4pIEZBSUxFRDogPGVycj4gPSAnaXNjc2lhZG06
IE5vIHNlc3Npb24gZm91bmQuXG4nOyA8cmM+ID0gMjENClRocmVhZC0xMTAzNzo6REVCVUc6OjIw
MTItMDgtMjggMTU6NDY6MjUsMTc3OjptaXNjOjoxMDIwOjpTYW1wbGluZ01ldGhvZDo6KF9fY2Fs
bF9fKSBSZXR1cm5pbmcgbGFzdCByZXN1bHQNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgt
MjggMTU6NDY6MjUsNTQyOjptdWx0aXBhdGg6OjYxOjpTdG9yYWdlLk1pc2MuZXhjQ21kOjoocmVz
Y2FuKSAnL3Vzci9iaW4vc3VkbyAtbiAvc2Jpbi9tdWx0aXBhdGgnIChjd2QgTm9uZSkNClRocmVh
ZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsNjU3OjptdWx0aXBhdGg6OjYxOjpT
dG9yYWdlLk1pc2MuZXhjQ21kOjoocmVzY2FuKSBTVUNDRVNTOiA8ZXJyPiA9ICcnOyA8cmM+ID0g
MA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw2NTg6Omx2bTo6NTQ3
OjpPcGVyYXRpb25NdXRleDo6KF9pbnZhbGlkYXRlQWxsUHZzKSBPcGVyYXRpb24gJ2x2bSBpbnZh
bGlkYXRlIG9wZXJhdGlvbicgZ290IHRoZSBvcGVyYXRpb24gbXV0ZXgNClRocmVhZC0xMTAzNzo6
REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsNjU5Ojpsdm06OjU0OTo6T3BlcmF0aW9uTXV0ZXg6
OihfaW52YWxpZGF0ZUFsbFB2cykgT3BlcmF0aW9uICdsdm0gaW52YWxpZGF0ZSBvcGVyYXRpb24n
IHJlbGVhc2VkIHRoZSBvcGVyYXRpb24gbXV0ZXgNClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTIt
MDgtMjggMTU6NDY6MjUsNjU5Ojpsdm06OjU1OTo6T3BlcmF0aW9uTXV0ZXg6OihfaW52YWxpZGF0
ZUFsbFZncykgT3BlcmF0aW9uICdsdm0gaW52YWxpZGF0ZSBvcGVyYXRpb24nIGdvdCB0aGUgb3Bl
cmF0aW9uIG11dGV4DQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2
MDo6bHZtOjo1NjE6Ok9wZXJhdGlvbk11dGV4OjooX2ludmFsaWRhdGVBbGxWZ3MpIE9wZXJhdGlv
biAnbHZtIGludmFsaWRhdGUgb3BlcmF0aW9uJyByZWxlYXNlZCB0aGUgb3BlcmF0aW9uIG11dGV4
DQpUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2MDo6bHZtOjo1ODA6
Ok9wZXJhdGlvbk11dGV4OjooX2ludmFsaWRhdGVBbGxMdnMpIE9wZXJhdGlvbiAnbHZtIGludmFs
aWRhdGUgb3BlcmF0aW9uJyBnb3QgdGhlIG9wZXJhdGlvbiBtdXRleA0KVGhyZWFkLTExMDM3OjpE
RUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw2NjA6Omx2bTo6NTgyOjpPcGVyYXRpb25NdXRleDo6
KF9pbnZhbGlkYXRlQWxsTHZzKSBPcGVyYXRpb24gJ2x2bSBpbnZhbGlkYXRlIG9wZXJhdGlvbicg
cmVsZWFzZWQgdGhlIG9wZXJhdGlvbiBtdXRleA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0w
OC0yOCAxNTo0NjoyNSw2NjE6Om1pc2M6OjEwMjA6OlNhbXBsaW5nTWV0aG9kOjooX19jYWxsX18p
IFJldHVybmluZyBsYXN0IHJlc3VsdA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAx
NTo0NjoyNSw2NjE6Omx2bTo6NDA2OjpPcGVyYXRpb25NdXRleDo6KF9yZWxvYWRwdnMpIE9wZXJh
dGlvbiAnbHZtIHJlbG9hZCBvcGVyYXRpb24nIGdvdCB0aGUgb3BlcmF0aW9uIG11dGV4DQpUaHJl
YWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDY2NDo6bHZtOjozNzQ6OlN0b3Jh
Z2UuTWlzYy5leGNDbWQ6OihjbWQpICcvdXNyL2Jpbi9zdWRvIC1uIC9zYmluL2x2bSBwdnMgLS1j
b25maWcgIiBkZXZpY2VzIHsgcHJlZmVycmVkX25hbWVzID0gW1xcIl4vZGV2L21hcHBlci9cXCJd
IGlnbm9yZV9zdXNwZW5kZWRfZGV2aWNlcz0xIHdyaXRlX2NhY2hlX3N0YXRlPTAgZGlzYWJsZV9h
ZnRlcl9lcnJvcl9jb3VudD0zIGZpbHRlciA9IFsgXFwiYSUzNjAwNTA4YjEwMDEwMzUzMzM5MjAy
MDIwMjAyMDAwMDV8MzYwMDYwMTYwMWNkZTFkMDBkNGNiMzczNGQyZWRlMTExJVxcIiwgXFwiciUu
KiVcXCIgXSB9ICBnbG9iYWwgeyAgbG9ja2luZ190eXBlPTEgIHByaW9yaXRpc2Vfd3JpdGVfbG9j
a3M9MSAgd2FpdF9mb3JfbG9ja3M9MSB9ICBiYWNrdXAgeyAgcmV0YWluX21pbiA9IDUwICByZXRh
aW5fZGF5cyA9IDAgfSAiIC0tbm9oZWFkaW5ncyAtLXVuaXRzIGIgLS1ub3N1ZmZpeCAtLXNlcGFy
YXRvciB8IC1vIHV1aWQsbmFtZSxzaXplLHZnX25hbWUsdmdfdXVpZCxwZV9zdGFydCxwZV9jb3Vu
dCxwZV9hbGxvY19jb3VudCxtZGFfY291bnQsZGV2X3NpemUnIChjd2QgTm9uZSkNClRocmVhZC0x
MTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsODkxOjpsdm06OjM3NDo6U3RvcmFnZS5N
aXNjLmV4Y0NtZDo6KGNtZCkgU1VDQ0VTUzogPGVycj4gPSAiICBDb3VsZG4ndCBmaW5kIGRldmlj
ZSB3aXRoIHV1aWQgNEU3RDhJLTY3cFItc0IxWC1mam8wLVFvUTItejJBeS00MVBIZzQuXG4iOyA8
cmM+ID0gMA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw4OTI6Omx2
bTo6NDI5OjpPcGVyYXRpb25NdXRleDo6KF9yZWxvYWRwdnMpIE9wZXJhdGlvbiAnbHZtIHJlbG9h
ZCBvcGVyYXRpb24nIHJlbGVhc2VkIHRoZSBvcGVyYXRpb24gbXV0ZXgNCk1haW5Qcm9jZXNzfFRo
cmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjggMTU6NDY6MjUsODkyOjpkZXZpY2VtYXBwZXI6
OjE0NDo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KF9nZXRQYXRoc1N0YXR1cykgJy9zYmluL2Rtc2V0
dXAgc3RhdHVzJyAoY3dkIE5vbmUpDQpNYWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoy
MDEyLTA4LTI4IDE1OjQ2OjI1LDkwOTo6ZGV2aWNlbWFwcGVyOjoxNDQ6OlN0b3JhZ2UuTWlzYy5l
eGNDbWQ6OihfZ2V0UGF0aHNTdGF0dXMpIFNVQ0NFU1M6IDxlcnI+ID0gJyc7IDxyYz4gPSAwDQpN
YWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDkxNTo6
bXVsdGlwYXRoOjoxNTk6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihnZXRTY3NpU2VyaWFsKSAnL3Ni
aW4vc2NzaV9pZCAtLXBhZ2U9MHg4MCAtLXdoaXRlbGlzdGVkIC0tZXhwb3J0IC0tcmVwbGFjZS13
aGl0ZXNwYWNlIC0tZGV2aWNlPS9kZXYvZG0tMCcgKGN3ZCBOb25lKQ0KTWFpblByb2Nlc3N8VGhy
ZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5MjU6Om11bHRpcGF0aDo6MTU5
OjpTdG9yYWdlLk1pc2MuZXhjQ21kOjooZ2V0U2NzaVNlcmlhbCkgU1VDQ0VTUzogPGVycj4gPSAn
JzsgPHJjPiA9IDANCk1haW5Qcm9jZXNzfFRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgtMjgg
MTU6NDY6MjUsOTM2OjptdWx0aXBhdGg6OjE1OTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KGdldFNj
c2lTZXJpYWwpICcvc2Jpbi9zY3NpX2lkIC0tcGFnZT0weDgwIC0td2hpdGVsaXN0ZWQgLS1leHBv
cnQgLS1yZXBsYWNlLXdoaXRlc3BhY2UgLS1kZXZpY2U9L2Rldi9kbS0yJyAoY3dkIE5vbmUpDQpN
YWluUHJvY2Vzc3xUaHJlYWQtMTEwMzc6OkRFQlVHOjoyMDEyLTA4LTI4IDE1OjQ2OjI1LDk0Njo6
bXVsdGlwYXRoOjoxNTk6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihnZXRTY3NpU2VyaWFsKSBTVUND
RVNTOiA8ZXJyPiA9ICcnOyA8cmM+ID0gMA0KVGhyZWFkLTExMDM3OjpXQVJOSU5HOjoyMDEyLTA4
LTI4IDE1OjQ2OjI1LDk0Nzo6bXVsdGlwYXRoOjoyNjE6OlN0b3JhZ2UuTXVsdGlwYXRoOjoocGF0
aExpc3RJdGVyKSBQcm9ibGVtIGdldHRpbmcgaGJ0bCBmcm9tIGRldmljZSBgY2Npc3MhYzBkMWAN
ClRocmVhZC0xMTAzNzo6V0FSTklORzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5NDk6OmhzbTo6NzM1
OjpTdG9yYWdlLkhTTTo6KF9nZXREZXZpY2VMaXN0KSBJZ25vcmluZyBwYXJ0aXRpb25lZCBkZXZp
Y2Ugeydwcm9kdWN0JzogJ0xPR0lDQUwgVk9MVU1FJywgJ2RtJzogJ2RtLTInLCAnZGV2dHlwZXMn
OiBbJ0ZDUCddLCAnZndyZXYnOiAnMi4wOCcsICdsb2dpY2FsYmxvY2tzaXplJzogJzUxMicsICdj
b25uZWN0aW9ucyc6IFtdLCAnZGV2dHlwZSc6ICdGQ1AnLCAncGh5c2ljYWxibG9ja3NpemUnOiAn
NTEyJywgJ3ZlbmRvcic6ICdIUCcsICdzZXJpYWwnOiAnU0hQX0xPR0lDQUxfVk9MVU1FX1BINzlN
Vzc1MzknLCAnZ3VpZCc6ICczNjAwNTA4YjEwMDEwMzUzMzM5MjAyMDIwMjAyMDAwMDUnLCAncGF0
aHMnOiBbeydkZXZudW0nOiBEZXZpY2VOdW1iZXIoTWFqb3I9MTA0LCBNaW5vcj0xNiksICdwaHlz
ZGV2JzogJ2NjaXNzIWMwZDEnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJ31dLCAn
Y2FwYWNpdHknOiAnMjIwMTIyMDcxMDQwJ30NClRocmVhZC0xMTAzNzo6REVCVUc6OjIwMTItMDgt
MjggMTU6NDY6MjUsOTUwOjp0YXNrOjo0OTU6OlRhc2tNYW5hZ2VyLlRhc2s6OihfZGVidWcpIFRh
c2sgMDcxNWI4OGUtMWZjZi00ZGRhLTkzMjgtMDQ2NTNlYTgyNTZlOiBmaW5pc2hlZDogeydkZXZM
aXN0JzogW3snZndyZXYnOiAnMDIyMCcsICd2Z1VVSUQnOiAnYUJ6SHNsLXhKcGUtNGRBRC05VVNB
LUdoY0stcnNpMi1EWlpHb1UnLCAncGF0aGxpc3QnOiBbXSwgJ2xvZ2ljYWxibG9ja3NpemUnOiAn
NTEyJywgJ2RldnR5cGUnOiAnRkNQJywgJ3BoeXNpY2FsYmxvY2tzaXplJzogJzUxMicsICdzZXJp
YWwnOiAnU0RHQ19SQUlEXzVfRkNOSFcwNzMzMDAwMDknLCAnR1VJRCc6ICczNjAwNjAxNjAxY2Rl
MWQwMGQ0Y2IzNzM0ZDJlZGUxMTEnLCAncHJvZHVjdElEJzogJ1JBSUQgNScsICd2ZW5kb3JJRCc6
ICdER0MnLCAnY2FwYWNpdHknOiAnNjk3OTMyMTg1NjAwJywgJ3BhcnRpdGlvbmVkJzogRmFsc2Us
ICdwYXRoc3RhdHVzJzogW3sncGh5c2Rldic6ICdzZGEnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5
cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGInLCAnc3RhdGUnOiAnYWN0
aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGUnLCAnc3Rh
dGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdz
ZGYnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5
c2Rldic6ICdzZGMnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcw
J30sIHsncGh5c2Rldic6ICdzZGQnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUnOiAnRkNQJywg
J2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGcnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ3R5cGUn
OiAnRkNQJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGgnLCAnc3RhdGUnOiAnYWN0aXZl
JywgJ3R5cGUnOiAnRkNQJywgJ2x1bic6ICcwJ31dLCAncHZVVUlEJzogJ01vSE0xdS1keGVoLUdP
MVYtYXhDby1MZlAyLTMzdVUtRjRpZmNEJ31dfQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0w
OC0yOCAxNTo0NjoyNSw5NTE6OnRhc2s6OjQ5NTo6VGFza01hbmFnZXIuVGFzazo6KF9kZWJ1Zykg
VGFzayAwNzE1Yjg4ZS0xZmNmLTRkZGEtOTMyOC0wNDY1M2VhODI1NmU6IG1vdmluZyBmcm9tIHN0
YXRlIHByZXBhcmluZyAtPiBzdGF0ZSBmaW5pc2hlZA0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAx
Mi0wOC0yOCAxNTo0NjoyNSw5NTE6OnJlc291cmNlTWFuYWdlcjo6Nzg2OjpSZXNvdXJjZU1hbmFn
ZXIuT3duZXI6OihyZWxlYXNlQWxsKSBPd25lci5yZWxlYXNlQWxsIHJlcXVlc3RzIHt9IHJlc291
cmNlcyB7fQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0yOCAxNTo0NjoyNSw5NTE6OnJl
c291cmNlTWFuYWdlcjo6ODIxOjpSZXNvdXJjZU1hbmFnZXIuT3duZXI6OihjYW5jZWxBbGwpIE93
bmVyLmNhbmNlbEFsbCByZXF1ZXN0cyB7fQ0KVGhyZWFkLTExMDM3OjpERUJVRzo6MjAxMi0wOC0y
OCAxNTo0NjoyNSw5NTI6OnRhc2s6OjQ5NTo6VGFza01hbmFnZXIuVGFzazo6KF9kZWJ1ZykgVGFz
ayAwNzE1Yjg4ZS0xZmNmLTRkZGEtOTMyOC0wNDY1M2VhODI1NmU6IHJlZiAwIGFib3J0aW5nIEZh
bHNlDQpUaHJlYWQtMTEwMzc6OklORk86OjIwMTItMDgtMjggMTU6NDY6MjUsOTUyOjpkaXNwYXRj
aGVyOjoxMDA6OlN0b3JhZ2UuRGlzcGF0Y2hlci5Qcm90ZWN0OjoocnVuKSBSdW4gYW5kIHByb3Rl
Y3Q6IGdldERldmljZUxpc3QsIFJldHVybiByZXNwb25zZTogeydzdGF0dXMnOiB7J21lc3NhZ2Un
OiAnT0snLCAnY29kZSc6IDB9LCAnZGV2TGlzdCc6IFt7J3ZlbmRvcklEJzogJ0RHQycsICdjYXBh
Y2l0eSc6ICc2OTc5MzIxODU2MDAnLCAnZndyZXYnOiAnMDIyMCcsICdwYXJ0aXRpb25lZCc6IEZh
bHNlLCAndmdVVUlEJzogJ2FCekhzbC14SnBlLTRkQUQtOVVTQS1HaGNLLXJzaTItRFpaR29VJywg
J3BhdGhsaXN0JzogW10sICdsb2dpY2FsYmxvY2tzaXplJzogJzUxMicsICdwYXRoc3RhdHVzJzog
W3sncGh5c2Rldic6ICdzZGEnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1
bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGInLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0
aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGUnLCAndHlwZSc6ICdGQ1AnLCAnc3Rh
dGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGYnLCAndHlwZSc6ICdG
Q1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6ICdzZGMnLCAn
dHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsncGh5c2Rldic6
ICdzZGQnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6ICcwJ30sIHsn
cGh5c2Rldic6ICdzZGcnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZlJywgJ2x1bic6
ICcwJ30sIHsncGh5c2Rldic6ICdzZGgnLCAndHlwZSc6ICdGQ1AnLCAnc3RhdGUnOiAnYWN0aXZl
JywgJ2x1bic6ICcwJ31dLCAnZGV2dHlwZSc6ICdGQ1AnLCAncGh5c2ljYWxibG9ja3NpemUnOiAn
NTEyJywgJ3B2VVVJRCc6ICdNb0hNMXUtZHhlaC1HTzFWLWF4Q28tTGZQMi0zM3VVLUY0aWZjRCcs
ICdzZXJpYWwnOiAnU0RHQ19SQUlEXzVfRkNOSFcwNzMzMDAwMDknLCAnR1VJRCc6ICczNjAwNjAx
NjAxY2RlMWQwMGQ0Y2IzNzM0ZDJlZGUxMTEnLCAncHJvZHVjdElEJzogJ1JBSUQgNSd9XX0NCg==
--_006_C8B8517ADA90DB40A482797D59EB8389641995EDCED01MBXS08repl_--
I was going to reply.. :)
It looks like it halts after an error due to cciss!c0d1, which btw is
dis=
played by multipath -ll.
That's just a local disk, isn't it? So it shouldn't even
be listed. I may=
have attached it by mistake, playing with /sys/class/fc_* tools.
So, if I remove that path, everything should go ok. Do you think so
too?
Yes, please remove and check again (make sure to clean device-mapper table =
with dmsetup remove $dm).
anyhow, vdsm should be more robust, so please file a bug for it(https://bug=
zilla.redhat.com/enter_bug.cgi?product=3DoVirt).
Haim
> it appears that vdsm fails to handle a device with '!' in
it
> (cciss!c0d1), but let's make sure its indeed the case
More than that the fact is that the path scsi_disk/ doesn't exist
in
/sys/block/cciss!c0d1/device And this must be due to the fact that
c0d1 is NOT a damn scsi disk
Anyway, here is your output
[root@pittor06vhxd020 ~]# ls -l /sys/block/ total 0 lrwxrwxrwx. 1
root
> root 0 2007-06-30 00:37 cciss!c0d0 -
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d0/block/ccis
> s!c0d0 lrwxrwxrwx. 1 root root 0 2007-06-30 00:32 cciss!c0d1 -
../devices/pci0000:00/0000:00:03.0/0000:06:00.0/cciss0/c0d1/block/ccis
> s!c0d1 lrwxrwxrwx. 1 root root 0 2007-06-30 01:17 dm-0 - ../devices/virtual/block/dm-0 lrwxrwxrwx. 1 root root 0 2007-06-30
00:40 dm-1 -> ../devices/virtual/block/dm-1 lrwxrwxrwx. 1 root
root 0
2007-06-30 01:17 dm-2 -> ../devices/virtual/block/dm-2 lrwxrwxrwx.
1
root root 0 2007-06-30 00:49 dm-3 -> ../devices/virtual/block/dm-3
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 dm-4 - ../devices/virtual/block/dm-4 lrwxrwxrwx. 1 root root 0 2007-06-30
01:00 dm-5 -> ../devices/virtual/block/dm-5 lrwxrwxrwx. 1 root
root 0
2007-06-30 00:49 dm-6 -> ../devices/virtual/block/dm-6 lrwxrwxrwx.
1
root root 0 2007-06-30 00:10 loop0 ->
../devices/virtual/block/loop0
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop1 - ../devices/virtual/block/loop1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop2 -> ../devices/virtual/block/loop2 lrwxrwxrwx. 1 root
root
0 2007-06-30 00:10 loop3 -> ../devices/virtual/block/loop3
lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 loop4 ->
../devices/virtual/block/loop4
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 loop5 - ../devices/virtual/block/loop5 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 loop6 -> ../devices/virtual/block/loop6 lrwxrwxrwx. 1 root
root
0 2007-06-30 00:10 loop7 -> ../devices/virtual/block/loop7
lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram0 ->
../devices/virtual/block/ram0
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram1 - ../devices/virtual/block/ram1 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram10 -> ../devices/virtual/block/ram10 lrwxrwxrwx. 1 root
root
0 2007-06-30 00:10 ram11 -> ../devices/virtual/block/ram11
lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram12 ->
../devices/virtual/block/ram12
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram13 - ../devices/virtual/block/ram13 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram14 -> ../devices/virtual/block/ram14 lrwxrwxrwx. 1 root
root
0 2007-06-30 00:10 ram15 -> ../devices/virtual/block/ram15
lrwxrwxrwx.
1 root root 0 2007-06-30 00:10 ram2 ->
../devices/virtual/block/ram2
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram3 - ../devices/virtual/block/ram3 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram4 -> ../devices/virtual/block/ram4 lrwxrwxrwx. 1 root
root 0
2007-06-30 00:10 ram5 -> ../devices/virtual/block/ram5 lrwxrwxrwx.
1
root root 0 2007-06-30 00:10 ram6 -> ../devices/virtual/block/ram6
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:10 ram7 - ../devices/virtual/block/ram7 lrwxrwxrwx. 1 root root 0 2007-06-30
00:10 ram8 -> ../devices/virtual/block/ram8 lrwxrwxrwx. 1 root
root 0
2007-06-30 00:10 ram9 -> ../devices/virtual/block/ram9 lrwxrwxrwx.
1
> root root 0 2007-06-30 00:36 sda -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sda
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdb -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-2/target2:0:2/2:0:2:0/block/sdb
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdc -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-3/target2:0:3/2:0:3:0/block/sdc
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdd -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.0/host2/rport-2:0-1/target2:0:1/2:0:1:0/block/sdd
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sde -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-0/target3:0:0/3:0:0:0/block/sde
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdf -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-1/target3:0:1/3:0:1:0/block/sdf
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdg -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/block/sdg
> lrwxrwxrwx. 1 root root 0 2007-06-29 18:59 sdh -
../devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:00.0/0000:0b:0
0.1/host3/rport-3:0-3/target3:0:3/3:0:3:0/block/sdh
> lrwxrwxrwx. 1 root root 0 2007-06-30 00:49 sr0 -
../devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sr0
[root@pittor06vhxd020 ~]# dmsetup table
3600601601cde1d0066b2fb054dece111: 0 1363148800 multipath 1
queue_if_no_path 1 emc 2 1 round-robin 0 4 1 8:0 1 8:48 1 8:64 1 8:80
1 round-robin 0 4 1 8:16 1 8:32 1 8:96 1 8:112 1
HostVG-Logging: 0 4194304 linear 104:4 24741888
HostVG-Swap: 0 24723456 linear 104:4 2048
3600508b1001035333920202020200005: 0 429925920 multipath 1
queue_if_no_path 0 1 1 round-robin 0 1 1 104:16 1
HostVG-Data: 0 40624128 linear 104:4 28936192
HostVG-Config: 0 16384 linear 104:4 24725504
live-rw: 0 2097152 snapshot 7:1 7:2 P 8
[root@pittor06vhxd020 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE
MOU=
NTPOINT
loop0 7:0 0 99.3M 1 loop
loop1 7:1 0 1G 1 loop
=E2=E2live-rw (dm-1) 253:1 0 1G 0
dm =
/
loop2 7:2 0 512M 0 loop
=E2=E2live-rw (dm-1) 253:1 0 1G 0
dm =
/
cciss!c0d0 104:0 0 33.9G 0 disk
=E2=E2cciss!c0d0p1 104:1 0 243M 0
part
=E2=E2cciss!c0d0p2 104:2 0 244M 0
part
=E2=E2cciss!c0d0p3 104:3 0 244M 0
part
=E2=E2cciss!c0d0p4 104:4 0 33.2G 0
part
=E2=E2HostVG-Swap (dm-3) 253:3 0 11.8G 0
lvm =
[SWAP]
=E2=E2HostVG-Config (dm-4) 253:4 0 8M 0
lvm =
/config
=E2=E2HostVG-Logging (dm-5) 253:5 0 2G 0
lvm =
/var/log
=E2=E2HostVG-Data (dm-6) 253:6 0 19.4G 0
lvm =
/data
cciss!c0d1 104:16 0 205G 0 disk
=E2=E23600508b1001035333920202020200005 (dm-0) 253:0 0 205G 0
mpath
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sda 8:0 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sdc 8:32 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sdd 8:48 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sde 8:64 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sdf 8:80 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sdg 8:96 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
sdh 8:112 0 650G 0 disk
=E2=E23600601601cde1d0066b2fb054dece111 (dm-2) 253:2 0 650G 0
mpath
Alberto Scotto
Blue Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
> al.scotto@reply.it<mailto:al.scotto@reply.it -----Original Message-----
> From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com] Sent: gioved=EC 23 agosto 2012 18:01
To: Scotto Alberto
> Cc: users@ovirt.org<mailto:users@ovirt.org Subject: Re: [Users] [rhev 3] add new domain fails: Could not
retrieve
LUNs
On 08/23/2012 06:20 PM, Scotto Alberto wrote:
> Here you are
thanks, can you run the following?
- ls -l /sys/block/
- dmsetup table
- lsblk (if exists)
it appears that vdsm fails to handle a device with '!' in it
(cciss!c0d1)=
, but let's make sure its indeed the case.
> Thread-47346::DEBUG::2007-06-30
>
00:37:10,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper)
> [10.16.250.216]
> Thread-47346::INFO::2007-06-30
> 00:37:10,269::dispatcher::94::Storage.Dispatcher.Protect::(run)
Run
> and protect: getDeviceList, args: ()
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state init
-> state
> preparing
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,269::misc::1010::SamplingMethod::(__call__) Trying to
enter
> sampling method (storage.sdc.refreshStorage)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1010::SamplingMethod::(__call__) Trying to
enter
> sampling method (storage.iscsi.rescan)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,270::misc::1012::SamplingMethod::(__call__) Got in to
> sampling method
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,271::iscsi::699::Storage.Misc.excCmd::(rescan)
> '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd
None)
> Thread-47346::DEBUG::2007-06-30
>> 00:37:10,300::iscsi::699::Storage.Misc.excCmd::(rescan) FAILED: <err> =3D 'iscsiadm: No session found.\n'; <rc> =3D 21
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,301::misc::1020::SamplingMethod::(__call__) Returning
last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,661::multipath::61::Storage.Misc.excCmd::(rescan)
> '/usr/bin/sudo -n /sbin/multipath' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,785::multipath::61::Storage.Misc.excCmd::(rescan)
SUCCESS:
> <err> =3D ''; <rc> =3D 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::547::OperationMutex::(_invalidateAllPvs)
Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::549::OperationMutex::(_invalidateAllPvs)
Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,786::lvm::559::OperationMutex::(_invalidateAllVgs)
Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::561::OperationMutex::(_invalidateAllVgs)
Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,787::lvm::580::OperationMutex::(_invalidateAllLvs)
Operation
> 'lvm invalidate operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::582::OperationMutex::(_invalidateAllLvs)
Operation
> 'lvm invalidate operation' released the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::misc::1020::SamplingMethod::(__call__) Returning
last
> result
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,788::lvm::406::OperationMutex::(_reloadpvs) Operation
'lvm
> reload operation' got the operation mutex
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,791::lvm::374::Storage.Misc.excCmd::(cmd)
'/usr/bin/sudo -n
> /sbin/lvm pvs --config " devices { preferred_names =3D
> [\\"^/dev/mapper/\\"] ignore_suspended_devices=3D1
write_cache_state=3D0
> disable_after_error_count=3D3 filter =3D [
>
\\"a%3600508b1001035333920202020200005|3600601601cde1d0066b2fb054dece<fi=
le:///\\%22a%253600508b1001035333920202020200005|3600601601cde1d0066b2fb054=
dece> 1 11%\\",
\\"r%.*%\\<file:///\\%22r%25.*%25\>" ] } global { locking_ty=
pe=3D1
> prioritise_write_locks=3D1 wait_for_locks=3D1 } backup {
retain_min =
=3D
> 50 retain_days =3D 0 } " --noheadings --units b --nosuffix
--separator
> | -o
>
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_c
> o
> unt,dev_size' (cwd None)
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,997::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS:
<err> =3D '
> /dev/sdh: read failed after 0 of 4096 at 0: Input/output error\n
> /dev/sdh: read failed after 0 of 4096 at 697932120064:
Input/output
> error\n /dev/sdh: read failed after 0 of 4096 at 697932177408:
> Input/output error\n WARNING: Error counts reached a limit of 3.
> Device /dev/sdh was disabled\n'; <rc> =3D 0
> Thread-47346::DEBUG::2007-06-30
> 00:37:10,998::lvm::429::OperationMutex::(_reloadpvs) Operation
'lvm
> reload operation' released the operation mutex
> MainProcess|Thread-47346::DEBUG::2007-06-30
>
MainProcess|00:37:11,005::devicemapper::144::Storage.Misc.excCmd::(_g
> MainProcess|e
> MainProcess|tPathsStatus) '/sbin/dmsetup status' (cwd
None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
>
MainProcess|00:37:11,014::devicemapper::144::Storage.Misc.excCmd::(_g
> MainProcess|e
> MainProcess|tPathsStatus) SUCCESS: <err> =3D '';
<rc> =3D 0
> MainProcess|Thread-47346::DEBUG::2007-06-30
>
MainProcess|00:37:11,019::multipath::159::Storage.Misc.excCmd::(getSc
> MainProcess|s
> MainProcess|iSerial) '/sbin/scsi_id --page=3D0x80
--whitelisted
> MainProcess|--export --replace-whitespace
--device=3D/dev/dm-0' (cwd
> MainProcess|None)
> MainProcess|Thread-47346::DEBUG::2007-06-30
>
MainProcess|00:37:11,026::multipath::159::Storage.Misc.excCmd::(getSc
> MainProcess|s
> MainProcess|iSerial) SUCCESS: <err> =3D '';
<rc> =3D 0
> Thread-47346::WARNING::2007-06-30
> 00:37:11,027::multipath::261::Storage.Multipath::(pathListIter)
Problem =
getting hbtl from device `cciss!c0d1` Traceback (most recent call last):
> File "/usr/share/vdsm/storage/multipath.py", line
259, in pathListIt=
er
> File "/usr/share/vdsm/storage/multipath.py", line
182, in getHBTL
> OSError: [Errno 2] No such file or directory:
'/sys/block/cciss!c0d1/dev=
ice/scsi_disk/'
> Thread-47346::ERROR::2007-06-30
> 00:37:11,029::task::868::TaskManager.Task::(_setError) Unexpected
error =
Traceback (most recent call last):
> File "/usr/share/vdsm/storage/task.py", line 876,
in _run
> File "/usr/share/vdsm/storage/hsm.py", line 696, in
public_getDevice=
List
> File "/usr/share/vdsm/storage/hsm.py", line 759, in
> _getDeviceList
> KeyError: 'hbtl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._run:
> 0be1d461-f8fa-4c20-861d-27fde8124408 () {} failed - stopping task
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: stopping in state preparing
> (force False)
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,030::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 1 aborting True
> Thread-47346::INFO::2007-06-30
> 00:37:11,031::task::1171::TaskManager.Task::(prepare) aborting:
Task
> is aborted: "'hbtl'" - code 100
> Thread-47346::DEBUG::2007-06-30
00:37:11,031::task::495::TaskManager.Tas=
k::(_debug) Task
0be1d461-f8fa-4c20-861d-27fde8124408: Prepare: aborted: 'h=
btl'
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,031::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: ref 0 aborting True
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: Task._doAbort: force False
> Thread-47346::DEBUG::2007-06-30
>
00:37:11,032::resourceManager::821::ResourceManager.Owner::(cancelAll
> )
> Owner.cancelAll requests {}
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,032::task::495::TaskManager.Task::(_debug) Task
>> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state preparing -> state aborting
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
> 0be1d461-f8fa-4c20-861d-27fde8124408: _aborting: recover policy
none
> Thread-47346::DEBUG::2007-06-30
> 00:37:11,033::task::495::TaskManager.Task::(_debug) Task
>> 0be1d461-f8fa-4c20-861d-27fde8124408: moving from state aborting -> state failed
> Thread-47346::DEBUG::2007-06-30
>
00:37:11,033::resourceManager::786::ResourceManager.Owner::(releaseAl
> l
> ) Owner.releaseAll requests {} resources {}
> Thread-47346::DEBUG::2007-06-30
>
00:37:11,034::resourceManager::821::ResourceManager.Owner::(cancelAll
> )
> Owner.cancelAll requests {}
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::106::Storage.D=
ispatcher.Protect::(run) 'hbtl'
> Thread-47346::ERROR::2007-06-30
00:37:11,034::dispatcher::107::Storage.D=
ispatcher.Protect::(run) Traceback (most
recent call last):
> File "/usr/share/vdsm/storage/dispatcher.py", line
96, in run
> File "/usr/share/vdsm/storage/task.py", line 1178,
in prepare
> KeyError: 'hbtl'
> Alberto Scotto
> Blue Reply
> Via Cardinal Massaia, 83
> 10147 - Torino - ITALY
> phone: +39 011 29100
>> al.scotto@reply.it<mailto:al.scotto@reply.it -----Original Message-----
>> From: Haim [mailto:hateya@redhat.com]<mailto:[mailto:hateya@redhat.com]> Sent: gioved=EC 23 agosto 2012 17:00
> To: Scotto Alberto
>> Cc: users@ovirt.org<mailto:users@ovirt.org> Subject: Re: [Users] [rhev 3] add new domain fails: Could not
> retrieve LUNs
>> On 08/23/2012 05:54 PM, Scotto Alberto wrote:
>> hi,
>> can you attach full vdsm log during the execution of
getDeviceList comma=
nd?
>> Hi all,
>>>> I'm trying to configure a FCP storage domain on RHEV 3.
>>>> I try to add a new domain from the console, but it can't
find any
>> LUNs: "Could not retrieve LUNs, please check your
storage"
>>>> Here is the output from /var/log/rhevm/rhevm.log:
>>>> ------------------------------------
>>>> 2007-06-29 21:50:07,811 WARN
>> [org.ovirt.engine.core.bll.GetConfigurationValueQuery]
>> (http-0.0.0.0-8443-1) calling GetConfigurationValueQuery with
null
>> version, using default general for version
>> 2007-06-29 21:50:07,911 INFO
>>
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) START, GetDeviceListVDSCommand(vdsId
=3D
>> 7e077f4c-25d8-11dc-bbcb-001cc4c2469a, storageType=3DFCP), log
id:
>> 60bdafe6
>> 2007-06-29 21:50:08,726 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Failed in GetDeviceListVDS method
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Error code BlockDeviceActionError and
error
>> message VDSGenericException: VDSErrorException: Failed to
>> GetDeviceListVDS, error =3D Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Command
>>
org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand
>> return value
>>>> Class Name:
>>
org.ovirt.engine.core.vdsbroker.vdsbroker.LUNListReturnForXmlRpc
>> lunList Null
>> mStatus Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>> mCode 600
>> mMessage Error block device action: ()
>>>> 2007-06-29 21:50:08,727 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (http-0.0.0.0-8443-1) Vds: pittor06vhxd020
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
>> (http-0.0.0.0-8443-1) Command GetDeviceListVDS execution
failed. Except=
ion:
>> VDSErrorException: VDSGenericException: VDSErrorException:
Failed to
>> GetDeviceListVDS, error =3D Error block device action: ()
>> 2007-06-29 21:50:08,727 INFO
>>
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand]
>> (http-0.0.0.0-8443-1) FINISH, GetDeviceListVDSCommand, log
id:
>> 60bdafe6
>> 2007-06-29 21:50:08,727 ERROR
>> [org.ovirt.engine.core.bll.storage.GetDeviceListQuery]
>> (http-0.0.0.0-8443-1) Query GetDeviceListQuery failed.
Exception
>> message is VdcBLLException:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
>> VDSGenericException: VDSErrorException: Failed to
GetDeviceListVDS,
>> error =3D Error block device action: ()
>>>> ----------------------------------------------
>>>> First question: do LUNs have to be visible from RHEV-H or
RHEV-M?
>>>> Currently they are visible only from the hypervisor.
>>>> ----------------------------------------
>>>> [root@pittor06vhxd020 log]# multipath -ll
>> 3600601601cde1d0066b2fb054dece111 dm-2 DGC,RAID 5 size=3D650G
>> features=3D'1 queue_if_no_path' hwhandler=3D'1
emc' wp=3Drw
>> |-+- policy=3D'round-robin 0' prio=3D1
status=3Dactive
>> | |- 2:0:0:0 sda 8:0 active ready running
>> | |- 2:0:1:0 sdd 8:48 active ready running
>> | |- 3:0:0:0 sde 8:64 active ready running
>> | `- 3:0:1:0 sdf 8:80 active ready running
>> `-+- policy=3D'round-robin 0' prio=3D0
status=3Denabled
>> |- 2:0:2:0 sdb 8:16 active ready running
>> |- 2:0:3:0 sdc 8:32 active ready running
>> |- 3:0:2:0 sdg 8:96 active ready running
>> `- 3:0:3:0 sdh 8:112 active ready running
>> 3600508b1001035333920202020200005 dm-0 HP,LOGICAL VOLUME
size=3D205G
>> features=3D'1 queue_if_no_path'
hwhandler=3D'0' wp=3Drw
>> `-+- policy=3D'round-robin 0' prio=3D1
status=3Dactive
>> `- 0:0:1:0 cciss!c0d1 104:16 active ready running
>> ------------------------------------------------------
>>>> Our SAN device is Clariion AX150. Is it compatible with
ovirt?
>>>> vdsClient -s 0 getDeviceListgives me:
>>>> Error block device action: ()
>>>> Could it be due to SPM turned off? (I have only one host)
>>>> [root@pittor06vhxd020 log]# ps axu | grep -i spm
>>>> root 16068 0.0 0.0 7888 868 pts/1 R+ 00:04 0:00 grep -i spm
>>>> How can I turn it on? I know the command but I don't know
what
>> paramaters append
>>>> spmStart
>>>> Thank you very much for any hints.
>>>> AS
>> Alberto Scotto
>>>> Blue
>> Via Cardinal Massaia, 83
>> 10147 - Torino - ITALY
>> phone: +39 011 29100
>>> al.scotto@reply.it<mailto:al.scotto@reply.it>>
--------------------------------------------------------------------
>> -
>> -
>> --
>>>> --
>> The information transmitted is intended for the person or
entity to
>> which it is addressed and may contain confidential and/or
privileged
>> material. Any review, retransmission, dissemination or other
use of,
>> or taking of any action in reliance upon, this information by
>> persons or entities other than the intended recipient is
prohibited.
>> If you received this in error, please contact the sender and
delete
>> the material from any computer.
>>>> _______________________________________________
>> Users mailing list
>>> Users@ovirt.org<mailto:Users@ovirt.org> ________________________________
>> --
> The information transmitted is intended for the person or entity
to whic=
h it is addressed and may contain confidential and/or privileged material.
=
Any review, retransmission, dissemination or other use of, or taking of any=
action in reliance upon, this information by persons or entities other tha=
n the intended recipient is prohibited. If you received this in error, plea=
se contact the sender and delete the material from any computer.
________________________________
--
The information transmitted is intended for the person or entity to
which=
it is addressed and may contain confidential and/or privileged material. A=
ny review, retransmission, dissemination or other use of, or taking of any =
action in reliance upon, this information by persons or entities other than=
the intended recipient is prohibited. If you received this in error, pleas=
e contact the sender and delete the material from any computer.
________________________________
--
The information transmitted is intended for the person or entity to which i=
t is addressed and may contain confidential and/or privileged material. Any=
review, retransmission, dissemination or other use of, or taking of any ac=
tion in reliance upon, this information by persons or entities other than t=
he intended recipient is prohibited. If you received this in error, please =
contact the sender and delete the material from any computer.
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
4468
days inactive
4473
days old
7 comments
2 participants
participants (2)
-
Haim -
Scotto Alberto