12:45 p.m.
--_000_7AFA66599AC41847AD8E021A1DBB9D1426FAA41057pandoreankama_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi again,
Having fixed LUNs and LUN_storage_server_connection_map tables varchar size=
s, adding my LIO storage is OK as expected.
create_tables.sql script should be fixed.
What's more, when the error happened, ovirt got in inconsistent state :
-the LUN appeared in ovirt but was locked. Trying to destroy it did nothing=
in the GUI. Engine logged "Failed to Acquire Lock to object EngineLock". A=
fter a while, the LUN disappeared by itself.
Ovirt should handle the exception correctly (by totally refusing the LUN re=
gistration) and outputting an explicit message (instead of "error 503") cou=
ld be nice.
Vince
De : Vincent Miszczak
Envoy=E9 : jeudi 22 novembre 2012 10:25
=C0 : users(a)ovirt.org
Objet : Adding LIO iSCSI target as storage fails
Hi guys,
I'm trying to add an LIO iSCSI target as data domain into ovirt and it fail=
s.
The GUI reports :
Error: A Request to the Server failed with the following Status Code: 503
And the engine log reports :
2012-11-22 10:05:10,154 INFO [org.ovirt.engine.core.bll.storage.AddSANStor=
ageDomainCommand] (ajp--0.0.0.0-8009-3) [7ff25919] Running command: AddSANS=
torageDomainCommand internal: false. Entities affected : ID: aaa00000-0000=
-0000-0000-123456789aaa Type: System
2012-11-22 10:05:10,177 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Cr=
eateVGVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] START, CreateVGVDSComman=
d(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da, storageDomainId=3D6e93ed9=
e-1dd7-42b7-9e93-dc0187cc6fde, deviceList=3D[50103ff106001405db2776165d3349=
2091bde115a020100344c494f2d4f52470049424c4f434b3a64623237373631362d35643333=
2d343932302d393162642d]), log id: 7bbd158c
2012-11-22 10:05:10,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Cr=
eateVGVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] FINISH, CreateVGVDSComma=
nd, return: LvKD1i-L68S-ix09-OYcr-Ytzd-93wh-8XFe8e, log id: 7bbd158c
2012-11-22 10:05:10,913 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Cr=
eateStorageDomainVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] START, Create=
StorageDomainVDSCommand(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da, sto=
rageDomain=3Dorg.ovirt.engine.core.common.businessentities.storage_domain_s=
tatic@cd3431b<mailto:storageDomain=3Dorg.ovirt.engine.core.common.businesse=
ntities.storage_domain_static@cd3431b>, args=3DLvKD1i-L68S-ix09-OYcr-Ytzd-9=
3wh-8XFe8e), log id: 56c4cbde
2012-11-22 10:05:17,332 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Cr=
eateStorageDomainVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] FINISH, Creat=
eStorageDomainVDSCommand, log id: 56c4cbde
2012-11-22 10:05:17,338 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tStorageDomainStatsVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] START, GetS=
torageDomainStatsVDSCommand(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da,=
storageDomainId=3D6e93ed9e-1dd7-42b7-9e93-dc0187cc6fde), log id: 5142061
2012-11-22 10:05:17,654 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tStorageDomainStatsVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] FINISH, Get=
StorageDomainStatsVDSCommand, return: org.ovirt.engine.core.common.business=
entities.storage_domains@68e03e79<mailto:org.ovirt.engine.core.common.busin=
essentities.storage_domains@68e03e79>, log id: 5142061
2012-11-22 10:05:17,674 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tVGInfoVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] START, GetVGInfoVDSComm=
and(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da, VGID=3DLvKD1i-L68S-ix09=
-OYcr-Ytzd-93wh-8XFe8e), log id: 52756b80
2012-11-22 10:05:17,739 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.Ge=
tVGInfoVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] FINISH, GetVGInfoVDSCom=
mand, return: [org.ovirt.engine.core.common.businessentities.LUNs@53a8c83f]=
, log id: 52756b80
2012-11-22 10:05:17,758 INFO [org.ovirt.engine.core.utils.transaction.Tran=
sactionSupport] (ajp--0.0.0.0-8009-3) [7ff25919] transaction rolled back
2012-11-22 10:05:17,759 ERROR [org.ovirt.engine.core.bll.storage.AddSANStor=
ageDomainCommand] (ajp--0.0.0.0-8009-3) [7ff25919] Command org.ovirt.engine=
.core.bll.storage.AddSANStorageDomainCommand throw exception: org.springfra=
mework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL =
[{call insertluns(?, ?, ?, ?, ?, ?, ?, ?)}]; ERROR: value too long for type=
character varying(50)
Where: SQL statement "INSERT INTO LUNs(LUN_id, physical_volume_id, volume=
_group_id, serial, lun_mapping, vendor_id, product_id, device_size) VALUES(=
$1 , $2 , $3 , $4 , $5 , $6 , $7 , $8 )"
PL/pgSQL function "insertluns" line 2 at SQL statement; nested exception is=
org.postgresql.util.PSQLException: ERROR: value too long for type characte=
r varying(50)
Where: SQL statement "INSERT INTO LUNs(LUN_id, physical_volume_id, volume=
_group_id, serial, lun_mapping, vendor_id, product_id, device_size) VALUES(=
$1 , $2 , $3 , $4 , $5 , $6 , $7 , $8 )"
Being registered to LIO mailing list, I know there are concerns about LUN I=
D length being pretty insane (127 chars here). Having used an Equallogic iS=
CSI SAN, I did not have this problem, so I suppose this come from the id le=
ngth. Until this is fixed at LIO side (if it has to be), ovirt should handl=
e such situations.
I'm gonna resize my database col size to test that.
My configuration :
oVirt Engine Version: 3.1.0-3.19.el6 (Dreyou repo) running on stock Centos6=
.3
LIO Target running on Centos6.3/3.6.3-1.el6.elrepo.x86_64 kernel
Ovirt node running on latest Fedora17
Vince
--=20
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--_000_7AFA66599AC41847AD8E021A1DBB9D1426FAA41057pandoreankama_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml"
xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word"
=
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml"
xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type
content=
=3D"text/html; charset=3Diso-8859-1"><meta name=3DGenerator
content=3D"Micr=
osoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Arial Unicode MS";
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@Arial Unicode MS";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-US;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.gwt-inlinelabel
{mso-style-name:gwt-inlinelabel;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DFR
link=3Dblue vlink=
=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal><span
lang=3DEN-US=
style=3D'color:black'>Hi
again,<o:p></o:p></span></p><p class=3DMsoNormal>=
<span lang=3DEN-US
style=3D'color:black'><o:p> </o:p></span></p><p
cla=
ss=3DMsoNormal><span lang=3DEN-US style=3D'color:black'>Having fixed LUNs
a=
nd </span><span lang=3DEN-US
style=3D'font-size:11.5pt;font-family:"Arial",=
"sans-serif";color:black'>LUN_storage_server_connection_map
</span><span la=
ng=3DEN-US style=3D'color:black'>tables varchar sizes, adding my LIO storag=
e is OK as expected.<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=
=3DEN-US
style=3D'color:black'><o:p> </o:p></span></p><p
class=3DMsoNo=
rmal><span lang=3DEN-US style=3D'color:black'>create_tables.sql script
shou=
ld be fixed.<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US s=
tyle=3D'color:black'><o:p> </o:p></span></p><p
class=3DMsoNormal><span=
lang=3DEN-US style=3D'color:black'>What’s more, when the error
happe=
ned, ovirt got in inconsistent state : <o:p></o:p></span></p><p
class=3DMso=
Normal><span lang=3DEN-US style=3D'color:black'>-the LUN appeared in
ovirt =
but was locked. Trying to destroy it did nothing in the GUI. Engine logged =
“Failed to Acquire Lock to object EngineLock”. After a while, t=
he LUN disappeared by itself.<o:p></o:p></span></p><p
class=3DMsoNormal><sp=
an lang=3DEN-US
style=3D'color:black'><o:p> </o:p></span></p><p
class=
=3DMsoNormal><span lang=3DEN-US style=3D'color:black'>Ovirt should handle
t=
he exception correctly (by totally refusing the LUN registration) and outpu=
tting an explicit message (instead of “error 503”) could be nic=
e.<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US style=3D'co=
lor:black'><o:p> </o:p></span></p><p
class=3DMsoNormal><span lang=3DEN=
-US style=3D'color:black'>Vince</span><span lang=3DEN-US
style=3D'color:#1F=
497D'><o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US style=
=3D'color:#1F497D'><o:p> </o:p></span></p><div><div
style=3D'border:no=
ne;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=3DMso=
Normal><b><span
style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"=
;mso-fareast-language:FR'>De :</span></b><span
style=3D'font-size:10.0=
pt;font-family:"Tahoma","sans-serif";mso-fareast-language:FR'>
Vincent Misz=
czak <br><b>Envoy=E9 :</b> jeudi 22 novembre 2012
10:25<br><b>=C0 =
;:</b> users@ovirt.org<br><b>Objet :</b> Adding LIO iSCSI
target as st=
orage fails<o:p></o:p></span></p></div></div><p
class=3DMsoNormal><o:p>&nbs=
p;</o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Hi
guys,<o:p></o:p></sp=
an></p><p class=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span></p>=
<p class=3DMsoNormal><span lang=3DEN-US>I’m trying to add an LIO
iSCS=
I target as data domain into ovirt and it
fails.<o:p></o:p></span></p><p cl=
ass=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span></p><p class=3DM=
soNormal><span lang=3DEN-US>The GUI reports
:<o:p></o:p></span></p><p class=
=3DMsoNormal><span lang=3DEN-US style=3D'font-family:"Arial Unicode
MS","sa=
ns-serif";color:black;background:white'>Error: A Request to the Server fail=
ed with the following Status Code: 503</span><span
lang=3DEN-US><o:p></o:p>=
</span></p><p class=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span>=
</p><p class=3DMsoNormal><span lang=3DEN-US>And the engine log reports
:<o:=
p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>2012-11-22 10:05=
:10,154 INFO [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCo=
mmand] (ajp--0.0.0.0-8009-3) [7ff25919] Running command: AddSANStorageDomai=
nCommand internal: false. Entities affected : ID: aaa00000-0000-0000-=
0000-123456789aaa Type: System<o:p></o:p></span></p><p
class=3DMsoNormal><s=
pan lang=3DEN-US>2012-11-22 10:05:10,177 INFO [org.ovirt.engine.core.=
vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] ST=
ART, CreateVGVDSCommand(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da, sto=
rageDomainId=3D6e93ed9e-1dd7-42b7-9e93-dc0187cc6fde, deviceList=3D[50103ff1=
06001405db2776165d33492091bde115a020100344c494f2d4f52470049424c4f434b3a6462=
3237373631362d356433332d343932302d393162642d]), log id: 7bbd158c<o:p></o:p>=
</span></p><p class=3DMsoNormal><span lang=3DEN-US>2012-11-22
10:05:10,904 =
INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (=
ajp--0.0.0.0-8009-3) [7ff25919] FINISH, CreateVGVDSCommand, return: LvKD1i-=
L68S-ix09-OYcr-Ytzd-93wh-8XFe8e, log id:
7bbd158c<o:p></o:p></span></p><p c=
lass=3DMsoNormal><span lang=3DEN-US>2012-11-22 10:05:10,913 INFO [org=
.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp-=
-0.0.0.0-8009-3) [7ff25919] START, CreateStorageDomainVDSCommand(vdsId =3D =
936326ca-3482-11e2-a6fc-6f050df067da, <a href=3D"mailto:storageDomain=3Dorg=
.ovirt.engine.core.common.businessentities.storage_domain_static@cd3431b">s=
torageDomain=3Dorg.ovirt.engine.core.common.businessentities.storage_domain=
_static@cd3431b</a>, args=3DLvKD1i-L68S-ix09-OYcr-Ytzd-93wh-8XFe8e), log id=
: 56c4cbde<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>201=
2-11-22 10:05:17,332 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.=
CreateStorageDomainVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] FINISH, Cre=
ateStorageDomainVDSCommand, log id:
56c4cbde<o:p></o:p></span></p><p class=
=3DMsoNormal><span lang=3DEN-US>2012-11-22 10:05:17,338 INFO [org.ovi=
rt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp--0=
.0.0.0-8009-3) [7ff25919] START, GetStorageDomainStatsVDSCommand(vdsId =3D =
936326ca-3482-11e2-a6fc-6f050df067da, storageDomainId=3D6e93ed9e-1dd7-42b7-=
9e93-dc0187cc6fde), log id: 5142061<o:p></o:p></span></p><p
class=3DMsoNorm=
al><span lang=3DEN-US>2012-11-22 10:05:17,654 INFO [org.ovirt.engine.=
core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp--0.0.0.0-800=
9-3) [7ff25919] FINISH, GetStorageDomainStatsVDSCommand, return: <a href=3D=
"mailto:org.ovirt.engine.core.common.businessentities.storage_domains@68e03=
e79">org.ovirt.engine.core.common.businessentities.storage_domains@68e03e79=
</a>, log id: 5142061<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=
=3DEN-US>2012-11-22 10:05:17,674 INFO [org.ovirt.engine.core.vdsbroke=
r.vdsbroker.GetVGInfoVDSCommand] (ajp--0.0.0.0-8009-3) [7ff25919] START, Ge=
tVGInfoVDSCommand(vdsId =3D 936326ca-3482-11e2-a6fc-6f050df067da, VGID=3DLv=
KD1i-L68S-ix09-OYcr-Ytzd-93wh-8XFe8e), log id:
52756b80<o:p></o:p></span></=
p><p class=3DMsoNormal><span lang=3DEN-US>2012-11-22 10:05:17,739
INFO =
; [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (ajp--0.0=
.0.0-8009-3) [7ff25919] FINISH, GetVGInfoVDSCommand, return: [org.ovirt.eng=
ine.core.common.businessentities.LUNs@53a8c83f], log id: 52756b80<o:p></o:p=
</span></p><p class=3DMsoNormal><span
lang=3DEN-US>2012-11-22 10:05:17,758=
INFO
[org.ovirt.engine.core.utils.transaction.TransactionSupport] (a=
jp--0.0.0.0-8009-3) [7ff25919] transaction rolled
back<o:p></o:p></span></p=
<p class=3DMsoNormal><span lang=3DEN-US>2012-11-22
10:05:17,759 ERROR [org=
.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand]
(ajp--0.0.0.0-80=
09-3) [7ff25919] Command org.ovirt.engine.core.bll.storage.AddSANStorageDom=
ainCommand throw exception: org.springframework.dao.DataIntegrityViolationE=
xception: CallableStatementCallback; SQL [{call insertluns(?, ?, ?, ?, ?, ?=
, ?, ?)}]; ERROR: value too long for type character
varying(50)<o:p></o:p><=
/span></p><p class=3DMsoNormal><span lang=3DEN-US> Where:
SQL stateme=
nt "INSERT INTO LUNs(LUN_id, physical_volume_id, volume_group_id, seri=
al, lun_mapping, vendor_id, product_id, device_size) VALUES( $1 , $2 =
, $3 , $4 , $5 , $6 , $7 , $8
)"<o=
:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>PL/pgSQL functi=
on "insertluns" line 2 at SQL statement; nested exception is org.=
postgresql.util.PSQLException: ERROR: value too long for type character var=
ying(50)<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US> =
; Where: SQL statement "INSERT INTO LUNs(LUN_id, physical_volume_id, v=
olume_group_id, serial, lun_mapping, vendor_id, product_id, device_size) VA=
LUES( $1 , $2 , $3 , $4 , $5 , $6
, $7 =
, $8 )"<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DE=
N-US><o:p> </o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>Be=
ing registered to LIO mailing list, I know there are concerns about LUN ID =
length being pretty insane (127 chars here). Having used an Equallogic iSCS=
I SAN, I did not have this problem, so I suppose this come from the id leng=
th. Until this is fixed at LIO side (if it has to be), ovirt should handle =
such situations.<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-=
US><o:p> </o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>I=
217;m gonna resize my database col size to test
that.<o:p></o:p></span></p>=
<p class=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span></p><p clas=
s=3DMsoNormal><span lang=3DEN-US>My configuration :
<o:p></o:p></span></p><=
p class=3DMsoNormal><span class=3Dgwt-inlinelabel><span lang=3DEN-US style=
=3D'font-family:"Arial Unicode
MS","sans-serif";color:black;background:whit=
e'>oVirt Engine Version:</span></span><span
class=3Dapple-converted-space><=
span lang=3DEN-US style=3D'font-family:"Arial Unicode
MS","sans-serif";colo=
r:black;background:white'> </span></span><span
class=3Dgwt-inlinelabel=
<span lang=3DEN-US style=3D'font-family:"Arial Unicode
MS","sans-serif";co=
lor:black;background:white'>3.1.0-3.19.el6 (Dreyou repo) running on stock C=
entos6.3</span></span><span class=3Dgwt-inlinelabel><span
style=3D'font-fam=
ily:"Arial Unicode
MS","sans-serif";color:black;background:white'><o:p></o:=
p></span></span></p><p class=3DMsoNormal><span
class=3Dgwt-inlinelabel><spa=
n lang=3DEN-US style=3D'font-family:"Arial Unicode
MS","sans-serif";color:b=
lack;background:white'>LIO Target running on Centos6.3/3.6.3-1.el6.elrepo.x=
86_64 kernel<o:p></o:p></span></span></p><p
class=3DMsoNormal><span class=
=3Dgwt-inlinelabel><span lang=3DEN-US style=3D'font-family:"Arial Unicode
M=
S","sans-serif";color:black;background:white'>Ovirt node running on
latest =
Fedora17<o:p></o:p></span></span></p><p
class=3DMsoNormal><span class=3Dgwt=
-inlinelabel><span lang=3DEN-US style=3D'font-family:"Arial Unicode
MS","sa=
ns-serif";color:black;background:white'><o:p> </o:p></span></span></p>=
<p class=3DMsoNormal><span class=3Dgwt-inlinelabel><span lang=3DEN-US
style=
=3D'font-family:"Arial Unicode
MS","sans-serif";color:black;background:whit=
e'>Vince</span></span><o:p></o:p></p></div><br
/>--=20
<br />This message has been scanned for viruses and
<br />dangerous content by
<a
href=3D"http://www.mailscanner.info/"><b>MailScanner</b></a>,
and is
<br />believed to be clean.
</body></html>
--_000_7AFA66599AC41847AD8E021A1DBB9D1426FAA41057pandoreankama_--