<div dir="auto"><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Jan 5, 2017 9:05 AM, "Pavel Gashev" <<a href="mailto:Pax@acronis.com">Pax@acronis.com</a>> wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12.0pt;color:#1f497d">
<p dir="ltr">Charles,</p>
<p dir="ltr">Please note the warning in logs about lvmetad. It's better to disable this service if you invoke LVM utilitities manually. It caches some LVM metadata but VDSM doesn't use it. So technically VDSM has different view on PVs.<br></p></div></div></div></blockquote></div></div></div><div dir="auto">+1.</div><div dir="auto">We are going to disable it entirely in coming version. </div><div dir="auto">Y. </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12.0pt;color:#1f497d"><p dir="ltr">
</p>
<div id="m_8430388033766773259x_signature-x" style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12.0pt;color:#1f497d">
<br>
</div>
</div>
<div id="m_8430388033766773259x_quoted_header" style="clear:both">
<hr style="border:none;height:1px;color:#e1e1e1;background-color:#e1e1e1">
<div style="border:none;padding:3.0pt 0cm 0cm 0cm"><span style="font-size:11.0pt;font-family:'Calibri','sans-serif'"><b>From:</b> Charles Tassell <<a href="mailto:charles@islandadmin.ca" target="_blank">charles@islandadmin.ca</a>><br>
<b>Sent:</b> Jan 5, 2017 7:33 AM<br>
<b>To:</b> Pavel Gashev; <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
<b>Subject:</b> Re: [ovirt-users] Storage Problem After Host Update (v3.6)<br>
</span></div>
</div>
<br type="attribution">
</div><div class="elided-text">
<font size="2"><span style="font-size:10pt">
<div class="m_8430388033766773259PlainText">Hi Pavel,<br>
<br>
Thanks, I added the 3.6-snapshot repo to upgrade to vdsm 4.17.36 and <br>
then diagnosed a bunch of LVM problems (PV's set as missing in pvs -v) <br>
and that got us fixed back up!<br>
<br>
<br>
On 2017-01-04 06:13 PM, Pavel Gashev wrote:<br>
> <a href="https://bugzilla.redhat.com/1363734" target="_blank">https://bugzilla.redhat.com/<wbr>1363734</a><br>
> <a href="https://bugzilla.redhat.com/1364339" target="_blank">https://bugzilla.redhat.com/<wbr>1364339</a><br>
> <a href="https://bugzilla.redhat.com/1366176" target="_blank">https://bugzilla.redhat.com/<wbr>1366176</a><br>
><br>
> Downgrade the lvm2 package.<br>
> Or upgrade vdsm to 4.17.35<br>
><br>
> -----Original Message-----<br>
> From: <<a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a>> on behalf of Charles Tassell <<a href="mailto:charles@islandadmin.ca" target="_blank">charles@islandadmin.ca</a>><br>
> Date: Wednesday 4 January 2017 at 23:22<br>
> To: "<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
> Subject: [ovirt-users] Storage Problem After Host Update (v3.6)<br>
><br>
> Hello,<br>
><br>
> One of our admins did a package update on one of our 3.6 hosts<br>
> through the web GUI and then rebooted the host. Since then, it hasn't<br>
> been able to properly mount storage and thus we can't move any VMs to<br>
> it. I rebooted it myself this afternoon and it took our whole cluster<br>
> down for about a half hour, which was kind of concerning.<br>
><br>
> We use multipath fibre channel for storage and when I do a vgscan or<br>
> vgdisplay -s everything shows up correctly but I get a lot of "Duplicate<br>
> of PV ..." warning messages. I also get the log entries below<br>
> constantly repeating in vdsm.log. Any ideas what's going on and how I<br>
> fix it?<br>
><br>
> CannotActivateLogicalVolumes: Cannot activate Logical Volumes: ('General<br>
> Storage Exception: (\'5 [] [\\\' WARNING: lvmetad is running but<br>
> disabled. Restart lvmetad before enabling it!\\\', " Refusing activation<br>
> of partial LV 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/metadata. Use<br>
> \\\'--activationmode partial\\\' to<br>
> override."]\\n8dfcc83b-64d7-<wbr>4ff7-9800-02ba3430ea7b/[\\\'<wbr>metadata\\\']\',)',)<br>
> Thread-1844::ERROR::2017-01-04<br>
> 16:03:43,649::dispatcher::76::<wbr>Storage.Dispatcher::(wrapper) {'status':<br>
> {'message': 'Cannot activate Logical Volumes: (\'General Storage<br>
> Exception: (\\\'5 [] [\\\\\\\' WARNING: lvmetad is running but<br>
> disabled. Restart lvmetad before enabling it!\\\\\\\', " Refusing<br>
> activation of partial LV 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/metadata.<br>
> Use \\\\\\\'--activationmode partial\\\\\\\' to<br>
> override."]\\\\n8dfcc83b-64d7-<wbr>4ff7-9800-02ba3430ea7b/[\\\\\\<wbr>\'metadata\\\\\\\']\\\',)\',)'<wbr>,<br>
> 'code': 570}}<br>
> Thread-101::ERROR::2017-01-04<br>
> 16:03:44,303::monitor::276::<wbr>Storage.Monitor::(_<wbr>monitorDomain) Error<br>
> monitoring domain 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b<br>
> Traceback (most recent call last):<br>
> File "/usr/share/vdsm/storage/<wbr>monitor.py", line 264, in _monitorDomain<br>
> self._produceDomain()<br>
> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/utils.py", line 769, in<br>
> wrapper<br>
> value = meth(self, *a, **kw)<br>
> File "/usr/share/vdsm/storage/<wbr>monitor.py", line 323, in _produceDomain<br>
> self.domain = sdCache.produce(self.sdUUID)<br>
> File "/usr/share/vdsm/storage/sdc.<wbr>py", line 100, in produce<br>
> domain.getRealDomain()<br>
> File "/usr/share/vdsm/storage/sdc.<wbr>py", line 52, in getRealDomain<br>
> return self._cache._realProduce(self.<wbr>_sdUUID)<br>
> File "/usr/share/vdsm/storage/sdc.<wbr>py", line 124, in _realProduce<br>
> domain = self._findDomain(sdUUID)<br>
> File "/usr/share/vdsm/storage/sdc.<wbr>py", line 143, in _findDomain<br>
> dom = findMethod(sdUUID)<br>
> File "/usr/share/vdsm/storage/<wbr>blockSD.py", line 1402, in findDomain<br>
> return BlockStorageDomain(<wbr>BlockStorageDomain.<wbr>findDomainPath(sdUUID))<br>
> File "/usr/share/vdsm/storage/<wbr>blockSD.py", line 455, in __init__<br>
> manifest = self.manifestClass(sdUUID)<br>
> File "/usr/share/vdsm/storage/<wbr>blockSD.py", line 407, in __init__<br>
> metadata = selectMetadata(self.sdUUID)<br>
> File "/usr/share/vdsm/storage/<wbr>blockSD.py", line 375, in selectMetadata<br>
> if len(mdProvider) > 0:<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 65, in __len__<br>
> return len(self.keys())<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 110, in keys<br>
> return list(self.__iter__())<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 107, in __iter__<br>
> self._dict.__iter__())<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 225, in __iter__<br>
> with self._accessWrapper():<br>
> File "/usr/lib64/python2.7/<wbr>contextlib.py", line 17, in __enter__<br>
> return self.gen.next()<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 152, in<br>
> _accessWrapper<br>
> self.refresh()<br>
> File "/usr/share/vdsm/storage/<wbr>persistentDict.py", line 230, in refresh<br>
> lines = self._metaRW.readlines()<br>
> File "/usr/share/vdsm/storage/<wbr>blockSD.py", line 335, in readlines<br>
> lvm.activateLVs(self._vgName, self._lvName)<br>
> File "/usr/share/vdsm/storage/lvm.<wbr>py", line 1191, in activateLVs<br>
> _setLVAvailability(vgName, toActivate, "y")<br>
> File "/usr/share/vdsm/storage/lvm.<wbr>py", line 826, in _setLVAvailability<br>
> raise error(str(e))<br>
><br>
> LVM output:<br>
> [root@ovirt-01 mapper]# vgchange -a y<br>
> WARNING: duplicate PV 1oDZGSqoQ4ueh0XEMzILNleRoNY8ZD<wbr>At is being used<br>
> from both devices /dev/sdac and /dev/sdk<br>
> Found duplicate PV 1oDZGSqoQ4ueh0XEMzILNleRoNY8ZD<wbr>At: using /dev/sdk<br>
> not /dev/sdac<br>
> Using duplicate PV /dev/sdk which is last seen, replacing /dev/sdac<br>
> Duplicate of PV EMfxl6-8coB-iUdZ-Dx4P-e5LZ-<wbr>UcIF-kvIfXx dev<br>
> /dev/mapper/<wbr>360060160ec732300fa5fdde41c66e<wbr>111p1 exists on unknown device<br>
> 65:177<br>
> Duplicate of PV EMfxl6-8coB-iUdZ-Dx4P-e5LZ-<wbr>UcIF-kvIfXx dev<br>
> /dev/mapper/<wbr>360060160ec732300fa5fdde41c66e<wbr>111p1 exists on unknown device<br>
> 8:145<br>
> Duplicate of PV i8z5fn-iYa0-ElZg-nL8h-Abul-<wbr>2HKU-Oh3F8w dev<br>
> /dev/mapper/<wbr>360060160ec7323000c61c43b3a66e<wbr>111p1 exists on unknown device<br>
> 65:225<br>
> Duplicate of PV i8z5fn-iYa0-ElZg-nL8h-Abul-<wbr>2HKU-Oh3F8w dev<br>
> /dev/mapper/<wbr>360060160ec7323000c61c43b3a66e<wbr>111p1 exists on unknown device<br>
> 8:193<br>
> Duplicate of PV f1IMBI-f2kl-vr1b-BbO1-Ydfw-<wbr>n5Ok-p1zPDd dev<br>
> /dev/mapper/<wbr>360060160ec7323001e5bcfa3026fe<wbr>111p1 exists on unknown device<br>
> 8:241<br>
> Duplicate of PV f1IMBI-f2kl-vr1b-BbO1-Ydfw-<wbr>n5Ok-p1zPDd dev<br>
> /dev/mapper/<wbr>360060160ec7323001e5bcfa3026fe<wbr>111p1 exists on unknown device<br>
> 66:17<br>
> Duplicate of PV N2B92Z-jpvT-RZVW-ot5M-4J9x-<wbr>wxjN-ZU6dJL dev<br>
> /dev/mapper/<wbr>360060160ec73230036b603a1db6ee<wbr>111p1 exists on unknown device<br>
> 65:241<br>
> Duplicate of PV N2B92Z-jpvT-RZVW-ot5M-4J9x-<wbr>wxjN-ZU6dJL dev<br>
> /dev/mapper/<wbr>360060160ec73230036b603a1db6ee<wbr>111p1 exists on unknown device<br>
> 8:209<br>
> Duplicate of PV UabEG9-XpDC-aE0j-gES1-SieC-<wbr>Ys3P-idn2zW dev<br>
> /dev/mapper/<wbr>360060160ec73230058c1b232906fe<wbr>111p1 exists on unknown device<br>
> 66:49<br>
> Duplicate of PV UabEG9-XpDC-aE0j-gES1-SieC-<wbr>Ys3P-idn2zW dev<br>
> /dev/mapper/<wbr>360060160ec73230058c1b232906fe<wbr>111p1 exists on unknown device<br>
> 65:17<br>
> Duplicate of PV lwe5su-6xQw-MiJu-Dh86-HNVz-<wbr>TsnG-YLoSG9 dev<br>
> /dev/mapper/<wbr>360060160ec732300be469e57ea6ee<wbr>111p1 exists on unknown device<br>
> 8:225<br>
> Duplicate of PV lwe5su-6xQw-MiJu-Dh86-HNVz-<wbr>TsnG-YLoSG9 dev<br>
> /dev/mapper/<wbr>360060160ec732300be469e57ea6ee<wbr>111p1 exists on unknown device<br>
> 66:1<br>
> Duplicate of PV tZsNnJ-LbSu-xOC2-uyVO-ATmH-<wbr>0BFk-lRPeE0 dev<br>
> /dev/mapper/<wbr>360060160ec732300c40d7ae05d6fe<wbr>111p1 exists on unknown device<br>
> 66:33<br>
> Duplicate of PV tZsNnJ-LbSu-xOC2-uyVO-ATmH-<wbr>0BFk-lRPeE0 dev<br>
> /dev/mapper/<wbr>360060160ec732300c40d7ae05d6fe<wbr>111p1 exists<br>
> on unknown device 65:1<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/metadata. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/outbox. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/leases. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/ids. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/inbox. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/master. Use '--activationmode<br>
> partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/389cceba-d9a4-<wbr>4452-b9a2-fd5a789084c0.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/873a161b-0933-<wbr>41ef-ad14-950a19dee77b.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/31dc9c3d-d316-<wbr>42ef-83c2-ed1bc6aa957e.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/d7c018c4-6488-<wbr>4a44-8ef5-42a4723fefaa.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/944a179a-35b3-<wbr>4bfe-ad21-43b1e902f6c0.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/ecfba68b-4fb4-<wbr>41d9-bd79-a570b08c02b2.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/c3b6fb85-1611-<wbr>453b-88c4-6d222089b100.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/6e1a0348-5305-<wbr>4d07-87cf-6ba74a9ad93c.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/121922ab-aa31-<wbr>4681-b268-61834eaffcb3.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/1ed17401-ba4f-<wbr>42de-8b47-86ea9a2938c5.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/36a0f2a4-25c2-<wbr>4201-b578-d67897083a2b.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/f20a2045-1d13-<wbr>4141-a544-75d4404c1847.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/7dc84aad-f4bb-<wbr>4f7a-8dce-72c5ec66cef1.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/68d74d9b-b8c1-<wbr>433c-8be7-735ab369eab2.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/59bcadd0-4fd9-<wbr>494b-a137-205588ba5ff6.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/f14f9a94-515c-<wbr>468e-b9aa-17acd1cf8c6f.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/4aa24cd6-e218-<wbr>4d31-be1f-d646fb5a52a5.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/23300e46-e1aa-<wbr>4b72-ba62-5833ca3780f7.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/5992ba58-6fb0-<wbr>421f-816c-04c162b6d0ab.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/9830bb60-960a-<wbr>432b-ad7c-279b37422ac7.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/3c4199fe-8ec3-<wbr>4b0f-83d6-78ec0ec23481.<br>
> Use '--activationmode partial' to override.<br>
> Refusing activation of partial LV<br>
> 8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/f409d03f-c1db-<wbr>420f-a8a3-5f445fab3adf.<br>
> Use '--activationmode partial' to override.<br>
> 24 logical volume(s) in volume group<br>
> "8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b" now active<br>
> 3 logical volume(s) in volume group "centos_ovirt-01" now active<br>
> WARNING: duplicate PV TgIsrjy7YlXlBrJJK1s30xs7AyY7Cw<wbr>mZ is being used<br>
> from both devices /dev/mapper/<wbr>360060160d5732300f01ea0bed593e<wbr>511 and /dev/sdu<br>
> WARNING: duplicate PV TgIsrjy7YlXlBrJJK1s30xs7AyY7Cw<wbr>mZ is being used<br>
> from both devices /dev/mapper/<wbr>360060160d5732300f01ea0bed593e<wbr>511 and /dev/sdc<br>
> 9 logical volume(s) in volume group<br>
> "383a1e7d-8b86-4a50-8bc3-<wbr>e4374c57a3e5" now active<br>
><br>
> [root@ovirt-01 mapper]# ls /dev/8dfcc83b-64d7-4ff7-9800-<wbr>02ba3430ea7b/<br>
> 04c87c50-1eb2-4453-bae2-<wbr>fe705e567976<br>
> 596c3099-2e41-4434-9902-<wbr>aa2aeab1fb85 bea2af77-301e-4f5e-942b-<wbr>43e12c30a33c<br>
> 1c77f2e1-d649-49e7-87c1-<wbr>52b7b2a4f172<br>
> 6324cfd7-0369-4b29-aca8-<wbr>0f83bf0593c8 cb155486-08f7-42a4-b46c-<wbr>48d98631fea1<br>
> 230da8da-373e-4dcc-bb79-<wbr>bd7c666d8ca6<br>
> 71bcdd9b-db35-46a6-9ba5-<wbr>94a7b4b5216d d1e9cefd-fb24-4cc1-af7c-<wbr>01debe42c40d<br>
> 2a4923c8-730b-474e-886a-<wbr>e75404739d71<br>
> 821862ea-0a09-4178-9625-<wbr>7c1e65021325 d638feb1-e3e4-4103-a47f-<wbr>5dc771092e62<br>
> 2bd2cb44-017c-4cb7-b710-<wbr>9c2b3fedbf55<br>
> 830c7f9d-6fd8-4feb-a70e-<wbr>00f4af25177f dec27ebe-2491-4a40-9c02-<wbr>82627448aa37<br>
> 33c1634c-cace-4c70-a59c-<wbr>51ff5ea0fe64<br>
> 893d55d1-032a-4c7d-90d1-<wbr>d2e2dcafd2b4 ea5471af-f1ab-43a0-a646-<wbr>787354366421<br>
> 39c29ad6-bb02-409c-9cc4-<wbr>6b2960812058<br>
> 8bbef2d3-2c27-41f8-82b2-<wbr>21b7e1ba17dd f425d6e0-8641-4207-84eb-<wbr>bb4d39d08bc2<br>
> 4924d2cc-8f98-4fa6-8169-<wbr>82931b05d22c<br>
> b09b84b7-875b-4a60-9f70-<wbr>8b0b8a57cc53 f904d381-7c0d-4850-8c92-<wbr>cf72bc5456f7<br>
><br>
> ______________________________<wbr>_________________<br>
> Users mailing list<br>
> <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
><br>
><br>
<br>
</div>
</span></font>
</div></div>
<br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
<br></blockquote></div><br></div></div></div>