Re: [ovirt-users] Some major problems after 4.2 upgrade, could really use some assistance

This is becoming critical for me, does anyone have any ideas or recommendations on what I can do to recover access to hosted VM? As of right now I have three hosts that are fully updated, they have the 4.2 repo and a full yum update was performed on them, there are no new updates to apply. The hosted engine had updates as well as a full and complete engine-setup, but did not return after being shut down. There must be some way I can get the engine running again? Please On Thu, Jan 11, 2018 at 8:24 AM, Jayme <jaymef@gmail.com> wrote:
The hosts have all ready been fully updated with 4.2 packages though.
ex.
ovirt-host.x86_64 4.2.0-1.el7.centos @ovirt-4.2 ovirt-host-dependencies.x86_64 4.2.0-1.el7.centos @ovirt-4.2 ovirt-host-deploy.noarch 1.7.0-1.el7.centos @ovirt-4.2 ovirt-hosted-engine-ha.noarch 2.2.2-1.el7.centos @ovirt-4.2 ovirt-hosted-engine-setup.noarch 2.2.3-1.el7.centos @ovirt-4.2
On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak <msivak@redhat.com> wrote:
Hi,
yes, you need to upgrade the hosts. Just take the ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from ovirt 4.2 repositories.
Martin
How do I upgrade the hosted engine packages when I can't reach it or do you mean upgrade the hosts if so how exactly do I do that. As for the missing VM it appears that the disk image is there but it's missing XML file I have no idea why or how to recreate it.
On Jan 11, 2018 4:43 AM, "Martin Sivak" <msivak@redhat.com> wrote:
Hi,
you hit one known issue we already have fixes for (4.1 hosts with 4.2 engine): https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hoste
d-engine-ha+branch:v2.1.z+topic:ovf_42_for_41
You can try hotfixing it by upgrading hosted engine packages to 4.2 or applying the patches manually and installing python-lxml.
I am not sure what happened to your other VM.
Best regards
Martin Sivak
On Thu, Jan 11, 2018 at 6:15 AM, Jayme <jaymef@gmail.com> wrote:
I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS shared storage. The shared storage is mounted from one of the hosts.
I upgraded the hosted engine first, downloading the 4.2 rpm, doing a
yum
update then engine setup which seemed to complete successfully, at
end it powered down the hosted VM but it never came back up. I was unable to start it.
I proceeded to upgrade the three hosts, ovirt 4.2 rpm and a full yum update. I also rebooted each of the three hosts.
After some time the hosts did come back and almost all of the VMs are running again and seem to be working ok with the exception of two:
1. The hosted VM still will not start, I've tried everything I can
On Thu, Jan 11, 2018 at 11:40 AM, Jayme <jaymef@gmail.com> wrote: the think
of.
2. A VM that I know existed is not running and does not appear to exist, I have no idea where it is or how to start it.
1. Hosted engine
From one of the hosts I get a weird error trying to start it:
# hosted-engine --vm-start Command VM.getStats with args {'vmID': '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: (code=1, message=Virtual machine does not exist: {'vmId': u'4013c829-c9d7-4b72-90d5-6fe58137504c'})
From the two other hosts I do not get the same error as above, sometimes it appears to start but --vm-status shows errors such as: Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}
Seeing these errors in syslog:
Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: error : qemuOpenFileAs:3183 : Failed to open file
'/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/ c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c- 4967-a1e5-fbe47c0cd705': No such file or directory
Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: error : qemuDomainStorageOpenStat:11492 : cannot stat file
'/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/ c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c- 4967-a1e5-fbe47c0cd705': Bad file descriptor
2. Missing VM. virsh -r list on each host does not show the VM at all. I know it existed and is important. The log on one of the hosts even shows that it started it recently then stopped in 10 or so minutes later:
Jan 10 18:47:17 host3 systemd-machined: New machine qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Started Virtual Machine qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Starting Virtual Machine qemu-9-Berna. Jan 10 18:54:45 host3 systemd-machined: Machine qemu-9-Berna terminated.
How can I find out the status of the "Berna" VM and get it running again?
Thanks so much!
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users

--Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Were you running gluster under you shared storage? If so, you probably = need to setup ganesha nfs yourself. If not, check your ha-agent logs and make sure it=E2=80=99s mounting the = storage properly and check for errors. Good luck!
From: Jayme <jaymef@gmail.com> Subject: Re: [ovirt-users] Some major problems after 4.2 upgrade, = could really use some assistance Date: January 11, 2018 at 12:28:32 PM CST To: Martin Sivak; users@ovirt.org =20 This is becoming critical for me, does anyone have any ideas or = recommendations on what I can do to recover access to hosted VM? As of = right now I have three hosts that are fully updated, they have the 4.2 = repo and a full yum update was performed on them, there are no new = updates to apply. The hosted engine had updates as well as a full and = complete engine-setup, but did not return after being shut down. There = must be some way I can get the engine running again? Please =20 On Thu, Jan 11, 2018 at 8:24 AM, Jayme <jaymef@gmail.com = <mailto:jaymef@gmail.com>> wrote: The hosts have all ready been fully updated with 4.2 packages though. =20 ex.=20 =20 ovirt-host.x86_64 = 4.2.0-1.el7.centos @ovirt-4.2 ovirt-host-dependencies.x86_64 = 4.2.0-1.el7.centos @ovirt-4.2 ovirt-host-deploy.noarch = 1.7.0-1.el7.centos @ovirt-4.2 ovirt-hosted-engine-ha.noarch = 2.2.2-1.el7.centos @ovirt-4.2 ovirt-hosted-engine-setup.noarch = 2.2.3-1.el7.centos @ovirt-4.2 =20 On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak <msivak@redhat.com = <mailto:msivak@redhat.com>> wrote: Hi, =20 yes, you need to upgrade the hosts. Just take the ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from ovirt 4.2 repositories. =20 Martin =20 On Thu, Jan 11, 2018 at 11:40 AM, Jayme <jaymef@gmail.com = <mailto:jaymef@gmail.com>> wrote:
How do I upgrade the hosted engine packages when I can't reach it or = do you mean upgrade the hosts if so how exactly do I do that. As for the = missing VM it appears that the disk image is there but it's missing XML file I = have no idea why or how to recreate it.
On Jan 11, 2018 4:43 AM, "Martin Sivak" <msivak@redhat.com = <mailto:msivak@redhat.com>> wrote:
Hi,
you hit one known issue we already have fixes for (4.1 hosts with =
4.2
engine): = https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hosted-engine-ha+br= anch:v2.1.z+topic:ovf_42_for_41 = <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hosted-engine-ha+b= ranch:v2.1.z+topic:ovf_42_for_41>
You can try hotfixing it by upgrading hosted engine packages to 4.2 = or applying the patches manually and installing python-lxml.
I am not sure what happened to your other VM.
Best regards
Martin Sivak
On Thu, Jan 11, 2018 at 6:15 AM, Jayme <jaymef@gmail.com = <mailto:jaymef@gmail.com>> wrote:
I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS shared storage. The shared storage is mounted from one of the hosts.
I upgraded the hosted engine first, downloading the 4.2 rpm, = doing a yum update then engine setup which seemed to complete successfully, = at the end it powered down the hosted VM but it never came back up. I was = unable to start it.
I proceeded to upgrade the three hosts, ovirt 4.2 rpm and a full = yum update. I also rebooted each of the three hosts.
After some time the hosts did come back and almost all of the VMs = are running again and seem to be working ok with the exception of = two:
1. The hosted VM still will not start, I've tried everything I = can think of.
2. A VM that I know existed is not running and does not appear to = exist, I have no idea where it is or how to start it.
1. Hosted engine
=46rom one of the hosts I get a weird error trying to start it:
# hosted-engine --vm-start Command VM.getStats with args {'vmID': '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: (code=3D1, message=3DVirtual machine does not exist: {'vmId': u'4013c829-c9d7-4b72-90d5-6fe58137504c'})
=46rom the two other hosts I do not get the same error as above, = sometimes it appears to start but --vm-status shows errors such as: Engine = status : {"reason": "failed liveliness check", "health": "bad", "vm": = "up", "detail": "Up"}
Seeing these errors in syslog:
Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: = 1910: error : qemuOpenFileAs:3183 : Failed to open file
= '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-= 4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705': No such file or directory
Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: = 1910: error : qemuDomainStorageOpenStat:11492 : cannot stat file
= '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-= 4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705': Bad file descriptor
2. Missing VM. virsh -r list on each host does not show the VM = at all. I know it existed and is important. The log on one of the hosts = even shows that it started it recently then stopped in 10 or so minutes = later:
Jan 10 18:47:17 host3 systemd-machined: New machine qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Started Virtual Machine = qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Starting Virtual Machine = qemu-9-Berna. Jan 10 18:54:45 host3 systemd-machined: Machine qemu-9-Berna = terminated.
How can I find out the status of the "Berna" VM and get it = running again?
Thanks so much!
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users = <http://lists.ovirt.org/mailman/listinfo/users>
=20 =20
Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Were = you running gluster under you shared storage? If so, you probably need = to setup ganesha nfs yourself.<div class=3D""><br class=3D""></div><div = class=3D"">If not, check your ha-agent logs and make sure it=E2=80=99s = mounting the storage properly and check for errors. Good luck!</div><div = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><hr = style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:0 0 0 = 0;margin:10px 0 5px 0;" class=3D""><span style=3D"margin: -1.3px 0.0px = 0.0px 0.0px" id=3D"RwhHeaderAttributes" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">From:</b> Jayme = <<a href=3D"mailto:jaymef@gmail.com" = class=3D"">jaymef@gmail.com</a>></font></span><br class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">Subject:</b> Re: = [ovirt-users] Some major problems after 4.2 upgrade, could really use = some assistance</font></span><br class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">Date:</b> January = 11, 2018 at 12:28:32 PM CST</font></span><br class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">To:</b> Martin = Sivak; <a href=3D"mailto:users@ovirt.org" = class=3D"">users@ovirt.org</a></font></span><br class=3D""> <br class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr" = class=3D"">This is becoming critical for me, does anyone have any ideas = or=20 recommendations on what I can do to recover access to hosted VM? = As of=20 right now I have three hosts that are fully updated, they have the 4.2=20= repo and a full yum update was performed on them, there are no new=20 updates to apply. The hosted engine had updates as well as a full = and=20 complete engine-setup, but did not return after being shut down. = There=20 must be some way I can get the engine running again? = Please</div><div class=3D"gmail_extra"><br class=3D""><div = class=3D"gmail_quote">On Thu, Jan 11, 2018 at 8:24 AM, Jayme <span = dir=3D"ltr" class=3D""><<a href=3D"mailto:jaymef@gmail.com" = target=3D"_blank" class=3D"">jaymef@gmail.com</a>></span> wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr" = class=3D"">The hosts have all ready been fully updated with 4.2 packages = though.<div class=3D""><br class=3D""></div><div = class=3D"">ex. </div><div class=3D""><br class=3D""></div><div = class=3D""><div class=3D"">ovirt-host.x86_64 = = = 4.2.0-1.el7.centos = = @ovirt-4.2</div><div = class=3D"">ovirt-host-dependencies.x86_<wbr class=3D"">64 = = = 4.2.0-1.el7.centos = = @ovirt-4.2</div><div class=3D"">ovirt-host-deploy.noarch = = = 1.7.0-1.el7.centos = = @ovirt-4.2</div><div class=3D"">ovirt-hosted-engine-ha.noarch = = = 2.2.2-1.el7.centos = = @ovirt-4.2</div><div class=3D"">ovirt-hosted-engine-setup.<wbr = class=3D"">noarch = = 2.2.3-1.el7.centos = = @ovirt-4.2</div></div></div><div class=3D"HOEnZb"><div = class=3D"h5"><div class=3D"gmail_extra"><br class=3D""><div = class=3D"gmail_quote">On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak = <span dir=3D"ltr" class=3D""><<a href=3D"mailto:msivak@redhat.com" = target=3D"_blank" class=3D"">msivak@redhat.com</a>></span> wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br class=3D""> <br class=3D""> yes, you need to upgrade the hosts. Just take the<br class=3D""> ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from<br = class=3D""> ovirt 4.2 repositories.<br class=3D""> <span class=3D"m_6061824483285160243HOEnZb"><font color=3D"#888888" = class=3D""><br class=3D""> Martin<br class=3D""> </font></span><div class=3D"m_6061824483285160243HOEnZb"><div = class=3D"m_6061824483285160243h5"><br class=3D""> On Thu, Jan 11, 2018 at 11:40 AM, Jayme <<a = href=3D"mailto:jaymef@gmail.com" target=3D"_blank" = class=3D"">jaymef@gmail.com</a>> wrote:<br class=3D""> > How do I upgrade the hosted engine packages when I can't reach it = or do you<br class=3D""> > mean upgrade the hosts if so how exactly do I do that. As for the = missing VM<br class=3D""> > it appears that the disk image is there but it's missing XML file I = have no<br class=3D""> > idea why or how to recreate it.<br class=3D""> ><br class=3D""> > On Jan 11, 2018 4:43 AM, "Martin Sivak" <<a = href=3D"mailto:msivak@redhat.com" target=3D"_blank" = class=3D"">msivak@redhat.com</a>> wrote:<br class=3D""> >><br class=3D""> >> Hi,<br class=3D""> >><br class=3D""> >> you hit one known issue we already have fixes for (4.1 hosts = with 4.2<br class=3D""> >> engine):<br class=3D""> >> <a = href=3D"https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hosted-engi= ne-ha+branch:v2.1.z+topic:ovf_42_for_41" rel=3D"noreferrer" = target=3D"_blank" class=3D"">https://gerrit.ovirt.org/#/q/s<wbr = class=3D"">tatus:open+project:ovirt-hoste<wbr = class=3D"">d-engine-ha+branch:v2.1.z+<wbr = class=3D"">topic:ovf_42_for_41</a><br class=3D""> >><br class=3D""> >> You can try hotfixing it by upgrading hosted engine packages to = 4.2 or<br class=3D""> >> applying the patches manually and installing python-lxml.<br = class=3D""> >><br class=3D""> >> I am not sure what happened to your other VM.<br class=3D""> >><br class=3D""> >> Best regards<br class=3D""> >><br class=3D""> >> Martin Sivak<br class=3D""> >><br class=3D""> >> On Thu, Jan 11, 2018 at 6:15 AM, Jayme <<a = href=3D"mailto:jaymef@gmail.com" target=3D"_blank" = class=3D"">jaymef@gmail.com</a>> wrote:<br class=3D""> >> > I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS = shared<br class=3D""> >> > storage.<br class=3D""> >> > The shared storage is mounted from one of the hosts.<br = class=3D""> >> ><br class=3D""> >> > I upgraded the hosted engine first, downloading the 4.2 = rpm, doing a yum<br class=3D""> >> > update then engine setup which seemed to complete = successfully, at the<br class=3D""> >> > end<br class=3D""> >> > it powered down the hosted VM but it never came back = up. I was unable<br class=3D""> >> > to<br class=3D""> >> > start it.<br class=3D""> >> ><br class=3D""> >> > I proceeded to upgrade the three hosts, ovirt 4.2 rpm and = a full yum<br class=3D""> >> > update.<br class=3D""> >> > I also rebooted each of the three hosts.<br class=3D""> >> ><br class=3D""> >> > After some time the hosts did come back and almost all of = the VMs are<br class=3D""> >> > running again and seem to be working ok with the exception = of two:<br class=3D""> >> ><br class=3D""> >> > 1. The hosted VM still will not start, I've tried = everything I can think<br class=3D""> >> > of.<br class=3D""> >> ><br class=3D""> >> > 2. A VM that I know existed is not running and does not = appear to exist,<br class=3D""> >> > I<br class=3D""> >> > have no idea where it is or how to start it.<br class=3D""> >> ><br class=3D""> >> > 1. Hosted engine<br class=3D""> >> ><br class=3D""> >> > =46rom one of the hosts I get a weird error trying to = start it:<br class=3D""> >> ><br class=3D""> >> > # hosted-engine --vm-start<br class=3D""> >> > Command VM.getStats with args {'vmID':<br class=3D""> >> > '4013c829-c9d7-4b72-90d5-6fe58<wbr class=3D"">137504c'} = failed:<br class=3D""> >> > (code=3D1, message=3DVirtual machine does not exist: = {'vmId':<br class=3D""> >> > u'4013c829-c9d7-4b72-90d5-6fe5<wbr class=3D"">8137504c'})<br= class=3D""> >> ><br class=3D""> >> > =46rom the two other hosts I do not get the same error as = above, sometimes<br class=3D""> >> > it<br class=3D""> >> > appears to start but --vm-status shows errors such = as: Engine status<br class=3D""> >> > : {"reason": "failed liveliness check", "health": "bad", = "vm": "up",<br class=3D""> >> > "detail": "Up"}<br class=3D""> >> ><br class=3D""> >> > Seeing these errors in syslog:<br class=3D""> >> ><br class=3D""> >> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 = 05:06:30.473+0000: 1910:<br class=3D""> >> > error :<br class=3D""> >> > qemuOpenFileAs:3183 : Failed to open file<br class=3D""> >> ><br class=3D""> >> > '/var/run/vdsm/storage/248f46f<wbr = class=3D"">0-d793-4581-9810-c9d965e2f286/<wbr = class=3D"">c2dde892-f978-4dfc-a421-<wbr = class=3D"">c8e04cf387f9/23aa0a66-fa6c-<wbr = class=3D"">4967-a1e5-fbe47c0cd705':<br class=3D""> >> > No such file or directory<br class=3D""> >> ><br class=3D""> >> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 = 05:06:30.473+0000: 1910:<br class=3D""> >> > error :<br class=3D""> >> > qemuDomainStorageOpenStat:1149<wbr class=3D"">2 : cannot = stat file<br class=3D""> >> ><br class=3D""> >> > '/var/run/vdsm/storage/248f46f<wbr = class=3D"">0-d793-4581-9810-c9d965e2f286/<wbr = class=3D"">c2dde892-f978-4dfc-a421-<wbr = class=3D"">c8e04cf387f9/23aa0a66-fa6c-<wbr = class=3D"">4967-a1e5-fbe47c0cd705':<br class=3D""> >> > Bad file descriptor<br class=3D""> >> ><br class=3D""> >> > 2. Missing VM. virsh -r list on each host does not = show the VM at all.<br class=3D""> >> > I<br class=3D""> >> > know it existed and is important. The log on one of = the hosts even<br class=3D""> >> > shows<br class=3D""> >> > that it started it recently then stopped in 10 or so = minutes later:<br class=3D""> >> ><br class=3D""> >> > Jan 10 18:47:17 host3 systemd-machined: New machine = qemu-9-Berna.<br class=3D""> >> > Jan 10 18:47:17 host3 systemd: Started Virtual Machine = qemu-9-Berna.<br class=3D""> >> > Jan 10 18:47:17 host3 systemd: Starting Virtual Machine = qemu-9-Berna.<br class=3D""> >> > Jan 10 18:54:45 host3 systemd-machined: Machine = qemu-9-Berna terminated.<br class=3D""> >> ><br class=3D""> >> > How can I find out the status of the "Berna" VM and get it = running<br class=3D""> >> > again?<br class=3D""> >> ><br class=3D""> >> > Thanks so much!<br class=3D""> >> ><br class=3D""> >> > ______________________________<wbr = class=3D"">_________________<br class=3D""> >> > Users mailing list<br class=3D""> >> > <a href=3D"mailto:Users@ovirt.org" target=3D"_blank" = class=3D"">Users@ovirt.org</a><br class=3D""> >> > <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" = rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://lists.ovirt.org/mailman<wbr = class=3D"">/listinfo/users</a><br class=3D""> >> ><br class=3D""> </div></div></blockquote></div><br class=3D""></div> </div></div></blockquote></div><br class=3D""></div> _______________________________________________<br class=3D"">Users = mailing list<br class=3D""><a href=3D"mailto:Users@ovirt.org" = class=3D"">Users@ovirt.org</a><br = class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br = class=3D""></div></blockquote></div><br class=3D""></div></body></html>= --Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271--
participants (2)
-
Darrell Budic
-
Jayme