--Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Were you running gluster under you shared storage? If so, you probably =
need to setup ganesha nfs yourself.
If not, check your ha-agent logs and make sure it=E2=80=99s mounting the =
storage properly and check for errors. Good luck!
From: Jayme <jaymef(a)gmail.com>
Subject: Re: [ovirt-users] Some major problems after 4.2 upgrade, =
could really
use some assistance
Date: January 11, 2018 at 12:28:32 PM CST
To: Martin Sivak; users(a)ovirt.org
=20
This is becoming critical for me, does anyone have any ideas or =
recommendations
on what I can do to recover access to hosted VM? As of =
right now I have three hosts that are fully updated, they have the 4.2 =
repo and a full yum update was performed on them, there are no new =
updates to apply. The hosted engine had updates as well as a full and =
complete engine-setup, but did not return after being shut down. There =
must be some way I can get the engine running again? Please
=20
On Thu, Jan 11, 2018 at 8:24 AM, Jayme <jaymef(a)gmail.com =
<mailto:jaymef@gmail.com>> wrote:
The hosts have all ready been fully updated with 4.2 packages
though.
=20
ex.=20
=20
ovirt-host.x86_64 =
4.2.0-1.el7.centos @ovirt-4.2
ovirt-host-dependencies.x86_64
=
4.2.0-1.el7.centos @ovirt-4.2
ovirt-host-deploy.noarch
=
1.7.0-1.el7.centos @ovirt-4.2
ovirt-hosted-engine-ha.noarch
=
2.2.2-1.el7.centos @ovirt-4.2
ovirt-hosted-engine-setup.noarch
=
2.2.3-1.el7.centos @ovirt-4.2
=20
On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak <msivak(a)redhat.com =
<mailto:msivak@redhat.com>> wrote:
Hi,
=20
yes, you need to upgrade the hosts. Just take the
ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from
ovirt 4.2 repositories.
=20
Martin
=20
On Thu, Jan 11, 2018 at 11:40 AM, Jayme <jaymef(a)gmail.com =
<mailto:jaymef@gmail.com>> wrote:
> How do I upgrade the hosted engine packages when I can't
reach it or =
do you
> mean upgrade the hosts if so how exactly do I do that. As for
the =
missing VM
> it appears that the disk image is there but it's missing XML
file I =
have no
> idea why or how to recreate it.
>
> On Jan 11, 2018 4:43 AM, "Martin Sivak" <msivak(a)redhat.com =
<mailto:msivak@redhat.com>> wrote:
>>
>> Hi,
>>
>> you hit one known issue we already have fixes for (4.1 hosts with =
4.2
>> engine):
>> =
https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hosted-engine-ha+br=
anch:v2.1.z+topic:ovf_42_for_41 =
<
https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hosted-engine-ha+b=
ranch:v2.1.z+topic:ovf_42_for_41>
>>
>> You can try hotfixing it by upgrading hosted engine packages to 4.2 =
or
>> applying the patches manually and installing python-lxml.
>>
>> I am not sure what happened to your other VM.
>>
>> Best regards
>>
>> Martin Sivak
>>
>> On Thu, Jan 11, 2018 at 6:15 AM, Jayme <jaymef(a)gmail.com =
<mailto:jaymef@gmail.com>> wrote:
>> > I performed Ovirt 4.2 upgrade on a 3 host cluster with
NFS shared
>> > storage.
>> > The shared storage is mounted from one of the hosts.
>> >
>> > I upgraded the hosted engine first, downloading the 4.2 rpm, =
doing
a yum
>> > update then engine setup which seemed to complete
successfully, =
at the
>> > end
>> > it powered down the hosted VM but it never came back up. I was =
unable
>> > to
>> > start it.
>> >
>> > I proceeded to upgrade the three hosts, ovirt 4.2 rpm and a full =
yum
>> > update.
>> > I also rebooted each of the three hosts.
>> >
>> > After some time the hosts did come back and almost all of the VMs =
are
>> > running again and seem to be working ok with the
exception of =
two:
>> >
>> > 1. The hosted VM still will not start, I've tried everything I =
can think
>> > of.
>> >
>> > 2. A VM that I know existed is not running and does not appear to =
exist,
>> > I
>> > have no idea where it is or how to start it.
>> >
>> > 1. Hosted engine
>> >
>> > =46rom one of the hosts I get a weird error trying to start it:
>> >
>> > # hosted-engine --vm-start
>> > Command VM.getStats with args {'vmID':
>> > '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed:
>> > (code=3D1, message=3DVirtual machine does not exist: {'vmId':
>> > u'4013c829-c9d7-4b72-90d5-6fe58137504c'})
>> >
>> > =46rom the two other hosts I do not get the same error as above, =
sometimes
>> > it
>> > appears to start but --vm-status shows errors such as: Engine =
status
>> > : {"reason": "failed liveliness
check", "health": "bad", "vm": =
"up",
>> > "detail": "Up"}
>> >
>> > Seeing these errors in syslog:
>> >
>> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: =
1910:
>> > error :
>> > qemuOpenFileAs:3183 : Failed to open file
>> >
>> > =
'/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-=
4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705':
>> > No such file or directory
>> >
>> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: =
1910:
>> > error :
>> > qemuDomainStorageOpenStat:11492 : cannot stat file
>> >
>> > =
'/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-=
4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705':
>> > Bad file descriptor
>> >
>> > 2. Missing VM. virsh -r list on each host does not show the VM =
at
all.
>> > I
>> > know it existed and is important. The log on one of the hosts =
even
>> > shows
>> > that it started it recently then stopped in 10 or so minutes =
later:
>> >
>> > Jan 10 18:47:17 host3 systemd-machined: New machine qemu-9-Berna.
>> > Jan 10 18:47:17 host3 systemd: Started Virtual Machine =
qemu-9-Berna.
>> > Jan 10 18:47:17 host3 systemd: Starting Virtual Machine
=
qemu-9-Berna.
>> > Jan 10 18:54:45 host3 systemd-machined: Machine
qemu-9-Berna =
terminated.
>> >
>> > How can I find out the status of the "Berna" VM and get it =
running
>> > again?
>> >
>> > Thanks so much!
>> >
>> > _______________________________________________
>> > Users mailing list
>> > Users(a)ovirt.org <mailto:Users@ovirt.org>
>> >
http://lists.ovirt.org/mailman/listinfo/users =
<
http://lists.ovirt.org/mailman/listinfo/users>
>> >
=20
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;"
class=3D"">Were =
you running gluster under you shared storage? If so, you probably need =
to setup ganesha nfs yourself.<div class=3D""><br
class=3D""></div><div =
class=3D"">If not, check your ha-agent logs and make sure it=E2=80=99s =
mounting the storage properly and check for errors. Good luck!</div><div =
class=3D""><br class=3D""><div><blockquote
type=3D"cite" class=3D""><hr =
style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:0 0 0 =
0;margin:10px 0 5px 0;" class=3D""><span style=3D"margin: -1.3px
0.0px =
0.0px 0.0px" id=3D"RwhHeaderAttributes" class=3D""><font =
face=3D"Helvetica" size=3D"4" color=3D"#000000"
style=3D"font: 13.0px =
Helvetica; color: #000000" class=3D""><b
class=3D"">From:</b> Jayme =
<<a href=3D"mailto:jaymef@gmail.com" =
class=3D"">jaymef(a)gmail.com</a>&gt;</font></span><br
class=3D"">
<span style=3D"margin: -1.3px 0.0px 0.0px 0.0px"
class=3D""><font =
face=3D"Helvetica" size=3D"4" color=3D"#000000"
style=3D"font: 13.0px =
Helvetica; color: #000000" class=3D""><b
class=3D"">Subject:</b> Re: =
[ovirt-users] Some major problems after 4.2 upgrade, could really use =
some assistance</font></span><br class=3D"">
<span style=3D"margin: -1.3px 0.0px 0.0px 0.0px"
class=3D""><font =
face=3D"Helvetica" size=3D"4" color=3D"#000000"
style=3D"font: 13.0px =
Helvetica; color: #000000" class=3D""><b
class=3D"">Date:</b> January =
11, 2018 at 12:28:32 PM CST</font></span><br class=3D"">
<span style=3D"margin: -1.3px 0.0px 0.0px 0.0px"
class=3D""><font =
face=3D"Helvetica" size=3D"4" color=3D"#000000"
style=3D"font: 13.0px =
Helvetica; color: #000000" class=3D""><b
class=3D"">To:</b> Martin =
Sivak; <a href=3D"mailto:users@ovirt.org" =
class=3D"">users(a)ovirt.org</a></font></span><br
class=3D"">
<br class=3D"Apple-interchange-newline"><div
class=3D""><div dir=3D"ltr" =
class=3D"">This is becoming critical for me, does anyone have any ideas =
or=20
recommendations on what I can do to recover access to hosted VM? =
As of=20
right now I have three hosts that are fully updated, they have the 4.2=20=
repo and a full yum update was performed on them, there are no new=20
updates to apply. The hosted engine had updates as well as a full =
and=20
complete engine-setup, but did not return after being shut down. =
There=20
must be some way I can get the engine running again? =
Please</div><div class=3D"gmail_extra"><br
class=3D""><div =
class=3D"gmail_quote">On Thu, Jan 11, 2018 at 8:24 AM, Jayme <span =
dir=3D"ltr" class=3D""><<a
href=3D"mailto:jaymef@gmail.com" =
target=3D"_blank"
class=3D"">jaymef(a)gmail.com</a>&gt;</span> wrote:<br =
class=3D""><blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr" =
class=3D"">The hosts have all ready been fully updated with 4.2 packages =
though.<div class=3D""><br class=3D""></div><div
=
class=3D"">ex. </div><div class=3D""><br
class=3D""></div><div =
class=3D""><div class=3D"">ovirt-host.x86_64
=
=
=
4.2.0-1.el7.centos
=
=
&nbsp;(a)ovirt-4.2</div><div =
class=3D"">ovirt-host-dependencies.x86_<wbr
class=3D"">64 =
=
=
4.2.0-1.el7.centos
=
=
&nbsp;(a)ovirt-4.2</div><div
class=3D"">ovirt-host-deploy.noarch =
=
=
1.7.0-1.el7.centos
=
=
&nbsp;(a)ovirt-4.2</div><div
class=3D"">ovirt-hosted-engine-ha.noarch =
=
=
2.2.2-1.el7.centos
=
=
&nbsp;(a)ovirt-4.2</div><div
class=3D"">ovirt-hosted-engine-setup.<wbr =
class=3D"">noarch
=
=
2.2.3-1.el7.centos
=
=
&nbsp;(a)ovirt-4.2</div></div></div><div
class=3D"HOEnZb"><div =
class=3D"h5"><div class=3D"gmail_extra"><br
class=3D""><div =
class=3D"gmail_quote">On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak =
<span dir=3D"ltr" class=3D""><<a
href=3D"mailto:msivak@redhat.com" =
target=3D"_blank"
class=3D"">msivak(a)redhat.com</a>&gt;</span> wrote:<br =
class=3D""><blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br
class=3D"">
<br class=3D"">
yes, you need to upgrade the hosts. Just take the<br class=3D"">
ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from<br =
class=3D"">
ovirt 4.2 repositories.<br class=3D"">
<span class=3D"m_6061824483285160243HOEnZb"><font
color=3D"#888888" =
class=3D""><br class=3D"">
Martin<br class=3D"">
</font></span><div
class=3D"m_6061824483285160243HOEnZb"><div =
class=3D"m_6061824483285160243h5"><br class=3D"">
On Thu, Jan 11, 2018 at 11:40 AM, Jayme <<a =
href=3D"mailto:jaymef@gmail.com" target=3D"_blank" =
class=3D"">jaymef(a)gmail.com</a>&gt; wrote:<br
class=3D"">
> How do I upgrade the hosted engine packages when I can't reach it =
or do you<br class=3D"">
> mean upgrade the hosts if so how exactly do I do that. As for the =
missing VM<br class=3D"">
> it appears that the disk image is there but it's missing XML file I =
have no<br class=3D"">
> idea why or how to recreate it.<br class=3D"">
><br class=3D"">
> On Jan 11, 2018 4:43 AM, "Martin Sivak" <<a =
href=3D"mailto:msivak@redhat.com" target=3D"_blank" =
class=3D"">msivak(a)redhat.com</a>&gt; wrote:<br
class=3D"">
>><br class=3D"">
>> Hi,<br class=3D"">
>><br class=3D"">
>> you hit one known issue we already have fixes for (4.1 hosts =
with 4.2<br class=3D"">
>> engine):<br class=3D"">
>> <a =
href=3D"https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hoste...
ne-ha+branch:v2.1.z+topic:ovf_42_for_41" rel=3D"noreferrer" =
target=3D"_blank"
class=3D"">https://gerrit.ovirt.org/#/q/s<wbr
=
class=3D"">tatus:open+project:ovirt-hoste<wbr =
class=3D"">d-engine-ha+branch:v2.1.z+<wbr =
class=3D"">topic:ovf_42_for_41</a><br class=3D"">
>><br class=3D"">
>> You can try hotfixing it by upgrading hosted engine packages to =
4.2 or<br class=3D"">
>> applying the patches manually and installing python-lxml.<br =
class=3D"">
>><br class=3D"">
>> I am not sure what happened to your other VM.<br
class=3D"">
>><br class=3D"">
>> Best regards<br class=3D"">
>><br class=3D"">
>> Martin Sivak<br class=3D"">
>><br class=3D"">
>> On Thu, Jan 11, 2018 at 6:15 AM, Jayme <<a =
href=3D"mailto:jaymef@gmail.com" target=3D"_blank" =
class=3D"">jaymef(a)gmail.com</a>&gt; wrote:<br
class=3D"">
>> > I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS =
shared<br class=3D"">
>> > storage.<br class=3D"">
>> > The shared storage is mounted from one of the hosts.<br =
class=3D"">
>> ><br class=3D"">
>> > I upgraded the hosted engine first, downloading the 4.2 =
rpm, doing a yum<br class=3D"">
>> > update then engine setup which seemed to complete =
successfully, at the<br class=3D"">
>> > end<br class=3D"">
>> > it powered down the hosted VM but it never came back =
up. I was unable<br class=3D"">
>> > to<br class=3D"">
>> > start it.<br class=3D"">
>> ><br class=3D"">
>> > I proceeded to upgrade the three hosts, ovirt 4.2 rpm and =
a full yum<br class=3D"">
>> > update.<br class=3D"">
>> > I also rebooted each of the three hosts.<br
class=3D"">
>> ><br class=3D"">
>> > After some time the hosts did come back and almost all of =
the VMs are<br class=3D"">
>> > running again and seem to be working ok with the exception =
of two:<br class=3D"">
>> ><br class=3D"">
>> > 1. The hosted VM still will not start, I've tried =
everything I can think<br class=3D"">
>> > of.<br class=3D"">
>> ><br class=3D"">
>> > 2. A VM that I know existed is not running and does not =
appear to exist,<br class=3D"">
>> > I<br class=3D"">
>> > have no idea where it is or how to start it.<br
class=3D"">
>> ><br class=3D"">
>> > 1. Hosted engine<br class=3D"">
>> ><br class=3D"">
>> > =46rom one of the hosts I get a weird error trying to =
start it:<br class=3D"">
>> ><br class=3D"">
>> > # hosted-engine --vm-start<br class=3D"">
>> > Command VM.getStats with args {'vmID':<br
class=3D"">
>> > '4013c829-c9d7-4b72-90d5-6fe58<wbr
class=3D"">137504c'} =
failed:<br class=3D"">
>> > (code=3D1, message=3DVirtual machine does not exist: =
{'vmId':<br class=3D"">
>> > u'4013c829-c9d7-4b72-90d5-6fe5<wbr
class=3D"">8137504c'})<br=
class=3D"">
>> ><br class=3D"">
>> > =46rom the two other hosts I do not get the same error as =
above, sometimes<br class=3D"">
>> > it<br class=3D"">
>> > appears to start but --vm-status shows errors such =
as: Engine status<br class=3D"">
>> > : {"reason": "failed liveliness check",
"health": "bad", =
"vm": "up",<br class=3D"">
>> > "detail": "Up"}<br
class=3D"">
>> ><br class=3D"">
>> > Seeing these errors in syslog:<br class=3D"">
>> ><br class=3D"">
>> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 =
05:06:30.473+0000: 1910:<br class=3D"">
>> > error :<br class=3D"">
>> > qemuOpenFileAs:3183 : Failed to open file<br
class=3D"">
>> ><br class=3D"">
>> > '/var/run/vdsm/storage/248f46f<wbr =
class=3D"">0-d793-4581-9810-c9d965e2f286/<wbr =
class=3D"">c2dde892-f978-4dfc-a421-<wbr =
class=3D"">c8e04cf387f9/23aa0a66-fa6c-<wbr =
class=3D"">4967-a1e5-fbe47c0cd705':<br class=3D"">
>> > No such file or directory<br class=3D"">
>> ><br class=3D"">
>> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 =
05:06:30.473+0000: 1910:<br class=3D"">
>> > error :<br class=3D"">
>> > qemuDomainStorageOpenStat:1149<wbr class=3D"">2
: cannot =
stat file<br class=3D"">
>> ><br class=3D"">
>> > '/var/run/vdsm/storage/248f46f<wbr =
class=3D"">0-d793-4581-9810-c9d965e2f286/<wbr =
class=3D"">c2dde892-f978-4dfc-a421-<wbr =
class=3D"">c8e04cf387f9/23aa0a66-fa6c-<wbr =
class=3D"">4967-a1e5-fbe47c0cd705':<br class=3D"">
>> > Bad file descriptor<br class=3D"">
>> ><br class=3D"">
>> > 2. Missing VM. virsh -r list on each host does not =
show the VM at all.<br class=3D"">
>> > I<br class=3D"">
>> > know it existed and is important. The log on one of =
the hosts even<br class=3D"">
>> > shows<br class=3D"">
>> > that it started it recently then stopped in 10 or so =
minutes later:<br class=3D"">
>> ><br class=3D"">
>> > Jan 10 18:47:17 host3 systemd-machined: New machine =
qemu-9-Berna.<br class=3D"">
>> > Jan 10 18:47:17 host3 systemd: Started Virtual Machine =
qemu-9-Berna.<br class=3D"">
>> > Jan 10 18:47:17 host3 systemd: Starting Virtual Machine =
qemu-9-Berna.<br class=3D"">
>> > Jan 10 18:54:45 host3 systemd-machined: Machine =
qemu-9-Berna terminated.<br class=3D"">
>> ><br class=3D"">
>> > How can I find out the status of the "Berna" VM and
get it =
running<br class=3D"">
>> > again?<br class=3D"">
>> ><br class=3D"">
>> > Thanks so much!<br class=3D"">
>> ><br class=3D"">
>> > ______________________________<wbr =
class=3D"">_________________<br class=3D"">
>> > Users mailing list<br class=3D"">
>> > <a href=3D"mailto:Users@ovirt.org"
target=3D"_blank" =
class=3D"">Users(a)ovirt.org</a><br class=3D"">
>> > <a
href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
rel=3D"noreferrer" target=3D"_blank" =
class=3D"">http://lists.ovirt.org/mailman<wbr =
class=3D"">/listinfo/users</a><br class=3D"">
>> ><br class=3D"">
</div></div></blockquote></div><br
class=3D""></div>
</div></div></blockquote></div><br
class=3D""></div>
_______________________________________________<br class=3D"">Users =
mailing list<br class=3D""><a href=3D"mailto:Users@ovirt.org"
=
class=3D"">Users(a)ovirt.org</a><br =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br =
class=3D""></div></blockquote></div><br
class=3D""></div></body></html>=
--Apple-Mail=_AD24D2E9-79CF-4789-A356-B91097F38271--