9:05 a.m.
--Apple-Mail=_F7E509C0-3A8A-4F46-A687-082B386460EC
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
I just tried running the most recent update that came out for Ovirt =
Hosted Engine setup. This is a glusterized setup with 4 hosts setup as =
per the Documentation. Each host had the icon showing upgrades =
available so I started clicking upgrade for each host doing them one at =
time. 3 of them went into maintenance fine, migrated their VMs and then =
activated when done. The last one to do was the one actually running =
the hosted-engine VM. This host also had one other VM. It migrated the =
other VM just fine, but stalled trying to migrate the hosted-engine. =
Eventually the web interface became non-responsive and I moved to =
command line. I have tried everything to get the engine to start up. I =
have tried putting all the hosts in maintenance so that I could =
selectively tell it to start the hosted-engine VM on one host =
specifically, but each fails with the =E2=80=9CFailed Liveliness error. =
I have tried restarting the ha agents, re-initializing the lockspace all =
with no success. All my VMs are still running fine, I just cannot get =
the hosted-engine to run. I can if necessary shutdown all the VMs and =
restart all the hosts, but was trying to avoid that unless I have to.
Ideas.
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
--Apple-Mail=_F7E509C0-3A8A-4F46-A687-082B386460EC
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">I just tried running the most recent update that came out for =
Ovirt Hosted Engine setup. This is a glusterized setup with 4 =
hosts setup as per the Documentation. Each host had the icon =
showing upgrades available so I started clicking upgrade for each host =
doing them one at time. 3 of them went into maintenance fine, migrated =
their VMs and then activated when done. The last one to do was the =
one actually running the hosted-engine VM. This host also had one =
other VM. It migrated the other VM just fine, but stalled trying =
to migrate the hosted-engine. Eventually the web interface became =
non-responsive and I moved to command line. I have tried =
everything to get the engine to start up. I have tried putting all =
the hosts in maintenance so that I could selectively tell it to start =
the hosted-engine VM on one host specifically, but each fails with the =
=E2=80=9CFailed Liveliness error. I have tried restarting the ha =
agents, re-initializing the lockspace all with no success. All my =
VMs are still running fine, I just cannot get the hosted-engine to run. =
I can if necessary shutdown all the VMs and restart all the =
hosts, but was trying to avoid that unless I have to.<div
class=3D""><br =
class=3D""></div><div
class=3D"">Ideas.</div><div class=3D""><br =
class=3D""></div><div
class=3D"">Gary</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""><div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""></div></body></html>=
--Apple-Mail=_F7E509C0-3A8A-4F46-A687-082B386460EC--
11:01 p.m.
--Apple-Mail=_82355EF7-10FA-498D-A0CF-51866FF509DE
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
After more testing, I can get the engine vm to run manually with =
ha-agent stopped. It still fails the liveliness check. Here is the =
repeating error in the engine.log from the engine vm
ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool =E2=80=
=94 46) [] Error in getting DB connection, database is inaccessible: =
Error retrieving database metadata; nested exception is =
org.springframework.jdbc.support.MetaDataAccessException: Could not get =
Connection for extracting meta data; nested exception is =
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get =
JDBC Connection; nested exception is java.sql.SQLException: =
javax.resource.ResourceException: IJ000453: Unable to get managed =
connection for java:/ENGINEDataSource
Found a suggestion about trying a test connection to the database and =
get the following.
[root@fai-kvm-engine ~]# psql --host localhost --port 5432 --user engine =
--password engine
Password for user engine:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and =
accepting
TCP/IP connections on port 5432?
[root@fai-kvm-engine ~]#
and checking that postgresql is running shows=20
[root@fai-kvm-engine ~]# systemctl status postgresql
=C3=A2=E2=80=94=7F postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; =
vendor preset: disabled)
Active: failed (Result: exit-code) since Sat 2017-01-07 20:58:04 UTC; =
2s ago
Process: 23867 ExecStart=3D/usr/bin/pg_ctl start -D ${PGDATA} -s -o -p =
${PGPORT} -w -t 300 (code=3Dexited, status=3D1/FAILURE)
Process: 23860 ExecStartPre=3D/usr/bin/postgresql-check-db-dir =
${PGDATA} (code=3Dexited, status=3D0/SUCCESS)
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net systemd[1]: Starting =
PostgreSQL database server...
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net pg_ctl[23867]: pg_ctl: =
another server might be running; trying to start server anyway
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net pg_ctl[23867]: FATAL: =
pre-existing shared memory block (key 5432001, ID 0) is still in use
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net pg_ctl[23867]: HINT: If =
you're sure there are no old server processes still running, remove the =
shared memory block or just delete the file "postmaster.pid".
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net pg_ctl[23867]: pg_ctl: =
could not start server
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net pg_ctl[23867]: Examine the =
log output.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net systemd[1]: =
postgresql.service: control process exited, code=3Dexited status=3D1
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net systemd[1]: Failed to =
start PostgreSQL database server.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net systemd[1]: Unit =
postgresql.service entered failed state.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net systemd[1]: =
postgresql.service failed.
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
On Jan 6, 2017, at 10:05 PM, Gary Pedretty
<gary(a)ravnalaska.net> =
wrote:
=20
I just tried running the most recent update that came out for Ovirt =
Hosted Engine
setup. This is a glusterized setup with 4 hosts setup as =
per the Documentation. Each host had the icon showing upgrades =
available so I started clicking upgrade for each host doing them one at =
time. 3 of them went into maintenance fine, migrated their VMs and then =
activated when done. The last one to do was the one actually running =
the hosted-engine VM. This host also had one other VM. It migrated the =
other VM just fine, but stalled trying to migrate the hosted-engine. =
Eventually the web interface became non-responsive and I moved to =
command line. I have tried everything to get the engine to start up. I =
have tried putting all the hosts in maintenance so that I could =
selectively tell it to start the hosted-engine VM on one host =
specifically, but each fails with the =E2=80=9CFailed Liveliness error. =
I have tried restarting the ha agents, re-initializing the lockspace all =
with no success. All my VMs are still running fine, I just cannot get =
the hosted-engine to run. I can if necessary shutdown all the VMs and =
restart all the hosts, but was trying to avoid that unless I have to.
=20
Ideas.
=20
Gary
=20
=20
=
------------------------------------------------------------------------
Gary Pedretty =
gary(a)ravnalaska.net <mailto:gary@eraalaska.net>
Systems Manager =
www.flyravn.com <http://www.flyravn.com/>
Ravn Alaska /\ =
907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238
=
fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest =
commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your
=
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D
=
Matt 22:39
=
------------------------------------------------------------------------
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_82355EF7-10FA-498D-A0CF-51866FF509DE
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">After more testing, I can get the engine vm to run manually =
with ha-agent stopped. It still fails the liveliness check. =
Here is the repeating error in the engine.log from the engine =
vm<div class=3D""><br class=3D""></div><div
class=3D"">ERROR =
[org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool =E2=80=94 =
46) [] Error in getting DB connection, database is inaccessible: Error =
retrieving database metadata; nested exception is =
org.springframework.jdbc.support.MetaDataAccessException: Could not get =
Connection for extracting meta data; nested exception is =
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get =
JDBC Connection; nested exception is java.sql.SQLException: =
javax.resource.ResourceException: IJ000453: Unable to get managed =
connection for java:/ENGINEDataSource</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""></div><div =
class=3D"">Found a suggestion about trying a test connection to the =
database and get the following.</div><div class=3D""><br =
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-engine =
~]# psql --host localhost --port 5432 --user engine --password =
engine</div><div class=3D"">Password for user
engine:</div><div =
class=3D"">psql: could not connect to server: Connection =
refused</div><div class=3D"">
Is the server =
running on host "localhost" (::1) and accepting</div><div =
class=3D""> TCP/IP connections on
port =
5432?</div><div class=3D"">could not connect to server: Connection
=
refused</div><div class=3D"">
Is the server =
running on host "localhost" (127.0.0.1) and accepting</div><div =
class=3D""> TCP/IP connections on
port =
5432?</div><div class=3D"">[root@fai-kvm-engine
~]#</div></div><div =
class=3D""><br class=3D""></div><div
class=3D"">and checking that =
postgresql is running shows </div><div class=3D""><br =
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-engine =
~]# systemctl status postgresql</div><div
class=3D"">=C3=A2=E2=80=94=7F =
postgresql.service - PostgreSQL database server</div><div =
class=3D""> Loaded: loaded =
(/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: =
disabled)</div><div class=3D""> Active: failed
(Result: =
exit-code) since Sat 2017-01-07 20:58:04 UTC; 2s ago</div><div =
class=3D""> Process: 23867 ExecStart=3D/usr/bin/pg_ctl start -D =
${PGDATA} -s -o -p ${PGPORT} -w -t 300 (code=3Dexited, =
status=3D1/FAILURE)</div><div class=3D""> Process: 23860
=
ExecStartPre=3D/usr/bin/postgresql-check-db-dir ${PGDATA} (code=3Dexited, =
status=3D0/SUCCESS)</div><div class=3D""><br
class=3D""></div><div =
class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Starting =
PostgreSQL database server...</div><div class=3D"">Jan 07 20:58:03
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: pg_ctl: =
another server might be running; trying to start server anyway</div><div =
class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: FATAL: =
pre-existing shared memory block (key 5432001, ID 0) is still in =
use</div><div class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: HINT: =
If you're sure there are no old server processes still running, =
remove the shared memory block or just delete the file =
"postmaster.pid".</div><div class=3D"">Jan 07 20:58:04
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: pg_ctl: =
could not start server</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: Examine the
=
log output.</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: =
postgresql.service: control process exited, code=3Dexited =
status=3D1</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Failed to
start =
PostgreSQL database server.</div><div class=3D"">Jan 07 20:58:04
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Unit =
postgresql.service entered failed state.</div><div class=3D"">Jan 07
=
20:58:04 <a href=3D"http://fai-kvm-engine.ravnalaska.net" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: =
postgresql.service failed.</div></div><div class=3D""><br
=
class=3D""></div><div class=3D""><br
class=3D""><div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div =
class=3D"">On Jan 6, 2017, at 10:05 PM, Gary Pedretty <<a =
href=3D"mailto:gary@ravnalaska.net"
class=3D"">gary(a)ravnalaska.net</a>&gt;=
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><meta=
http-equiv=3D"Content-Type" content=3D"text/html charset=3Dutf-8" =
class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space;" class=3D"">I just tried
=
running the most recent update that came out for Ovirt Hosted Engine =
setup. This is a glusterized setup with 4 hosts setup as per the =
Documentation. Each host had the icon showing upgrades available =
so I started clicking upgrade for each host doing them one at time. 3 of =
them went into maintenance fine, migrated their VMs and then activated =
when done. The last one to do was the one actually running the =
hosted-engine VM. This host also had one other VM. It =
migrated the other VM just fine, but stalled trying to migrate the =
hosted-engine. Eventually the web interface became non-responsive =
and I moved to command line. I have tried everything to get the =
engine to start up. I have tried putting all the hosts in =
maintenance so that I could selectively tell it to start the =
hosted-engine VM on one host specifically, but each fails with the =
=E2=80=9CFailed Liveliness error. I have tried restarting the ha =
agents, re-initializing the lockspace all with no success. All my =
VMs are still running fine, I just cannot get the hosted-engine to run. =
I can if necessary shutdown all the VMs and restart all the =
hosts, but was trying to avoid that unless I have to.<div
class=3D""><br =
class=3D""></div><div
class=3D"">Ideas.</div><div class=3D""><br =
class=3D""></div><div
class=3D"">Gary</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""><div class=3D"">
<div style=3D"letter-spacing: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"letter-spacing:
normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px; letter-spacing: normal; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com/" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really loving the =
record green up date! Summmer!! yourself=E2=80=9D Matt =
22:39</span></font><div style=3D"letter-spacing: normal;
text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-family: Menlo;" class=3D""></div><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""></font><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><font face=3D"Menlo"
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"letter-spacing:
normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""><br =
class=3D""></font></div></div><span
style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br =
class=3D""></div></div>_______________________________________________<br
=
class=3D"">Users mailing list<br class=3D""><a =
href=3D"mailto:Users@ovirt.org"
class=3D"">Users(a)ovirt.org</a><br =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br =
class=3D""></div></blockquote></div><br
class=3D""></div></body></html>=
--Apple-Mail=_82355EF7-10FA-498D-A0CF-51866FF509DE--
11:33 p.m.
New subject: Hosted Engine Update problems [SOLVED]
--Apple-Mail=_CC370DE9-49A0-4353-AF88-4867FD6C6211
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
I tried the suggestion of deleting the postmaster.pid file after =
verifying that it was not locked by any process. =20
Then I restarted postgres which succeeded and then restarted the engine =
VM.
The vm came up normally and passed the liveliness test.
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
On Jan 7, 2017, at 12:01 PM, Gary Pedretty
<gary(a)ravnalaska.net> =
wrote:
=20
After more testing, I can get the engine vm to run manually with =
ha-agent
stopped. It still fails the liveliness check. Here is the =
repeating error in the engine.log from the engine vm
=20
ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool =
=E2=80=94
46) [] Error in getting DB connection, database is =
inaccessible: Error retrieving database metadata; nested exception is =
org.springframework.jdbc.support.MetaDataAccessException: Could not get =
Connection for extracting meta data; nested exception is =
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get =
JDBC Connection; nested exception is java.sql.SQLException: =
javax.resource.ResourceException: IJ000453: Unable to get managed =
connection for java:/ENGINEDataSource
=20
=20
Found a suggestion about trying a test connection to the database and =
get the
following.
=20
[root@fai-kvm-engine ~]# psql --host localhost --port 5432 --user =
engine
--password engine
Password for user engine:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and =
accepting
TCP/IP connections on port 5432?
[root@fai-kvm-engine ~]#
=20
and checking that postgresql is running shows=20
=20
[root@fai-kvm-engine ~]# systemctl status postgresql
=C3=A2=E2=80=94=7F postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; =
enabled; vendor
preset: disabled)
Active: failed (Result: exit-code) since Sat 2017-01-07 20:58:04 =
UTC; 2s ago
Process: 23867 ExecStart=3D/usr/bin/pg_ctl start -D ${PGDATA} -s -o
=
-p ${PGPORT} -w -t 300 (code=3Dexited, status=3D1/FAILURE)
Process: 23860 ExecStartPre=3D/usr/bin/postgresql-check-db-dir =
${PGDATA} (code=3Dexited, status=3D0/SUCCESS)
=20
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> systemd[1]: Starting PostgreSQL =
database server...
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> pg_ctl[23867]: pg_ctl: another =
server might be running; trying to start server anyway
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> pg_ctl[23867]: FATAL: =
pre-existing shared memory block (key 5432001, ID 0) is still in use
Jan 07 20:58:03 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> pg_ctl[23867]: HINT: If you're =
sure there are no old server processes still running, remove the shared =
memory block or just delete the file "postmaster.pid".
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> pg_ctl[23867]: pg_ctl: could not =
start server
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> pg_ctl[23867]: Examine the log =
output.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> systemd[1]: postgresql.service: =
control process exited, code=3Dexited status=3D1
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> systemd[1]: Failed to start =
PostgreSQL database server.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> systemd[1]: Unit =
postgresql.service entered failed state.
Jan 07 20:58:04 fai-kvm-engine.ravnalaska.net =
<http://fai-kvm-engine.ravnalaska.net/> systemd[1]: postgresql.service =
failed.
=20
=20
=
------------------------------------------------------------------------
Gary Pedretty =
gary(a)ravnalaska.net <mailto:gary@eraalaska.net>
Systems Manager =
www.flyravn.com <http://www.flyravn.com/>
Ravn Alaska /\ =
907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238
=
fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest =
commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your
=
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D
=
Matt 22:39
=
------------------------------------------------------------------------
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
> On Jan 6, 2017, at 10:05 PM, Gary Pedretty <gary(a)ravnalaska.net =
<mailto:gary@ravnalaska.net>> wrote:
>=20
> I just tried running the most recent update that came out for Ovirt =
Hosted
Engine setup. This is a glusterized setup with 4 hosts setup as =
per the Documentation. Each host had the icon showing upgrades =
available so I started clicking upgrade for each host doing them one at =
time. 3 of them went into maintenance fine, migrated their VMs and then =
activated when done. The last one to do was the one actually running =
the hosted-engine VM. This host also had one other VM. It migrated the =
other VM just fine, but stalled trying to migrate the hosted-engine. =
Eventually the web interface became non-responsive and I moved to =
command line. I have tried everything to get the engine to start up. I =
have tried putting all the hosts in maintenance so that I could =
selectively tell it to start the hosted-engine VM on one host =
specifically, but each fails with the =E2=80=9CFailed Liveliness error. =
I have tried restarting the ha agents, re-initializing the lockspace all =
with no success. All my VMs are still running fine, I just cannot get =
the hosted-engine to run. I can if necessary shutdown all the VMs and =
restart all the hosts, but was trying to avoid that unless I have to.
>>=20
>> Ideas.
>>=20
>> Gary
>>=20
>>=20
> =
------------------------------------------------------------------------
> Gary Pedretty =
gary(a)ravnalaska.net <mailto:gary@eraalaska.net>
> Systems Manager =
www.flyravn.com <http://www.flyravn.com/>
> Ravn Alaska /\ =
907-450-7251
> 5245 Airport Industrial Road / \/\ 907-450-7238
=
fax
> Fairbanks, Alaska 99709 /\ / \ \ Second greatest =
commandment
> Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your
=
neighbor as
> Really loving the record green up date! Summmer!! yourself=E2=80=9D
=
Matt 22:39
> =
------------------------------------------------------------------------
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_CC370DE9-49A0-4353-AF88-4867FD6C6211
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">I tried the suggestion of deleting the postmaster.pid file =
after verifying that it was not locked by any process. <div =
class=3D""><br class=3D""></div><div
class=3D"">Then I restarted =
postgres which succeeded and then restarted the engine VM.</div><div =
class=3D""><br class=3D""></div><div
class=3D"">The vm came up normally =
and passed the liveliness test.</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""></div><div =
class=3D"">Gary</div><div class=3D""><br
class=3D""></div><div =
class=3D""><br class=3D""><div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div =
class=3D"">On Jan 7, 2017, at 12:01 PM, Gary Pedretty <<a =
href=3D"mailto:gary@ravnalaska.net"
class=3D"">gary(a)ravnalaska.net</a>&gt;=
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><meta=
http-equiv=3D"Content-Type" content=3D"text/html charset=3Dutf-8" =
class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space;" class=3D"">After more =
testing, I can get the engine vm to run manually with ha-agent stopped. =
It still fails the liveliness check. Here is the repeating =
error in the engine.log from the engine vm<div class=3D""><br =
class=3D""></div><div class=3D"">ERROR =
[org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool =E2=80=94 =
46) [] Error in getting DB connection, database is inaccessible: Error =
retrieving database metadata; nested exception is =
org.springframework.jdbc.support.MetaDataAccessException: Could not get =
Connection for extracting meta data; nested exception is =
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get =
JDBC Connection; nested exception is java.sql.SQLException: =
javax.resource.ResourceException: IJ000453: Unable to get managed =
connection for java:/ENGINEDataSource</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""></div><div =
class=3D"">Found a suggestion about trying a test connection to the =
database and get the following.</div><div class=3D""><br =
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-engine =
~]# psql --host localhost --port 5432 --user engine --password =
engine</div><div class=3D"">Password for user
engine:</div><div =
class=3D"">psql: could not connect to server: Connection =
refused</div><div class=3D"">
Is the server =
running on host "localhost" (::1) and accepting</div><div =
class=3D""> TCP/IP connections on
port =
5432?</div><div class=3D"">could not connect to server: Connection
=
refused</div><div class=3D"">
Is the server =
running on host "localhost" (127.0.0.1) and accepting</div><div =
class=3D""> TCP/IP connections on
port =
5432?</div><div class=3D"">[root@fai-kvm-engine
~]#</div></div><div =
class=3D""><br class=3D""></div><div
class=3D"">and checking that =
postgresql is running shows </div><div class=3D""><br =
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-engine =
~]# systemctl status postgresql</div><div
class=3D"">=C3=A2=E2=80=94=7F =
postgresql.service - PostgreSQL database server</div><div =
class=3D""> Loaded: loaded =
(/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: =
disabled)</div><div class=3D""> Active: failed
(Result: =
exit-code) since Sat 2017-01-07 20:58:04 UTC; 2s ago</div><div =
class=3D""> Process: 23867 ExecStart=3D/usr/bin/pg_ctl start -D =
${PGDATA} -s -o -p ${PGPORT} -w -t 300 (code=3Dexited, =
status=3D1/FAILURE)</div><div class=3D""> Process: 23860
=
ExecStartPre=3D/usr/bin/postgresql-check-db-dir ${PGDATA} (code=3Dexited, =
status=3D0/SUCCESS)</div><div class=3D""><br
class=3D""></div><div =
class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Starting =
PostgreSQL database server...</div><div class=3D"">Jan 07 20:58:03
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: pg_ctl: =
another server might be running; trying to start server anyway</div><div =
class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: FATAL: =
pre-existing shared memory block (key 5432001, ID 0) is still in =
use</div><div class=3D"">Jan 07 20:58:03 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: HINT: =
If you're sure there are no old server processes still running, =
remove the shared memory block or just delete the file =
"postmaster.pid".</div><div class=3D"">Jan 07 20:58:04
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: pg_ctl: =
could not start server</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> pg_ctl[23867]: Examine the
=
log output.</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: =
postgresql.service: control process exited, code=3Dexited =
status=3D1</div><div class=3D"">Jan 07 20:58:04 <a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Failed to
start =
PostgreSQL database server.</div><div class=3D"">Jan 07 20:58:04
<a =
href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: Unit =
postgresql.service entered failed state.</div><div class=3D"">Jan 07
=
20:58:04 <a href=3D"http://fai-kvm-engine.ravnalaska.net/" =
class=3D"">fai-kvm-engine.ravnalaska.net</a> systemd[1]: =
postgresql.service failed.</div></div><div class=3D""><br
=
class=3D""></div><div class=3D""><br
class=3D""><div class=3D"">
<div style=3D"letter-spacing: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"letter-spacing:
normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px; letter-spacing: normal; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com/" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really loving the =
record green up date! Summmer!! yourself=E2=80=9D Matt =
22:39</span></font><div style=3D"letter-spacing: normal;
text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-family: Menlo;" class=3D""></div><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""></font><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><font face=3D"Menlo"
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"letter-spacing:
normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""><br =
class=3D""></font></div></div><span
style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div class=3D""><blockquote
type=3D"cite" class=3D""><div =
class=3D"">On Jan 6, 2017, at 10:05 PM, Gary Pedretty <<a =
href=3D"mailto:gary@ravnalaska.net"
class=3D"">gary(a)ravnalaska.net</a>&gt;=
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><meta=
http-equiv=3D"Content-Type" content=3D"text/html charset=3Dutf-8" =
class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space;" class=3D"">I just tried
=
running the most recent update that came out for Ovirt Hosted Engine =
setup. This is a glusterized setup with 4 hosts setup as per the =
Documentation. Each host had the icon showing upgrades available =
so I started clicking upgrade for each host doing them one at time. 3 of =
them went into maintenance fine, migrated their VMs and then activated =
when done. The last one to do was the one actually running the =
hosted-engine VM. This host also had one other VM. It =
migrated the other VM just fine, but stalled trying to migrate the =
hosted-engine. Eventually the web interface became non-responsive =
and I moved to command line. I have tried everything to get the =
engine to start up. I have tried putting all the hosts in =
maintenance so that I could selectively tell it to start the =
hosted-engine VM on one host specifically, but each fails with the =
=E2=80=9CFailed Liveliness error. I have tried restarting the ha =
agents, re-initializing the lockspace all with no success. All my =
VMs are still running fine, I just cannot get the hosted-engine to run. =
I can if necessary shutdown all the VMs and restart all the =
hosts, but was trying to avoid that unless I have to.<div
class=3D""><br =
class=3D""></div><div
class=3D"">Ideas.</div><div class=3D""><br =
class=3D""></div><div
class=3D"">Gary</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""><div class=3D"">
<div style=3D"letter-spacing: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"letter-spacing:
normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px; letter-spacing: normal; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com/" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really loving the =
record green up date! Summmer!! yourself=E2=80=9D Matt =
22:39</span></font><div style=3D"letter-spacing: normal;
text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-family: Menlo;" class=3D""></div><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""></font><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><font face=3D"Menlo"
class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"letter-spacing:
normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><font
face=3D"Menlo" =
style=3D"font-size: 12px;" class=3D""><br =
class=3D""></font></div></div><span
style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br =
class=3D""></div></div>_______________________________________________<br
=
class=3D"">Users mailing list<br class=3D""><a =
href=3D"mailto:Users@ovirt.org"
class=3D"">Users(a)ovirt.org</a><br =
class=3D""><a
href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<... =
class=3D""></div></blockquote></div><br =
class=3D""></div></div>_______________________________________________<br
=
class=3D"">Users mailing list<br class=3D""><a =
href=3D"mailto:Users@ovirt.org"
class=3D"">Users(a)ovirt.org</a><br =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br =
class=3D""></div></blockquote></div><br
class=3D""></div></body></html>=
--Apple-Mail=_CC370DE9-49A0-4353-AF88-4867FD6C6211--
2877
days inactive
2877
days old
2 comments
1 participants
participants (1)
-
Gary Pedretty