<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jun 23, 2016 at 6:36 PM, Stefano Danzi <span dir="ltr"><<a href="mailto:s.danzi@hawai.it" target="_blank">s.danzi@hawai.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
Hi!<br>
I've just upgrade oVirt from 3.6 to 4.0 and I'm not able to start the self hosted engine.<br></blockquote><div><br></div><div>Hi Stefano, can you please try "yum clean metadata" "yum update" again?</div><div>You should get vdsm 4.18.4.1, please let us know if this solve your issue.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
first thing is that the host network lose the degaut gateway configuration. But this is not the problem.<br>
<br>
Logs:<br>
<br>
==> /var/log/ovirt-hosted-engine-ha/agent.log <==<br>
MainThread::INFO::2016-06-23 18:28:40,833::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Reloading vm.conf from the shared storage domain<br>
MainThread::INFO::2016-06-23 18:28:40,833::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Trying to get a fresher copy of vm configuration from the OVF_STORE<br>
MainThread::INFO::2016-06-23 18:28:44,535::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:8d07965c-a5c4-4057-912d-901f80cf246c, volUUID:ce3aa63e-e1c4-498e-bdca-9d2e9f47f0f9<br>
MainThread::INFO::2016-06-23 18:28:44,582::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a, volUUID:3c477b06-063e-4f01-bd05-84c7d467742b<br>
MainThread::INFO::2016-06-23 18:28:44,674::ovf_store::111::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE<br>
MainThread::INFO::2016-06-23 18:28:44,675::ovf_store::118::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/ovirt01.hawai.lan:_engine/46f55a31-f35f-465c-b3e2-df45c05e06a7/images/bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a/3c477b06-063e-4f01-bd05-84c7d467742b<br>
MainThread::INFO::2016-06-23 18:28:44,682::config::226::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Found an OVF for HE VM, trying to convert<br>
MainThread::INFO::2016-06-23 18:28:44,684::config::231::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Got vm.conf from OVF_STORE<br>
MainThread::INFO::2016-06-23 18:28:44,684::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Initializing ha-broker connection<br>
MainThread::INFO::2016-06-23 18:28:44,685::brokerlink::129::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor ping, options {'addr': '192.168.1.254'}<br>
<br>
==> /var/log/ovirt-hosted-engine-ha/broker.log <==<br>
Thread-25::ERROR::2016-06-23 18:28:44,697::listener::182::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle) Error while serving connection<br>
Traceback (most recent call last):<br>
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle<br>
data)<br>
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 299, in _dispatch<br>
.set_storage_domain(client, sd_type, **options)<br>
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 66, in set_storage_domain<br>
self._backends[client].connect()<br>
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 400, in connect<br>
volUUID=volume.volume_uuid<br>
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 245, in _get_volume_path<br>
volUUID<br>
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__<br>
return self.__send(self.__name, args)<br>
File "/usr/lib64/python2.7/xmlrpclib.py", line 1587, in __request<br>
verbose=self.__verbose<br>
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request<br>
return self.single_request(host, handler, request_body, verbose)<br>
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request<br>
self.send_content(h, request_body)<br>
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content<br>
connection.endheaders(request_body)<br>
File "/usr/lib64/python2.7/httplib.py", line 975, in endheaders<br>
self._send_output(message_body)<br>
File "/usr/lib64/python2.7/httplib.py", line 835, in _send_output<br>
self.send(msg)<br>
File "/usr/lib64/python2.7/httplib.py", line 797, in send<br>
self.connect()<br>
File "/usr/lib/python2.7/site-packages/vdsm/m2cutils.py", line 203, in connect<br>
sock = socket.create_connection((self.host, self.port), self.timeout)<br>
File "/usr/lib64/python2.7/socket.py", line 571, in create_connection<br>
raise err<br>
error: [Errno 101] Network is unreachable<br>
<br>
==> /var/log/ovirt-hosted-engine-ha/agent.log <==<br>
MainThread::INFO::2016-06-23 18:28:44,697::hosted_engine::602::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed set the storage domain: 'Failed to set storage domain VdsmBackend, options {'hosted-engine.lockspace': '7B22696D6167655F75756964223A202265663131373139322D623564662D346534362D383939622D626666333362663862663135222C202270617468223A206E756C6C2C2022766F6C756D655F75756964223A202230613363393433652D633032392D343134372D623364342D396366353364663161356262227D', 'sp_uuid': '00000000-0000-0000-0000-000000000000', 'dom_type': 'nfs3', 'hosted-engine.metadata': '7B22696D6167655F75756964223A202233353066396361302D363933302D346336342D626438632D613262656631656336383934222C202270617468223A206E756C6C2C2022766F6C756D655F75756964223A202235393335373437392D656664612D343563312D386363392D656131653739633231646561227D', 'sd_uuid': '46f55a31-f35f-465c-b3e2-df45c05e06a7'}: Request failed: <class 'socket.error'>'. Waiting '5's before the next attempt<br>
<br>
==> /var/log/vdsm/vdsm.log <==<br>
periodic/3::WARNING::2016-06-23 18:34:41,731::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/2::WARNING::2016-06-23 18:34:43,657::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.BlockjobMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/0::WARNING::2016-06-23 18:34:43,732::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/3::WARNING::2016-06-23 18:34:45,732::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/2::WARNING::2016-06-23 18:34:47,723::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/1::WARNING::2016-06-23 18:34:49,724::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/0::WARNING::2016-06-23 18:34:51,725::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/3::WARNING::2016-06-23 18:34:53,726::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/2::WARNING::2016-06-23 18:34:55,727::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
<br>
==> /var/log/vdsm/mom.log <==<br>
2016-06-23 18:34:56,376 - mom.vdsmInterface - ERROR - Cannot connect to VDSM! [Errno 101] Network is unreachable<br>
<br>
==> /var/log/vdsm/vdsm.log <==<br>
periodic/1::WARNING::2016-06-23 18:34:57,727::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/2::WARNING::2016-06-23 18:34:58,652::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.BlockjobMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
periodic/2::WARNING::2016-06-23 18:34:59,728::periodic::269::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['b66ae2c5-de0f-4361-953b-f10226da7eb8']<br>
Reactor thread::INFO::2016-06-23 18:35:00,048::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:<a href="http://192.168.1.50:60604" rel="noreferrer" target="_blank">192.168.1.50:60604</a><br>
Reactor thread::DEBUG::2016-06-23 18:35:00,057::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11<br>
Reactor thread::INFO::2016-06-23 18:35:00,058::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol stomp from ::ffff:<a href="http://192.168.1.50:60604" rel="noreferrer" target="_blank">192.168.1.50:60604</a><br>
Reactor thread::INFO::2016-06-23 18:35:00,058::stompreactor::101::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request<br>
Reactor thread::DEBUG::2016-06-23 18:35:00,059::stompreactor::483::protocoldetector.StompDetector::(handle_socket) Stomp detected from ('::ffff:192.168.1.50', 60604)<br>
JsonRpc (StompReactor)::INFO::2016-06-23 18:35:00,059::stompreactor::128::Broker.StompAdapter::(_cmd_subscribe) Subscribe command received<br>
jsonrpc.Executor/2::DEBUG::2016-06-23 18:35:01,688::__init__::522::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {}<br>
<br>
<br>
[root@ovirt01 ~]# netstat -lnp<br>
Active Internet connections (only servers)<br>
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name<br>
tcp 0 0 <a href="http://0.0.0.0:111" rel="noreferrer" target="_blank">0.0.0.0:111</a> 0.0.0.0:* LISTEN 985/rpcbind<br>
tcp 0 0 <a href="http://0.0.0.0:882" rel="noreferrer" target="_blank">0.0.0.0:882</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:54322" rel="noreferrer" target="_blank">0.0.0.0:54322</a> 0.0.0.0:* LISTEN 925/python<br>
tcp 0 0 <a href="http://0.0.0.0:22" rel="noreferrer" target="_blank">0.0.0.0:22</a> 0.0.0.0:* LISTEN 1600/sshd<br>
tcp 0 0 <a href="http://0.0.0.0:49152" rel="noreferrer" target="_blank">0.0.0.0:49152</a> 0.0.0.0:* LISTEN 1978/glusterfsd<br>
tcp 0 0 <a href="http://0.0.0.0:2049" rel="noreferrer" target="_blank">0.0.0.0:2049</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:38465" rel="noreferrer" target="_blank">0.0.0.0:38465</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:49153" rel="noreferrer" target="_blank">0.0.0.0:49153</a> 0.0.0.0:* LISTEN 1963/glusterfsd<br>
tcp 0 0 <a href="http://0.0.0.0:32802" rel="noreferrer" target="_blank">0.0.0.0:32802</a> 0.0.0.0:* LISTEN 2193/rpc.statd<br>
tcp 0 0 <a href="http://0.0.0.0:38466" rel="noreferrer" target="_blank">0.0.0.0:38466</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:16514" rel="noreferrer" target="_blank">0.0.0.0:16514</a> 0.0.0.0:* LISTEN 1633/libvirtd<br>
tcp 0 0 <a href="http://0.0.0.0:38468" rel="noreferrer" target="_blank">0.0.0.0:38468</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:38469" rel="noreferrer" target="_blank">0.0.0.0:38469</a> 0.0.0.0:* LISTEN 1970/glusterfs<br>
tcp 0 0 <a href="http://0.0.0.0:24007" rel="noreferrer" target="_blank">0.0.0.0:24007</a> 0.0.0.0:* LISTEN 1652/glusterd<br>
tcp6 0 0 :::54321 :::* LISTEN 1921/python<br>
tcp6 0 0 :::22 :::* LISTEN 1600/sshd<br>
tcp6 0 0 :::16514 :::* LISTEN 1633/libvirtd<br>
udp 0 0 <a href="http://0.0.0.0:40503" rel="noreferrer" target="_blank">0.0.0.0:40503</a> 0.0.0.0:* 937/avahi-daemon: r<br>
udp 0 0 <a href="http://0.0.0.0:40873" rel="noreferrer" target="_blank">0.0.0.0:40873</a> 0.0.0.0:* 2193/rpc.statd<br>
udp 0 0 <a href="http://0.0.0.0:111" rel="noreferrer" target="_blank">0.0.0.0:111</a> 0.0.0.0:* 985/rpcbind<br>
udp 0 0 <a href="http://127.0.0.1:323" rel="noreferrer" target="_blank">127.0.0.1:323</a> 0.0.0.0:* 952/chronyd<br>
udp 0 0 <a href="http://0.0.0.0:694" rel="noreferrer" target="_blank">0.0.0.0:694</a> 0.0.0.0:* 985/rpcbind<br>
udp 0 0 <a href="http://127.0.0.1:703" rel="noreferrer" target="_blank">127.0.0.1:703</a> 0.0.0.0:* 2193/rpc.statd<br>
udp 0 0 <a href="http://0.0.0.0:879" rel="noreferrer" target="_blank">0.0.0.0:879</a> 0.0.0.0:* 1970/glusterfs<br>
udp 0 0 <a href="http://0.0.0.0:5353" rel="noreferrer" target="_blank">0.0.0.0:5353</a> 0.0.0.0:* 937/avahi-daemon: r<br>
udp6 0 0 ::1:323 :::* 952/chronyd<br>
Active UNIX domain sockets (only servers)<br>
Proto RefCnt Flags Type State I-Node PID/Program name Path<br>
unix 2 [ ACC ] STREAM LISTENING 19455 1633/libvirtd /var/run/libvirt/libvirt-sock<br>
unix 2 [ ACC ] STREAM LISTENING 22529 1633/libvirtd /var/run/libvirt/libvirt-sock-ro<br>
unix 2 [ ACC ] STREAM LISTENING 16902 1/systemd /var/run/rpcbind.sock<br>
unix 2 [ ACC ] STREAM LISTENING 24077 1978/glusterfsd /var/run/gluster/changelog-36d1d6bf77f0ad218a0db9ada4237737.sock<br>
unix 2 [ ACC ] STREAM LISTENING 20494 992/gssproxy /run/gssproxy.sock<br>
unix 2 [ ACC ] STREAM LISTENING 14641 682/multipathd @/org/kernel/linux/storage/multipathd<br>
unix 2 [ ACC ] STREAM LISTENING 16914 1/systemd /var/run/avahi-daemon/socket<br>
unix 2 [ ACC ] STREAM LISTENING 19484 993/sanlock /var/run/sanlock/sanlock.sock<br>
unix 2 [ ACC ] STREAM LISTENING 24095 1963/glusterfsd /var/run/gluster/changelog-c54f38f29c5c1e84d2766564ea967b8d.sock<br>
unix 2 [ ACC ] STREAM LISTENING 16929 1/systemd /var/run/dbus/system_bus_socket<br>
unix 2 [ ACC ] STREAM LISTENING 12093 1/systemd /run/systemd/private<br>
unix 2 [ ACC ] STREAM LISTENING 16924 1/systemd @ISCSID_UIP_ABSTRACT_NAMESPACE<br>
unix 2 [ ACC ] STREAM LISTENING 18255 925/python /var/run/vdsm/ovirt-imageio-daemon.sock<br>
unix 2 [ ACC ] STREAM LISTENING 23905 1963/glusterfsd /var/run/gluster/b3162364606c81381a4c768484c6f091.socket<br>
unix 2 [ ACC ] SEQPACKET LISTENING 13409 1/systemd /run/udev/control<br>
unix 2 [ ACC ] STREAM LISTENING 90477 1970/glusterfs /var/run/gluster/b6606e2cc1634ac3c57dae961f6a6cc3.socket<br>
unix 2 [ ACC ] STREAM LISTENING 13428 1/systemd /run/lvm/lvmetad.socket<br>
unix 2 [ ACC ] STREAM LISTENING 18302 920/python /var/run/ovirt-hosted-engine-ha/broker.socket<br>
unix 2 [ ACC ] STREAM LISTENING 23947 1978/glusterfsd /var/run/gluster/65f923a8c0294836aec3ffce760511cb.socket<br>
unix 2 [ ACC ] STREAM LISTENING 20112 1652/glusterd /var/run/glusterd.socket<br>
unix 2 [ ACC ] STREAM LISTENING 18321 939/python /var/run/vdsm/svdsm.sock<br>
unix 2 [ ACC ] STREAM LISTENING 20493 992/gssproxy /var/lib/gssproxy/default.sock<br>
unix 2 [ ACC ] STREAM LISTENING 13470 1/systemd /run/lvm/lvmpolld.socket<br>
unix 2 [ ACC ] STREAM LISTENING 16909 1/systemd @ISCSIADM_ABSTRACT_NAMESPACE<br>
unix 2 [ ACC ] STREAM LISTENING 18160 990/wdmd /var/run/wdmd/wdmd.sock<br>
unix 2 [ ACC ] STREAM LISTENING 2291 1/systemd /run/systemd/journal/stdout<br>
unix 2 [ ACC ] STREAM LISTENING 90622 1923/python /var/run/vdsm/mom-vdsm.sock<br>
<br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Sandro Bonazzola<br>Better technology. Faster innovation. Powered by community collaboration.<br>See how it works at <a href="http://redhat.com" target="_blank">redhat.com</a><br></div></div></div></div>
</div></div>