<HTML><BODY><div class="js-helper js-readmsg-msg">
<style type="text/css"></style>
<div>
<base target="_self" href="https://e.mail.ru/">
<div id="style_15093744340000000937_BODY"><div class="class_1509379577">Thank you all!<br><br>Francesco,<br><br>[root@alpha ~]# journalctl -u libvirtd<br>-- Logs begin at Sun 2017-10-29 09:39:58 EET, end at Mon 2017-10-30 16:35:36 EET. --<br>Oct 29 09:41:24 alpha.datamax.bg systemd[1]: Starting Virtualization daemon...<br>Oct 29 09:41:26 alpha.datamax.bg systemd[1]: Started Virtualization daemon.<br>Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: libvirt version: 2.0.0, package: 10.el7_3.9 (CentOS BuildSystem <<a href="http://bugs.centos.org" target="_blank">http://bugs.centos.org</a>>, 2017-05-25-20:52:28, c1bm.rdu2.centos.org)<br>Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: hostname: alpha.datamax.bg<br><span style="color: rgb(251, 44, 44);">Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)</span><br>Oct 29 22:41:30 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:17:08 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:22:47 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:26:11 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:42:42 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:45:09 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:45:43 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:56:52 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:56:58 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 29 23:59:13 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br>Oct 30 16:34:48 alpha.datamax.bg libvirtd[4207]: XML error: Attempted double use of PCI slot 0000:00:06.0 (may need "multifunction='on'" for device on function 0)<br><br><span id="result_box" lang="en"><span>the first mistake for XML error in red matches exactly the time when we restarted it.<br><span id="result_box" lang="en"><span>Тhese errors have at all three nodes and appears when trying to start engine virtual machine.</span></span> </span></span><br><br><span id="result_box" class="short_text" lang="en"><span>Also attaching vdsm.log.</span></span><br><br>Have a nice day!<br><blockquote style="border-left:1px solid #0857A6;margin:10px;padding:0 0 0 10px;">
Понедельник, 30 октября 2017, 16:32 +02:00 от Francesco Romani <<a href="mailto:fromani@redhat.com">fromani@redhat.com</a>>:<br>
<br>
<div id="">
<div class="js-helper_mailru_css_attribute_postfix js-readmsg-msg_mailru_css_attribute_postfix">
<style></style>
<div>
<div id="style_15093739220000000466_BODY_mailru_css_attribute_postfix"><div class="class_1509406060_mailru_css_attribute_postfix">
<p>Hi,</p>
<p><br>
</p>
<p>anything in the journal about libvirt? (journalctl -u libvirtd)</p>
<p><br>
</p>
<p>could you share a bigger chunk of the vdsm log, demonstrating the
failed VM start?</p>
<p><br>
</p>
<p>Bests,<br>
</p>
<br>
<div class="moz-cite-prefix_mailru_css_attribute_postfix_mailru_css_attribute_postfix">On 10/30/2017 03:28 PM, Hristo Pavlov
wrote:<br>
</div>
<blockquote type="cite" cite="mid:1509373682.39477110@f387.i.mail.ru">
<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>I tried
it already, <span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>doesn't start on any of the nodes. <br>
<br>
<br>
In a log /var/log/libvirt/qemu/HostedEngine.log at all<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>
three nodes</span></span> <span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>has nothing to do with starting, as if
it had not started.</span></span></span></span></span></span><br>
<br>
<br>
<blockquote style="border-left:1px solid #0857A6;margin:10px;padding:0 0 0 10px;"> Понедельник, 30 октября 2017, 16:11 +02:00
от Nathanaël Blanchet <a class="moz-txt-link-rfc2396E_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="//e.mail.ru/compose/?mailto=mailto%3ablanchet@abes.fr" target="_blank" rel="noopener noreferrer noopener noreferrer"><blanchet@abes.fr></a>:<br>
<br>
<div id="">
<div class="js-helper_mailru_css_attribute_postfix_mailru_css_attribute_postfix js-readmsg-msg_mailru_css_attribute_postfix_mailru_css_attribute_postfix">
<style></style>
<div>
<div id="style_15093726620000000687_BODY_mailru_css_attribute_postfix_mailru_css_attribute_postfix">
<div class="class_1509374974_mailru_css_attribute_postfix_mailru_css_attribute_postfix">
<p>Hi,</p>
<p>It happened to me too, after a live migration of
it, I shut down the hosted engine on the targeted
host, and I couldn't restart it anymore on this
specific host. But I was able to start it on the
initial one, where I initially where I deployed the
HE. It was like the lease and the libvirt host
definition staid on the first host after migration.</p>
<p>So try you may try <span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>hosted-engine --vm-start on one of
other hosts...<br>
</span></span></p>
<br>
<div class="moz-cite-prefix_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix">Le
30/10/2017 à 12:28, Hristo Pavlov a écrit :<br>
</div>
<blockquote type="cite" cite="mid:1509362918.145880938@f267.i.mail.ru"> Hi
All,<br>
<br>
Our oVirt cluster is with 3 nodes with shared fibre
channel storage, the engine virtual machine is self
hosted.<br>
<br>
Hypervisors OS: CentOS Linux release 7.3 / x86_64,
oVirt version is <span class="st_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix"><span class="st_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix">4.1.2.2.
<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>The environment has been
working for about a year without any
problems</span></span>.<br>
<br>
After</span></span><span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span> shutdown of the hosted engine
virtual machine, it doesn't start. <br>
<br>
<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>Тhese commands that were
executed:<br>
<br>
hosted-engine --set-maintenance
--mode=global<br>
hosted-engine --vm-shutdown<br>
</span></span><br>
after the status of engine vm was down, we
executed start.<br>
<br>
[root@alpha] hosted-engine --vm-start<br>
VM exists and is down, destroying it<br>
Exception in thread Client localhost:54321 (most
likely raised during interpreter shutdown):</span></span><span class="st_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix"><br>
<br>
we noticied that at vdsm.log <br>
<br>
017-10-30 13:11:04,863+0200 INFO (jsonrpc/1)
[jsonrpc.JsonRpcServer] RPC call
StorageDomain.getStats succeeded in 0.26 seconds
(__init__:533)<br>
2017-10-30 13:11:05,802+0200 INFO (jsonrpc/6)
[jsonrpc.JsonRpcServer] RPC call
Host.getAllVmStats succeeded in 0.01 seconds
(__init__:533)<br>
2017-10-30 13:11:05,825+0200 WARN (jsonrpc/2)
[virt.vm]
(vmId='da98112d-b9fb-4098-93fa-1f1374b41e46')
Failed to get metadata, domain not connected.
(vm:2765)<br>
2017-10-30 13:11:05,825+0200 ERROR (jsonrpc/2)
[jsonrpc.JsonRpcServer] Internal server error
(__init__:570)<br>
Traceback (most recent call last):<br>
File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py",
line 565, in _handle_request<br>
res = method(**params)<br>
File
"/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py",
line 202, in _dynamicMethod<br>
result = fn(*methodArgs)<br>
File "/usr/share/vdsm/API.py", line 1454, in
getAllVmIoTunePolicies<br>
io_tune_policies_dict =
self._cif.getAllVmIoTunePolicies()<br>
File "/usr/share/vdsm/clientIF.py", line 448, in
getAllVmIoTunePolicies<br>
'current_values': v.getIoTune()}<br>
File "/usr/share/vdsm/virt/vm.py", line 2803, in
getIoTune<br>
result = self.getIoTuneResponse()<br>
File "/usr/share/vdsm/virt/vm.py", line 2816, in
getIoTuneResponse<br>
res = self._dom.blockIoTune(<br>
File
"/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py",
line 47, in __getattr__<br>
% self.vmid)<br>
NotConnectedError: VM
u'da98112d-b9fb-4098-93fa-1f1374b41e46' was not
started yet or was shut down<br>
<br>
<br>
The storage of self hosted engine multipath, pvs,
lvs, seems ok... <br>
<br>
<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>At the moment of the three nodes
there is a working about 100 virtual machines
and we can't manage them.<br>
<br>
<span id="result_box_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" class="short_text_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" lang="en"><span>Does
anyone have any ideas, what can be done</span></span>
то recover self hosted engine virtual machine?<br>
<br>
Thahk You!<br>
Have a nice day!<br>
</span></span><br>
<br>
<br>
<br>
<br>
<br>
<br>
<span></span></span><br>
<br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="//e.mail.ru/compose/?mailto=mailto%3aUsers@ovirt.org" target="_blank" moz-do-not-send="true" rel="noopener noreferrer noopener noreferrer noopener noreferrer">Users@ovirt.org</a>
<a class="moz-txt-link-freetext_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true" rel="noopener noreferrer noopener noreferrer noopener noreferrer">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" cols="72">--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
<a class="moz-txt-link-abbreviated_mailru_css_attribute_postfix_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="//e.mail.ru/compose/?mailto=mailto%3ablanchet@abes.fr" target="_blank" moz-do-not-send="true" rel="noopener noreferrer noopener noreferrer noopener noreferrer">blanchet@abes.fr</a> </pre>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader_mailru_css_attribute_postfix_mailru_css_attribute_postfix"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="//e.mail.ru/compose/?mailto=mailto%3aUsers@ovirt.org" target="_blank" rel="noopener noreferrer noopener noreferrer">Users@ovirt.org</a>
<a class="moz-txt-link-freetext_mailru_css_attribute_postfix_mailru_css_attribute_postfix" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" rel="noopener noreferrer noopener noreferrer">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature_mailru_css_attribute_postfix_mailru_css_attribute_postfix" cols="72">--
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh</pre>
</div></div>
<div>_______________________________________________<br>
Users mailing list<br>
<a href="//e.mail.ru/compose/?mailto=mailto%3aUsers@ovirt.org" target="_blank" rel=" noopener noreferrer">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" rel=" noopener noreferrer">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<br><br>
</div></div>
<base target="_self" href="https://e.mail.ru/">
</div>
</div></BODY></HTML>