1:45 p.m.
I have a hardware node that I recently upgraded from 3.6.4 to 4.1.0.4-1.
It is having issues that certain VMs I cannot deploy to it because ovirt
says host is not available.
However that host is status up and has 14 running VMs on it.
How do I tell why ovirt thinks the host is not available?
Attached is output from "vdsClient -s ovirt4.test.j2noc.com getVdsStats"
vdsm-4.19.4-1.el7.centos.x86_64
ovirt-engine-4.1.0.4-1.el7.centos.noarch
engine.log:
2017-03-07 11:06:49,269-08 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJo
inPool-1-worker-0) [] EVENT_ID: USER_RUN_VM_ON_NON_DEFAULT_VDS(152),
Correlation ID: 81490f5b-3029-44d4-b223-df
6bb506efb3, Job ID: 76900dc9-ed9a-4fda-a17a-31825aa515c9, Call Stack:
null, Custom Event ID: -1, Message: Guest
billj4 started on Host ovirt7.test.j2noc.com. (Default Host parameter
was ignored - assigned Host was not available).
1:21 a.m.
We have tried re-kickstarting the server, and removing it from ovirt and
reinstalling it.
Ovirt still thinks its not available to run VMs.
I even tried calling it by a different name, still no luck.
Starting a VM using "run-once" will deploy to the host just fine.
WHY?
I thought when host is in status UP its available.
How do I find out why ovirt thinks its not available for VMs even though
it marked it as up?
On 3/7/17 11:45 AM, Bill James wrote:
1:57 a.m.
only thing different on the VM config is that I says Deploy to host
ovirt4.test, which ovirt then ignores because "host is not available".
I attached logs from ovirt engine and ovirt4.test.
VM I tried to start is called "billj4-2".
I also tried to migrate another VM that is suppose to run on ovirt4
called "billj4".
This time it successfully migrated and is running on ovrit4.test.
Thank you for your help!
[root@ovirt4 test vdsm]# virsh -r list --all
Id Name State
----------------------------------------------------
2 billj4 running
On 3/12/17 6:16 AM, Yaniv Kaul wrote:
4:41 a.m.
--------------A47812010390970F420D5690
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
I'm not sure if its related or not but I noticed 3 other hosts are doing
similar things.
They all have in common that they are earlier hardware nodes from the
other 3 nodes in the cluster.
All are HP DL360's, but that later 3 have 256 GB ram, older ones have
128GB ram.
The other 3 nodes are still running ovirt 3.6.4.
On 3/12/17 11:57 PM, Bill James wrote:
--------------A47812010390970F420D5690
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I'm not sure if its related or not but I noticed 3 other hosts are
doing similar things.<br>
They all have in common that they are earlier hardware nodes from
the other 3 nodes in the cluster.<br>
All are HP DL360's, but that later 3 have 256 GB ram, older ones
have 128GB ram.<br>
The other 3 nodes are still running ovirt 3.6.4.<br>
<br>
<br>
<div class="moz-cite-prefix">On 3/12/17 11:57 PM, Bill James
wrote:<br>
</div>
<blockquote cite="mid:663b7ab1-82b0-d000-149f-50ef1959126b@j2.com"
type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
only thing different on the VM config is that I says Deploy to
host ovirt4.test, which ovirt then ignores because "host is not
available".<br>
<br>
I attached logs from ovirt engine and ovirt4.test.<br>
VM I tried to start is called "billj4-2".<br>
I also tried to migrate another VM that is suppose to run on
ovirt4 called "billj4".<br>
This time it successfully migrated and is running on ovrit4.test.
<br>
<br>
Thank you for your help!<br>
<br>
<br>
[root@ovirt4 test vdsm]# virsh -r list --all<br>
Id Name State<br>
----------------------------------------------------<br>
2 billj4 running<br>
<br>
<br>
<div class="moz-cite-prefix">On 3/12/17 6:16 AM, Yaniv Kaul
wrote:<br>
</div>
<blockquote
cite="mid:CAJgorsYsnueuqrkTZHwJVCyuTquxuFOKfuDoGsg2T64N0BOEvQ@mail.gmail.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html; ">
<div dir="ltr"><br>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Mar 7, 2017 at 9:50 PM Bill James
<<a moz-do-not-send="true"
href="mailto:bill.james@j2.com">bill.james@j2.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote">I have a hardware node that
I recently upgraded from 3.6.4 to 4.1.0.4-1.<br
class="gmail_msg">
It is having issues that certain VMs I cannot deploy to it
because ovirt<br class="gmail_msg">
says host is not available.<br class="gmail_msg">
However that host is status up and has 14 running VMs on
it.<br class="gmail_msg">
<br class="gmail_msg">
How do I tell why ovirt thinks the host is not available?<br
class="gmail_msg">
</blockquote>
<div><br>
</div>
<div>Anything different on the VM's configuration?</div>
<div>If some VMs do run on it, look like the issue is with
the VM configuration. Can you provide full logs?</div>
<div>Y.</div>
<div> </div>
<blockquote class="gmail_quote"> <br
class="gmail_msg">
Attached is output from "vdsClient -s <a
moz-do-not-send="true"
href="http://ovirt4.test.j2noc.com" rel="noreferrer"
class="gmail_msg"
target="_blank">ovirt4.test.j2noc.com</a>
getVdsStats"<br class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
vdsm-4.19.4-1.el7.centos.x86_64<br class="gmail_msg">
ovirt-engine-4.1.0.4-1.el7.centos.noarch<br
class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
engine.log:<br class="gmail_msg">
2017-03-07 11:06:49,269-08 WARN<br class="gmail_msg">
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br
class="gmail_msg">
(ForkJo<br class="gmail_msg">
inPool-1-worker-0) [] EVENT_ID:
USER_RUN_VM_ON_NON_DEFAULT_VDS(152),<br class="gmail_msg">
Correlation ID: 81490f5b-3029-44d4-b223-df<br
class="gmail_msg">
6bb506efb3, Job ID: 76900dc9-ed9a-4fda-a17a-31825aa515c9,
Call Stack:<br class="gmail_msg">
null, Custom Event ID: -1, Message: Guest<br
class="gmail_msg">
billj4 started on Host <a moz-do-not-send="true"
href="http://ovirt7.test.j2noc.com" rel="noreferrer"
class="gmail_msg"
target="_blank">ovirt7.test.j2noc.com</a>.
(Default Host parameter<br class="gmail_msg">
was ignored - assigned Host was not available).<br
class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br
class="gmail_msg">
Users mailing list<br class="gmail_msg">
<a moz-do-not-send="true"
href="mailto:Users@ovirt.org"
class="gmail_msg"
target="_blank">Users(a)ovirt.org</a><br
class="gmail_msg">
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer" class="gmail_msg"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
class="gmail_msg">
</blockquote>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>
--------------A47812010390970F420D5690--
1:29 p.m.
--------------2F5D52418C6C993367FE08C5
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
We found if we move the rebuilt host to a new cluster setup for 4.1
compatibility it could startup VMs just fine.
So we are guessing its just a 3.6 compat mode issue.
I'll try upgrading the rest of the nodes to 4.1 and change compat mode.
Hopefully all will just work then.
We'll see this weekend....
Thanks.
On 03/14/2017 02:41 AM, Bill James wrote:
--------------2F5D52418C6C993367FE08C5
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
We found if we move the rebuilt host to a new cluster setup for 4.1
compatibility it could startup VMs just fine.<br>
So we are guessing its just a 3.6 compat mode issue.<br>
I'll try upgrading the rest of the nodes to 4.1 and change compat
mode.<br>
Hopefully all will just work then.<br>
We'll see this weekend....<br>
<br>
Thanks.<br>
<br>
<br>
<div class="moz-cite-prefix">On 03/14/2017 02:41 AM, Bill James
wrote:<br>
</div>
<blockquote cite="mid:f6be257b-dacc-f2cc-5aaf-464bc6ab9487@j2.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
I'm not sure if its related or not but I noticed 3 other hosts are
doing similar things.<br>
They all have in common that they are earlier hardware nodes from
the other 3 nodes in the cluster.<br>
All are HP DL360's, but that later 3 have 256 GB ram, older ones
have 128GB ram.<br>
The other 3 nodes are still running ovirt 3.6.4.<br>
<br>
<br>
<div class="moz-cite-prefix">On 3/12/17 11:57 PM, Bill James
wrote:<br>
</div>
<blockquote cite="mid:663b7ab1-82b0-d000-149f-50ef1959126b@j2.com"
type="cite"> only thing different on the VM config is that I
says Deploy to host ovirt4.test, which ovirt then ignores
because "host is not available".<br>
<br>
I attached logs from ovirt engine and ovirt4.test.<br>
VM I tried to start is called "billj4-2".<br>
I also tried to migrate another VM that is suppose to run on
ovirt4 called "billj4".<br>
This time it successfully migrated and is running on
ovrit4.test. <br>
<br>
Thank you for your help!<br>
<br>
<br>
[root@ovirt4 test vdsm]# virsh -r list --all<br>
Id Name State<br>
----------------------------------------------------<br>
2 billj4 running<br>
<br>
<br>
<div class="moz-cite-prefix">On 3/12/17 6:16 AM, Yaniv Kaul
wrote:<br>
</div>
<blockquote
cite="mid:CAJgorsYsnueuqrkTZHwJVCyuTquxuFOKfuDoGsg2T64N0BOEvQ@mail.gmail.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html;
">
<div dir="ltr"><br>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Mar 7, 2017 at 9:50 PM Bill James
<<a moz-do-not-send="true"
href="mailto:bill.james@j2.com">bill.james@j2.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote">I have a hardware node
that I recently upgraded from 3.6.4 to 4.1.0.4-1.<br
class="gmail_msg">
It is having issues that certain VMs I cannot deploy to
it because ovirt<br class="gmail_msg">
says host is not available.<br class="gmail_msg">
However that host is status up and has 14 running VMs on
it.<br class="gmail_msg">
<br class="gmail_msg">
How do I tell why ovirt thinks the host is not
available?<br class="gmail_msg">
</blockquote>
<div><br>
</div>
<div>Anything different on the VM's configuration?</div>
<div>If some VMs do run on it, look like the issue is with
the VM configuration. Can you provide full logs?</div>
<div>Y.</div>
<div> </div>
<blockquote class="gmail_quote"> <br
class="gmail_msg">
Attached is output from "vdsClient -s <a
moz-do-not-send="true"
href="http://ovirt4.test.j2noc.com"
rel="noreferrer"
class="gmail_msg"
target="_blank">ovirt4.test.j2noc.com</a>
getVdsStats"<br class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
vdsm-4.19.4-1.el7.centos.x86_64<br class="gmail_msg">
ovirt-engine-4.1.0.4-1.el7.centos.noarch<br
class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
engine.log:<br class="gmail_msg">
2017-03-07 11:06:49,269-08 WARN<br class="gmail_msg">
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br
class="gmail_msg">
(ForkJo<br class="gmail_msg">
inPool-1-worker-0) [] EVENT_ID:
USER_RUN_VM_ON_NON_DEFAULT_VDS(152),<br
class="gmail_msg">
Correlation ID: 81490f5b-3029-44d4-b223-df<br
class="gmail_msg">
6bb506efb3, Job ID:
76900dc9-ed9a-4fda-a17a-31825aa515c9, Call Stack:<br
class="gmail_msg">
null, Custom Event ID: -1, Message: Guest<br
class="gmail_msg">
billj4 started on Host <a moz-do-not-send="true"
href="http://ovirt7.test.j2noc.com"
rel="noreferrer"
class="gmail_msg"
target="_blank">ovirt7.test.j2noc.com</a>.
(Default Host parameter<br class="gmail_msg">
was ignored - assigned Host was not available).<br
class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br
class="gmail_msg">
Users mailing list<br class="gmail_msg">
<a moz-do-not-send="true"
href="mailto:Users@ovirt.org"
class="gmail_msg"
target="_blank">Users(a)ovirt.org</a><br
class="gmail_msg">
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer" class="gmail_msg"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
class="gmail_msg">
</blockquote>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>
--------------2F5D52418C6C993367FE08C5--
6:09 p.m.
well, upgrading to ovirt 4.1.0.4-1 did not fix the problem.
This cluster has 7 hardware nodes plus a management server, all ruining
4.1.0.4-1.
For some reason ovirt thinks 5 out of the 7 nodes are not available,
even though they have VMs running on them.
The 2 nodes that do deploy VM fine I just rebooted today. All nodes were
rebooted 2 days ago for the 4.1 upgrade.
My test was to take the same VM and specify host to deploy to, for each
of the 7 nodes.
hostId='e8c7567b-7ccb-4d82-8aea-beb1cbf05bf6' = ovirt2 (ok)
hostId='b6db5aa3-55f5-4424-9222-d69199ffd59d' = ovirt7 (ok)
the rest all fail:
hostId='4b3c7000-fe36-4b5a-97ab-b3cf28a238a0' = ovirt1
hostId='6ed53fd2-f552-4dd2-b42c-c0e8401b6aa8' = ovirt3
hostId='64a663bb-148b-4d3f-8e9c-3db27bc7eee7' = ovirt4
hostId='d15998e3-7801-4a66-9f4e-6978c747d6bb' = ovirt5
hostId='70213e88-5c85-4931-8781-64841e7a6a19' = ovirt6
2017-03-20 15:56:05,761-07 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQu
artzScheduler1) [519095c3-83ed-4478-8dd4-f432db6db140] EVENT_ID:
USER_RUN_VM_ON_NON_DEFAULT_VDS(152), Correlation
ID: 11ec1687-30f1-4543-b5a3-4760a19071ae, Job ID:
4cca04b8-e7e5-45b3-853f-d69cb56d8e7c, Call Stack: null, Custom E
vent ID: -1, Message: Guest billj4 started on Host
ovirt7.test.j2noc.com. (Default Host parameter was ignored - as
signed Host was not available).
How do I determine why ovirt thinks the host is not available?
attached is engine log and picture of host tab, and snippets of engine
log where the error occurs.
libvirt-client-2.0.0-10.el7_3.5.x86_64
ovirt-engine-tools-4.1.0.4-1.el7.centos.noarch
vdsm-4.19.4-1.el7.centos.x86_64
On 03/14/2017 11:29 AM, Bill James wrote:
2940
days inactive
2953
days old
6 comments
2 participants
participants (2)
-
Bill James -
Yaniv Kaul