12:15 p.m.
--_000_DM5PR01MB2506CA22D55C58A5210C6EA6FFCC0DM5PR01MB2506prod_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi,
The log does't indicate HV reboot, and i see lots of errors in the logs.
During the reboot, what happened to the VM inside of the HV ? migrated ? pa=
used ? what about the system's logs ? does it indicate a graceful shutdown =
?
--
Respectfully
Mahdi A. Mahdi
________________________________
From: Erekle Magradze <erekle.magradze(a)recogizer.de Sent:
Friday, February 23, 2018 2:48 PM
To: Mahdi Adnan; users(a)ovirt.org
Subject: Re: [ovirt-users] rebooting hypervisors from time to time
Thanks for the reply,
I've attached all the logs from yesterday, reboot has happened during the d=
ay but this is not the first time and this is not the only one hypervisor.
Kind Regards
Erekle
On 02/23/2018 09:00 AM, Mahdi Adnan wrote:
Hi,
Can you post the VDSM and Engine logs ?
--
Respectfully
Mahdi A. Mahdi
________________________________
From: users-bounces@ovirt.org<mailto:users-bounces@ovirt.org> <users-bounce=
s@ovirt.org><mailto:users-bounces@ovirt.org> on behalf of Erekle Magradze <=
erekle.magradze@recogizer.de><mailto:erekle.magradze@recogizer.de Sent: Thursday, February 22, 2018 11:48 PM
To: users@ovirt.org<mailto:users@ovirt.org Subject:
Re: [ovirt-users] rebooting hypervisors from time to time
Dear all,
It would be great if someone will share any experience regarding the
similar case, would be great to have a hint where to start investigation.
Thanks again
Cheers
Erekle
On 02/22/2018 05:05 PM, Erekle Magradze wrote:
recogizer.com
-----------------------------------------------------------------
Recogizer Group GmbH
Gesch=E4ftsf=FChrer: Oliver Habisch, Carsten Kreutze
Handelsregister: Amtsgericht Bonn HRB 20724
Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993
Diese E-Mail enth=E4lt vertrauliche und/oder rechtlich gesch=FCtzte Informa=
tionen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt=FC=
mlich erhalten haben, informieren Sie bitte sofort den Absender und l=F6sch=
en Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe d=
ieser Mail und der darin enthaltenen Informationen ist nicht gestattet.
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Recogizer Group GmbH
Dr.rer.nat. Erekle Magradze
Lead Big Data Engineering & DevOps
Rheinwerkallee 2, 53227 Bonn
Tel: +49 228 29974555
E-Mail erekle.magradze@recogizer.de<mailto:erekle.magradze@recogizer.de recogizer.com
-----------------------------------------------------------------
Recogizer Group GmbH
Gesch=E4ftsf=FChrer: Oliver Habisch, Carsten Kreutze
Handelsregister: Amtsgericht Bonn HRB 20724
Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993
Diese E-Mail enth=E4lt vertrauliche und/oder rechtlich gesch=FCtzte Informa=
tionen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt=FC=
mlich erhalten haben, informieren Sie bitte sofort den Absender und l=F6sch=
en Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe d=
ieser Mail und der darin enthaltenen Informationen ist nicht gestattet.
--_000_DM5PR01MB2506CA22D55C58A5210C6EA6FFCC0DM5PR01MB2506prod_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Diso-8859-=
1" <style type=3D"text/css"
style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style </head <body dir=3D"ltr" <div
style=3D"font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;=
color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);" Hi,</div <div
style=3D"font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;=
color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);" <br </div <div style=3D"font-family: Calibri, Helvetica,
sans-serif; font-size: 12pt;=
color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);" The log does't indicate HV reboot, and i see lots of errors
in the logs.</d=
iv <div style=3D"font-family: Calibri, Helvetica,
sans-serif; font-size: 12pt;=
color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);" During the reboot, what happened to the VM inside of the HV ?
migrated ? pa=
used ? what about the system's logs ? does it indicate a graceful shutdown =
?</div <div style=3D"font-family:
Calibri, Helvetica, sans-serif; font-size: 12pt;=
color: rgb(0, 0, 0);" <br
</div <div
id=3D"signature"><br <div
class=3D"ecxmoz-signature">-- <br
<br <font
color=3D"#3366ff"><font
color=3D"#000000">Respectfully<b><br
</b><b>Mahdi A. Mahdi</b></font></font><font
color=3D"#3366ff"><br
<br </font><font
color=3D"#3366ff"></font></div
</div <hr
style=3D"display:inline-block;width:98%" tabindex=3D"-1" <div id=3D"divRplyFwdMsg"
dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b>
Erekle Magradze <e=
rekle.magradze(a)recogizer.de&gt;<br
<b>Sent:</b> Friday, February 23, 2018 2:48 PM<br <b>To:</b> Mahdi Adnan; users(a)ovirt.org<br <b>Subject:</b> Re: [ovirt-users] rebooting
hypervisors from time to time</=
font <div> </div
</div <div
style=3D"background-color:#FFFFFF"
<p>Thanks for the reply,</p
<p>I've attached all the logs from yesterday, reboot has happened during
th=
e day but this is not the first time and this is not the only one hyperviso=
r.</p <p>Kind Regards</p <p>Erekle</p
<br <div
class=3D"x_moz-cite-prefix">On 02/23/2018 09:00 AM, Mahdi Adnan wrote:=
<br </div
<blockquote type=3D"cite"><style type=3D"text/css"
style=3D"display:none" <!--
p
{margin-top:0;
margin-bottom:0}
-- </style <div
style=3D"font-family:Calibri,Helvetica,sans-serif; font-size:12pt; col=
or:rgb(0,0,0); background-color:rgba(0,0,0,0)"
Hi,</div <div
style=3D"font-family:Calibri,Helvetica,sans-serif; font-size:12pt; col=
or:rgb(0,0,0); background-color:rgba(0,0,0,0)"
<br </div <div style=3D"font-family:Calibri,Helvetica,sans-serif;
font-size:12pt; col=
or:rgb(0,0,0); background-color:rgba(0,0,0,0)" Can you
post the VDSM and Engine logs ?</div <div
style=3D"font-family:Calibri,Helvetica,sans-serif; font-size:12pt; col=
or:rgb(0,0,0)" <br
</div <div
id=3D"x_signature"><br <div
class=3D"x_ecxmoz-signature">-- <br
<br <font
color=3D"#3366ff"><font
color=3D"#000000">Respectfully<b><br
</b><b>Mahdi A. Mahdi</b></font></font><font
color=3D"#3366ff"><br
<br </font></div </div <hr
tabindex=3D"-1" style=3D"display:inline-block; width:98%" <div id=3D"x_divRplyFwdMsg"
dir=3D"ltr"><font face=3D"Calibri, sans-serif" =
color=3D"#000000" style=3D"font-size:11pt"><b>From:</b <a class=3D"x_moz-txt-link-abbreviated"
href=3D"mailto:users-bounces@ovirt.=
org">users-bounces(a)ovirt.org</a <a
class=3D"x_moz-txt-link-rfc2396E" href=3D"mailto:users-bounces@ovirt.org=
">&lt;users-bounces(a)ovirt.org&gt;</a> on behalf of Erekle Magradze
<a class=3D"x_moz-txt-link-rfc2396E"
href=3D"mailto:erekle.magradze@recogiz=
er.de">&lt;erekle.magradze(a)recogizer.de&gt;</a><br <b>Sent:</b> Thursday, February 22, 2018 11:48
PM<br <b>To:</b> <a
class=3D"x_moz-txt-link-abbreviated" href=3D"mailto:users@ovi=
rt.org">users(a)ovirt.org</a><br
<b>Subject:</b> Re: [ovirt-users] rebooting hypervisors from time to
time</=
font <div> </div
</div <div
class=3D"x_BodyFragment"><font size=3D"2"><span
style=3D"font-size:11p=
t" <div class=3D"x_PlainText">Dear all,<br <br It would be great if someone will
share any experience regarding the <br similar
case, would be great to have a hint where to start investigation.<b=
r <br Thanks again<br <br Cheers<br <br Erekle<br <br <br On
02/22/2018 05:05 PM, Erekle Magradze wrote:<br >
Hello there,<br ><br > I am facing the following problem from time to time
one of the <br > hypervisor (there are 3 of
them)s is rebooting, I am using <br >
ovirt-release42-4.2.1-1.el7.centos.noarch and glsuter as a storage <br=
> backend (glusterfs-3.12.5-2.el7.x86_64).<br ><br >
I am suspecting gluster because of the e.g. message bellow from one of=
<br > the volumes,<br
><br > Could you please help and
suggest to which direction should <br >
investigation go?<br ><br > Thanks in advance<br
><br > Cheers<br ><br >
Erekle<br ><br ><br >
[2018-02-22 15:36:10.011687] and [2018-02-22 15:37:10.955013]<br > [2018-02-22 15:41:10.198701] I [MSGID: 109063]
<br > [dht-layout.c:716:dht_layout_normalize]
0-virtimages-dht: Found <br > anomalies in (null) (gfid
=3D 00000000-0000-0000-0000-000000000000). <=
br > Holes=3D1 overlaps=3D0<br > [2018-02-22 15:41:10.198704] I [MSGID: 109063]
<br > [dht-layout.c:716:dht_layout_normalize]
0-virtimages-dht: Found <br > anomalies in (null) (gfid
=3D 00000000-0000-0000-0000-000000000000). <=
br > Holes=3D1 overlaps=3D0<br > [2018-02-22 15:42:11.293608] I [MSGID: 109063]
<br > [dht-layout.c:716:dht_layout_normalize]
0-virtimages-dht: Found <br > anomalies in (null) (gfid
=3D 00000000-0000-0000-0000-000000000000). <=
br > Holes=3D1 overlaps=3D0<br > [2018-02-22 15:53:16.245720] I [MSGID: 100030]
<br > [glusterfsd.c:2524:main] 0-/usr/sbin/glusterfs:
Started running <br > /usr/sbin/glusterfs version
3.12.5 (args: /usr/sbin/glusterfs <br >
--volfile-server=3D10.0.0.21 --volfi<br >
le-server=3D10.0.0.22 --volfile-server=3D10.0.0.23 <br >
--volfile-id=3D/virtimages <br >
/rhev/data-center/mnt/glusterSD/10.0.0.21:_virtimages)<br >
[2018-02-22 15:53:16.263712] W [MSGID: 101002] <br >
[options.c:995:xl_opt_validate] 0-glusterfs: option 'address-family' <=
br > is deprecated, preferred is
'transport.address-family', continuing <br=
> with correction<br >
[2018-02-22 15:53:16.269595] I [MSGID: 101190] <br >
[event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started <br > thread with index 1<br >
[2018-02-22 15:53:16.273483] I [MSGID: 101190] <br >
[event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started <br > thread with index 2<br >
[2018-02-22 15:53:16.273594] W [MSGID: 101174] <br >
[graph.c:363:_log_if_unknown_option] 0-virtimages-readdir-ahead: <br > option 'parallel-readdir' is not
recognized<br > [2018-02-22
15:53:16.273703] I [MSGID: 114020] [client.c:2360:notify] =
<br > 0-virtimages-client-0: parent translators are ready,
attempting <br > connect on
transport<br > [2018-02-22
15:53:16.276455] I [MSGID: 114020] [client.c:2360:notify] =
<br > 0-virtimages-client-1: parent translators are ready,
attempting <br > connect on
transport<br > [2018-02-22
15:53:16.276683] I [rpc-clnt.c:1986:rpc_clnt_reconfig] <br=
> 0-virtimages-client-0: changing port to 49152 (from
0)<br > [2018-02-22
15:53:16.279191] I [MSGID: 114020] [client.c:2360:notify] =
<br > 0-virtimages-client-2: parent translators are ready,
attempting <br > connect on
transport<br > [2018-02-22
15:53:16.282126] I [MSGID: 114057] <br >
[client-handshake.c:1478:select_server_supported_programs] <br > 0-virtimages-client-0: Using Program GlusterFS 3.3,
Num (1298437), <br=
> Version (330)<br >
[2018-02-22 15:53:16.282573] I [MSGID: 114046] <br >
[client-handshake.c:1231:client_setvolume_cbk] 0-virtimages-client-0: =
<br > Connected to virtimages-client-0, attached to remote
volume <br >
'/mnt/virtimages/virtimgs'.<br >
[2018-02-22 15:53:16.282584] I [MSGID: 114047] <br >
[client-handshake.c:1242:client_setvolume_cbk] 0-virtimages-client-0: =
<br > Server and Client lk-version numbers are not same,
reopening the fds<b=
r > [2018-02-22 15:53:16.282665] I [MSGID: 108005]
<br > [afr-common.c:4929:__afr_handle_child_up_event]
<br > 0-virtimages-replicate-0: Subvolume
'virtimages-client-0' came back <b=
r > up; going online.<br >
[2018-02-22 15:53:16.282877] I [rpc-clnt.c:1986:rpc_clnt_reconfig] <br=
> 0-virtimages-client-1: changing port to 49152 (from
0)<br > [2018-02-22
15:53:16.282934] I [MSGID: 114035] <br >
[client-handshake.c:202:client_set_lk_version_cbk] <br >
0-virtimages-client-0: Server lk version =3D 1<br
><br >
_______________________________________________<br >
Users mailing list<br > <a
class=3D"x_moz-txt-link-abbreviated" href=3D"mailto:Users@ovirt.org=
">Users(a)ovirt.org</a><br >
<a href=3D"http://lists.ovirt.org/mailman/listinfo/users">http:...
.ovirt.org/mailman/listinfo/users</a><br
<br -- <br Recogizer Group GmbH<br
<br Dr.rer.nat. Erekle
Magradze<br Lead Big Data Engineering &
DevOps<br Rheinwerkallee 2, 53227
Bonn<br Tel: +49 228
29974555<br <br E-Mail
<a class=3D"x_moz-txt-link-abbreviated" href=3D"mailto:erekle.magrad=
ze(a)recogizer.de"
erekle.magradze(a)recogizer.de</a><br
recogizer.com<br <br
-----------------------------------------------------------------<br <br Recogizer Group GmbH<br Gesch=E4ftsf=FChrer: Oliver Habisch, Carsten Kreutze<br Handelsregister: Amtsgericht Bonn HRB 20724<br Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993<br Diese E-Mail enth=E4lt vertrauliche und/oder rechtlich
gesch=FCtzte Informa=
tionen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt=FC=
mlich erhalten haben, informieren Sie bitte sofort den Absender und l=F6sch=
en Sie diese Mail. Das unerlaubte Kopieren
sowie die unbefugte Weitergabe dieser Mail und der darin enthaltenen Infor=
mationen ist nicht gestattet.<br
<br
_______________________________________________<br Users
mailing list<br <a
class=3D"x_moz-txt-link-abbreviated"
href=3D"mailto:Users@ovirt.org">Use=
rs(a)ovirt.org</a><br <a
href=3D"http://lists.ovirt.org/mailman/listinfo/users">http:...
t.org/mailman/listinfo/users</a><br
</div
</span></font></div
</blockquote <br <pre
class=3D"x_moz-signature" cols=3D"72">--=20
Recogizer Group GmbH
Dr.rer.nat. Erekle Magradze
Lead Big Data Engineering & DevOps
Rheinwerkallee 2, 53227 Bonn
Tel: +49 228 29974555
E-Mail <a class=3D"x_moz-txt-link-abbreviated"
href=3D"mailto:erekle.magrad=
ze@recogizer.de">erekle.magradze(a)recogizer.de</a
recogizer.com
-----------------------------------------------------------------
Recogizer Group GmbH
Gesch=E4ftsf=FChrer: Oliver Habisch, Carsten Kreutze
Handelsregister: Amtsgericht Bonn HRB 20724
Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993
Diese E-Mail enth=E4lt vertrauliche und/oder rechtlich gesch=FCtzte Informa=
tionen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt=FC=
mlich erhalten haben, informieren Sie bitte sofort den Absender und l=F6sch=
en Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe d=
ieser Mail und der darin enthaltenen Informationen ist nicht gestattet.</pr=
e </div
</body </html
--_000_DM5PR01MB2506CA22D55C58A5210C6EA6FFCC0DM5PR01MB2506prod_--
Hello there,
I am facing the following problem from time to time one of the
hypervisor (there are 3 of them)s is rebooting, I am using
ovirt-release42-4.2.1-1.el7.centos.noarch and glsuter as a storage
backend (glusterfs-3.12.5-2.el7.x86_64).
I am suspecting gluster because of the e.g. message bellow from one of
the volumes,
Could you please help and suggest to which direction should
investigation go?
Thanks in advance
Cheers
Erekle
[2018-02-22 15:36:10.011687] and [2018-02-22 15:37:10.955013]
[2018-02-22 15:41:10.198701] I [MSGID: 109063]
[dht-layout.c:716:dht_layout_normalize] 0-virtimages-dht: Found
anomalies in (null) (gfid =3D 00000000-0000-0000-0000-000000000000).
Holes=3D1 overlaps=3D0
[2018-02-22 15:41:10.198704] I [MSGID: 109063]
[dht-layout.c:716:dht_layout_normalize] 0-virtimages-dht: Found
anomalies in (null) (gfid =3D 00000000-0000-0000-0000-000000000000).
Holes=3D1 overlaps=3D0
[2018-02-22 15:42:11.293608] I [MSGID: 109063]
[dht-layout.c:716:dht_layout_normalize] 0-virtimages-dht: Found
anomalies in (null) (gfid =3D 00000000-0000-0000-0000-000000000000).
Holes=3D1 overlaps=3D0
[2018-02-22 15:53:16.245720] I [MSGID: 100030]
[glusterfsd.c:2524:main] 0-/usr/sbin/glusterfs: Started running
/usr/sbin/glusterfs version 3.12.5 (args: /usr/sbin/glusterfs
--volfile-server=3D10.0.0.21 --volfi
le-server=3D10.0.0.22 --volfile-server=3D10.0.0.23
--volfile-id=3D/virtimages
/rhev/data-center/mnt/glusterSD/10.0.0.21:_virtimages)
[2018-02-22 15:53:16.263712] W [MSGID: 101002]
[options.c:995:xl_opt_validate] 0-glusterfs: option 'address-family'
is deprecated, preferred is 'transport.address-family', continuing
with correction
[2018-02-22 15:53:16.269595] I [MSGID: 101190]
[event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started
thread with index 1
[2018-02-22 15:53:16.273483] I [MSGID: 101190]
[event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started
thread with index 2
[2018-02-22 15:53:16.273594] W [MSGID: 101174]
[graph.c:363:_log_if_unknown_option] 0-virtimages-readdir-ahead:
option 'parallel-readdir' is not recognized
[2018-02-22 15:53:16.273703] I [MSGID: 114020] [client.c:2360:notify]
0-virtimages-client-0: parent translators are ready, attempting
connect on transport
[2018-02-22 15:53:16.276455] I [MSGID: 114020] [client.c:2360:notify]
0-virtimages-client-1: parent translators are ready, attempting
connect on transport
[2018-02-22 15:53:16.276683] I [rpc-clnt.c:1986:rpc_clnt_reconfig]
0-virtimages-client-0: changing port to 49152 (from 0)
[2018-02-22 15:53:16.279191] I [MSGID: 114020] [client.c:2360:notify]
0-virtimages-client-2: parent translators are ready, attempting
connect on transport
[2018-02-22 15:53:16.282126] I [MSGID: 114057]
[client-handshake.c:1478:select_server_supported_programs]
0-virtimages-client-0: Using Program GlusterFS 3.3, Num (1298437),
Version (330)
[2018-02-22 15:53:16.282573] I [MSGID: 114046]
[client-handshake.c:1231:client_setvolume_cbk] 0-virtimages-client-0:
Connected to virtimages-client-0, attached to remote volume
'/mnt/virtimages/virtimgs'.
[2018-02-22 15:53:16.282584] I [MSGID: 114047]
[client-handshake.c:1242:client_setvolume_cbk] 0-virtimages-client-0:
Server and Client lk-version numbers are not same, reopening the fds
[2018-02-22 15:53:16.282665] I [MSGID: 108005]
[afr-common.c:4929:__afr_handle_child_up_event]
0-virtimages-replicate-0: Subvolume 'virtimages-client-0' came back
up; going online.
[2018-02-22 15:53:16.282877] I [rpc-clnt.c:1986:rpc_clnt_reconfig]
0-virtimages-client-1: changing port to 49152 (from 0)
[2018-02-22 15:53:16.282934] I [MSGID: 114035]
[client-handshake.c:202:client_set_lk_version_cbk]
0-virtimages-client-0: Server lk version =3D 1
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Recogizer Group GmbH
Dr.rer.nat. Erekle Magradze
Lead Big Data Engineering & DevOps
Rheinwerkallee 2, 53227 Bonn
Tel: +49 228 29974555
E-Mail erekle.magradze@recogizer.de<mailto:erekle.magradze@recogizer.de