--Apple-Mail=_7EF46EB5-FF10-4695-889D-DC90FC772541 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8

On 26 Oct 2017, at 12:32, Roberto Nunin <robnunin@gmail.com> wrote: =20 Hi Michael =20 By frozen I mean the action to put host in maintenance while some VM = were running on it. This action wasn't completed after more than one hour.

ok, and was the problem in this last VM not finishing the migration? Was = it migrating at all? If yes, what was the progress in UI, any failures? = There are various timeouts which should have been triggered, so if they = were not triggered it would indeed point to some internal issue. Would = be great to attach source and destination vdsm.log

Thinking that shutting down the VM could help, I've done it. Looking = at results, not.

=20 Yes, I've restarted the ovirt-engine service, I've still not restarted =

What was the result? Did it fail to shut down? Did you use Power Off to = force immediate shutdown? If it was migrating, did you try to cancel the = migration first? the hosted-engine VM. well, it=E2=80=99s not only a universal =E2=80=9Cfix=E2=80=9D of various = things, sometimes it does more harm than benefit too. Logs would be = helpful.

Hosts still not restarted. Do you think can help ?

hard to say. Either way please salvage logs first

=20 Obviously we will migrate, this activities are enabling us to have = redundancy at the storage level, then we will migrate to 4.1.x

=20 Thanks =20 2017-10-26 12:26 GMT+02:00 Michal Skrivanek = <michal.skrivanek@redhat.com <mailto:michal.skrivanek@redhat.com>>: =20

...

On 26 Oct 2017, at 10:20, Roberto Nunin <robnunin@gmail.com = <mailto:robnunin@gmail.com>> wrote: =20 We are running 4.0.1.1-1.el7.centos =20 Hi, any reason not to upgrade to 4.1? =20 =20 After a frozen migration attempt, we have two VM that after shutdown, = are not anymore able to be started up again. =20 what do you mean by frozen? Are you talking about "VM live migration" = or =E2=80=9Clive storage migration=E2=80=9D? How exactly did you resolve that situation, you only shut down those = VMs? No other troubleshooting steps, e.g. restarting engine, hosts, =

great:) Thanks, michal things like that?

=20 Thanks, michal

...

=20 Message returned is : =20 Bad volume specification {'index': '0', u'domainID': = u'731d95a9-61a7-4c7a-813b-fb1c3dde47ea', 'reqsize': '0', u'format': = u'cow', u'optional': u'false', u'address': {u'function': u'0x0', u'bus': = u'0x00', u'domain': u'0x0000', u'type': u'pci', u'slot': u'0x05'}, = u'volumeID': u'cffc70ff-ed72-46ef-a369-4be95de72260', 'apparentsize': = '3221225472', u'imageID': u'3fe5a849-bcc2-42d3-93c5aca4c504515b', = u'specParams': {}, u'readonly': u'false', u'iface': u'virtio', = u'deviceId': u'3fe5a849bcc2-42d3-93c5-aca4c504515b', 'truesize': = '3221225472', u'poolID': u'00000001-0001-0001-0001-0000000001ec', = u'device': u'disk', u'shared': u'false', u'propagateErrors': = u'off',u'type':u'disk'} =20 Probably this is caused by a wrong pointer into the database that = still refer to the migration image-id. =20 If we search within all_disks view, we can find that parentid field = isn't 00000000-0000-0000-0000-000000000000 like all other running vm, = but it has a value: =20 vm_names | parentid ----------------------+-------------------------------------- working01.company.xx | 00000000-0000-0000-0000-000000000000 working02.company.xx | 00000000-0000-0000-0000-000000000000 working03.company.xx | 00000000-0000-0000-0000-000000000000 working04.company.xx | 00000000-0000-0000-0000-000000000000 broken001.company.xx | 30533842-2c83-4d0e-95d2-48162dbe23bd = <<<<<<<<< working05.company.xx | 00000000-0000-0000-0000-000000000000 =20 =20 How we can recover from this ? =20 Thanks in advance Regards, =20 --=20 Robert=E2=80=8Bo=E2=80=8B =20 =20 =20 _______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users = <http://lists.ovirt.org/mailman/listinfo/users> =20 =20 =20 =20 --=20 Roberto =20 =20 =20

--Apple-Mail=_7EF46EB5-FF10-4695-889D-DC90FC772541 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 26 Oct 2017, at 12:32, Roberto Nunin <<a = href=3D"mailto:robnunin@gmail.com" class=3D"">robnunin@gmail.com</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_default" style=3D"">Hi = Michael</div><div class=3D"gmail_default" style=3D""><br = class=3D""></div><div class=3D"gmail_default" style=3D"">By frozen I = mean the action to put host in maintenance while some VM were running on = it.</div><div class=3D"gmail_default" style=3D"">This action wasn't = completed after more than one = hour.</div></div></div></blockquote><div><br class=3D""></div>ok, and = was the problem in this last VM not finishing the migration? Was it = migrating at all? If yes, what was the progress in UI, any failures? = There are various timeouts which should have been triggered, so if they = were not triggered it would indeed point to some internal issue. Would = be great to attach source and destination vdsm.log</div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_default" style=3D"">Thinking = that shutting down the VM could help, I've done it. Looking at results, = not.</div></div></div></blockquote><div><br class=3D""></div>What was = the result? Did it fail to shut down? Did you use Power Off to force = immediate shutdown? If it was migrating, did you try to cancel the = migration first?</div><div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_default" style=3D""><br class=3D""></div><div = class=3D"gmail_default" style=3D"">Yes, I've restarted the ovirt-engine = service, I've still not restarted the hosted-engine = VM.</div></div></div></blockquote><div><br class=3D""></div>well, it=E2=80= =99s not only a universal =E2=80=9Cfix=E2=80=9D of various things, = sometimes it does more harm than benefit too. Logs would be = helpful.</div><div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_default" style=3D"">Hosts still not restarted. Do you = think can help ?</div></div></div></blockquote><div><br = class=3D""></div>hard to say. Either way please salvage logs = first</div><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D""><div dir=3D"ltr" class=3D""><div class=3D"gmail_default" = style=3D""><br class=3D""></div><div class=3D"gmail_default" = style=3D"">Obviously we will migrate, this activities are enabling us to = have redundancy at the storage level, then we will migrate to = 4.1.x</div></div></div></blockquote><div><br = class=3D""></div>great:)</div><div><br = class=3D""></div><div>Thanks,</div><div>michal</div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_default" style=3D""><br = class=3D""></div><div class=3D"gmail_default" style=3D"">Thanks</div><div = class=3D"gmail_extra"><br class=3D""><div class=3D"gmail_quote">2017-10-26= 12:26 GMT+02:00 Michal Skrivanek <span dir=3D"ltr" class=3D""><<a = href=3D"mailto:michal.skrivanek@redhat.com" target=3D"_blank" = class=3D"">michal.skrivanek@redhat.com</a>></span>:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div = style=3D"word-wrap:break-word" class=3D""><br class=3D""><div = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D"">On 26 = Oct 2017, at 10:20, Roberto Nunin <<a = href=3D"mailto:robnunin@gmail.com" target=3D"_blank" = class=3D"">robnunin@gmail.com</a>> wrote:</div><br = class=3D"m_-7560322293096975458Apple-interchange-newline"><div = class=3D""><div dir=3D"ltr" class=3D""><div class=3D"">We are = running<span style=3D"font-size:large" class=3D""> <span = style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D"">4.0.1.1-1.el7.centos</span></span></div></div></div></blockquot= e><div class=3D""><br class=3D""></div>Hi,</div><div class=3D"">any = reason not to upgrade to 4.1?</div><div class=3D""><span class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div style=3D"font-size:large" class=3D""><span = style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D""><br class=3D""></span></div><div style=3D"font-size:large" = class=3D""><span style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D"">After a frozen migration attempt, we have two VM that after = shutdown, are not anymore able to be started up = again.</span></div></div></div></blockquote><div class=3D""><br = class=3D""></div></span>what do you mean by frozen? Are you talking = about "VM live migration" or =E2=80=9Clive storage = migration=E2=80=9D?</div><div class=3D"">How exactly did you resolve = that situation, you only shut down those VMs? No other troubleshooting = steps, e.g. restarting engine, hosts, things like that?</div><div = class=3D""><br class=3D""></div><div class=3D"">Thanks,</div><div = class=3D"">michal</div><div class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><span class=3D""><div dir=3D"ltr" = class=3D""><div style=3D"font-size:large" class=3D""><span = style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D""><br class=3D""></span></div><div style=3D"font-size:large" = class=3D""><span style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D"">Message returned is :</span></div><div = style=3D"font-size:large" class=3D""><span style=3D"font-family:'Arial = Unicode MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D""><br class=3D""></span></div><div class=3D""><span = style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center" class=3D""><div = style=3D"text-align:left" class=3D"">Bad volume specification {'index': = '0', u'domainID': u'731d95a9-61a7-4c7a-813b-<wbr class=3D"">fb1c3dde47ea',= 'reqsize': '0', u'format': u'cow', u'optional': u'false', u'address': = {u'function': u'0x0', u'bus': u'0x00', u'domain': u'0x0000', u'type': = u'pci', u'slot': u'0x05'}, u'volumeID': u'cffc70ff-ed72-46ef-a369-<wbr = class=3D"">4be95de72260', 'apparentsize': '3221225472', u'imageID': = u'3fe5a849-bcc2-42d3-<wbr class=3D"">93c5aca4c504515b', u'specParams': = {}, u'readonly': u'false', u'iface': u'virtio', u'deviceId': = u'3fe5a849bcc2-42d3-93c5-<wbr class=3D"">aca4c504515b', 'truesize': = '3221225472', u'poolID': u'00000001-0001-0001-0001-<wbr = class=3D"">0000000001ec', u'device': u'disk', u'shared': u'false', = u'propagateErrors': u'off',u'type':u'disk'}</div><div = style=3D"text-align:left;font-size:small" class=3D""><br = class=3D""></div></span></div><div style=3D"font-size:large" = class=3D""><span style=3D"font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D"">Probably this is caused by a wrong pointer into the database = that still refer to the migration image-id.</span></div><div = style=3D"font-size:large" class=3D""><span style=3D"font-family:'Arial = Unicode MS',Arial,sans-serif;text-align:-webkit-center;font-size:small" = class=3D""><br class=3D""></span></div><div class=3D""><span = style=3D"font-size:small;font-family:'Arial Unicode = MS',Arial,sans-serif;text-align:-webkit-center" class=3D"">If we search = within all_disks view, we can find that parentid field = isn't </span><font face=3D"Arial Unicode MS, Arial, sans-serif" = class=3D"">00000000-0000-0000-0000-<wbr class=3D"">000000000000 like all = other running vm, but it has a value:</font></div><div class=3D""><font = face=3D"Arial Unicode MS, Arial, sans-serif" class=3D""><br = class=3D""></font></div><div class=3D""><font class=3D""><div = class=3D""><font face=3D"monospace, monospace" class=3D"">    =    vm_names       |      =          parentid</font></div><div = class=3D""><font face=3D"monospace, monospace" = class=3D"">----------------------+-------<wbr = class=3D"">------------------------------<wbr = class=3D"">-</font></div><div class=3D""><font face=3D"monospace, = monospace" class=3D""> working01.company.xx | = 00000000-0000-0000-0000-<wbr class=3D"">000000000000</font></div><div = class=3D""><font face=3D"monospace, monospace" = class=3D""> working02.company.xx | 00000000-0000-0000-0000-<wbr = class=3D"">000000000000</font></div><div class=3D""><font = face=3D"monospace, monospace" class=3D""> working03.company.xx | = 00000000-0000-0000-0000-<wbr class=3D"">000000000000</font></div><div = class=3D""><font face=3D"monospace, monospace" = class=3D""> working04.company.xx | 00000000-0000-0000-0000-<wbr = class=3D"">000000000000</font></div><div class=3D""><font = face=3D"monospace, monospace" class=3D""> broken001.company.xx | = 30533842-2c83-4d0e-95d2-<wbr class=3D"">48162dbe23bd = <<<<<<<<<</font></div><div class=3D""><font = face=3D"monospace, monospace" class=3D""> working05.company.xx | = 00000000-0000-0000-0000-<wbr class=3D"">000000000000</font></div><div = style=3D"font-family:"Arial Unicode MS",Arial,sans-serif" = class=3D""><br class=3D""></div></font></div><div class=3D""><font = face=3D"Arial Unicode MS, Arial, sans-serif" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Arial Unicode MS, = Arial, sans-serif" class=3D"">How we can recover from this = ?</font></div><div class=3D""><font face=3D"Arial Unicode MS, Arial, = sans-serif" class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Arial Unicode MS, Arial, sans-serif" class=3D"">Thanks in = advance</font></div><div class=3D""><font face=3D"Arial Unicode MS, = Arial, sans-serif" class=3D"">Regards,</font></div><div class=3D""><br = class=3D""></div>-- <br class=3D""><div = class=3D"m_-7560322293096975458gmail_signature"><div dir=3D"ltr" = class=3D""><div dir=3D"ltr" class=3D""><font size=3D"4" = class=3D"">Robert<div style=3D"font-size:large;display:inline" = class=3D"">=E2=80=8Bo=E2=80=8B</div><br class=3D""><br = class=3D""></font><span class=3D""><font size=3D"4" class=3D""><br = class=3D""><br class=3D""></font></span></div></div></div> </div></span> ______________________________<wbr class=3D"">_________________<br = class=3D"">Users mailing list<br class=3D""><a = href=3D"mailto:Users@ovirt.org" target=3D"_blank" = class=3D"">Users@ovirt.org</a><br class=3D""><a = href=3D"http://lists.ovirt.org/mailman/listinfo/users" target=3D"_blank" = class=3D"">http://lists.ovirt.org/<wbr = class=3D"">mailman/listinfo/users</a><br = class=3D""></div></blockquote></div><br = class=3D""></div></blockquote></div><br class=3D""><br clear=3D"all" = class=3D""><div class=3D""><br class=3D""></div>-- <br class=3D""><div = class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div = dir=3D"ltr" class=3D""><div dir=3D"ltr" class=3D""><font size=3D"4" = class=3D"">Roberto<br class=3D""><br class=3D""></font><span = class=3D""><font size=3D"4" class=3D""><br class=3D""><br = class=3D""></font></span></div></div></div> </div></div> </div></blockquote></div><br class=3D""></body></html>= --Apple-Mail=_7EF46EB5-FF10-4695-889D-DC90FC772541--