<div dir="ltr">The revert was merged.<br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 25, 2018 at 12:32 PM, Daniel Belenky <span dir="ltr"><<a href="mailto:dbelenky@redhat.com" target="_blank">dbelenky@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div dir="auto">Have you tried running OST with rpms from the suspected patch to reproduce?</div></div></blockquote><div><br></div><div>I meant reproducing it manually, not though OST.<br></div><div></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div class="h5"><br><div class="gmail_quote"><div>On Thu, Jan 25, 2018 at 12:24 PM Edward Haas <<a href="mailto:ehaas@redhat.com" target="_blank">ehaas@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div><div>We have two options, a revert or a fix:<br>Revert: <a href="https://gerrit.ovirt.org/#/c/86789/" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>86789/</a><br>Fix: <a href="https://gerrit.ovirt.org/#/c/86785/" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>86785/</a><br><br></div>We are not sure about the fix because we cannot reproduce the problem manually.<br></div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 25, 2018 at 10:45 AM, Eyal Edri <span><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Once you have RPMs, you can run the upgrade suite from the manual job.</div><div class="m_1520784753263556806m_8729624379495480961HOEnZb"><div class="m_1520784753263556806m_8729624379495480961h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 25, 2018 at 10:43 AM, Edward Haas <span><<a href="mailto:ehaas@redhat.com" target="_blank">ehaas@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Can we test if this one fixes this problem?<br><a href="https://gerrit.ovirt.org/#/c/86781" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>86781</a></div><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172HOEnZb"><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 25, 2018 at 10:00 AM, Eyal Edri <span><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>Indeed, the patch looks relevant, </div><div>Dan, can we revert it or send a fix ASAP to avoid building up a large queue? </div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198h5">On Thu, Jan 25, 2018 at 9:29 AM, Daniel Belenky <span><<a href="mailto:dbelenky@redhat.com" target="_blank">dbelenky@redhat.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198h5"><div><div><div>Hi,<br><br></div>We failed to setup host in OST upgrade from 4.1 to master suite.</div><div>Please note that the upgrade suite installs 4.1 engine, then upgrades it to master and then tries to set up a host.<br></div><br><b>Links:</b><br><ol><li><a href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5093/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-002_bootstrap.py/" target="_blank">Link to failed job</a></li><li><span id="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605m_5382805945259122848gmail-goog_997625503"></span><span id="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605m_5382805945259122848gmail-goog_997625504"></span><a href="https://gerrit.ovirt.org/#/c/86474/33" target="_blank">Suspected patch: Gerrit 86474/33</a></li></ol><p><b>Error snippet from engine.log (engine):</b></p><pre><span style="font-family:monospace,monospace">2018-01-24 15:13:20,257-05 ERROR [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(<wbr>511), An error has occurred during installation of Host lago-upgrade-from-release-<wbr>suite-master-host0: Failed to execute stage 'Closing up': Failed to start service 'vdsmd'.
2018-01-24 15:13:20,301-05 INFO [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-<wbr>suite-master-host0. Stage: Clean up.
2018-01-24 15:13:20,304-05 INFO [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-<wbr>suite-master-host0. Stage: Pre-termination.
2018-01-24 15:13:20,332-05 INFO [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-<wbr>suite-master-host0. Retrieving installation logs to: '/var/log/ovirt-engine/host-<wbr>deploy/ovirt-host-deploy-<wbr>20180124151320-lago-upgrade-<wbr>from-release-suite-master-<wbr>host0-34609a2f.log'.
2018-01-24 15:13:29,227-05 INFO [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-<wbr>suite-master-host0. Stage: Termination.
2018-01-24 15:13:29,321-05 ERROR [org.ovirt.engine.core.uutils.<wbr>ssh.SSHDialog] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] SSH error running command root@lago-upgrade-from-<wbr>release-suite-master-host0:'<wbr>umask 0077; MYTMP="$(TMPDIR="${OVIRT_<wbr>TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:<wbr>True': IOException: Command returned failure code 1 during SSH session 'root@lago-upgrade-from-<wbr>release-suite-master-host0'
2018-01-24 15:13:29,322-05 ERROR [org.ovirt.engine.core.bll.<wbr>hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] Error during host lago-upgrade-from-release-<wbr>suite-master-host0 install
2018-01-24 15:13:29,324-05 ERROR [org.ovirt.engine.core.bll.<wbr>hostdeploy.<wbr>InstallVdsInternalCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] Host installation failed for host '4d681c3b-e8db-4a71-b5e3-<wbr>0db096e3ae9c', 'lago-upgrade-from-release-<wbr>suite-master-host0': Command returned failure code 1 during SSH session 'root@lago-upgrade-from-<wbr>release-suite-master-host0'
2018-01-24 15:13:29,330-05 INFO [org.ovirt.engine.core.<wbr>vdsbroker.<wbr>SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] START, SetVdsStatusVDSCommand(<wbr>HostName = lago-upgrade-from-release-<wbr>suite-master-host0, SetVdsStatusVDSCommandParamete<wbr>rs:{hostId='4d681c3b-e8db-<wbr>4a71-b5e3-0db096e3ae9c', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 5e6c4a3e
2018-01-24 15:13:29,339-05 INFO [org.ovirt.engine.core.<wbr>vdsbroker.<wbr>SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] FINISH, SetVdsStatusVDSCommand, log id: 5e6c4a3e
2018-01-24 15:13:29,346-05 ERROR [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (EE-ManagedThreadFactory-<wbr>engine-Thread-1) [34609a2f] EVENT_ID: VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-<wbr>suite-master-host0 installation failed. Command returned failure code 1 during SSH session 'root@lago-upgrade-from-<wbr>release-suite-master-host0'.</span></pre><p><b>Error snippet from /var/log/messages (host0):</b></p><pre><span style="font-family:monospace,monospace"></span><span style="font-family:monospace,monospace">15:13:19 host0 NetworkManager[580]: <info> (bondscan-Ncw7DP): new Bond device (carrier: OFF, driver: 'bonding', ifindex: 4)<br>15:13:19 host0 vdsm-tool: Traceback (most recent call last):<br>15:13:19 host0 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main<br>15:13:19 host0 vdsm-tool: return tool_command[cmd]["command"](*<wbr>args)<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/tool/network.py"<wbr>, line 97, in dump_bonding_options<br>15:13:19 host0 vdsm-tool: sysfs_options_mapper.dump_<wbr>bonding_options()<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/link/<wbr>bond/sysfs_options_mapper.py", line 48, in dump_bonding_options<br>15:13:19 host0 vdsm-tool: jdump(_get_default_bonding_<wbr>options(), f)<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/link/<wbr>bond/sysfs_options_mapper.py", line 60, in _get_default_bonding_options<br>15:13:19 host0 vdsm-tool: with _bond_device(bond_name):<br>15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/<wbr>contextlib.py", line 17, in __enter__<br>15:13:19 host0 vdsm-tool: return self.gen.next()<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/link/<wbr>bond/sysfs_options_mapper.py", line 102, in _bond_device<br>15:13:19 host0 vdsm-tool: _unmanage_nm_device(bond_name)<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/link/<wbr>bond/sysfs_options_mapper.py", line 116, in _unmanage_nm_device<br>15:13:19 host0 vdsm-tool: dev.managed = False<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/nm/<wbr>networkmanager.py", line 90, in managed<br>15:13:19 host0 vdsm-tool: self._device.managed = value<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/nm/<wbr>nmdbus/device.py", line 81, in managed<br>15:13:19 host0 vdsm-tool: return self._set_property('Managed', value)<br>15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-<wbr>packages/vdsm/network/nm/<wbr>nmdbus/device.py", line 88, in _set_property<br>15:13:19 host0 vdsm-tool: self.IF_NAME, property_name, property_value)<br>15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-<wbr>packages/dbus/proxies.py", line 70, in __call__<br>15:13:19 host0 vdsm-tool: return self._proxy_method(*args, **keywords)<br>15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-<wbr>packages/dbus/proxies.py", line 145, in __call__<br>15:13:19 host0 vdsm-tool: **keywords)<br>15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-<wbr>packages/dbus/connection.py", line 651, in call_blocking<br>15:13:19 host0 vdsm-tool: message, timeout)<br>15:13:19 host0 vdsm-tool: DBusException: org.freedesktop.DBus.Error.<wbr>AccessDenied: Property "Managed" of interface "org.freedesktop.<wbr>NetworkManager.Device" is not settable</span></pre><div><div><div><div><br></div><div>Thanks,<span class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605HOEnZb"><font color="#888888"><br></font></span></div><span class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605HOEnZb"><font color="#888888"><div>-- <br><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605m_5382805945259122848gmail_signature"><div><div><div><div><span><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>DANIEL</span> <span>BELENKY</span><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span style="font-weight:normal">RHV DEVOPS</span><br></p></span></div></div></div></div></div>
</div></font></span></div></div></div></div>
<br></div></div>______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/devel</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172m_6063433550416238198m_-2606821731357439605gmail_signature" data-smartmail="gmail_signature"><div><div><div><div><div><div><div><div><div><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium" border="0"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div></div></div>
</div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_1520784753263556806m_8729624379495480961m_-7600488180264582172gmail_signature" data-smartmail="gmail_signature"><div><div><div><div><div><div><div><div><div><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium" border="0"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</blockquote></div></div></div></div><span class="HOEnZb"><font color="#888888"><div dir="ltr">-- <br></div><div class="m_1520784753263556806gmail_signature" data-smartmail="gmail_signature">Daniel Belenky<br>RHV DevOps</div>
</font></span></blockquote></div><br></div></div>