The revert was merged.

On Thu, Jan 25, 2018 at 12:32 PM, Daniel Belenky <dbelenky@redhat.com> wrote:
Have you tried running OST with rpms from the suspected patch to reproduce?

I meant reproducing it manually, not though OST.


On Thu, Jan 25, 2018 at 12:24 PM Edward Haas <ehaas@redhat.com> wrote:
We have two options, a revert or a fix:
Revert: https://gerrit.ovirt.org/#/c/86789/
Fix: https://gerrit.ovirt.org/#/c/86785/

We are not sure about the fix because we cannot reproduce the problem manually.


On Thu, Jan 25, 2018 at 10:45 AM, Eyal Edri <eedri@redhat.com> wrote:
Once you have RPMs, you can run the upgrade suite from the manual job.

On Thu, Jan 25, 2018 at 10:43 AM, Edward Haas <ehaas@redhat.com> wrote:
Can we test if this one fixes this problem?
https://gerrit.ovirt.org/#/c/86781

On Thu, Jan 25, 2018 at 10:00 AM, Eyal Edri <eedri@redhat.com> wrote:
Indeed, the patch looks relevant, 
Dan, can we revert it or send a fix ASAP to avoid building up a large queue? 

On Thu, Jan 25, 2018 at 9:29 AM, Daniel Belenky <dbelenky@redhat.com> wrote:
Hi,

We failed to setup host in OST upgrade from 4.1 to master suite.
Please note that the upgrade suite installs 4.1 engine, then upgrades it to master and then tries to set up a host.

Links:
  1. Link to failed job
  2. Suspected patch: Gerrit 86474/33

Error snippet from engine.log (engine):

2018-01-24 15:13:20,257-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-upgrade-from-release-suite-master-host0: Failed to execute stage 'Closing up': Failed to start service 'vdsmd'.
2018-01-24 15:13:20,301-05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host0. Stage: Clean up.
2018-01-24 15:13:20,304-05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host0. Stage: Pre-termination.
2018-01-24 15:13:20,332-05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host0. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-20180124151320-lago-upgrade-from-release-suite-master-host0-34609a2f.log'.
2018-01-24 15:13:29,227-05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host0. Stage: Termination.
2018-01-24 15:13:29,321-05 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] SSH error running command root@lago-upgrade-from-release-suite-master-host0:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x &&  "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True': IOException: Command returned failure code 1 during SSH session 'root@lago-upgrade-from-release-suite-master-host0'
2018-01-24 15:13:29,322-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] Error during host lago-upgrade-from-release-suite-master-host0 install
2018-01-24 15:13:29,324-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] Host installation failed for host '4d681c3b-e8db-4a71-b5e3-0db096e3ae9c', 'lago-upgrade-from-release-suite-master-host0': Command returned failure code 1 during SSH session 'root@lago-upgrade-from-release-suite-master-host0'
2018-01-24 15:13:29,330-05 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] START, SetVdsStatusVDSCommand(HostName = lago-upgrade-from-release-suite-master-host0, SetVdsStatusVDSCommandParameters:{hostId='4d681c3b-e8db-4a71-b5e3-0db096e3ae9c', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 5e6c4a3e
2018-01-24 15:13:29,339-05 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] FINISH, SetVdsStatusVDSCommand, log id: 5e6c4a3e
2018-01-24 15:13:29,346-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] EVENT_ID: VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-suite-master-host0 installation failed. Command returned failure code 1 during SSH session 'root@lago-upgrade-from-release-suite-master-host0'.

Error snippet from /var/log/messages (host0):

15:13:19 host0 NetworkManager[580]: <info>  (bondscan-Ncw7DP): new Bond device (carrier: OFF, driver: 'bonding', ifindex: 4)
15:13:19 host0 vdsm-tool: Traceback (most recent call last):
15:13:19 host0 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main
15:13:19 host0 vdsm-tool: return tool_command[cmd]["command"](*args)
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 97, in dump_bonding_options
15:13:19 host0 vdsm-tool: sysfs_options_mapper.dump_bonding_options()
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", line 48, in dump_bonding_options
15:13:19 host0 vdsm-tool: jdump(_get_default_bonding_options(), f)
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", line 60, in _get_default_bonding_options
15:13:19 host0 vdsm-tool: with _bond_device(bond_name):
15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
15:13:19 host0 vdsm-tool: return self.gen.next()
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", line 102, in _bond_device
15:13:19 host0 vdsm-tool: _unmanage_nm_device(bond_name)
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", line 116, in _unmanage_nm_device
15:13:19 host0 vdsm-tool: dev.managed = False
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/nm/networkmanager.py", line 90, in managed
15:13:19 host0 vdsm-tool: self._device.managed = value
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/nm/nmdbus/device.py", line 81, in managed
15:13:19 host0 vdsm-tool: return self._set_property('Managed', value)
15:13:19 host0 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/nm/nmdbus/device.py", line 88, in _set_property
15:13:19 host0 vdsm-tool: self.IF_NAME, property_name, property_value)
15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 70, in __call__
15:13:19 host0 vdsm-tool: return self._proxy_method(*args, **keywords)
15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 145, in __call__
15:13:19 host0 vdsm-tool: **keywords)
15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/site-packages/dbus/connection.py", line 651, in call_blocking
15:13:19 host0 vdsm-tool: message, timeout)
15:13:19 host0 vdsm-tool: DBusException: org.freedesktop.DBus.Error.AccessDenied: Property "Managed" of interface "org.freedesktop.NetworkManager.Device" is not settable

Thanks,
--

DANIEL BELENKY

RHV DEVOPS


_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel



--

Eyal edri


MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D


Red Hat EMEA

TRIED. TESTED. TRUSTED.
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)




--

Eyal edri


MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D


Red Hat EMEA

TRIED. TESTED. TRUSTED.
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)

--
Daniel Belenky
RHV DevOps