Re: [ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.2 (ovirt-ansible-hosted-engine-setup) ] [ 14-01-2019 ] [ 001_upgrade_engine.test_initialize_engine]

Any update on this? On Tue, Jan 15, 2019 at 7:28 PM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > On Tue, Jan 15, 2019 at 6:28 PM Martin Perina <mperina(a)redhat.com&gt; wrote: > >> There is way how to replace directory from previous package version with >> a symlink in new version, we are already using this in other roles. Ondro, >> could you please share the details about this hack? >> > > Ahh, sorry, I see that your are already using this hack: > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/ov... > But why is this not working? We have already used that in ovirt-infra > role several times: > > 1. during the upgrade from 4.1, when we split roles to multiple packages > (1 role per RPM) > > https://github.com/oVirt/ovirt-ansible-infra/commit/79b62e77f8d1d1cf24f04... > > 2. when we needed to overcome latest case sensitive issues in galaxy. And > in both cases packages were upgraded successfully > > https://github.com/oVirt/ovirt-ansible-infra/commit/a80002d4966241b819625... > > So what's different in the hosted-engine role case? > >> >> >> On Tue, 15 Jan 2019, 13:24 Simone Tiraboschi <stirabos(a)redhat.com wrote: >> >>> >>> >>> On Tue, Jan 15, 2019 at 11:15 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>> wrote: >>> >>>> On Tue, Jan 15, 2019 at 11:38 AM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> > >>>> > The last ovirt-engine package ran on Jan 11th and this failure >>>> happened on the 14th so the appliance package is suppose to be newer then >>>> the engine and should be running fine. >>>> > if this is not the case, then I suggest that the maintainers of the >>>> project see what is needed to be updated and trigger a new package build >>>> (which is why I was alerting to the failure) >>>> > >>>> > Thanks, >>>> > Dafna >>>> > >>>> > >>>> > On Tue, Jan 15, 2019 at 6:51 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>>> wrote: >>>> >> >>>> >> On Mon, Jan 14, 2019 at 2:15 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >> > >>>> >> > Hi, >>>> >> > >>>> >> > we are failing test 001_upgrade_engine.test_initialize_engine on >>>> ovirt 4.2 for project ovirt-ansible-hosted-engine-setup. >>>> >> > >>>> >> > Patch reported by CQ: >>>> >> > >>>> >> > >>>> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/860f6a6... >>>> - Rename the role - replacing '-' with '_' >>>> >> > >>>> >> > Ido, can you please check this issue? >>>> >> > >>>> >> > failed build: >>>> >> > >>>> >> > http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3712/ >>>> >>>> This build has a single failure: >>>> >>>> >>>> https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3712/testRepo... >>>> >>>> which is in the upgrade_from_release suite, not hosted-engine. >>>> >>>> However, I looked at the logs, and the root cause indeed seems to be >>>> related to hosted-engine, as the CQ identified (Nice!), which is: >>>> >>>> >>>> https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3712/artifact... >>>> >>>> 2019-01-14 10:38:38,168::ssh.py::ssh::58::lago.ssh::DEBUG::Running >>>> 8d4833e8 on lago-upgrade-from-release-suite-4-2-engine: yum -y update >>>> ovirt-*setup* >>>> 2019-01-14 10:38:39,732::ssh.py::ssh::81::lago.ssh::DEBUG::Command >>>> 8d4833e8 on lago-upgrade-from-release-suite-4-2-engine returned with 1 >>>> 2019-01-14 10:38:39,733::ssh.py::ssh::89::lago.ssh::DEBUG::Command >>>> 8d4833e8 on lago-upgrade-from-release-suite-4-2-engine output: >>>> Loaded plugins: fastestmirror, versionlock >>>> Determining fastest mirrors >>>> Excluding 9 updates due to versionlock (use "yum versionlock status" >>>> to show them) >>>> Resolving Dependencies >>>> --> Running transaction check >>>> ---> Package ovirt-ansible-engine-setup.noarch 0:1.1.5-1.el7 will be >>>> updated >>>> ---> Package ovirt-ansible-engine-setup.noarch >>>> 0:1.1.7-0.1.master.20181219093302.el7 will be an update >>>> ---> Package ovirt-ansible-hosted-engine-setup.noarch 0:1.0.2-1.el7 >>>> will be updated >>>> ---> Package ovirt-ansible-hosted-engine-setup.noarch >>>> 0:1.0.6-0.1.master.20190114102323.el7 will be an update >>>> ---> Package ovirt-engine-dwh-setup.noarch 0:4.2.4.3-1.el7 will be >>>> updated >>>> ---> Package ovirt-engine-dwh-setup.noarch >>>> 0:4.2.5-0.0.master.20180708055101.el7 will be an update >>>> ---> Package ovirt-engine-extension-aaa-ldap-setup.noarch >>>> 0:1.3.8-1.el7 will be updated >>>> ---> Package ovirt-engine-extension-aaa-ldap-setup.noarch >>>> 0:1.3.9-0.0.master.gitc39177c.el7 will be an update >>>> --> Processing Dependency: ovirt-engine-extension-aaa-ldap = >>>> 1.3.9-0.0.master.gitc39177c.el7 for package: >>>> >>>> ovirt-engine-extension-aaa-ldap-setup-1.3.9-0.0.master.gitc39177c.el7.noarch >>>> ---> Package ovirt-engine-setup.noarch 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-setup.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> ---> Package ovirt-engine-setup-base.noarch 0:4.2.7.5-1.el7 will be >>>> updated >>>> ---> Package ovirt-engine-setup-base.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> --> Processing Dependency: ovirt-engine-lib >= >>>> 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 for package: >>>> >>>> ovirt-engine-setup-base-4.2.8.3-0.0.master.20190111080213.git7965c2e.el7.noarch >>>> ---> Package ovirt-engine-setup-plugin-ovirt-engine.noarch >>>> 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-setup-plugin-ovirt-engine.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> ---> Package ovirt-engine-setup-plugin-ovirt-engine-common.noarch >>>> 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-setup-plugin-ovirt-engine-common.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> ---> Package ovirt-engine-setup-plugin-vmconsole-proxy-helper.noarch >>>> 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-setup-plugin-vmconsole-proxy-helper.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> ---> Package ovirt-engine-setup-plugin-websocket-proxy.noarch >>>> 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-setup-plugin-websocket-proxy.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> ---> Package ovirt-imageio-proxy-setup.noarch 0:1.4.5-0.el7 will be >>>> updated >>>> ---> Package ovirt-imageio-proxy-setup.noarch 0:1.4.6-1.el7 will be an >>>> update >>>> ---> Package ovirt-setup-lib.noarch 0:1.1.5-1.el7 will be updated >>>> ---> Package ovirt-setup-lib.noarch >>>> 0:1.1.6-0.0.master.20180921125403.git90612e6.el7 will be an update >>>> --> Running transaction check >>>> ---> Package ovirt-engine-extension-aaa-ldap.noarch 0:1.3.8-1.el7 will >>>> be updated >>>> ---> Package ovirt-engine-extension-aaa-ldap.noarch >>>> 0:1.3.9-0.0.master.gitc39177c.el7 will be an update >>>> ---> Package ovirt-engine-lib.noarch 0:4.2.7.5-1.el7 will be updated >>>> ---> Package ovirt-engine-lib.noarch >>>> 0:4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 will be an update >>>> --> Finished Dependency Resolution >>>> >>>> Dependencies Resolved >>>> >>>> >>>> ================================================================================ >>>> Package >>>> Arch Version >>>> Repository Size >>>> >>>> ================================================================================ >>>> Updating: >>>> ovirt-ansible-engine-setup >>>> noarch 1.1.7-0.1.master.20181219093302.el7 >>>> alocalsync 15 k >>>> ovirt-ansible-hosted-engine-setup >>>> noarch 1.0.6-0.1.master.20190114102323.el7 >>>> alocalsync 45 k >>>> ovirt-engine-dwh-setup >>>> noarch 4.2.5-0.0.master.20180708055101.el7 >>>> alocalsync 78 k >>>> ovirt-engine-extension-aaa-ldap-setup >>>> noarch 1.3.9-0.0.master.gitc39177c.el7 >>>> alocalsync 37 k >>>> ovirt-engine-setup >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 11 k >>>> ovirt-engine-setup-base >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 104 k >>>> ovirt-engine-setup-plugin-ovirt-engine >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 179 k >>>> ovirt-engine-setup-plugin-ovirt-engine-common >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 109 k >>>> ovirt-engine-setup-plugin-vmconsole-proxy-helper >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 30 k >>>> ovirt-engine-setup-plugin-websocket-proxy >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 28 k >>>> ovirt-imageio-proxy-setup >>>> noarch 1.4.6-1.el7 >>>> alocalsync 23 k >>>> ovirt-setup-lib >>>> noarch 1.1.6-0.0.master.20180921125403.git90612e6.el7 >>>> alocalsync 19 k >>>> Updating for dependencies: >>>> ovirt-engine-extension-aaa-ldap >>>> noarch 1.3.9-0.0.master.gitc39177c.el7 >>>> alocalsync 117 k >>>> ovirt-engine-lib >>>> noarch 4.2.8.3-0.0.master.20190111080213.git7965c2e.el7 >>>> alocalsync 31 k >>>> >>>> Transaction Summary >>>> >>>> ================================================================================ >>>> Upgrade 12 Packages (+2 Dependent packages) >>>> >>>> Total download size: 827 k >>>> Downloading packages: >>>> Delta RPMs disabled because /usr/bin/applydeltarpm not installed. >>>> >>>> -------------------------------------------------------------------------------- >>>> Total 8.7 MB/s | 827 kB >>>> 00:00 >>>> Running transaction check >>>> Running transaction test >>>> Transaction test succeeded >>>> Running transaction >>>> >>>> 2019-01-14 10:38:39,733::ssh.py::ssh::96::lago.ssh::DEBUG::Command >>>> 8d4833e8 on lago-upgrade-from-release-suite-4-2-engine errors: >>>> Transaction couldn't start: >>>> file /usr/share/ansible/roles/oVirt.hosted-engine-setup from install >>>> of >>>> ovirt-ansible-hosted-engine-setup-1.0.6-0.1.master.20190114102323.el7.noarch >>>> conflicts with file from package >>>> ovirt-ansible-hosted-engine-setup-1.0.2-1.el7.noarch >>>> >>>> So: >>>> >>>> 1. Yum failed to update ovirt-ansible-hosted-engine-setup, likely >>>> because we (tried to) rename stuff there. Seems like we >>>> tried handling it, based on the comment in the spec file, but perhaps >>>> this didn't work. Ido? Adding also Simone. >>>> >>>> >>> This comes from here: >>> >>> https://docs.fedoraproject.org/en-US/packaging-guidelines/Directory_Repla... >>> >>> "Due to a known limitation with RPM, it is not possible to replace a >>> directory with any kind of file or symlink, nor is it possible to replace a >>> symlink to a directory with a directory, without RPM producing file >>> conflict errors while trying to install the package. For more information >>> on the issues involved, refer to bug 447156 and bug 646523." >>> >>> Due to galaxy 3.0 prescriptions, we should rename the role from from >>> oVirt.hosted-engine-setup in ovirt.hosted_engine_setup and we are trying to >>> keep a symlink there for backward compatibility. >>> The point is that rpm doesn't allow to replace a directory with a >>> symlink during the upgrade; we should have a workaround in place but this >>> is probably not enough. >>> Let's check what we can do on that. >>> >>> >>>> 2. We should probably patch OST to fail right after the failed >>>> yum update, no need to run also engine-setup and confuse people... >>>> Pushed this: https://gerrit.ovirt.org/96906 . >>>> >>>> Best regards, >>>> >>>> >> > >>>> >> > error: >>>> >> > >>>> >> > [ INFO ] Checking for an update for Setup... >>>> >> > An update for the Setup packages ovirt-engine-setup >>>> ovirt-engine-setup-plugin-websocket-proxy ovirt-engine-dwh-setup was found. >>>> Please update that package by running: >>>> >> > "yum update ovirt-engine-setup >>>> ovirt-engine-setup-plugin-websocket-proxy ovirt-engine-dwh-setup" >>>> >> > and then execute Setup again. >>>> >> > [ ERROR ] Failed to execute stage 'Environment customization': >>>> Please update the Setup packages >>>> >> > [ INFO ] Stage: Clean up >>>> >> > Log file is located at >>>> /var/log/ovirt-engine/setup/ovirt-engine-setup-20190114053840-4ysu6y.log >>>> >> > [ INFO ] Generating answer file >>>> '/var/lib/ovirt-engine/setup/answers/20190114053855-setup.conf' >>>> >> > [ INFO ] Stage: Pre-termination >>>> >> > [ INFO ] Stage: Termination >>>> >> > [ ERROR ] Execution of setup failed >>>> >> > ('FATAL Internal error (main): Please update the Setup packages',) >>>> >> > >>>> >> > lago.ssh: DEBUG: Command 8e6b18d0 on >>>> lago-upgrade-from-release-suite-4-2-engine errors: >>>> >> > Traceback (most recent call last): >>>> >> > File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line >>>> 88, in main >>>> >> > installer.execute() >>>> >> > File "/usr/lib/python2.7/site-packages/otopi/main.py", line >>>> 157, in execute >>>> >> > self.context.runSequence() >>>> >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line >>>> 771, in runSequence >>>> >> > util.raiseExceptionInformation(infos[0]) >>>> >> > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 81, >>>> in raiseExceptionInformation >>>> >> > exec('raise info[1], None, info[2]') >>>> >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line >>>> 133, in _executeMethod >>>> >> > method['method']() >>>> >> > File >>>> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-common/distro-rpm/packages.py", >>>> line 410, in _customization >>>> >> > raise RuntimeError(_('Please update the Setup packages')) >>>> >> > RuntimeError: Please update the Setup packages >>>> >> >>>> >> I think this is self-explanatory, no? >>>> >> >>>> >> Most likely this happens because we have a new engine and the >>>> appliance >>>> >> is not updated. If it's just a timing issue, should be solved >>>> without >>>> >> intervention on next appliance build. You (CI) might consider, >>>> though, >>>> >> forcing an appliance build and using it in each CQ batch that >>>> updates >>>> >> the engine. >>>> >> >>>> >> If it's something else, need to check appliance build status. >>>> >> >>>> >> Best regards, >>>> >> -- >>>> >> Didi >>>> >>>> >>>> >>>> -- >>>> Didi >>>> >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ITRH6TAWRYN... >>> >> > > -- > Martin Perina > Associate Manager, Software Engineering > Red Hat Czech s.r.o. > _______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/infra@ovirt.org/message/FCY2K3XRWQR... >