[ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ Aug. 8th 2017 ] [ 001_upgrade_engine.test_initialize_engine ]

Tue Aug 8 10:56:23 UTC 2017

There is something going on withe the artifacts logs, they are only
partially downloaded (I sent a mail to lago team to have a look).

Here is the link to the setup logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/

The error would appear in the ovirt-engine-setup-20170808040357-4iiirj.log:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170808040357-4iiirj.log

Hope this helps.

On 08/08/2017 01:30 PM, Yedidyah Bar David wrote:
> On Tue, Aug 8, 2017 at 12:27 PM, Yaniv Kaul <ykaul at redhat.com> wrote:
>>
>> On Tue, Aug 8, 2017 at 12:21 PM, Dafna Ron <dron at redhat.com> wrote:
>>> Hi,
>>>
>>> We see a sporadic failure in the upgrade test. from what I can see from
>>> the log it is related to the firewalD package.
>>>
>>> Test failed: 001_upgrade_engine.test_initialize_engine
>>>
>>> Link to suspected patches:
>>>
>>> Link to Job:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/
>>>
>>> Link to all logs:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/
> Downloaded all exported artifacts, unzipped, and:
>
>>>
>>> Error snippet from the log:
>>>
>>> <error>
>>>
>>>
>>>
>>> 2017-08-08 04:09:11,006-0400 DEBUG otopi.plugins.otopi.network.firewalld
>>> plugin.execute:926 execute-output: ('/bin/firewall-cmd', '--zone',
>>> u'public', '--permanent', '--add-service', 'ovirt-postgres') stderr:
>>> ESC[91mError: Action org.fedoraproject.FirewallD1.all is not
>>> registeredESC[00m
>>>
>>> 2017-08-08 04:09:11,007-0400 DEBUG otopi.context
>>> context._executeMethod:142 method exception
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
>>> _executeMethod
>>>     method['method']()
>>>   File "/usr/share/otopi/plugins/otopi/network/firewalld.py", line 334, in
>>> _closeup
>>>     '--add-service', service,
>>>   File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 931, in
>>> execute
>>>     command=args[0],
>>> RuntimeError: Command '/bin/firewall-cmd' failed to execute
>>>
>>> </error>
> can't find this error.
>
> In the console log [1] I see:
>
> 08:09:14 [upgrade-from-prevrelease-suit] Error occured, aborting
> 08:09:14 [upgrade-from-prevrelease-suit] Traceback (most recent call last):
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 360, in
> do_run
> 08:09:14 [upgrade-from-prevrelease-suit]
> self.cli_plugins[args.ovirtverb].do_run(args)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in
> do_run
> 08:09:14 [upgrade-from-prevrelease-suit]     self._do_run(**vars(args))
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper
> 08:09:14 [upgrade-from-prevrelease-suit]     return func(*args, **kwargs)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper
> 08:09:14 [upgrade-from-prevrelease-suit]     return func(*args,
> prefix=prefix, **kwargs)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in
> do_ovirt_runtest
> 08:09:14 [upgrade-from-prevrelease-suit]     raise RuntimeError('Some
> tests failed')
> 08:09:14 [upgrade-from-prevrelease-suit] RuntimeError: Some tests failed
>
> But it's hard to understand what failed.
>
> If I look around in the web interface, I can find this in [2].
>
> So for some reason, downloading "all files in zip" misses many files.
> Not sure why. IIRC we already had similar cases in the past.
> When I press "(all files in zip)" in [3], I get a 134016 bytes
> archive.zip, which only has inside it "basic-suit-master-el7" (and
> "JenkinsTestedChangeList.dat"), no "upgrade-from-prevrelease-suit-master-el7"
> or "upgrade-from-release-suit-master-el7". Please check.
>
> [1] http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/consoleFull
> [2] http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170808040357-4iiirj.log
> [3] http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/
>
>>>
>> Google shows we are not the only one suffering from it - see [1] and the
>> workarounds suggested.
> The workaround [4] seems to have been to restart polkitd.
> Do we want to do this?
> Where? If in otopi, I'd consider this a somewhat ugly hack, and
> would rather not, unless we have a clear and realistic reproducer.
> If in CI, not sure where/when exactly.
>
> Best,
>
> [4] https://github.com/openshift/openshift-ansible/pull/3831/commits/457605bc7b8ba57b653eb14491f988a5565e3252
>
>> Y.
>> [1] https://github.com/openshift/openshift-ansible/issues/3213
>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>>
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>
>