[ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ bootstrap.verify_add_hosts ] [ 18/10/17 ]

Thu Oct 19 14:48:32 UTC 2017

Fix merged based on Alona and Martin's reviews.
It seems to do the trick with my testing on my local engine, let's hope
that's really it.

On Thu, Oct 19, 2017 at 4:37 PM, Allon Mureinik <amureini at redhat.com> wrote:

> Bloody hell. The original was also completely broken, and worked by
> chance. Damn it.
>
> This should fix it:
> https://gerrit.ovirt.org/#/c/82989/
>
> On Thu, Oct 19, 2017 at 3:49 PM, Martin Perina <mperina at redhat.com> wrote:
>
>> So the real issue on adding a host is the same as I've today described in
>> [2] and most probably caused by [3] (I reverted engine in my dev env prior
>> this patch and host deploy finished successfully).
>>
>> Allon, do you have time to post a fix? If not I'll try to dig into your
>> change and related networking code to post it ...
>>
>>
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1504005
>> [3] https://gerrit.ovirt.org/#/c/82545/
>>
>> On Thu, Oct 19, 2017 at 11:12 AM, Martin Perina <mperina at redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Oct 19, 2017 at 11:04 AM, Martin Perina <mperina at redhat.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Oct 19, 2017 at 10:58 AM, Dan Kenigsberg <danken at redhat.com>
>>>> wrote:
>>>>
>>>>> On Thu, Oct 19, 2017 at 10:29 AM, Martin Perina <mperina at redhat.com>
>>>>> wrote:
>>>>> >
>>>>> >
>>>>> > On Thu, Oct 19, 2017 at 7:35 AM, Dan Kenigsberg <danken at redhat.com>
>>>>> wrote:
>>>>> >>
>>>>> >> On Wed, Oct 18, 2017 at 2:40 PM, Daniel Belenky <
>>>>> dbelenky at redhat.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Hi all,
>>>>> >>>
>>>>> >>> The following test is failing: 002_bootstrap.verify_add_hosts
>>>>> >>> All logs from failing job
>>>>> >>> Only 2 engine patches participated in the test, so the suspected
>>>>> patches
>>>>> >>> are:
>>>>> >>>
>>>>> >>> https://gerrit.ovirt.org/#/c/82542/2
>>>>> >>> https://gerrit.ovirt.org/#/c/82545/3
>>>>> >>>
>>>>> >>> Due to the fact that when this error first introduced we had
>>>>> another
>>>>> >>> error, the CI can't automatically detect the specific patch.
>>>>> >>>
>>>>> >>> Error snippet from logs: ovirt-host-deploy-ansible log (Full log)
>>>>> >>>
>>>>> >>> TASK [ovirt-host-deploy-firewalld : Enable firewalld rules]
>>>>> >>> ********************
>>>>> >>> failed: [lago-basic-suite-master-host-0] (item={u'service':
>>>>> >>> u'glusterfs'}) => {"changed": false, "failed": true, "item":
>>>>> {"service":
>>>>> >>> "glusterfs"}, "msg": "ERROR: Exception caught:
>>>>> >>> org.fedoraproject.FirewallD1.Exception: INVALID_SERVICE:
>>>>> 'glusterfs' not
>>>>> >>> among existing services Permanent and Non-Permanent(immediate)
>>>>> operation,
>>>>> >>> Services are defined by port/tcp relationship and named as they
>>>>> are in
>>>>> >>> /etc/services (on most systems)"}
>>>>> >>>
>>>>> >>>
>>>>> >>> Error from HOST 0 firewalld log:
>>>>> >>> lago-basic-suite-master-host-0/_var_log/firewalld/ (Full log)
>>>>> >>>
>>>>> >>> 2017-10-15 16:51:24 ERROR: INVALID_SERVICE: 'glusterfs' not among
>>>>> >>> existing services
>>>>> >>
>>>>> >>
>>>>> >> Ondra, would such an error propagate through the playbook to Engine
>>>>> and
>>>>> >> fail the add-host flow? (I think it should!)
>>>>> >
>>>>> >
>>>>> > We didn't do that so far, because of EL 7.3
>>>>> > . We need firewalld from 7.4 to have all available services in place
>>>>> (I
>>>>> > don't remember but I think imageio service was the one delivered
>>>>> only in
>>>>> > firewalld from 7.4). So  up until now we ingore non-existent
>>>>> firewalld
>>>>> > service, but if needed we can turn this on and fail host deploy.
>>>>>
>>>>> Ok, so for now your "luckily" off the hook and not the reason of
>>>>> failure.
>>>>>
>>>>> >>
>>>>> >>
>>>>> >> Do you know which package provide the glusterfs firewalld service,
>>>>> and why
>>>>> >> it is missing from the host?
>>>>> >
>>>>> >
>>>>> > So we have used 'glusterfs' firewalld service per Sahina
>>>>> recommendation,
>>>>> > which is included in glusterfs-server package from version 3.7.6
>>>>> [1]. But
>>>>> > this package is not installed when installing packages for cluster
>>>>> with
>>>>> > gluster capabilities enabled. So now I'm confused: don't we need
>>>>> > glusterfs-server package? If not and we need those ports open
>>>>> because they
>>>>> > are used by services from different already installed glusterfs
>>>>> packages,
>>>>> > shouldn't the firewalld configuration be moved from glusterfs-server
>>>>> to
>>>>> > glusterfs package?
>>>>>
>>>>> glusterfs-cli.rpm is required to consume gluster storage (virt use
>>>>> case), but I don't recall that it needs open ports.
>>>>>
>>>>
>>>> It was there even for IPTables, if gluster support is enabled on
>>>> cluster, then gluster specific ports were enabled even with IPTables.
>>>> FirewallD feature continues to use that.
>>>> 
>>>>
>>>>
>>>>> glusterfs-server.rpm is required to provide gluster storage (gluster
>>>>> use case).
>>>>> If I recall correctly, firewalld feature has differentiated between
>>>>> the two; opening needed ports only when relevant.
>>>>>
>>>>
>>>> Right, but if gluster services are configured for firewalld it means
>>>> that the host has been added to the cluster with gluster feature enabled
>>>> and not only virt
>>>> 
>>>>
>>>>
>>>>>
>>>>> >
>>>>> >
>>>>> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1057295
>>>>> >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20171019/36bc3396/attachment-0001.html>