Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ bootstrap.verify_add_hosts ] [ 18/10/17 ]

So the real issue on adding a host is the same as I've today described in [2] and most probably caused by [3] (I reverted engine in my dev env prior this patch and host deploy finished successfully). Allon, do you have time to post a fix? If not I'll try to dig into your change and related networking code to post it ... [2] https://bugzilla.redhat.com/show_bug.cgi?id=1504005 [3] https://gerrit.ovirt.org/#/c/82545/ On Thu, Oct 19, 2017 at 11:12 AM, Martin Perina <mperina(a)redhat.com&gt; wrote: > > > On Thu, Oct 19, 2017 at 11:04 AM, Martin Perina <mperina(a)redhat.com&gt; > wrote: > >> >> >> On Thu, Oct 19, 2017 at 10:58 AM, Dan Kenigsberg <danken(a)redhat.com&gt; >> wrote: >> >>> On Thu, Oct 19, 2017 at 10:29 AM, Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> > >>> > >>> > On Thu, Oct 19, 2017 at 7:35 AM, Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> >> >>> >> On Wed, Oct 18, 2017 at 2:40 PM, Daniel Belenky <dbelenky(a)redhat.com >>> > >>> >> wrote: >>> >>> >>> >>> Hi all, >>> >>> >>> >>> The following test is failing: 002_bootstrap.verify_add_hosts >>> >>> All logs from failing job >>> >>> Only 2 engine patches participated in the test, so the suspected >>> patches >>> >>> are: >>> >>> >>> >>> https://gerrit.ovirt.org/#/c/82542/2 >>> >>> https://gerrit.ovirt.org/#/c/82545/3 >>> >>> >>> >>> Due to the fact that when this error first introduced we had another >>> >>> error, the CI can't automatically detect the specific patch. >>> >>> >>> >>> Error snippet from logs: ovirt-host-deploy-ansible log (Full log) >>> >>> >>> >>> TASK [ovirt-host-deploy-firewalld : Enable firewalld rules] >>> >>> ******************** >>> >>> failed: [lago-basic-suite-master-host-0] (item={u'service': >>> >>> u'glusterfs'}) => {"changed": false, "failed": true, "item": >>> {"service": >>> >>> "glusterfs"}, "msg": "ERROR: Exception caught: >>> >>> org.fedoraproject.FirewallD1.Exception: INVALID_SERVICE: >>> 'glusterfs' not >>> >>> among existing services Permanent and Non-Permanent(immediate) >>> operation, >>> >>> Services are defined by port/tcp relationship and named as they are >>> in >>> >>> /etc/services (on most systems)"} >>> >>> >>> >>> >>> >>> Error from HOST 0 firewalld log: >>> >>> lago-basic-suite-master-host-0/_var_log/firewalld/ (Full log) >>> >>> >>> >>> 2017-10-15 16:51:24 ERROR: INVALID_SERVICE: 'glusterfs' not among >>> >>> existing services >>> >> >>> >> >>> >> Ondra, would such an error propagate through the playbook to Engine >>> and >>> >> fail the add-host flow? (I think it should!) >>> > >>> > >>> > We didn't do that so far, because of EL 7.3 >>> > . We need firewalld from 7.4 to have all available services in place >>> (I >>> > don't remember but I think imageio service was the one delivered only >>> in >>> > firewalld from 7.4). So up until now we ingore non-existent firewalld >>> > service, but if needed we can turn this on and fail host deploy. >>> >>> Ok, so for now your "luckily" off the hook and not the reason of >>> failure. >>> >>> >> >>> >> >>> >> Do you know which package provide the glusterfs firewalld service, >>> and why >>> >> it is missing from the host? >>> > >>> > >>> > So we have used 'glusterfs' firewalld service per Sahina >>> recommendation, >>> > which is included in glusterfs-server package from version 3.7.6 [1]. >>> But >>> > this package is not installed when installing packages for cluster >>> with >>> > gluster capabilities enabled. So now I'm confused: don't we need >>> > glusterfs-server package? If not and we need those ports open because >>> they >>> > are used by services from different already installed glusterfs >>> packages, >>> > shouldn't the firewalld configuration be moved from glusterfs-server >>> to >>> > glusterfs package? >>> >>> glusterfs-cli.rpm is required to consume gluster storage (virt use >>> case), but I don't recall that it needs open ports. >>> >> >> ​It was there even for IPTables, if gluster support is enabled on >> cluster, then gluster specific ports were enabled even with IPTables. >> FirewallD feature continues to use that. >> ​ >> >> >>> glusterfs-server.rpm is required to provide gluster storage (gluster >>> use case). >>> If I recall correctly, firewalld feature has differentiated between >>> the two; opening needed ports only when relevant. >>> >> >> ​Right, but if gluster services are configured for firewalld it means >> that the host has been added to the cluster with gluster feature enabled >> and not only virt >> ​ >> >> >>> >>> > >>> > >>> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1057295 >>> > >>> > >>> >> >> >