Could not get access to ACL tech driver 'ebiptables'
From /var/log/vdsm/vdsm.log on the host where it tried to start: Thread-23385::ERROR::2016-11-19 02:12:41,907::vm::765::virt.vm::(_startUnderlyingVm) vmId=`c3125d32-ae2a-4d2f-af4c-13661d90ddf9`::*The vm start process failed* Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 706, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 1996, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 917, in wrapper return func(inst, *args, **kwargs) File "*/usr/lib64/python2.7/site-packages/libvirt.py*", line 3611, in *createXML* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
I imported a guest from its iscsi storage domain and clicked the green UP button, but the guest failed to start. This was the first time vdsm tried to create a temporary storage domain for a host other than hosted_engine. I'm using the same chap credential that was used with the same iscsi storage domain with the old instance of Manager. It looks like it wasn't able to get permission to do something, but everything was set up with sudo-as-root. I used CentOS 7.2 with the ovirt-release repository and hosted-engine script to deploy Manager. libvirtError: internal error: *Could not get access to ACL tech driver 'ebiptables'*
From /usr/lib64/python2.7/site-packages/libvirt.py which raised the error: def createXML(self, xmlDesc, flags=0): """Launch a new guest domain, based on an XML description similar to the one returned by virDomainGetXMLDesc() This function may require privileged access to the hypervisor. The domain is not persistent, so its definition will disappear when it is destroyed, or if the host is restarted (see virDomainDefineXML() to define persistent domains).
If the VIR_DOMAIN_START_PAUSED flag is set, the guest domain will be started, but its CPUs will remain paused. The CPUs can later be manually started using virDomainResume. If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest domain will be automatically destroyed when the virConnectPtr object is finally released. This will also happen if the client application crashes / loses its connection to the libvirtd daemon. Any domains marked for auto destroy will block attempts at migration, save-to-file, or snapshots. virDomainFree should be used to free the resources after the domain object is no longer needed. """ ret = libvirtmod.virDomainCreateXML(self._o, xmlDesc, flags) if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) __tmp = virDomain(self,_obj=ret) return __tmp
I suspect this has something to do with macspoofing because I found that I was able to start a guest by changing the virtual network interface profile to remove network filtering. I verified the guests are able to start with filtering enabled on the vnic profile if it is set to false in engine-config and ovirt-engine service bounced. I'd prefer to leaf macspoofing disabled globally and only enable for things like VRR, CARP; but I'll have to leave it enabled for now. Could it be that the macs of the imported guests, being from the foreign mac pool, are being blocked by an ebtables policy? I wonder if I add their Ethernet range to the pools of the gaining Manager... On Fri, Nov 18, 2016 at 9:35 PM Kenneth Bingham <w@qrk.us> wrote:
I imported a guest from its iscsi storage domain and clicked the green UP button, but the guest failed to start. This was the first time vdsm tried to create a temporary storage domain for a host other than hosted_engine. I'm using the same chap credential that was used with the same iscsi storage domain with the old instance of Manager. It looks like it wasn't able to get permission to do something, but everything was set up with sudo-as-root. I used CentOS 7.2 with the ovirt-release repository and hosted-engine script to deploy Manager.
From /var/log/vdsm/vdsm.log on the host where it tried to start: Thread-23385::ERROR::2016-11-19 02:12:41,907::vm::765::virt.vm::(_startUnderlyingVm) vmId=`c3125d32-ae2a-4d2f-af4c-13661d90ddf9`::*The vm start process failed* Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 706, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 1996, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 917, in wrapper return func(inst, *args, **kwargs) File "*/usr/lib64/python2.7/site-packages/libvirt.py*", line 3611, in *createXML* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: *Could not get access to ACL tech driver 'ebiptables'*
From /usr/lib64/python2.7/site-packages/libvirt.py which raised the error: def createXML(self, xmlDesc, flags=0): """Launch a new guest domain, based on an XML description similar to the one returned by virDomainGetXMLDesc() This function may require privileged access to the hypervisor. The domain is not persistent, so its definition will disappear when it is destroyed, or if the host is restarted (see virDomainDefineXML() to define persistent domains).
If the VIR_DOMAIN_START_PAUSED flag is set, the guest domain will be started, but its CPUs will remain paused. The CPUs can later be manually started using virDomainResume.
If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest domain will be automatically destroyed when the virConnectPtr object is finally released. This will also happen if the client application crashes / loses its connection to the libvirtd daemon. Any domains marked for auto destroy will block attempts at migration, save-to-file, or snapshots.
virDomainFree should be used to free the resources after the domain object is no longer needed. """ ret = libvirtmod.virDomainCreateXML(self._o, xmlDesc, flags) if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) __tmp = virDomain(self,_obj=ret) return __tmp
On Sat, Nov 19, 2016 at 7:35 AM, Kenneth Bingham <w@qrk.us> wrote:
I suspect this has something to do with macspoofing because I found that I was able to start a guest by changing the virtual network interface profile to remove network filtering. I verified the guests are able to start with filtering enabled on the vnic profile if it is set to false in engine-config and ovirt-engine service bounced. I'd prefer to leaf macspoofing disabled globally and only enable for things like VRR, CARP; but I'll have to leave it enabled for now. Could it be that the macs of the imported guests, being from the foreign mac pool, are being blocked by an ebtables policy? I wonder if I add their Ethernet range to the pools of the gaining Manager...
On Fri, Nov 18, 2016 at 9:35 PM Kenneth Bingham <w@qrk.us> wrote:
I imported a guest from its iscsi storage domain and clicked the green UP button, but the guest failed to start. This was the first time vdsm tried to create a temporary storage domain for a host other than hosted_engine. I'm using the same chap credential that was used with the same iscsi storage domain with the old instance of Manager. It looks like it wasn't able to get permission to do something, but everything was set up with sudo-as-root. I used CentOS 7.2 with the ovirt-release repository and hosted-engine script to deploy Manager.
From /var/log/vdsm/vdsm.log on the host where it tried to start: Thread-23385::ERROR::2016-11-19 02:12:41,907::vm::765::virt.vm::(_startUnderlyingVm) vmId=`c3125d32-ae2a-4d2f-af4c-13661d90ddf9`::*The vm start process failed* Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 706, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 1996, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 917, in wrapper return func(inst, *args, **kwargs) File "*/usr/lib64/python2.7/site-packages/libvirt.py*", line 3611, in *createXML* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: *Could not get access to ACL tech driver 'ebiptables'*
From /usr/lib64/python2.7/site-packages/libvirt.py which raised the error: def createXML(self, xmlDesc, flags=0): """Launch a new guest domain, based on an XML description similar to the one returned by virDomainGetXMLDesc() This function may require privileged access to the hypervisor. The domain is not persistent, so its definition will disappear when it is destroyed, or if the host is restarted (see virDomainDefineXML() to define persistent domains).
If the VIR_DOMAIN_START_PAUSED flag is set, the guest domain will be started, but its CPUs will remain paused. The CPUs can later be manually started using virDomainResume.
If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest domain will be automatically destroyed when the virConnectPtr object is finally released. This will also happen if the client application crashes / loses its connection to the libvirtd daemon. Any domains marked for auto destroy will block attempts at migration, save-to-file, or snapshots.
virDomainFree should be used to free the resources after the domain object is no longer needed. """ ret = libvirtmod.virDomainCreateXML(self._o, xmlDesc, flags) if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) __tmp = virDomain(self,_obj=ret) return __tmp
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Please include the Kernel, libvirt and ebtables versions you run with. In addition, the logs from libvirt and vdsm (vdsm.log, supervdsm.log) would help. Laine, Thomas, can this be related to https://bugzilla.redhat.com/show_bug.cgi?id=1396032 ?
On 11/20/2016 03:40 AM, Edward Haas wrote:
On Sat, Nov 19, 2016 at 7:35 AM, Kenneth Bingham <w@qrk.us <mailto:w@qrk.us>> wrote:
I suspect this has something to do with macspoofing because I found that I was able to start a guest by changing the virtual network interface profile to remove network filtering. I verified the guests are able to start with filtering enabled on the vnic profile if it is set to false in engine-config and ovirt-engine service bounced. I'd prefer to leaf macspoofing disabled globally and only enable for things like VRR, CARP; but I'll have to leave it enabled for now. Could it be that the macs of the imported guests, being from the foreign mac pool, are being blocked by an ebtables policy? I wonder if I add their Ethernet range to the pools of the gaining Manager...
On Fri, Nov 18, 2016 at 9:35 PM Kenneth Bingham <w@qrk.us <mailto:w@qrk.us>> wrote:
I imported a guest from its iscsi storage domain and clicked the green UP button, but the guest failed to start. This was the first time vdsm tried to create a temporary storage domain for a host other than hosted_engine. I'm using the same chap credential that was used with the same iscsi storage domain with the old instance of Manager. It looks like it wasn't able to get permission to do something, but everything was set up with sudo-as-root. I used CentOS 7.2 with the ovirt-release repository and hosted-engine script to deploy Manager.
From /var/log/vdsm/vdsm.log on the host where it tried to start: Thread-23385::ERROR::2016-11-19 02:12:41,907::vm::765::virt.vm::(_startUnderlyingVm) vmId=`c3125d32-ae2a-4d2f-af4c-13661d90ddf9`::*The vm start process failed* Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 706, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 1996, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 917, in wrapper return func(inst, *args, **kwargs) File "*/usr/lib64/python2.7/site-packages/libvirt.py*", line 3611, in *createXML* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: *Could not get access to ACL tech driver 'ebiptables'*
From /usr/lib64/python2.7/site-packages/libvirt.py which raised the error: def createXML(self, xmlDesc, flags=0): """Launch a new guest domain, based on an XML description similar to the one returned by virDomainGetXMLDesc() This function may require privileged access to the hypervisor. The domain is not persistent, so its definition will disappear when it is destroyed, or if the host is restarted (see virDomainDefineXML() to define persistent domains).
If the VIR_DOMAIN_START_PAUSED flag is set, the guest domain will be started, but its CPUs will remain paused. The CPUs can later be manually started using virDomainResume.
If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest domain will be automatically destroyed when the virConnectPtr object is finally released. This will also happen if the client application crashes / loses its connection to the libvirtd daemon. Any domains marked for auto destroy will block attempts at migration, save-to-file, or snapshots.
virDomainFree should be used to free the resources after the domain object is no longer needed. """ ret = libvirtmod.virDomainCreateXML(self._o, xmlDesc, flags) if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) __tmp = virDomain(self,_obj=ret) return __tmp
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
Please include the Kernel, libvirt and ebtables versions you run with. In addition, the logs from libvirt and vdsm (vdsm.log, supervdsm.log) would help.
Laine, Thomas, can this be related to https://bugzilla.redhat.com/show_bug.cgi?id=1396032 ?
I doubt it. The end result is the same (guest fails to start if nwfilter rules are in place, and error message mentions ebtables), but the error is different and means that the initial failure was different. In the BZ you reference, libvirt found both ebtables and iptables and initizlized its internal driver, but when it attempted to add a rule to the ebtables "nat' table, this failed because the kernel module ebtable_nat wasn't autoloaded as it should have been. The error message above indicates that libvirt couldn't even properly initialize its internal driver that it uses to add ebtables and iptables rules. You should restart libvirtd and look for any errors or warnings in the log during libvirtd's startup. Possibly you've somehow uninstalled the package(s) containing the iptables, ip6tables, or ebtables commands. Or possibly you have alternate builds of those packages that have some things disabled (e.g. ipv6). It's also possible that you had firewalld running when you first started libvirtd (so that libvirt's driver chose to use dbus calls to firewalld rather than executing iptables or ebtables commands), but then later stopped firewalld; I *think* the error would be different in that case though (too tired to check for myself right now :-)
participants (3)
-
Edward Haas -
Kenneth Bingham -
Laine Stump