OST is failing - Last successful run was Dec-13-2019

Seems we have NFS permissions issue for el8 vdsm in some of the runs. Example from https://jenkins.ovirt.org/view/Amit/job/ovirt-system-tests_manual/6302/artif... : 2020-01-03 12:07:34,169-0500 INFO (MainThread) [vds] (PID: 1264) I am the actual vdsm 4.40.0.1458.git1fca84350 lago-basic-suite-master-host-1 (4.18.0-80.11.2.el8_0.x86_64) (vdsmd:152)... 2020-01-03 12:50:29,662-0500 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata (monitor:501) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 499, in _pathChecked delay = result.delay() File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line 391, in delay raise exception.MiscFileReadException(self.path, self.rc, self.err) vdsm.storage.exception.MiscFileReadException: Internal file read failure: ('/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata', 1, bytearray(b"/usr/bin/dd: failed to open \'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata\': Operation not permitted\n")) 2020-01-03 12:50:30,112-0500 DEBUG (jsonrpc/7) [jsonrpc.JsonRpcServer] Calling 'StoragePool.disconnect' in bridge with {'storagepoolID': 'c90b137f-6e1f-4b9a-9612-da58910a2439', 'hostID': 2, 'scsiKey': 'c90b137f-6e1f-4b9a-9612-da58910a2439'} (__init__:329) 2020-01-03 12:50:30,114-0500 INFO (jsonrpc/7) [vdsm.api] START disconnectStoragePool(spUUID='c90b137f-6e1f-4b9a-9612-da58910a2439', hostID=2, remove=False, options=None) from=::ffff:192.168.201.4,38786, flow_id=8d05a1, task_id=95573498-d1c7-41ad-ad33-28f2192b2b60 (api:48) Probably need to set NFS server export options as in https://bugzilla.redhat.com/show_bug.cgi?id=1776843#c7

On Sun, Jan 5, 2020 at 10:08 AM Amit Bawer <abawer@redhat.com> wrote:
Seems we have NFS permissions issue for el8 vdsm in some of the runs.
Example from https://jenkins.ovirt.org/view/Amit/job/ovirt-system-tests_manual/6302/artif... :
2020-01-03 12:07:34,169-0500 INFO (MainThread) [vds] (PID: 1264) I am the actual vdsm 4.40.0.1458.git1fca84350 lago-basic-suite-master-host-1 (4.18.0-80.11.2.el8_0.x86_64) (vdsmd:152)... 2020-01-03 12:50:29,662-0500 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata (monitor:501) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 499, in _pathChecked delay = result.delay() File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line 391, in delay raise exception.MiscFileReadException(self.path, self.rc, self.err) vdsm.storage.exception.MiscFileReadException: Internal file read failure: ('/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata', 1, bytearray(b"/usr/bin/dd: failed to open \'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata\': Operation not permitted\n")) 2020-01-03 12:50:30,112-0500 DEBUG (jsonrpc/7) [jsonrpc.JsonRpcServer] Calling 'StoragePool.disconnect' in bridge with {'storagepoolID': 'c90b137f-6e1f-4b9a-9612-da58910a2439', 'hostID': 2, 'scsiKey': 'c90b137f-6e1f-4b9a-9612-da58910a2439'} (__init__:329) 2020-01-03 12:50:30,114-0500 INFO (jsonrpc/7) [vdsm.api] START disconnectStoragePool(spUUID='c90b137f-6e1f-4b9a-9612-da58910a2439', hostID=2, remove=False, options=None) from=::ffff:192.168.201.4,38786, flow_id=8d05a1, task_id=95573498-d1c7-41ad-ad33-28f2192b2b60 (api:48)
Probably need to set NFS server export options as in https://bugzilla.redhat.com/show_bug.cgi?id=1776843#c7
Here is fix for NFS server on EL8: https://gerrit.ovirt.org/106120 Should this be changes also for NFS server on EL7? _______________________________________________
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WLLVWPZU6HFDWV...
-- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.

AF On Mon, Jan 6, 2020 at 11:48 AM Martin Perina <mperina@redhat.com> wrote:
On Sun, Jan 5, 2020 at 10:08 AM Amit Bawer <abawer@redhat.com> wrote:
Seems we have NFS permissions issue for el8 vdsm in some of the runs.
Example from https://jenkins.ovirt.org/view/Amit/job/ovirt-system-tests_manual/6302/artif... :
2020-01-03 12:07:34,169-0500 INFO (MainThread) [vds] (PID: 1264) I am the actual vdsm 4.40.0.1458.git1fca84350 lago-basic-suite-master-host-1 (4.18.0-80.11.2.el8_0.x86_64) (vdsmd:152)... 2020-01-03 12:50:29,662-0500 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata (monitor:501) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 499, in _pathChecked delay = result.delay() File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line 391, in delay raise exception.MiscFileReadException(self.path, self.rc, self.err) vdsm.storage.exception.MiscFileReadException: Internal file read failure: ('/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata', 1, bytearray(b"/usr/bin/dd: failed to open \'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_exported/b92b26cf-fac4-4ccf-ba31-f6fb4184e302/dom_md/metadata\': Operation not permitted\n")) 2020-01-03 12:50:30,112-0500 DEBUG (jsonrpc/7) [jsonrpc.JsonRpcServer] Calling 'StoragePool.disconnect' in bridge with {'storagepoolID': 'c90b137f-6e1f-4b9a-9612-da58910a2439', 'hostID': 2, 'scsiKey': 'c90b137f-6e1f-4b9a-9612-da58910a2439'} (__init__:329) 2020-01-03 12:50:30,114-0500 INFO (jsonrpc/7) [vdsm.api] START disconnectStoragePool(spUUID='c90b137f-6e1f-4b9a-9612-da58910a2439', hostID=2, remove=False, options=None) from=::ffff:192.168.201.4,38786, flow_id=8d05a1, task_id=95573498-d1c7-41ad-ad33-28f2192b2b60 (api:48)
Probably need to set NFS server export options as in https://bugzilla.redhat.com/show_bug.cgi?id=1776843#c7
Here is fix for NFS server on EL8: https://gerrit.ovirt.org/106120
Should this be changes also for NFS server on EL7?
AFAICT the need to change the NFS options mostly arises from changes in vdsm dependencies for el8, such as libvirt, requiring a different access for NFS shares than before. So if we are testing el8 hosts with el7 NFS server that might be relvent for el7 NFS server as well.
_______________________________________________
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WLLVWPZU6HFDWV...
-- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.
participants (2)
-
Amit Bawer
-
Martin Perina