On Tue, Jun 20, 2017 at 10:28 AM, Eyal Edri <eedri@redhat.com> wrote:

On Tue, Jun 20, 2017 at 9:56 AM, Yaniv Kaul <ykaul@redhat.com> wrote:

On Mon, Jun 19, 2017 at 4:48 PM, Gil Shinar <gshinar@redhat.com> wrote:
Test failed: 002_bootstrap.add_secondary_storage_domains
Link to suspected patches:
Link to Job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7264
Link to all logs: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7264/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/

We have an open bug on this, the mount to the ISO domain gets stuck and never returns:
call mount with (u'192.168.201.3:/exports/nfs/iso', u'/rhev/data-center/mnt/192.168.201.3:_exports_nfs_iso') {'vfstype': 'nfs', 'mntOpts': 'soft,nosharecache,timeo=600,retrans=6,nfsvers=3', 'timeout': None, 'cgroup': None}

We'll need probably a live system where it reproduces to see where the mount command gets stuck.
Sounds like an OS issue to me.

Its a bit problematic since its not failing all the time, what we can do is export the ENV once its fails so we can import it offline and debug it.
We are planning to release a new Lago version next week with some fixes that hopefully will allow this option.

BTW, since we are not using the ISO domain, perhaps we should remove its creation. It'll save time.
(note however that this is the only NFSv3 mount we perform though).

Should we open a bug / ticket on it? who can take ownership on modifying the test?

There's an open bug about the issue itself. I'm just too lazy to find it.

We can open a bug on removing the ISO domain creation, since we don't use it. I'd like us to switch one of the other domains to NFSv3 if possible, though.

Error snippet from the log:
Host 0:
2017-06-19 08:46:48,457-0400 WARN  (jsonrpc/1) [storage.HSM] getPV failed for guid: 360014050f775dd654404bd39d061693c (hsm:1928)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 1925, in _getDeviceList
    pv = lvm.getPV(guid)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 852, in getPV
    raise se.InaccessiblePhysDev((pvName,))
InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'360014050f775dd654404bd39d061693c',)"
2017-06-19 08:46:48,546-0400 WARN  (jsonrpc/1) [storage.LVM] lvm pvs failed: 5 [] ['  Failed to find physical volume "/dev/mapper/360014050fae4a3a8e8047e6933531876".'] (lvm:322)
2017-06-19 08:46:48,546-0400 WARN  (jsonrpc/1) [storage.HSM] getPV failed for guid: 360014050fae4a3a8e8047e6933531876 (hsm:1928)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 1925, in _getDeviceList
    pv = lvm.getPV(guid)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 852, in getPV
    raise se.InaccessiblePhysDev((pvName,))
InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'360014050fae4a3a8e8047e6933531876',)"

Host 1:

2017-06-19 08:49:03,151-0400 ERROR (monitor/b92f872) [storage.Monitor] Setting up monitor for b92f8727-8c51-4e12-a3fe-5441531c13ca failed (monitor:329)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/monitor.py", line 326, in _setupLoop
    self._setupMonitor()
  File "/usr/share/vdsm/storage/monitor.py", line 348, in _setupMonitor
    self._produceDomain()
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 237, in wrapper
    value = meth(self, *a, **kw)
  File "/usr/share/vdsm/storage/monitor.py", line 366, in _produceDomain
    self.domain = sdCache.produce(self.sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 108, in produce
    domain.getRealDomain()
  File "/usr/share/vdsm/storage/sdc.py", line 49, in getRealDomain
    return self._cache._realProduce(self._sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 132, in _realProduce
    domain = self._findDomain(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 149, in _findDomain
    return findMethod(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 174, in _findUnfetchedDomain
    raise se.StorageDomainDoesNotExist(sdUUID)
StorageDomainDoesNotExist: Storage domain does not exist: (u'b92f8727-8c51-4e12-a3fe-5441531c13ca',)

_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Eyal edri

ASSOCIATE MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D

Red Hat EMEA

TRIED. TESTED. TRUSTED.

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)