[ovirt-users] HE setup failing: Cannot set temporary password for console connection

Wed May 13 07:29:24 UTC 2015

Hello Didi,
this is a clean install of the first host. (I am trying a disaster scenario and will later restore the backup)

--
Daniel Helgenberger
m box bewegtbild GmbH

P: +49/30/2408781-22<tel:+49/30/2408781-22>
F: +49/30/2408781-10<tel:+49/30/2408781-10>

ACKERSTR. 19
D-10115 BERLIN

www.m-box.de<http://www.m-box.de> www.monkeymen.tv<http://www.monkeymen.tv>

Geschäftsführer: Martin Retschitzegger / Michaela Göllner
Handeslregister: Amtsgericht Charlottenburg / HRB 112767

On Wed, May 13, 2015 at 12:22 AM -0700, "Yedidyah Bar David" <didi at redhat.com<mailto:didi at redhat.com>> wrote:

----- Original Message -----
> From: "Daniel Helgenberger" <daniel.helgenberger at m-box.de>
> To: users at ovirt.org
> Sent: Tuesday, May 12, 2015 8:48:42 PM
> Subject: [ovirt-users] HE setup failing: Cannot set temporary password for console connection
>
> Hello,
>
> a week ago I set up a HE (3.5.2) with no problems; now this fails.

You mean you re-deploy the same host (clean setup), or adding another host?

> I
> suspect storage trouble but cannot pinpoint it. Did someth. change there
> with the recent async release?
>
> The storage domain is NFS3 via a 1G link to a ZFS storage appliance. The
> export is working quite fast; tested with iozone running for over 24h
> without problems.
>
> It starts with the "Verifying sanlock lockspace initialization" phase;
> which lasts about 7min (!).
>
> In the end it seems sanlock is locking the domain down. I can only get
> it back by stopping sanlock and vdsm.
>
> Attached is the full vdsm lock.
>
>
> Thanks!
>
> vdsm:
>
> Thread-27::ERROR::2015-05-12
> 19:17:51,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain)
> Error while collecting domain 2f83ea82-e110-465d-a2a6-ac30534a9f6a
> monitoring information
> Traceback (most recent call last):
>   File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in
>   _monitorDomain
>     self.domain.selftest()
>   File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__
>     return getattr(self.getRealDomain(), attrName)
>   File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>     return self._cache._realProduce(self._sdUUID)
>   File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>     domain = self._findDomain(sdUUID)
>   File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>     dom = findMethod(sdUUID)
>   File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain
>     return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID))
>   File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath
>     raise se.StorageDomainDoesNotExist(sdUUID)
> StorageDomainDoesNotExist: Storage domain does not exist:
> ('2f83ea82-e110-465d-a2a6-ac30534a9f6a',)
>
>
> Detector thread::DEBUG::2015-05-12
> 19:24:33,163::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http
> detected from ('127.0.0.1', 47076)
> Thread-93::DEBUG::2015-05-12
> 19:24:33,166::BindingXMLRPC::1133::vds::(wrapper) client [127.0.0.1]::call
> vmSetTicket with ('e3d350c5-6254-4908-99e9-d60ca2e046f7', '5435uOVP',
> '10800', 'disconnect', {}) {}
> Thread-93::ERROR::2015-05-12
> 19:24:33,166::BindingXMLRPC::1152::vds::(wrapper) unexpected error
> Traceback (most recent call last):
>   File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1136, in wrapper
>     res = f(*args, **kwargs)
>   File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 383, in vmSetTicket
>     return vm.setTicket(password, ttl, existingConnAction, params)
>   File "/usr/share/vdsm/API.py", line 642, in setTicket
>     return v.setTicket(password, ttl, existingConnAction, params)
>   File "/usr/share/vdsm/virt/vm.py", line 4834, in setTicket
>     graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \
> AttributeError: 'NoneType' object has no attribute 'XMLDesc'
>
> and:
>
> Thread-91::DEBUG::2015-05-12
> 19:25:15,152::libvirtconnection::143::root::(wrapper) Unknown libvirterror:
> ecode: 38 edom: 42 level: 2 message: Failed to acquire lock: No space left
> on device
> Thread-91::DEBUG::2015-05-12
> 19:25:15,154::vm::2294::vm.Vm::(_startUnderlyingVm)
> vmId=`e3d350c5-6254-4908-99e9-d60ca2e046f7`::_ongoingCreations released
> Thread-91::ERROR::2015-05-12
> 19:25:15,154::vm::2331::vm.Vm::(_startUnderlyingVm)
> vmId=`e3d350c5-6254-4908-99e9-d60ca2e046f7`::The vm start process failed
> Traceback (most recent call last):
>   File "/usr/share/vdsm/virt/vm.py", line 2271, in _startUnderlyingVm
>     self._run()
>   File "/usr/share/vdsm/virt/vm.py", line 3335, in _run
>     self._connection.createXML(domxml, flags),
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
>   111, in wrapper
>     ret = f(*args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3427, in
>   createXML
>     if ret is None:raise libvirtError('virDomainCreateXML() failed',
>     conn=self)
> libvirtError: Failed to acquire lock: No space left on device
>
> sanlock:
> 2015-05-12 19:11:27+0200 4338 [708]: s2 lockspace
> 2f83ea82-e110-465d-a2a6-ac30534a9f6a:1:/rhev/data-center/mnt/10.11.0.30:_volumes_ovirt_engine/2f83ea82-e110-465d-a2a6-ac30534a9f6a/dom_md/ids:0
> 2015-05-12 19:11:48+0200 4359 [702]: s2 host 1 1 4338
> 9f9fb274-9ae8-490e-bfba-72a743b0ebb2.hv01.sec.i
> 2015-05-12 19:11:48+0200 4359 [702]: s2 host 250 1 0
> 9f9fb274-9ae8-490e-bfba-72a743b0ebb2.hv01.sec.i
> 2015-05-12 19:11:48+0200 4359 [707]: s2:r2 resource
> 2f83ea82-e110-465d-a2a6-ac30534a9f6a:SDM:/rhev/data-center/mnt/10.11.0.30:_volumes_ovirt_engine/2f83ea82-e110-465d-a2a6-ac30534a9f6a/dom_md/leases:1048576
> for 3,11,4167
> 2015-05-12 19:23:26+0200 5057 [708]: s3 lockspace
> 2f83ea82-e110-465d-a2a6-ac30534a9f6a:1:/rhev/data-center/mnt/10.11.0.30:_volumes_ovirt_engine/2f83ea82-e110-465d-a2a6-ac30534a9f6a/dom_md/ids:0
> 2015-05-12 19:23:48+0200 5079 [702]: s3 host 1 2 5058
> 9f9fb274-9ae8-490e-bfba-72a743b0ebb2.hv01.sec.i
> 2015-05-12 19:23:48+0200 5079 [702]: s3 host 250 1 0
> 9f9fb274-9ae8-490e-bfba-72a743b0ebb2.hv01.sec.i
> 2015-05-12 19:24:53+0200 5144 [702]: cmd 9 target pid 4893 not found
> 2015-05-12 19:25:14+0200 5165 [708]: r3 cmd_acquire 2,9,4893 invalid
> lockspace found -1 failed 0 name 2f83ea82-e110-465d-a2a6-ac30534a9f6a
>
> --
> Daniel Helgenberger
> m box bewegtbild GmbH
>
> P: +49/30/2408781-22
> F: +49/30/2408781-10
>
> ACKERSTR. 19
> D-10115 BERLIN
>
>
> www.m-box.de<http://www.m-box.de>  www.monkeymen.tv<http://www.monkeymen.tv>
>
> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>

--
Didi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150513/4d91d36e/attachment-0001.html>