[ovirt-users] oVirt 3.5.1 - VM "hostedengine" Failing to start

Wed Jan 28 16:39:44 UTC 2015

Hi Sandro

I was was trying both

Here is the debug version...

[root at powerhost1 ~]# engine-iso-uploader -v  upload -i iso /tmp/rhel-server-6.6-x86_64-dvd.iso
Please provide the REST API password for the admin at internal oVirt Engine user (CTRL+D to abort):
DEBUG: API Vendor(ovirt.org<http://ovirt.org>) API Version(3.5.0)
DEBUG: id=1328bbfe-f05a-4779-ac34-560b213ece11 address=192.168.1.32 path=/Volumes/Raid1/iso
Uploading, please wait...
DEBUG: local NFS mount point is /tmp/tmpCjZhA_
DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft 192.168.1.32:/Volumes/Raid1/iso /tmp/tmpCjZhA_)
DEBUG: /bin/mount -t nfs -o rw,sync,soft 192.168.1.32:/Volumes/Raid1/iso /tmp/tmpCjZhA_
DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', '192.168.1.32:/Volumes/Raid1/iso', '/tmp/tmpCjZhA_'])
DEBUG: returncode(32)
DEBUG: STDOUT()
DEBUG: STDERR(mount.nfs: Connection timed out
)
ERROR: mount.nfs: Connection timed out

DEBUG: /bin/umount -t nfs -f  /tmp/tmpCjZhA_
DEBUG: /bin/umount -t nfs -f  /tmp/tmpCjZhA_
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpCjZhA_'])
DEBUG: returncode(1)
DEBUG: STDOUT()
DEBUG: STDERR(umount2: Invalid argument
umount: /tmp/tmpCjZhA_: not mounted
)
DEBUG: umount2: Invalid argument
umount: /tmp/tmpCjZhA_: not mounted

Could this be associated with the auto negotiation of the nfs version?  Is there a way I can specify nfs3 on the mount or on the export(os x)

On Jan 28, 2015, at 6:11 AM, Sandro Bonazzola <sbonazzo at redhat.com<mailto:sbonazzo at redhat.com>> wrote:

Il 28/01/2015 16:05, Maor Lipchuk ha scritto:

----- Original Message -----
From: "Roy Golan" <rgolan at redhat.com<mailto:rgolan at redhat.com>>
To: "Mikola Rose" <mrose at power-soft.com<mailto:mrose at power-soft.com>>, "Maor Lipchuk" <mlipchuk at redhat.com<mailto:mlipchuk at redhat.com>>
Cc: users at ovirt.org<mailto:users at ovirt.org>
Sent: Wednesday, January 28, 2015 12:02:50 PM
Subject: Re: [ovirt-users] oVirt 3.5.1 - VM "hostedengine" Failing to start

On 01/28/2015 11:04 AM, Mikola Rose wrote:
-rwxr-xr-x   1 vdsm  kvm  0 Jan 28 00:59 /Volumes/Raid1/__DIRECT_IO_TEST__
-rwxr-xr-x   1 vdsm  kvm  0 Jan 28 00:55
/Volumes/Raid1/data/__DIRECT_IO_TEST__
-rwxrwxrwx   1 vdsm  kvm  0 Jan 28 00:55
/Volumes/Raid1/iso/__DIRECT_IO_TEST__
-rwxr-xr-x   1 vdsm  kvm  0 Jan 26 09:43
/Volumes/Raid1/vm/__DIRECT_IO_TEST__

Note,  since removing the file the hosted_engine1 is starting up and
everything seems fine but then all of a sudden it restarts.

please attach some logs.
Also may be related...

root at powerhost1 ~]# engine-iso-uploader --ssh-user=36 upload -i iso
rhel-server-6.6-x86_64-dvd.iso
Please provide the REST API password for the admin at internal oVirt
Engine user (CTRL+D to abort):
Uploading, please wait...
INFO: Start uploading rhel-server-6.6-x86_64-dvd.iso
ERROR: Unable to copy rhel-server-6.6-x86_64-dvd.iso to ISO storage
domain on iso.
ERROR: Error message is "unable to test the available space on
/Volumes/Raid1/iso"

Maor?

Hi,

I think this is related to the engine-iso-uploader tool,
Sandro, do you have any insights about this?

Please don't use UIDs like 36 as user.

a command line like:

ovirt-iso-uploader --ssh-user=root upload -i iso rhel-server-6.6-x86_64-dvd.iso

or like

ovirt-iso-uploader --ssh-user=your_user_in_kvm_group upload -i iso rhel-server-6.6-x86_64-dvd.iso

should work.

Regards,
Maor

On Jan 28, 2015, at 12:53 AM, Roy Golan <rgolan at redhat.com<mailto:rgolan at redhat.com>
<mailto:rgolan at redhat.com>> wrote:

On 01/28/2015 03:10 AM, Mikola Rose wrote:
Hi there,

I seem to have run into a problem.  my hosted engine vm is failing
to start;

vdsm.log;

Thread-20::DEBUG::2015-01-27
16:53:37,999::fileSD::152::Storage.StorageDomain::(__init__) Reading
domain in path
/rhev/data-center/mnt/192.168.1.32:_Volumes_Raid1/443b4931-667f-441f-98d8-51384e67a0af
Thread-20::ERROR::2015-01-27
16:53:38,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain)
Error while collecting domain 443b4931-667f-441f-98d8-51384e67a0af
monitoring information
Traceback (most recent call last):
 File "/usr/share/vdsm/storage/domainMonitor.py", line 221, in
_monitorDomain
   self.domain = sdCache.produce(self.sdUUID)
 File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
   domain.getRealDomain()
 File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
   return self._cache._realProduce(self._sdUUID)
 File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
   domain = self._findDomain(sdUUID)
 File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
   dom = findMethod(sdUUID)
 File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain
   return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID))
 File "/usr/share/vdsm/storage/fileSD.py", line 159, in __init__
validateFileSystemFeatures(sdUUID, self.mountpoint)
 File "/usr/share/vdsm/storage/fileSD.py", line 88, in
validateFileSystemFeatures
oop.getProcessPool(sdUUID).directTouch(testFilePath)
 File "/usr/share/vdsm/storage/outOfProcess.py", line 320, in
directTouch
   ioproc.touch(path, flags, mode)
 File "/usr/lib/python2.6/site-packages/ioprocess/__init__.py",
line 507, in touch
   self.timeout)
 File "/usr/lib/python2.6/site-packages/ioprocess/__init__.py",
line 391, in _sendCommand
   raise OSError(errcode, errstr)
OSError: [Errno 13] Permission denied

I assume this is an nfs issue so I checked to see if I could create
a file in the mounts from the host machine,  which I could.

My test bed is usign an old OS X NFS via xraid adn the export is ;
/Volumes/Raid1 -maproot=root:wheel -network 192.168.1.0 -mask
255.255.255.0

drwxr-xr-x    6 vdsm   kvm      204 Jan 27 16:30
443b4931-667f-441f-98d8-51384e67a0af
drwxr-xr-x    4 vdsm   kvm      136 Jan 27 12:32 data
drwxr-xr-x    4 vdsm   kvm      136 Jan 27 00:18 iso
drwxr-xr-x    3 vdsm   kvm      102 Jan 26 09:43 vm

the host machine seems to mount the shares fine....

drwxr-xr-x  5 vdsm kvm 4096 Jan 27 12:28 .
drwxr-xr-x  3 vdsm kvm 4096 Jan 27 10:04 ..
drwxr-xr-x 13 vdsm kvm  544 Jan 27 00:12 192.168.1.32:_Volumes_Raid1
drwxr-xr-x  2 vdsm kvm 4096 Jan 27 00:12
192.168.1.32:_Volumes_Raid1_data
drwxr-xr-x  2 vdsm kvm 4096 Jan 27 00:24 192.168.1.32:_Volumes_Raid1_iso

and as I said above I can create files in any one of those mounts

Is there a place I can look at to find the offending file?  If that
is the issue...  Oddly enough everything worked until i rebooted, so
I must have either changed something or something is buggered.

please echo the output of

find  /Volumes/Raid1 -name "__DIRECT_IO_TEST__" | xargs ls -la

the failure is to create this file.

_______________________________________________
Users mailing list
Users at ovirt.org<mailto:Users at ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

Mik Rose | Manager, IT - Support Services | *PowerSoft Development Corp*
1 (250) 642-0295 x23 http://www.power-soft.com
<http://www.power-soft.com/> Live Support
<https://secure.logmeinrescue.com/Customer/Download.aspx?EntryID=15095831>
This e-mail may be privileged and/or confidential, and the sender does
not waive
any related rights and obligations. Any distribution, use or copying
of this e-mail or the information
it contains by other than an intended recipient is unauthorized.
If you received this e-mail in error, please advise me (by return
e-mail or otherwise) immediately.

--
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com<http://redhat.com/>

Mik Rose | Manager, IT - Support Services | PowerSoft Development Corp
1 (250) 642-0295 x23   http://www.power-soft.com<http://www.power-soft.com/>   Live Support<https://secure.logmeinrescue.com/Customer/Download.aspx?EntryID=15095831>

This e-mail may be privileged and/or confidential, and the sender does not waive
any related rights and obligations. Any distribution, use or copying of this e-mail or the information
it contains by other than an intended recipient is unauthorized.
If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150128/6f93ab0d/attachment-0001.html>