I try to deploy hosted-engine to a NFS Share accessible by (currently) two hosts. The host is running latest ovirt-node-ng 4.2.7.

hosted-engine --deploy fails constantly in late stage when trying to run engine from NFS. It already ran as "HostedEngineLocal" and I think is then migrated to NFS storage.

[root@epycdphv02 ~]# ls -al /rhev/data-center/mnt/storage01.office.databay.de:_ovirt_enginetotal 23drwxrwxrwx 3 vdsm kvm 4 Dec 3 13:01 .drwxr-xr-x 3 vdsm kvm 4096 Dec 1 17:11 ..drwxr-xr-x 6 vdsm kvm 6 Dec 3 13:09 1dacf1ea-0934-4840-bed4-e9d023572f59-rwxr-xr-x 1 vdsm kvm 0 Dec 3 13:42 __DIRECT_IO_TEST__

storage01.office.databay.de:/ovirt/engine on /rhev/data-center/mnt/storage01.office.databay.de:_ovirt_engine type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=192.168.1.121,local_lock=none,addr=192.168.1.3)

Could not open '/var/run/vdsm/storage/1dacf1ea-0934-4840-bed4-e9d023572f59/2b1332f6-3bb6-495b-87fe-c5b85e0ac495/39d45b33-5f29-430b-8b58-14a8ea20fb08': Permission denied

Even permissions of mentioned file seem to be ok. SELINUX is disabled since I had a lots of problems with earlier versions trying to deploy hosted-engine.

[root@epycdphv02 ~]# ls -al '/var/run/vdsm/storage/1dacf1ea-0934-4840-bed4-e9d023572f59/2b1332f6-3bb6-495b-87fe-c5b85e0ac495/39d45b33-5f29-430b-8b58-14a8ea20fb08'-rw-rw---- 1 vdsm kvm 53687091200 Dec 3 13:09 /var/run/vdsm/storage/1dacf1ea-0934-4840-bed4-e9d023572f59/2b1332f6-3bb6-495b-87fe-c5b85e0ac495/39d45b33-5f29-430b-8b58-14a8ea20fb08

[ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": "0:00:00.218320", "end": "2018-12-03 13:20:19.139919", "rc": 0, "start": "2018-12-03 13:20:18.921599", "stderr": "", "stderr_lines": [], "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=156443 (Mon Dec 3 13:20:16 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=156443 (Mon Dec 3 13:20:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUnexpectedlyDown\\nstopped=False\\ntimeout=Fri Jan 2 20:29:01 1970\\n\", \"hostname\": \"epycdphv02.office.databay.de\", \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": \"d3355c40\", \"local_conf_timestamp\": 156443, \"host-ts\": 156443}, \"global_maintenance\": false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=156443 (Mon Dec 3 13:20:16 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=156443 (Mon Dec 3 13:20:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUnexpectedlyDown\\nstopped=False\\ntimeout=Fri Jan 2 20:29:01 1970\\n\", \"hostname\": \"epycdphv02.office.databay.de\", \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": \"d3355c40\", \"local_conf_timestamp\": 156443, \"host-ts\": 156443}, \"global_maintenance\": false}"]}[ INFO ] TASK [Check VM status at virt level][ INFO ] TASK [Fail if engine VM is not running][ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Engine VM is not running, please check vdsm logs"}[ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook[ INFO ] Stage: Clean up[ INFO ] Cleaning temporary resources[ INFO ] TASK [Gathering Facts][ INFO ] ok: [localhost][ INFO ] TASK [Fetch logs from the engine VM][ INFO ] ok: [localhost][ INFO ] TASK [Set destination directory path][ INFO ] ok: [localhost][ INFO ] TASK [Create destination directory][ INFO ] changed: [localhost][ INFO ] TASK [include_tasks][ INFO ] ok: [localhost][ INFO ] TASK [Find the local appliance image][ INFO ] ok: [localhost][ INFO ] TASK [Set local_vm_disk_path][ INFO ] ok: [localhost][ INFO ] TASK [Give the vm time to flush dirty buffers][ INFO ] ok: [localhost][ INFO ] TASK [Copy engine logs][ INFO ] TASK [include_tasks][ INFO ] ok: [localhost][ INFO ] TASK [Remove local vm dir][ INFO ] changed: [localhost][ INFO ] TASK [Remove temporary entry in /etc/hosts for the local VM][ INFO ] ok: [localhost][ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20181203132110.conf'[ INFO ] Stage: Pre-termination[ INFO ] Stage: Termination[ ERROR ] Hosted Engine deployment failed: please check the logs for the issue, fix accordingly or re-deploy from scratch. Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20181203124703-45t4ja.log


Ralf Schenk fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail rs@databay.de		Databay AG Jens-Otto-Krag-Straße 11 D-52146 Würselen www.databay.de
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Wilhelm Dohmen