On Wed, Jun 3, 2020 at 4:15 PM Yedidyah Bar David <didi@redhat.com> wrote:
[snip]

This looks simply like libvirt was restarted too much too quickly. Did
you restart it yourself before deploy?

No. I installed from iso and after restart I connected to cockpit and run the gluster wizard for single host


If not, and it's only due to the deploy process restarting it, then either:
1. That's ok, and we should simply configure systemd to allow that,
2. or there is some other problem, and restarting libvirt is just a symptom.

> Jun 03 15:13:35 ovirt01.mydomain.local systemd[1]: libvirtd.service: Failed with result 'exit-code'.
> Jun 03 15:13:35 ovirt01.mydomain.local systemd[1]: Failed to start Virtualization daemon.
> [root@ovirt01 g.cecchi]#
>
> Let me know what files do you want to analyze the problem
> Under /var/log/ovirt-hosted-engine-setup I have:
>
> [root@ovirt01 ovirt-hosted-engine-setup]# ls -lrt
> total 632
> -rw-r--r--. 1 root root 123814 Jun  3 15:07 ovirt-hosted-engine-setup-ansible-get_network_interfaces-20205315737-ooohyb.log
> -rw-r--r--. 1 root root 126674 Jun  3 15:08 ovirt-hosted-engine-setup-ansible-validate_hostnames-20205315737-oqixuw.log
> -rw-r--r--. 1 root root 127548 Jun  3 15:10 ovirt-hosted-engine-setup-ansible-validate_hostnames-202053151022-yls4qo.log
> -rw-r--r--. 1 root root 261482 Jun  3 15:13 ovirt-hosted-engine-setup-ansible-initial_clean-20205315123-7x25zv.log
> [root@ovirt01 ovirt-hosted-engine-setup]#

Perhaps check these for libvirt restarts, also /var/log/messages (or
journalctl).
Do you see there other errors?

The latest file contains these last lines with the summary:

2020-06-03 15:13:34,567+0200 INFO SUMMARY:
Duration        Task Name
--------        --------
[ < 1 sec ]     Execute just a specific set of steps
[  00:01  ]     Force facts gathering
[  00:01  ]     Install oVirt Hosted Engine packages
[ < 1 sec ]     System configuration validations
[  00:01  ]     Get all active network interfaces
[ < 1 sec ]     Filter bonds with bad naming
[ < 1 sec ]     Generate output list
[  00:01  ]     Collect interface types
[ < 1 sec ]     Get list of Team devices
[ < 1 sec ]     Filter unsupported interface types
[ < 1 sec ]     Prepare getent key
[ < 1 sec ]     Get full hostname
[ < 1 sec ]     Set hostname variable if not defined
[ < 1 sec ]     Define host address variable if not defined
[ < 1 sec ]     Get host address resolution
[ < 1 sec ]     Parse host address resolution
[ < 1 sec ]     Get target address from selected interface (IPv4)
[ < 1 sec ]     Get target address from selected interface (IPv6)
[ < 1 sec ]     Check for alias
[ < 1 sec ]     Filter resolved address list
[ < 1 sec ]     Get engine FQDN resolution
[ < 1 sec ]     Parse engine he_fqdn resolution
[ < 1 sec ]     Define he_cloud_init_host_name
[ < 1 sec ]     Get uuid
[ < 1 sec ]     Set he_vm_uuid
[ < 1 sec ]     Get uuid
[ < 1 sec ]     Set he_nic_uuid
[ < 1 sec ]     Get uuid
[ < 1 sec ]     Set he_cdrom_uuid
[ < 1 sec ]     get timezone
[ < 1 sec ]     Set he_time_zone
[  00:01  ]     Check firewalld status
[ < 1 sec ]     Get default gateway IPv4
[ < 1 sec ]     Get default gateway IPv6
[ < 1 sec ]     Set he_gateway
[ < 1 sec ]     Generate unicast MAC address
[ < 1 sec ]     Set he_vm_mac_addr
[ < 1 sec ]     Get free memory
[ < 1 sec ]     Get cached memory
[ < 1 sec ]     Set Max memory
[ < 1 sec ]     set he_mem_size_MB to max available if not defined
[ < 1 sec ]     Populate service facts
[ < 1 sec ]     get max cpus
[ < 1 sec ]     Set he_maxvcpus
[ < 1 sec ]     Set he_vcpus to maximum amount if not defined
[ < 1 sec ]     Stop libvirt service
[  00:01  ]     Drop vdsm config statements
[  00:01  ]     Restore initial abrt config files
[  00:01  ]     Restart abrtd service
[  00:01  ]     Drop libvirt sasl2 configuration by vdsm
[  00:01  ]     Stop and disable services
[ < 1 sec ]     Restore initial libvirt default network configuration
[ FAILED  ]     Start libvirt
2020-06-03 15:13:34,567+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7f7156a11b70> kwargs


Previous lines of the same file contain:
...
2020-06-03 15:13:23,336+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Check number of chosen CPUs kwargs
2020-06-03 15:13:23,631+0200 INFO ansible skipped {'status': 'SKIPPED', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-
hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'Check number of chosen CPUs', 'ansible_host': 'localhost'}
2020-06-03 15:13:23,632+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f7154057748> kwargs
2020-06-03 15:13:23,944+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Stop libvirt service'}
2020-06-03 15:13:23,945+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Stop libvirt service kwargs is_conditional:False
2020-06-03 15:13:23,945+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Stop libvirt service kwargs
2020-06-03 15:13:24,625+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Stop libvirt service', 'task_duration': 0}
2020-06-03 15:13:24,626+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71541c4f98> kwargs
2020-06-03 15:13:24,930+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Drop vdsm config statements'}
2020-06-03 15:13:24,930+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Drop vdsm config statements kwargs is_conditional:False
2020-06-03 15:13:24,930+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Drop vdsm config statements kwargs
2020-06-03 15:13:25,166+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715412fbe0> kwargs
2020-06-03 15:13:25,385+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715412f198> kwargs
2020-06-03 15:13:25,600+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71540e8668> kwargs
2020-06-03 15:13:25,815+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715412fb38> kwargs
2020-06-03 15:13:26,096+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Drop vdsm config statements', 'task_duration': 1}
2020-06-03 15:13:26,096+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715402c780> kwargs
2020-06-03 15:13:26,417+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Restore initial abrt config files'}
2020-06-03 15:13:26,417+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Restore initial abrt config files kwargs is_conditional:False
2020-06-03 15:13:26,417+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Restore initial abrt config files kwargs
2020-06-03 15:13:26,878+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715404ba20> kwargs
2020-06-03 15:13:27,138+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f7154138320> kwargs
2020-06-03 15:13:27,391+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f7154061588> kwargs
2020-06-03 15:13:27,643+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715404b278> kwargs
2020-06-03 15:13:27,945+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Restore initial abrt config files', 'task_duration': 1}
2020-06-03 15:13:27,945+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f7154061ac8> kwargs
2020-06-03 15:13:28,268+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Restart abrtd service'}
2020-06-03 15:13:28,268+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Restart abrtd service kwargs is_conditional:False
2020-06-03 15:13:28,268+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Restart abrtd service kwargs
2020-06-03 15:13:29,045+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Restart abrtd service', 'task_duration': 1}
2020-06-03 15:13:29,046+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f715413e860> kwargs
2020-06-03 15:13:29,358+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Drop libvirt sasl2 configuration by vdsm'}
2020-06-03 15:13:29,358+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Drop libvirt sasl2 configuration by vdsm kwargs is_conditional:False
2020-06-03 15:13:29,358+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Drop libvirt sasl2 configuration by vdsm kwargs
2020-06-03 15:13:30,873+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Drop libvirt sasl2 configuration by vdsm', 'task_duration': 1}
2020-06-03 15:13:30,873+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71541f2d30> kwargs
2020-06-03 15:13:31,212+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Stop and disable services'}
2020-06-03 15:13:31,212+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Stop and disable services kwargs is_conditional:False
2020-06-03 15:13:31,212+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Stop and disable services kwargs
2020-06-03 15:13:31,614+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71541c4518> kwargs
2020-06-03 15:13:31,978+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f714ffc0c88> kwargs
2020-06-03 15:13:32,353+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f714ffc0128> kwargs
2020-06-03 15:13:32,644+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Stop and disable services', 'task_duration': 1}
2020-06-03 15:13:32,644+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f7154199240> kwargs
2020-06-03 15:13:32,955+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Restore initial libvirt default network configuration'}
2020-06-03 15:13:32,956+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Restore initial libvirt default network configuration kwargs is_conditional:False
2020-06-03 15:13:32,956+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Restore initial libvirt default network configuration kwargs
2020-06-03 15:13:33,475+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Restore initial libvirt default network configuration', 'task_duration': 0}
2020-06-03 15:13:33,476+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71541380f0> kwargs
2020-06-03 15:13:33,785+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Start libvirt'}
2020-06-03 15:13:33,786+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Start libvirt kwargs is_conditional:False
2020-06-03 15:13:33,786+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Start libvirt kwargs
2020-06-03 15:13:34,564+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]"
2020-06-03 15:13:34,565+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]"
2020-06-03 15:13:34,565+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]"
2020-06-03 15:13:34,565+0200 ERROR ansible failed {
    "ansible_host": "localhost",
    "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
    "ansible_result": {
        "_ansible_no_log": false,
        "changed": false,
        "invocation": {
            "module_args": {
                "daemon_reexec": false,
                "daemon_reload": false,
                "enabled": true,
                "force": null,
                "masked": null,
                "name": "libvirtd",
                "no_block": false,
                "scope": null,
                "state": "started",
                "user": null
            }
        },
        "msg": "Unable to start service libvirtd: Job for libvirtd.service failed because the control process exited with error code.\nSee \"systemctl status libvirtd.service\" and \"journalctl -xe\" for details.\n"
    },
    "ansible_task": "Start libvirt",
    "ansible_type": "task",
    "status": "FAILED",
    "task_duration": 1
}
2020-06-03 15:13:34,565+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f71541c4240> kwargs ignore_errors:None
2020-06-03 15:13:34,567+0200 INFO ansible stats {
    "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
    "ansible_playbook_duration": "01:29 Minutes",
    "ansible_result": "type: <class 'dict'>\nstr: {'localhost': {'ok': 96, 'failures': 1, 'unreachable': 0, 'changed': 23, 'skipped': 37, 'rescued': 0, 'ignored': 0}}",
    "ansible_type": "finish",
    "status": "FAILED"

I confirm that what suggested by Joop worked for me too:

"
I have had this too and saw that during the test week it was also reported.
The 'solution' was to run /usr/sbin/ovirt-hosted-engine-cleanup and let the wizard stay where it is and afterwards click Redeploy VM and then it will run to its conclusion. Don't know what precisely it fixes. Had a look and couldn't find it.
"

I confirm also the eventi for host already reported and due to missing packages:
Gluster command [<UNKNOWN>] failed on server <UNKNOWN>.

Gianluca