Happy to say that I just passed this "Get local VM IP" step There were a lot of leftover from previous failed attempts (cf log I sent earlier : "internal error: Failed to autostart storage pool..." ) Those were not cleaned up by ovirt-hosted-engine-cleanup I had to do the followinf so libvirt would be happy again : rm -rf /etc/libvirt/storage/*.xml rm -rf /etc/libvirt/storage/autostart/* rm -rf /var/tmp/local* ovirt-hosted-engine-cleanup is not doing a really good job Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group On Tue, Feb 26, 2019 at 3:49 AM Guillaume Pavese < guillaume.pavese@interactiv-group.com> wrote:
journalctl -u libvirtd.service :
févr. 25 18:47:24 vs-inf-int-kvm-fr-301-210.hostics.fr systemd[1]: Stopping Virtualization daemon... févr. 25 18:47:24 vs-inf-int-kvm-fr-301-210.hostics.fr systemd[1]: Stopped Virtualization daemon. févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr systemd[1]: Starting Virtualization daemon... févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr systemd[1]: Started Virtualization daemon. févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq[6310]: read /etc/hosts - 4 addresses févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq[6310]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq-dhcp[6310]: read /var/lib/libvirt/dnsmasq/default.hostsfile févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.739+0000: 13551: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.739+0000: 13551: info : hostname: vs-inf-int-kvm-fr-301-210.hostics.fr févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.739+0000: 13551: error : virDirOpenInternal:2936 : cannot open directory '/var/tmp/localvmgmyYik/images/15023c8a-e3a7-4851-a97d-3b90996b423b': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : storageDriverAutostartCallback:209 : internal error: Failed to autostart storage pool '15023c8a-e3a7-4851-a97d-3b90996b423b': cannot open directory '/var/tmp/localvmgmyYik/images/15023c8a-e3a7-4851-a97d-3b90996b423b': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : virDirOpenInternal:2936 : cannot open directory '/var/tmp/localvmdRIozH': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : storageDriverAutostartCallback:209 : internal error: Failed to autostart storage pool 'localvmdRIozH': cannot open directory '/var/tmp/localvmdRIozH': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : virDirOpenInternal:2936 : cannot open directory '/var/tmp/localvmdRIozH/images/15023c8a-e3a7-4851-a97d-3b90996b423b': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : storageDriverAutostartCallback:209 : internal error: Failed to autostart storage pool '15023c8a-e3a7-4851-a97d-3b90996b423b-1': cannot open directory '/var/tmp/localvmdRIozH/images/15023c8a-e3a7-4851-a97d-3b90996b423b': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : virDirOpenInternal:2936 : cannot open directory '/var/tmp/localvmgmyYik': No such file or directory févr. 25 18:47:34 vs-inf-int-kvm-fr-301-210.hostics.fr libvirtd[13535]: 2019-02-25 17:47:34.740+0000: 13551: error : storageDriverAutostartCallback:209 : internal error: Failed to autostart storage pool 'localvmgmyYik': cannot open directory '/var/tmp/localvmgmyYik': No such file or directory
/var/log/libvirt/qemu/HostedEngineLocal.log :
2019-02-25 17:50:08.694+0000: starting up libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org), qemu version: 2.12.0qemu-kvm-ev-2.12.0-18.el7_6.3.1, kernel: 3.10.0-957.5.1.el7.x86_64, hostname: vs-inf-int-kvm-fr-301-210.hostics.fr LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name guest=HostedEngineLocal,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-HostedEngineLocal/master-key.aes -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off -cpu Haswell-noTSX,+kvmclock -m 16384 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 8ba608c8-b721-4b5b-b839-b62f5e919814 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=27,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=off,strict=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/tmp/localvmlF5yTM/images/15023c8a-e3a7-4851-a97d-3b90996b423b/07fdcff3-11ce-4f7c-af05-0a878593e78e,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/tmp/localvmlF5yTM/seed.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:1d:4b:b6,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=31,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -vnc 127.0.0.1:0 -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 2019-02-25T17:50:08.904663Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/4 (label charserial0) 2019-02-25T17:50:08.911239Z qemu-kvm: warning: host doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10] 2019-02-25T17:50:08.917723Z qemu-kvm: warning: host doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10] 2019-02-25T17:50:08.918494Z qemu-kvm: warning: host doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10] 2019-02-25T17:50:08.919217Z qemu-kvm: warning: host doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10]
I guess there is something about those last warnings? It should be noted that I previously successfully deployed oVirt 4.2 in the same Nested environment
Running libvirt in debug mode will need to wait tomorrow ; my night is already cut to nothing much anymore XD
Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group
On Tue, Feb 26, 2019 at 3:33 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Mon, Feb 25, 2019 at 7:15 PM Guillaume Pavese < guillaume.pavese@interactiv-group.com> wrote:
No, as indicated previously, still :
[root@vs-inf-int-kvm-fr-301-210 ~]# virsh -r net-dhcp-leases default Expiry Time MAC address Protocol IP address Hostname Client ID or DUID
-------------------------------------------------------------------------------------------------------------------
[root@vs-inf-int-kvm-fr-301-210 ~]#
I did not see any relevant log on the HE vm. Is there something I should look for there?
This smells really bad: I'd suggest to check /var/log/messages and /var/log/libvirt/qemu/HostedEngineLocal.log for libvirt errors; if nothing is there can I ask you to try reexecuting with libvirt debug logs (edit /etc/libvirt/libvirtd.conf).
Honestly I'm not able to reproduce it on my side.
Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group
On Tue, Feb 26, 2019 at 3:12 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Mon, Feb 25, 2019 at 7:04 PM Guillaume Pavese < guillaume.pavese@interactiv-group.com> wrote:
I still can't connect with VNC remotely but locally with X forwarding it works. However my connection has too high latency for that to be usable (I'm in Japan, my hosts in France, ~250 ms ping)
But I could see that the VM is booted!
and in Hosts logs there is :
févr. 25 18:51:12 vs-inf-int-kvm-fr-301-210.hostics.fr python[14719]: ansible-command Invoked with warn=True executable=None _uses_shell=True _raw_params=virsh -r net-dhcp-leases default | grep -i 00:16:3e:1d:4b:b6 | awk '{ print $5 }' | cut -f1 -d'/' removes=None argv=None creates=None chdir=None stdin=None févr. 25 18:51:30 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq-dhcp[6310]: DHCPDISCOVER(virbr0) 00:16:3e:1d:4b:b6 févr. 25 18:51:30 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq-dhcp[6310]: DHCPOFFER(virbr0) 192.168.122.14 00:16:3e:1d:4b:b6 févr. 25 18:51:30 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq-dhcp[6310]: DHCPREQUEST(virbr0) 192.168.122.14 00:16:3e:1d:4b:b6 févr. 25 18:51:30 vs-inf-int-kvm-fr-301-210.hostics.fr dnsmasq-dhcp[6310]: DHCPACK(virbr0) 192.168.122.14 00:16:3e:1d:4b:b6 vs-inf-int-ovt-fr-301-210 févr. 25 18:51:42 vs-inf-int-kvm-fr-301-210.hostics.fr python[14757]: ansible-command Invoked with warn=True executable=None _uses_shell=True _raw_params=virsh -r net-dhcp-leases default | grep -i 00:16:3e:1d:4b:b6 | awk '{ print $5 }' | cut -f1 -d'/' removes=None argv=None creates=None chdir=None stdin=None févr. 25 18:52:12 vs-inf-int-kvm-fr-301-210.hostics.fr python[14789]: ansible-command Invoked with warn=True executable=None _uses_shell=True _raw_params=virsh -r net-dhcp-leases default | grep -i 00:16:3e:1d:4b:b6 | awk '{ print $5 }' | cut -f1 -d'/' removes=None argv=None creates=None chdir=None stdin=None févr. 25 18:52:43 vs-inf-int-kvm-fr-301-210.hostics.fr python[14818]: ansible-command Invoked with warn=True executable=None _uses_shell=True _raw_params=virsh -r net-dhcp-leases default | grep -i 00:16:3e:1d:4b:b6 | awk '{ print $5 }' | cut -f1 -d'/' removes=None argv=None creates=None chdir=None stdin=None ....
ssh to the vm works too :
[root@vs-inf-int-kvm-fr-301-210 ~]# ssh root@192.168.122.14 The authenticity of host '192.168.122.14 (192.168.122.14)' can't be established. ECDSA key fingerprint is SHA256:+/pUzTGVA4kCyICb7XgqrxWYYkqzmDjVmdAahiBFgOQ. ECDSA key fingerprint is MD5:4b:ef:ff:4a:7c:1a:af:c2:af:4a:0f:14:a3:c5:31:fb. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.122.14' (ECDSA) to the list of known hosts. root@192.168.122.14's password: [root@vs-inf-int-ovt-fr-301-210 ~]#
But the test that the playbook tries still fails with empty result :
[root@vs-inf-int-kvm-fr-301-210 ~]# virsh -r net-dhcp-leases default Expiry Time MAC address Protocol IP address Hostname Client ID or DUID
-------------------------------------------------------------------------------------------------------------------
[root@vs-inf-int-kvm-fr-301-210 ~]#
This smells like a bug to me: and nothing at all in the output of virsh -r net-dhcp-leases default
?
Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group
On Tue, Feb 26, 2019 at 1:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Mon, Feb 25, 2019 at 5:50 PM Guillaume Pavese < guillaume.pavese@interactiv-group.com> wrote:
> I did that but no success yet. > > I see that "Get local VM IP" task tries the following : > > virsh -r net-dhcp-leases default | grep -i {{ he_vm_mac_addr }} | > awk '{ print $5 }' | cut -f1 -d'/' > > > However while the task is running, and vm running in qemu, "virsh -r > net-dhcp-leases default" never returns anything : >
Yes, I think that libvirt will never provide a DHCP lease since the appliance OS never correctly complete the boot process. I'd suggest to connect to the running VM via vnc DURING the boot process and check what's wrong.
> [root@vs-inf-int-kvm-fr-301-210 ~]# virsh -r net-dhcp-leases default > Expiry Time MAC address Protocol IP address > Hostname Client ID or DUID > > ------------------------------------------------------------------------------------------------------------------- > > [root@vs-inf-int-kvm-fr-301-210 ~]# > > > > > Guillaume Pavese > Ingénieur Système et Réseau > Interactiv-Group > > > On Tue, Feb 26, 2019 at 12:33 AM Simone Tiraboschi < > stirabos@redhat.com> wrote: > >> OK, try this: >> temporary >> edit /usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/02_create_local_vm.yml >> around line 120 >> and edit tasks "Get local VM IP" >> changing from "retries: 50" to "retries: 500" so that you have >> more time to debug it >> >> >> >> On Mon, Feb 25, 2019 at 4:20 PM Guillaume Pavese < >> guillaume.pavese@interactiv-group.com> wrote: >> >>> I retried after killing the remaining qemu process and >>> doing ovirt-hosted-engine-cleanup >>> The new attempt failed again at the same step. Then after it >>> fails, it cleans the temporary files (and vm disk) but *qemu >>> still runs!* : >>> >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Get local VM IP] >>> >>> [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 50, >>> "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i >>> 00:16:3e:6c:e8:f9 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": >>> "0:00:00.092436", "end": "2019-02-25 16:09:38.863263", "rc": 0, "start": >>> "2019-02-25 16:09:38.770827", "stderr": "", "stderr_lines": [], "stdout": >>> "", "stdout_lines": []} >>> [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] >>> [ INFO ] ok: [localhost] >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] >>> [ INFO ] changed: [localhost] >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry >>> in /etc/hosts for the local VM] >>> [ INFO ] ok: [localhost] >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Notify the user about >>> a failure] >>> [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": >>> "The system may not be provisioned according to the playbook results: >>> please check the logs for the issue, fix accordingly or re-deploy from >>> scratch.\n"} >>> [ ERROR ] Failed to execute stage 'Closing up': Failed executing >>> ansible-playbook >>> [ INFO ] Stage: Clean up >>> [ INFO ] Cleaning temporary resources >>> ... >>> >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] >>> [ INFO ] ok: [localhost] >>> [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry >>> in /etc/hosts for the local VM] >>> [ INFO ] ok: [localhost] >>> [ INFO ] Generating answer file >>> '/var/lib/ovirt-hosted-engine-setup/answers/answers-20190225161011.conf' >>> [ INFO ] Stage: Pre-termination >>> [ INFO ] Stage: Termination >>> [ ERROR ] Hosted Engine deployment failed: please check the logs >>> for the issue, fix accordingly or re-deploy from scratch. >>> >>> >>> >>> [root@vs-inf-int-kvm-fr-301-210 ~]# ps aux | grep qemu >>> root 4021 0.0 0.0 24844 1788 ? Ss févr.22 0:00 >>> /usr/bin/qemu-ga --method=virtio-serial >>> --path=/dev/virtio-ports/org.qemu.guest_agent.0 >>> --blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush,guest-exec,guest-exec-status >>> -F/etc/qemu-ga/fsfreeze-hook >>> qemu 26463 22.9 4.8 17684512 1088844 ? Sl 16:01 3:09 >>> /usr/libexec/qemu-kvm -name guest=HostedEngineLocal,debug-threads=on -S >>> -object >>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-HostedEngineLocal/master-key.aes >>> -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off -cpu >>> Haswell-noTSX,+kvmclock -m 16384 -realtime mlock=off -smp >>> 4,sockets=4,cores=1,threads=1 -uuid 316eca5f-81de-4a0b-af1f-58f910402a8e >>> -no-user-config -nodefaults -chardev >>> socket,id=charmonitor,fd=27,server,nowait -mon >>> chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown >>> -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot >>> menu=off,strict=on -device >>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive >>> file=/var/tmp/localvmdRIozH/images/15023c8a-e3a7-4851-a97d-3b90996b423b/07fdcff3-11ce-4f7c-af05-0a878593e78e,format=qcow2,if=none,id=drive-virtio-disk0 >>> -device >>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 >>> -drive >>> file=/var/tmp/localvmdRIozH/seed.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on >>> -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev >>> tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device >>> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:6c:e8:f9,bus=pci.0,addr=0x3 >>> -chardev pty,id=charserial0 -device >>> isa-serial,chardev=charserial0,id=serial0 -chardev >>> socket,id=charchannel0,fd=31,server,nowait -device >>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 >>> -vnc 127.0.0.1:0 -device >>> VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 -object >>> rng-random,id=objrng0,filename=/dev/random -device >>> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 -sandbox >>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny >>> -msg timestamp=on >>> root 28416 0.0 0.0 112712 980 pts/3 S+ 16:14 0:00 >>> grep --color=auto qemu >>> >>> >>> Before the first Error, while the vm was running for sure and the >>> disk was there, I also unsuccessfuly tried to connect to it with VNC and >>> got the same error I got before : >>> >>> [root@vs-inf-int-kvm-fr-301-210 ~]# debug1: Connection to port >>> 5900 forwarding to vs-inf-int-kvm-fr-301-210.hostics.fr port 5900 >>> requested. >>> debug1: channel 3: new [direct-tcpip] >>> channel 3: open failed: connect failed: Connection refused >>> debug1: channel 3: free: direct-tcpip: listening port 5900 for >>> vs-inf-int-kvm-fr-301-210.hostics.fr port 5900, connect from >>> 127.0.0.1 port 37002 to 127.0.0.1 port 5900, nchannels 4 >>> >>> >>> Guillaume Pavese >>> Ingénieur Système et Réseau >>> Interactiv-Group >>> >>> >>> On Mon, Feb 25, 2019 at 11:57 PM Guillaume Pavese < >>> guillaume.pavese@interactiv-group.com> wrote: >>> >>>> Something was definitely wrong ; as indicated, qemu process >>>> for guest=HostedEngineLocal was running but the disk file did not exist >>>> anymore... >>>> No surprise I could not connect >>>> >>>> I am retrying >>>> >>>> >>>> Guillaume Pavese >>>> Ingénieur Système et Réseau >>>> Interactiv-Group >>>> >>>> >>>> On Mon, Feb 25, 2019 at 11:15 PM Guillaume Pavese < >>>> guillaume.pavese@interactiv-group.com> wrote: >>>> >>>>> It fails too : >>>>> I made sure PermitTunnel=yes in sshd config but when I try to >>>>> connect to the forwarded port I get the following error on the openened >>>>> host ssh session : >>>>> >>>>> [gpavese@sheepora-X230 ~]$ ssh -v -L 5900: >>>>> vs-inf-int-kvm-fr-301-210.hostics.fr:5900 >>>>> root@vs-inf-int-kvm-fr-301-210.hostics.fr >>>>> ... >>>>> [root@vs-inf-int-kvm-fr-301-210 ~]# >>>>> debug1: channel 3: free: direct-tcpip: listening port 5900 for >>>>> vs-inf-int-kvm-fr-301-210.hostics.fr port 5900, connect from >>>>> ::1 port 42144 to ::1 port 5900, nchannels 4 >>>>> debug1: Connection to port 5900 forwarding to >>>>> vs-inf-int-kvm-fr-301-210.hostics.fr port 5900 requested. >>>>> debug1: channel 3: new [direct-tcpip] >>>>> channel 3: open failed: connect failed: Connection refused >>>>> debug1: channel 3: free: direct-tcpip: listening port 5900 for >>>>> vs-inf-int-kvm-fr-301-210.hostics.fr port 5900, connect from >>>>> 127.0.0.1 port 32778 to 127.0.0.1 port 5900, nchannels 4 >>>>> >>>>> >>>>> and in journalctl : >>>>> >>>>> févr. 25 14:55:38 vs-inf-int-kvm-fr-301-210.hostics.fr >>>>> sshd[19595]: error: connect_to >>>>> vs-inf-int-kvm-fr-301-210.hostics.fr port 5900: failed. >>>>> >>>>> >>>>> Guillaume Pavese >>>>> Ingénieur Système et Réseau >>>>> Interactiv-Group >>>>> >>>>> >>>>> On Mon, Feb 25, 2019 at 10:44 PM Simone Tiraboschi < >>>>> stirabos@redhat.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 25, 2019 at 2:35 PM Guillaume Pavese < >>>>>> guillaume.pavese@interactiv-group.com> wrote: >>>>>> >>>>>>> I made sure of everything and even stopped firewalld but still >>>>>>> can't connect : >>>>>>> >>>>>>> [root@vs-inf-int-kvm-fr-301-210 ~]# cat >>>>>>> /var/run/libvirt/qemu/HostedEngineLocal.xml >>>>>>> <graphics type='vnc' port='*5900*' autoport='yes' >>>>>>> *listen='127.0.0.1*'> >>>>>>> <listen type='address' address='*127.0.0.1*' >>>>>>> fromConfig='1' autoGenerated='no'/> >>>>>>> >>>>>>> [root@vs-inf-int-kvm-fr-301-210 ~]# netstat -pan | grep 59 >>>>>>> tcp 0 0 127.0.0.1:5900 0.0.0.0:* >>>>>>> LISTEN 13376/qemu-kvm >>>>>>> >>>>>> >>>>>> >>>>>> I suggest to try ssh tunneling, run >>>>>> ssh -L 5900:vs-inf-int-kvm-fr-301-210.hostics.fr:5900 >>>>>> root@vs-inf-int-kvm-fr-301-210.hostics.fr >>>>>> >>>>>> and then >>>>>> remote-viewer vnc://localhost:5900 >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> [root@vs-inf-int-kvm-fr-301-210 ~]# systemctl status >>>>>>> firewalld.service >>>>>>> ● firewalld.service - firewalld - dynamic firewall daemon >>>>>>> Loaded: loaded (/usr/lib/systemd/system/firewalld.service; >>>>>>> enabled; vendor preset: enabled) >>>>>>> *Active: inactive (dead)* >>>>>>> *févr. 25 14:24:03 vs-inf-int-kvm-fr-301-210.hostics.fr >>>>>>> <http://vs-inf-int-kvm-fr-301-210.hostics.fr> systemd[1]: Stopped firewalld >>>>>>> - dynamic firewall daemon.* >>>>>>> >>>>>>> From my laptop : >>>>>>> [gpavese@sheepora-X230 ~]$ telnet >>>>>>> vs-inf-int-kvm-fr-301-210.hostics.fr *5900* >>>>>>> Trying 10.199.210.11... >>>>>>> [*nothing gets through...*] >>>>>>> ^C >>>>>>> >>>>>>> For making sure : >>>>>>> [gpavese@sheepora-X230 ~]$ telnet >>>>>>> vs-inf-int-kvm-fr-301-210.hostics.fr *9090* >>>>>>> Trying 10.199.210.11... >>>>>>> *Connected* to vs-inf-int-kvm-fr-301-210.hostics.fr. >>>>>>> Escape character is '^]'. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Guillaume Pavese >>>>>>> Ingénieur Système et Réseau >>>>>>> Interactiv-Group >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 25, 2019 at 10:24 PM Parth Dhanjal < >>>>>>> dparth@redhat.com> wrote: >>>>>>> >>>>>>>> Hey! >>>>>>>> >>>>>>>> You can check under /var/run/libvirt/qemu/HostedEngine.xml >>>>>>>> Search for 'vnc' >>>>>>>> From there you can look up the port on which the HE VM is >>>>>>>> available and connect to the same. >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 25, 2019 at 6:47 PM Guillaume Pavese < >>>>>>>> guillaume.pavese@interactiv-group.com> wrote: >>>>>>>> >>>>>>>>> 1) I am running in a Nested env, but under libvirt/kvm on >>>>>>>>> remote Centos 7.4 Hosts >>>>>>>>> >>>>>>>>> Please advise how to connect with VNC to the local HE vm. I >>>>>>>>> see it's running, but this is on a remote host, not my local machine : >>>>>>>>> qemu 13376 100 3.7 17679424 845216 ? Sl 12:46 >>>>>>>>> 85:08 /usr/libexec/qemu-kvm -name guest=HostedEngineLocal,debug-threads=on >>>>>>>>> -S -object >>>>>>>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-HostedEngineLocal/master-key.aes >>>>>>>>> -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off -cpu >>>>>>>>> Haswell-noTSX,+kvmclock -m 16384 -realtime mlock=off -smp >>>>>>>>> 4,sockets=4,cores=1,threads=1 -uuid 6fe7c1c3-ea93-4343-a385-0d9e14bb563a >>>>>>>>> -no-user-config -nodefaults -chardev >>>>>>>>> socket,id=charmonitor,fd=27,server,nowait -mon >>>>>>>>> chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown >>>>>>>>> -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot >>>>>>>>> menu=off,strict=on -device >>>>>>>>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive >>>>>>>>> file=/var/tmp/localvmgmyYik/images/15023c8a-e3a7-4851-a97d-3b90996b423b/07fdcff3-11ce-4f7c-af05-0a878593e78e,format=qcow2,if=none,id=drive-virtio-disk0 >>>>>>>>> -device >>>>>>>>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 >>>>>>>>> -drive >>>>>>>>> file=/var/tmp/localvmgmyYik/seed.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on >>>>>>>>> -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev >>>>>>>>> tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device >>>>>>>>> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:3e:fe:28,bus=pci.0,addr=0x3 >>>>>>>>> -chardev pty,id=charserial0 -device >>>>>>>>> isa-serial,chardev=charserial0,id=serial0 -chardev >>>>>>>>> socket,id=charchannel0,fd=31,server,nowait -device >>>>>>>>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 >>>>>>>>> *-vnc 127.0.0.1:0 <http://127.0.0.1:0> -device VGA*,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 >>>>>>>>> -object rng-random,id=objrng0,filename=/dev/random -device >>>>>>>>> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 -sandbox >>>>>>>>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny >>>>>>>>> -msg timestamp=on >>>>>>>>> >>>>>>>>> >>>>>>>>> 2) [root@vs-inf-int-kvm-fr-301-210 ~]# cat >>>>>>>>> /etc/libvirt/qemu/networks/default.xml >>>>>>>>> <!-- >>>>>>>>> WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE >>>>>>>>> LIKELY TO BE >>>>>>>>> OVERWRITTEN AND LOST. Changes to this xml configuration >>>>>>>>> should be made using: >>>>>>>>> virsh net-edit default >>>>>>>>> or other application using the libvirt API. >>>>>>>>> --> >>>>>>>>> >>>>>>>>> <network> >>>>>>>>> <name>default</name> >>>>>>>>> <uuid>ba7bbfc8-28b8-459e-a42d-c2d6218e2cb6</uuid> >>>>>>>>> <forward mode='nat'/> >>>>>>>>> <bridge name='virbr0' stp='on' delay='0'/> >>>>>>>>> <mac address='52:54:00:e5:fe:3b'/> >>>>>>>>> <ip address='192.168.122.1' netmask='255.255.255.0'> >>>>>>>>> <dhcp> >>>>>>>>> <range start='192.168.122.2' end='192.168.122.254'/> >>>>>>>>> </dhcp> >>>>>>>>> </ip> >>>>>>>>> </network> >>>>>>>>> You have new mail in /var/spool/mail/root >>>>>>>>> [root@vs-inf-int-kvm-fr-301-210 ~] >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Guillaume Pavese >>>>>>>>> Ingénieur Système et Réseau >>>>>>>>> Interactiv-Group >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 25, 2019 at 9:57 PM Simone Tiraboschi < >>>>>>>>> stirabos@redhat.com> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Feb 25, 2019 at 1:14 PM Guillaume Pavese < >>>>>>>>>> guillaume.pavese@interactiv-group.com> wrote: >>>>>>>>>> >>>>>>>>>>> He deployment with "hosted-engine --deploy" fails at TASK >>>>>>>>>>> [ovirt.hosted_engine_setup : Get local VM IP] >>>>>>>>>>> >>>>>>>>>>> See following Error : >>>>>>>>>>> >>>>>>>>>>> 2019-02-25 12:46:50,154+0100 INFO >>>>>>>>>>> otopi.ovirt_hosted_engine_setup.ansible_utils >>>>>>>>>>> ansible_utils._process_output:109 TASK [ovirt.hosted_engine_setup : Get >>>>>>>>>>> local VM IP] >>>>>>>>>>> 2019-02-25 12:55:26,823+0100 DEBUG >>>>>>>>>>> otopi.ovirt_hosted_engine_setup.ansible_utils >>>>>>>>>>> ansible_utils._process_output:103 {u'_ansible_parsed': True, >>>>>>>>>>> u'stderr_lines': [], u'cmd': u"virsh -r net-dhcp-leases default | grep -i 00 >>>>>>>>>>> :16:3e:3e:fe:28 | awk '{ print $5 }' | cut -f1 -d'/'", >>>>>>>>>>> u'end': u'2019-02-25 12:55:26.666925', u'_ansible_no_log': False, >>>>>>>>>>> u'stdout': u'', u'changed': True, u'invocation': {u'module_args': {u'warn': >>>>>>>>>>> True, u'executable': >>>>>>>>>>> None, u'_uses_shell': True, u'_raw_params': u"virsh -r >>>>>>>>>>> net-dhcp-leases default | grep -i 00:16:3e:3e:fe:28 | awk '{ print $5 }' | >>>>>>>>>>> cut -f1 -d'/'", u'removes': None, u'argv': None, u'creates': None, >>>>>>>>>>> u'chdir': None, u'std >>>>>>>>>>> in': None}}, u'start': u'2019-02-25 12:55:26.584686', >>>>>>>>>>> u'attempts': 50, u'stderr': u'', u'rc': 0, u'delta': u'0:00:00.082239', >>>>>>>>>>> u'stdout_lines': []} >>>>>>>>>>> 2019-02-25 12:55:26,924+0100 ERROR >>>>>>>>>>> otopi.ovirt_hosted_engine_setup.ansible_utils >>>>>>>>>>> ansible_utils._process_output:107 fatal: [localhost]: FAILED! => >>>>>>>>>>> {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default >>>>>>>>>>> | grep -i 00:16:3e:3e:fe:28 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": >>>>>>>>>>> "0:00:00.082239", "end": "2019-02-25 12:55:26.666925", "rc": 0, "start": >>>>>>>>>>> "2019-02-25 12:55:26.584686", "stderr": "", "stderr_lines": [], "stdout": >>>>>>>>>>> "", "stdout_lines": []} >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here we are just waiting for the bootstrap engine VM to >>>>>>>>>> fetch an IP address from default libvirt network over DHCP but it your case >>>>>>>>>> it never happened. >>>>>>>>>> Possible issues: something went wrong in the bootstrap >>>>>>>>>> process for the engine VM or the default libvirt network is not correctly >>>>>>>>>> configured. >>>>>>>>>> >>>>>>>>>> 1. can you try to reach the engine VM via VNC and check >>>>>>>>>> what's happening there? (another question, are you running it nested? AFAIK >>>>>>>>>> it will not work if nested over ESXi) >>>>>>>>>> 2. can you please share the output of >>>>>>>>>> cat /etc/libvirt/qemu/networks/default.xml >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Guillaume Pavese >>>>>>>>>>> Ingénieur Système et Réseau >>>>>>>>>>> Interactiv-Group >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Users mailing list -- users@ovirt.org >>>>>>>>>>> To unsubscribe send an email to users-leave@ovirt.org >>>>>>>>>>> Privacy Statement: >>>>>>>>>>> https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>>> List Archives: >>>>>>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/VXRMU3SQWTMB2Y... >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>> Users mailing list -- users@ovirt.org >>>>>>>>> To unsubscribe send an email to users-leave@ovirt.org >>>>>>>>> Privacy Statement: >>>>>>>>> https://www.ovirt.org/site/privacy-policy/ >>>>>>>>> oVirt Code of Conduct: >>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>> List Archives: >>>>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/45UR44ITQTV7YV... >>>>>>>>> >>>>>>>>