<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 23, 2015 at 4:56 PM, Gianluca Cecchi <span dir="ltr"><<a href="mailto:gianluca.cecchi@gmail.com" target="_blank">gianluca.cecchi@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Fri, Oct 23, 2015 at 4:42 PM, Simone Tiraboschi <span dir="ltr"><<a href="mailto:stirabos@redhat.com" target="_blank">stirabos@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div><br><div><br></div><div><br></div></div></div><div>Are ovirt-ha-agent and ovirt-ha-broker up and running?</div><div>Can you please try to restart them via systemd?</div><div> </div></div><br></div></div></blockquote></span><div>In the mean time I found inside the logs they failed to start.</div><div><br></div><div>I found in broker log the message</div><div><div>Thread-1730::ERROR::2015-10-22 17:31:47,016::listener::192::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle) Error handling request, data: 'set-storage-domain FilesystemBackend dom_type=nfs3 sd_uuid=f53854cd-8767-4011-9564-36dc36e0a5d1'</div><span class=""><div>Traceback (most recent call last):</div></span></div><div>...</div><div><div>BackendFailureException: path to storage domain f53854cd-8767-4011-9564-36dc36e0a5d1 not found in /rhev/data-center/mnt</div></div><div><br></div><div>so probably the NFS part was not in lace yet when the broker attempted to start?</div><div>I saw that actually I had now</div><div><br></div><div><div>[root@ovc71 ovirt-hosted-engine-ha]# ll /rhev/data-center/mnt/ovc71.localdomain.local:_NFS__DOMAIN</div><div>total 0</div><div>-rwxr-xr-x. 1 vdsm kvm 0 Oct 23 16:46 __DIRECT_IO_TEST__</div><div>drwxr-xr-x. 5 vdsm kvm 47 Oct 22 15:49 f53854cd-8767-4011-9564-36dc36e0a5d1</div></div><div><br></div></div><div>and I was able to run </div><div><br></div><div>systemctl start ovirt-ha-broker.service<br></div><div>and verify it correctly started.</div><div>and the same for</div><div>systemctl start ovirt-ha-agent<br></div><div><br></div><div>after a couple of minutes the sh engine VM was powered on and I was able to access web admin portal.</div><div><br></div></div></div></blockquote><div><br></div><div>OK, can you please try again the whole reboot procedure just to ensure that it was just a temporary NFS glitch?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div></div><div>But if I try to connect to its console with</div><div><br></div><div><div>[root@ovc71 ovirt-hosted-engine-ha]# hosted-engine --add-console-password</div><div>Enter password: </div><div><span style="white-space:pre-wrap"> </span>code = 0</div><div><span style="white-space:pre-wrap"> </span>message = 'Done'</div></div><div><br></div><div>and then</div><div><div># remote-viewer --spice-ca-file=/etc/pki/vdsm/libvirt-spice/ca-cert.pem spice://localhost?tls-port=5900 --spice-host-subject="C=EN, L=Test, O=Test, CN=Test"</div><div><br></div><div>** (remote-viewer:7173): WARNING **: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-Gb5xXSKiKK: Connection refused</div><span class=""><div>GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications.</div></span><div>(/usr/bin/remote-viewer:7173): Spice-Warning **: ssl_verify.c:492:openssl_verify: ssl: subject 'C=EN, L=Test, O=Test, CN=Test' verification failed</div><div>(/usr/bin/remote-viewer:7173): Spice-Warning **: ssl_verify.c:494:openssl_verify: ssl: verification failed</div><div><br></div></div></div></div></blockquote><div><br></div><div>The issue was here: --spice-host-subject="C=EN, L=Test, O=Test, CN=Test"</div><div>This one was just the temporary subject used by hosted-engine-setup during the bootstrap sequence when your engine was still to come.</div><div>At the end that cert got replace by the engine CA signed ones and so you have to substitute that subject to match the one you used during your setup.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div><div></div><div>(remote-viewer:7173): GSpice-WARNING **: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1)</div></div><div><br></div><div>I get an error window with</div><div>Unable to connect to the graphic server spice://localhost?tls-port=5900<br></div><div><br></div><div><div>[root@ovc71 ovirt-hosted-engine-ha]# netstat -tan | grep 5900</div><div>tcp 0 0 <a href="http://0.0.0.0:5900" target="_blank">0.0.0.0:5900</a> 0.0.0.0:* LISTEN</div></div><div><br></div><div><br></div><div>the qemu command line of the sh engine is:</div><div><div>qemu 4489 1 23 16:41 ? 00:02:35 /usr/libexec/qemu-kvm -name HostedEngine -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Nehalem -m 8192 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 9e654c4a-925c-48ba-9818-6908b7714d3a -smbios type=1,manufacturer=oVirt,product=oVirt Node,version=7-1.1503.el7.centos.2.8,serial=97F39B57-FA7D-2A47-9E0E-304705DE227D,uuid=9e654c4a-925c-48ba-9818-6908b7714d3a -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/HostedEngine.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2015-10-23T14:41:23,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-reboot -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw,serial= -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/var/run/vdsm/storage/f53854cd-8767-4011-9564-36dc36e0a5d1/45ae3a4a-2190-4494-9419-b7c2af8a7aef/52b97c5b-96ae-4efc-b2e0-f56cde243384,if=none,id=drive-virtio-disk0,format=raw,serial=45ae3a4a-2190-4494-9419-b7c2af8a7aef,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:16:6a:b6,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/9e654c4a-925c-48ba-9818-6908b7714d3a.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/9e654c4a-925c-48ba-9818-6908b7714d3a.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/9e654c4a-925c-48ba-9818-6908b7714d3a.org.ovirt.hosted-engine-setup.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -chardev pty,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -spice tls-port=5900,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -msg timestamp=on</div></div><div><br></div></div></div>
</blockquote></div><br></div></div>