Issues deploying 4.4 with HE on new EPYC hosts
Hello all, I have some EPYC servers that are not yet in production, so I wanted to go ahead and move them off of 4.3 (which was working) to 4.4. I flattened and reinstalled the hosts with CentOS 8.1 Minimal and installed all updates. Some very simple networking, just a bond and two iSCSI interfaces. After adding the oVirt 4.4 repo and installing the requirements, I run 'hosted-engine --deploy' and proceed through the setup. Everything looks as though it is going nicely and the local HE starts and runs perfectly. After copying the HE disks out to storage, the system tries to start it there but is using a different CPU definition and it's impossible to start it. At this point I'm stuck but hoping someone knows the fix, because this is as vanilla a deployment as I could attempt and it appears EPYC CPUs are a no-go right now with 4.4. When the HostedEngineLocal VM is running, the CPU definition is: <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC-IBPB</model> <vendor>AMD</vendor> <feature policy='require' name='x2apic'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='clwb'/> <feature policy='require' name='umip'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='wbnoinvd'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='svm'/> <feature policy='require' name='topoext'/> </cpu> Once the HostedEngine VM is defined and trying to start, the CPU definition is simply: <cpu mode='custom' match='exact' check='partial'> <model fallback='allow'>EPYC</model> <topology sockets='16' cores='4' threads='1'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='virt-ssbd'/> <numa> <cell id='0' cpus='0-63' memory='16777216' unit='KiB'/> </numa> </cpu> On attempts to start it, the host is logging this error: "CPU is incompatible with host CPU: Host CPU does not provide required features: virt-ssbd". So, the HostedEngineLocal VM works because it has a requirement set for 'amd-ssbd' instead of 'virt-ssbd', and a VM requiring 'virt-ssbd' can't run on EPYC CPUs with CentOS 8.1. As mentioned, the HostedEngine ran fine on oVirt 4.3 with CentOS 7.8, and on 4.3 the cpu definition also required 'virt-ssbd', so I can only imagine that perhaps this is due to the more recent 4.x kernel that I now need HE to require 'amd-ssbd' instead? Any clues to help with this? I can completely wipe/reconfigure the hosts as needed so I'm willing to try whatever so that I can move forward with a 4.4 deployment. Thanks! Mark
Does 8.1 enable AVIC by default? May be related to this: https://bugzilla.redhat.com/show_bug.cgi?id=1694170 Can try disabling avic in the kvm module on the hosts and see if that allows them to activate. Regards, Ryan On Tue, May 26, 2020 at 9:11 AM Mark R <ovirtlist@beanz.33mail.com> wrote:
Hello all,
I have some EPYC servers that are not yet in production, so I wanted to go ahead and move them off of 4.3 (which was working) to 4.4. I flattened and reinstalled the hosts with CentOS 8.1 Minimal and installed all updates. Some very simple networking, just a bond and two iSCSI interfaces. After adding the oVirt 4.4 repo and installing the requirements, I run 'hosted-engine --deploy' and proceed through the setup. Everything looks as though it is going nicely and the local HE starts and runs perfectly. After copying the HE disks out to storage, the system tries to start it there but is using a different CPU definition and it's impossible to start it. At this point I'm stuck but hoping someone knows the fix, because this is as vanilla a deployment as I could attempt and it appears EPYC CPUs are a no-go right now with 4.4.
When the HostedEngineLocal VM is running, the CPU definition is: <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC-IBPB</model> <vendor>AMD</vendor> <feature policy='require' name='x2apic'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='clwb'/> <feature policy='require' name='umip'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='wbnoinvd'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='svm'/> <feature policy='require' name='topoext'/> </cpu>
Once the HostedEngine VM is defined and trying to start, the CPU definition is simply:
<cpu mode='custom' match='exact' check='partial'> <model fallback='allow'>EPYC</model> <topology sockets='16' cores='4' threads='1'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='virt-ssbd'/> <numa> <cell id='0' cpus='0-63' memory='16777216' unit='KiB'/> </numa> </cpu>
On attempts to start it, the host is logging this error: "CPU is incompatible with host CPU: Host CPU does not provide required features: virt-ssbd".
So, the HostedEngineLocal VM works because it has a requirement set for 'amd-ssbd' instead of 'virt-ssbd', and a VM requiring 'virt-ssbd' can't run on EPYC CPUs with CentOS 8.1. As mentioned, the HostedEngine ran fine on oVirt 4.3 with CentOS 7.8, and on 4.3 the cpu definition also required 'virt-ssbd', so I can only imagine that perhaps this is due to the more recent 4.x kernel that I now need HE to require 'amd-ssbd' instead?
Any clues to help with this? I can completely wipe/reconfigure the hosts as needed so I'm willing to try whatever so that I can move forward with a 4.4 deployment.
Thanks! Mark _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KZHDCDE6JYADDM...
Thank you for the suggestion, Ryan.... it looks like AVIC is disabled by default though (/sys/module/kvm_amd/parameters/avic is 0).
Replying to my own post here, I've verified that it's currently not possible to do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the system setting a requirement of 'virt-ssbd' for the final HE definition after the move to shared storage. The local HE runs perfectly through the setup process because it correctly uses 'amd-ssbd' but unfortunately that doesn't stick after the disks are moved. You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin interface and if you want to run something hacky could at this point change the cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything down and bring it back up cleanly... hosted engine will now work because the requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or presumably RHEL 8, amd-ssbd is needed) is gone.
Hi, The cluster CPU type should be set to AMD EPYC, because your system does not support Secure AMD EPYC type (it requires the virt-ssbd). Did you by any chance specify the Secure variant during HE setup? Lucia On Wed, May 27, 2020 at 4:13 PM Mark R <ovirtlist@beanz.33mail.com> wrote:
Replying to my own post here, I've verified that it's currently not possible to do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the system setting a requirement of 'virt-ssbd' for the final HE definition after the move to shared storage. The local HE runs perfectly through the setup process because it correctly uses 'amd-ssbd' but unfortunately that doesn't stick after the disks are moved.
You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin interface and if you want to run something hacky could at this point change the cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything down and bring it back up cleanly... hosted engine will now work because the requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or presumably RHEL 8, amd-ssbd is needed) is gone. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VFZT6QCQLEDHCR...
On Wed, May 27, 2020 at 4:13 PM Mark R <ovirtlist@beanz.33mail.com> wrote:
Replying to my own post here, I've verified that it's currently not possible to do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the system setting a requirement of 'virt-ssbd' for the final HE definition after the move to shared storage. The local HE runs perfectly through the setup process because it correctly uses 'amd-ssbd' but unfortunately that doesn't stick after the disks are moved.
You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin interface and if you want to run something hacky could at this point change the cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything down and bring it back up cleanly... hosted engine will now work because the requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or presumably RHEL 8, amd-ssbd is needed) is gone.
I would like to test it during final stage of deployment, but the "virsh create" command requires a password. [root@novirt2 log]# virsh create /tmp/he.xml Please enter your authentication name: I don't remember how to setup a user so that I can run it on oVirt host.. any input? Just for my testing lab in 4.4. obviously... Thans, Gianluca
This might help: https://lists.ovirt.org/pipermail/users/2017-January/079157.html On Thu, May 28, 2020 at 10:17 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Wed, May 27, 2020 at 4:13 PM Mark R <ovirtlist@beanz.33mail.com> wrote:
Replying to my own post here, I've verified that it's currently not possible to do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the system setting a requirement of 'virt-ssbd' for the final HE definition after the move to shared storage. The local HE runs perfectly through the setup process because it correctly uses 'amd-ssbd' but unfortunately that doesn't stick after the disks are moved.
You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin interface and if you want to run something hacky could at this point change the cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything down and bring it back up cleanly... hosted engine will now work because the requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or presumably RHEL 8, amd-ssbd is needed) is gone.
I would like to test it during final stage of deployment, but the "virsh create" command requires a password.
[root@novirt2 log]# virsh create /tmp/he.xml Please enter your authentication name:
I don't remember how to setup a user so that I can run it on oVirt host.. any input?
Just for my testing lab in 4.4. obviously... Thans, Gianluca _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BOOXPB5BR73D6M...
On Thu, May 28, 2020 at 10:22 AM Lucia Jelinkova <ljelinko@redhat.com> wrote:
This might help: https://lists.ovirt.org/pipermail/users/2017-January/079157.html
On Thu, May 28, 2020 at 10:17 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Wed, May 27, 2020 at 4:13 PM Mark R <ovirtlist@beanz.33mail.com> wrote:
Replying to my own post here, I've verified that it's currently not possible to do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the system setting a requirement of 'virt-ssbd' for the final HE definition after the move to shared storage. The local HE runs perfectly through the setup process because it correctly uses 'amd-ssbd' but unfortunately that doesn't stick after the disks are moved.
You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin interface and if you want to run something hacky could at this point change the cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything down and bring it back up cleanly... hosted engine will now work because the requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or presumably RHEL 8, amd-ssbd is needed) is gone.
I would like to test it during final stage of deployment, but the "virsh create" command requires a password.
[root@novirt2 log]# virsh create /tmp/he.xml Please enter your authentication name:
I don't remember how to setup a user so that I can run it on oVirt host.. any input?
Just for my testing lab in 4.4. obviously... Thans, Gianluca
Thanks Lucia, in the mean time I found another e-mail reporting the same credentials stored in that file (that are static and not randomly generated..).
Unfortunately the strategy works to have the hosted engine vm started, but it seems it is not recognized... So I'm in the case of being during single host HCI deployment and in the final stage with the unsupported tsx-ctrl cpu flag, see this thread tto: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/5LBCJGWTVRVTEWC... Strange that in /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-create_target_vm-202042810039-c8h9k9.log I have this 2020-05-28 10:01:16,234+0200 DEBUG var changed: host "localhost" var "server_cpu_dict" type "<class 'dict'>" value: "{ "AMD EPYC": "EPYC", "AMD Opteron G4": "Opteron_G4", "AMD Opteron G5": "Opteron_G5", "IBM POWER8": "POWER8", "IBM POWER9": "POWER9", "IBM z114, z196": "z196-base", "IBM z13s, z13": "z13-base", "IBM z14": "z14-base", "IBM zBC12, zEC12": "zEC12-base", "Intel Broadwell Family": "Broadwell-noTSX", "Intel Cascadelake Server Family": "Cascadelake-Server,-hle,-rtm,+arch-capabilities", "Intel Haswell Family": "Haswell-noTSX", "Intel IvyBridge Family": "IvyBridge", "Intel Nehalem Family": "Nehalem", "Intel SandyBridge Family": "SandyBridge", "Intel Skylake Client Family": "Skylake-Client,-hle,-rtm", "Intel Skylake Server Family": "Skylake-Server,-hle,-rtm", "Intel Westmere Family": "Westmere", "Secure AMD EPYC": "EPYC,+ibpb,+virt-ssbd", "Secure Intel Broadwell Family": "Broadwell-noTSX,+spec-ctrl,+ssbd,+md-clear", "Secure Intel Cascadelake Server Family": "Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+tsx-ctrl,+arch-capabilities", "Secure Intel Haswell Family": "Haswell-noTSX,+spec-ctrl,+ssbd,+md-clear", "Secure Intel IvyBridge Family": "IvyBridge,+pcid,+spec-ctrl,+ssbd,+md-clear", "Secure Intel Nehalem Family": "Nehalem,+spec-ctrl,+ssbd,+md-clear", "Secure Intel SandyBridge Family": "SandyBridge,+pcid,+spec-ctrl,+ssbd,+md-clear", "Secure Intel Skylake Client Family": "Skylake-Client,+spec-ctrl,+ssbd,+md-clear,-hle,-rtm", "Secure Intel Skylake Server Family": "Skylake-Server,+spec-ctrl,+ssbd,+md-clear,-hle,-rtm", "Secure Intel Westmere Family": "Westmere,+pcid,+spec-ctrl,+ssbd,+md-clear" }" and the tsx-ctrl flag is there..... but when during deployment it starts the final engine vm after copying the local one to gluster storage, it gets: May 28 10:06:59 novirt2 journal[13368]: unsupported configuration: unknown CPU feature: tsx-ctrl The VM is started and then stopped and so on, so I try some times the dump command virsh -r dumpxml HostedEngine > /tmp/he.xml until I catch it then I edit the file and remove the line with the tsx-ctrl flag and start the vm [root@novirt2 log]# virsh create /tmp/he.xml Please enter your authentication name: vdsm@ovirt Please enter your password: Domain HostedEngine created from /tmp/he.xml So far so good the engine vm starts and it is reachable on its final hostname and engine service started and accessible web admin gui, but it is not recognized by the host The qemu-kvm command is this one: qemu 20826 1 99 10:16 ? 00:00:07 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,md-clear=on,mds-no=on,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T08:16:29,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-f6a70e5e-ebea-4468-a59a-cc760b9bcea4,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=42:43,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=44:45 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=46,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=47,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -vnc 172.19.0.224:0 -k en-us -spice port=5901,tls-port=5902,addr=172.19.0.224,disable-ticketing,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on The status remains as down [root@novirt2 log]# hosted-engine --vm-status --== Host novirt2.example.net (id: 1) status ==-- Host ID : 1 Host timestamp : 34180 Score : 0 Engine status : {"vm": "down_unexpected", "health": "bad", "detail": "Down", "reason": "bad vm status"} Hostname : novirt2.example.net Local maintenance : False stopped : False crc32 : b297faaa conf_on_shared_storage : True local_conf_timestamp : 34180 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=34180 (Thu May 28 10:17:57 2020) host-id=1 score=0 vm_conf_refresh_time=34180 (Thu May 28 10:17:57 2020) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Thu Jan 1 10:30:29 1970 [root@novirt2 log]# Also restarting ovirt-ha-agent on host doesn't change the things.... In agent log i have MainThread::ERROR::2020-05-28 10:23:45,817::hosted_engine::953::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Failed to stop engine VM: Command VM.destroy with args {'vmID': 'b572d924-b278-41c7-a9da-52c4f590aac1'} failed: (code=1, message=Virtual machine does not exist: {'vmId': 'b572d924-b278-41c7-a9da-52c4f590aac1'}) MainThread::INFO::2020-05-28 10:23:45,843::brokerlink::73::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineForceStop-ReinitializeFSM) sent? ignored MainThread::INFO::2020-05-28 10:23:45,850::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state ReinitializeFSM (score: 0) that is strange because the VM has started with that uuid. The qemu-kvm command infact contains the same uuid referred in agent.log: "uuid=b572d924-b278-41c7-a9da-52c4f590aac1" In the mean time the deploy fails because from host point of view it was not able to reach up state..... and to test I put in global maintenance. I can access the engine web admin portal but it seems it is recognized as a sort of external engine....? Perhaps I have to remove some other line inside the generated he.xml file to have the engine recognized? See screenshots from web admin view about hosted engine vm, the cluster type that has been setup by the deploy and the storage view (data and vm storage domains not appearing, only the engine one): https://drive.google.com/file/d/1by4mEOo3iQv1fMbRsqGvZCEOUFUJbPk4/view?usp=s... https://drive.google.com/file/d/199tOdLfSnWIb_rxCGM0zpgOjF3XFLUFm/view?usp=s... https://drive.google.com/file/d/1jVaMGvImRhyf3xQx5gn6bCu6OyfZ1Fq_/view?usp=s... and content of he.xml https://drive.google.com/file/d/1vOBB0t-vKD5f_7wUUMaIcHxVDXQ4Cfbe/view?usp=s... any input to solve and at least try some features of 4.4 on this hw env? Thanks, Gianluca
On Thu, May 28, 2020 at 11:00 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
any input to solve and at least try some features of 4.4 on this hw env?
Thanks, Gianluca
it seems I was able to have it recognized this way: strip these lines from he.xml
<ovirt-vm:exitCode type="int">1</ovirt-vm:exitCode> <ovirt-vm:exitMessage>unsupported configuration: unknown CPU feature:
tsx-ctrl</ovirt-vm:exitMessage>
<ovirt-vm:exitReason type="int">1</ovirt-vm:exitReason>
13a17
<ovirt-vm:startTime
type="float">1590653346.0201075</ovirt-vm:startTime> 246a251,264
<graphics type='vnc' port='-1' autoport='yes' keymap='en-us'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> </graphics> <graphics type='spice' autoport='yes'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> <channel name='main' mode='secure'/> <channel name='display' mode='secure'/> <channel name='inputs' mode='secure'/> <channel name='cursor' mode='secure'/> <channel name='playback' mode='secure'/> <channel name='record' mode='secure'/> <channel name='smartcard' mode='secure'/> <channel name='usbredir' mode='secure'/> </graphics>
I have to understand what to use for graphics but it is not important at the moment.... After failure of deployment the link to the engine storage domain under /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/ has been deleted. So I go there and create it: [root@novirt2 ~]# cd /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/ [root@novirt2 ~]# ln -s /rhev/data-center/mnt/glusterSD/novirt2st.storage.local\:_engine/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/images/df02bff9-2c4b-4e14-a0a3-591a84ccaed9 df02bff9-2c4b-4e14-a0a3-591a84ccaed9 Start VM [root@novirt2 ~]# virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf create he.xml Domain HostedEngine created from he.xml Access its webadmin gui and now all is as expected, with only one engine vm... Volumes for data and vmstore are there, but not the storage domains, but I can create them without problems Also if I exit the global maintenance the state passes from ReinitializeFSM to EngineUp [root@novirt2 ~]# hosted-engine --vm-status --== Host novirt2.example.net (id: 1) status ==-- Host ID : 1 Host timestamp : 38553 Score : 3400 Engine status : {"vm": "up", "health": "good", "detail": "Up"} Hostname : novirt2.example.net Local maintenance : False stopped : False crc32 : 5a3b40e1 conf_on_shared_storage : True local_conf_timestamp : 38553 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=38553 (Thu May 28 11:30:51 2020) host-id=1 score=3400 vm_conf_refresh_time=38553 (Thu May 28 11:30:51 2020) conf_on_shared_storage=True maintenance=False state=EngineUp stopped=False [root@novirt2 ~]# Now go to test further functionalities..... Gianluca
I think you have the same problem as Mark - the cluster is set to use the secure variant of the CPU type but your host does not support all the necessary flags. Intel Cascadelake Server Family - the VM is run with Cascadelake-Server,-hle,-rtm,+arch-capabilities Secure Intel Cascadelake Server Family - the VM is run with Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+tsx-ctrl,+arch-capabilities The cluster should be set to Intel Cascadelake Server Family. I am not familiar with the HE setup process - have you specified the cluster CPU type manually or is it auto-assigned? Lucia On Thu, May 28, 2020 at 11:32 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 11:00 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
any input to solve and at least try some features of 4.4 on this hw env?
Thanks, Gianluca
it seems I was able to have it recognized this way:
strip these lines from he.xml
<ovirt-vm:exitCode type="int">1</ovirt-vm:exitCode> <ovirt-vm:exitMessage>unsupported configuration: unknown CPU
feature: tsx-ctrl</ovirt-vm:exitMessage>
<ovirt-vm:exitReason type="int">1</ovirt-vm:exitReason>
13a17
<ovirt-vm:startTime
type="float">1590653346.0201075</ovirt-vm:startTime> 246a251,264
<graphics type='vnc' port='-1' autoport='yes' keymap='en-us'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> </graphics> <graphics type='spice' autoport='yes'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> <channel name='main' mode='secure'/> <channel name='display' mode='secure'/> <channel name='inputs' mode='secure'/> <channel name='cursor' mode='secure'/> <channel name='playback' mode='secure'/> <channel name='record' mode='secure'/> <channel name='smartcard' mode='secure'/> <channel name='usbredir' mode='secure'/> </graphics>
I have to understand what to use for graphics but it is not important at the moment.... After failure of deployment the link to the engine storage domain under /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/ has been deleted. So I go there and create it:
[root@novirt2 ~]# cd /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/
[root@novirt2 ~]# ln -s /rhev/data-center/mnt/glusterSD/novirt2st.storage.local\:_engine/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/images/df02bff9-2c4b-4e14-a0a3-591a84ccaed9 df02bff9-2c4b-4e14-a0a3-591a84ccaed9
Start VM [root@novirt2 ~]# virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf create he.xml Domain HostedEngine created from he.xml
Access its webadmin gui and now all is as expected, with only one engine vm... Volumes for data and vmstore are there, but not the storage domains, but I can create them without problems
Also if I exit the global maintenance the state passes from ReinitializeFSM to EngineUp
[root@novirt2 ~]# hosted-engine --vm-status
--== Host novirt2.example.net (id: 1) status ==--
Host ID : 1 Host timestamp : 38553 Score : 3400 Engine status : {"vm": "up", "health": "good", "detail": "Up"} Hostname : novirt2.example.net Local maintenance : False stopped : False crc32 : 5a3b40e1 conf_on_shared_storage : True local_conf_timestamp : 38553 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=38553 (Thu May 28 11:30:51 2020) host-id=1 score=3400 vm_conf_refresh_time=38553 (Thu May 28 11:30:51 2020) conf_on_shared_storage=True maintenance=False state=EngineUp stopped=False [root@novirt2 ~]#
Now go to test further functionalities.....
Gianluca
On Thu, May 28, 2020 at 12:02 PM Lucia Jelinkova <ljelinko@redhat.com> wrote:
I think you have the same problem as Mark - the cluster is set to use the secure variant of the CPU type but your host does not support all the necessary flags.
Intel Cascadelake Server Family - the VM is run with Cascadelake-Server,-hle,-rtm,+arch-capabilities Secure Intel Cascadelake Server Family - the VM is run with Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+tsx-ctrl,+arch-capabilities
The cluster should be set to Intel Cascadelake Server Family. I am not familiar with the HE setup process - have you specified the cluster CPU type manually or is it auto-assigned?
No, the user is not allowed to force a specific CPU type for the cluster: hosted-engine-setup process will simply register the first host into the Default cluster and the engine will detect the CPU type and configure the cluster according to the detected CPU. I tend to think that this is a bug in the code that chooses the CPU type for the cluster according to what the host reports.
Lucia
On Thu, May 28, 2020 at 11:32 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 11:00 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
any input to solve and at least try some features of 4.4 on this hw env?
Thanks, Gianluca
it seems I was able to have it recognized this way:
strip these lines from he.xml
<ovirt-vm:exitCode type="int">1</ovirt-vm:exitCode> <ovirt-vm:exitMessage>unsupported configuration: unknown CPU
feature: tsx-ctrl</ovirt-vm:exitMessage>
<ovirt-vm:exitReason type="int">1</ovirt-vm:exitReason>
13a17
<ovirt-vm:startTime
type="float">1590653346.0201075</ovirt-vm:startTime> 246a251,264
<graphics type='vnc' port='-1' autoport='yes' keymap='en-us'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> </graphics> <graphics type='spice' autoport='yes'
passwdValidTo='1970-01-01T00:00:01'>
<listen type='network' network='vdsm-ovirtmgmt'/> <channel name='main' mode='secure'/> <channel name='display' mode='secure'/> <channel name='inputs' mode='secure'/> <channel name='cursor' mode='secure'/> <channel name='playback' mode='secure'/> <channel name='record' mode='secure'/> <channel name='smartcard' mode='secure'/> <channel name='usbredir' mode='secure'/> </graphics>
I have to understand what to use for graphics but it is not important at the moment.... After failure of deployment the link to the engine storage domain under /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/ has been deleted. So I go there and create it:
[root@novirt2 ~]# cd /var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/
[root@novirt2 ~]# ln -s /rhev/data-center/mnt/glusterSD/novirt2st.storage.local\:_engine/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/images/df02bff9-2c4b-4e14-a0a3-591a84ccaed9 df02bff9-2c4b-4e14-a0a3-591a84ccaed9
Start VM [root@novirt2 ~]# virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf create he.xml Domain HostedEngine created from he.xml
Access its webadmin gui and now all is as expected, with only one engine vm... Volumes for data and vmstore are there, but not the storage domains, but I can create them without problems
Also if I exit the global maintenance the state passes from ReinitializeFSM to EngineUp
[root@novirt2 ~]# hosted-engine --vm-status
--== Host novirt2.example.net (id: 1) status ==--
Host ID : 1 Host timestamp : 38553 Score : 3400 Engine status : {"vm": "up", "health": "good", "detail": "Up"} Hostname : novirt2.example.net Local maintenance : False stopped : False crc32 : 5a3b40e1 conf_on_shared_storage : True local_conf_timestamp : 38553 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=38553 (Thu May 28 11:30:51 2020) host-id=1 score=3400 vm_conf_refresh_time=38553 (Thu May 28 11:30:51 2020) conf_on_shared_storage=True maintenance=False state=EngineUp stopped=False [root@novirt2 ~]#
Now go to test further functionalities.....
Gianluca
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5OC6B4AF52T4T4...
Il Gio 28 Mag 2020, 11:59 Lucia Jelinkova <ljelinko@redhat.com> ha scritto:
The cluster should be set to Intel Cascadelake Server Family. I am not familiar with the HE setup process - have you specified the cluster CPU type manually or is it auto-assigned?
Lucia
Autoassigned. In fact in another thread I asked if possible to get option to choose during hosted engine wizard...
It is really strange, the host should not even be in UP state if it does not comply with the cluster cpu type settings. Would you mind sharing the engine.log? And if that's not much of a trouble, could you have a look into the database? select cpu_flags from vds_dynamic; select name, cpu_name, cpu_flags, cpu_verb from cluster; Thanks, Lucia On Thu, May 28, 2020 at 12:10 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
Il Gio 28 Mag 2020, 11:59 Lucia Jelinkova <ljelinko@redhat.com> ha scritto:
The cluster should be set to Intel Cascadelake Server Family. I am not familiar with the HE setup process - have you specified the cluster CPU type manually or is it auto-assigned?
Lucia
Autoassigned. In fact in another thread I asked if possible to get option to choose during hosted engine wizard...
On Thu, May 28, 2020 at 3:01 PM Lucia Jelinkova <ljelinko@redhat.com> wrote:
It is really strange, the host should not even be in UP state if it does not comply with the cluster cpu type settings.
Would you mind sharing the engine.log?
And if that's not much of a trouble, could you have a look into the database?
select cpu_flags from vds_dynamic; select name, cpu_name, cpu_flags, cpu_verb from cluster;
Thanks,
Lucia
engine.log.gz here: https://drive.google.com/file/d/1fclwgYhFUU1qgFodv4V_sQauIvmMvSS2/view?usp=s... cpu_flags | mce,xsavec,sep,fma,rtm,xgetbv1,arat,lm,mpx,tsc,avx512vl,avx512f,ssbd,avx,vpid,pdpe1gb,cpuid,vme,tsc_deadline_timer,avx2,mca,clwb,tsc_adjust,lahf_lm,ibrs,pcid,erms,x2apic,skip-l1dfl-vmentry,de,sse4_2,msr,syscall,invpcid,clflushopt,pat,sse4_1,avx512_vnni,hypervisor,fsgsbase,nopl,xsaveopt,vmx,md-clear,ibpb,pae,md_clear,pse36,pclmulqdq,avx512dq,movbe,hle,adx,rep_good,3dnowprefetch,bmi1,bmi2,avx512bw,umip,mtrr,pni,xsave,pse,vnmi,fxsr,aes,popcnt,rdtscp,rdseed,sse2,avx512cd,cx8,smap,nx,invpcid_single,arch-capabilities,arch_perfmon,ibrs-all,rdrand,abm,tsc_known_freq,mds-no,apic,rdctl-no,cmov,stibp,smep,mmx,constant_tsc,pge,pku,ss,ospke,clflush,tpr_shadow,cx16,ssse3,ept,xtopology,ht,arch_capabilities,flexpriority,fpu,f16c,sse,ibrs_enhanced,model_kvm64,model_Skylake-Server,model_core2duo,model_Opteron_G2,model_Skylake-Server-IBRS,model_Broadwell,model_Haswell,model_kvm32,model_IvyBridge-IBRS,model_Cascadelake-Server,model_Nehalem-IBRS,model_Broadwell-IBRS,model_qemu64,model_qemu32,model_Haswell-noTSX,model_Nehalem,model_pentium,model_SandyBridge,model_IvyBridge,model_Westmere-IBRS,model_Haswell-IBRS,model_Westmere,model_Broadwell-noTSX,model_Skylake-Client-IBRS,model_pentium3,model_coreduo,model_486,model_Skylake-Client,model_Haswell-noTSX-IBRS,model_Broadwell-noTSX-IBRS,model_Conroe,model_n270,model_pentium2,model_SandyBridge-IBRS,model_Penryn,model_Opteron_G1 for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations 1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag.... so the query returns now: engine=# select name, cpu_name, cpu_flags, cpu_verb from cluster; -[ RECORD 1 ]---------------------------------------------- name | Default cpu_name | Intel Cascadelake Server Family cpu_flags | vmx,model_Cascadelake-Server cpu_verb | Cascadelake-Server,-hle,-rtm,+arch-capabilities Gianluca
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote: [snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type. I'm also able to connect to its console from web admin gui The command line generated now is: qemu 29450 1 43 23:38 ? 00:03:09 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T21:38:21,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-b7696ce2-fd8c-4856-8c38-197fc520271b,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=43:44,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=45:46 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=47,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=50,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -device usb-tablet,id=input0,bus=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67.0,port=1 -vnc 172.19.0.224:3,password -k en-us -spice port=5904,tls-port=5905,addr=172.19.0.224,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on Only aftermath is that in web admin gui I still see the temporary engine marked as "external-HostedEngineLocal" and I'm quite afraid of deleting it and having some effects on real engine one...
When you look into the detail of external-HostedEngineLocal, Genral tab, what is its Origin? On Thu, May 28, 2020 at 11:53 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type. I'm also able to connect to its console from web admin gui
The command line generated now is:
qemu 29450 1 43 23:38 ? 00:03:09 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T21:38:21,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-b7696ce2-fd8c-4856-8c38-197fc520271b,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=43:44,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=45:46 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=47,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=50,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -device usb-tablet,id=input0,bus=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67.0,port=1 -vnc 172.19.0.224:3,password -k en-us -spice port=5904,tls-port=5905,addr=172.19.0.224,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
Only aftermath is that in web admin gui I still see the temporary engine marked as "external-HostedEngineLocal" and I'm quite afraid of deleting it and having some effects on real engine one...
Based on my vague memories from Dec 2018, I think I got a similar situation where I had to delete that external Engine. Of course that was on 4.2.7 and the story here can be different. If you use gluster, you can power off the engine (Global Maintenance) and then create a gluster snapshot before wiping the remnants. Best Regards, Strahil Nikolov На 29 май 2020 г. 0:52:46 GMT+03:00, Gianluca Cecchi <gianluca.cecchi@gmail.com> написа:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to
Cascadelake Server Family" from web admin gui and now I have to try
"Intel these
steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type. I'm also able to connect to its console from web admin gui
The command line generated now is:
qemu 29450 1 43 23:38 ? 00:03:09 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T21:38:21,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-b7696ce2-fd8c-4856-8c38-197fc520271b,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=43:44,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=45:46 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=47,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=50,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -device usb-tablet,id=input0,bus=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67.0,port=1 -vnc 172.19.0.224:3,password -k en-us -spice port=5904,tls-port=5905,addr=172.19.0.224,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
Only aftermath is that in web admin gui I still see the temporary engine marked as "external-HostedEngineLocal" and I'm quite afraid of deleting it and having some effects on real engine one...
On Thu, May 28, 2020 at 11:56 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type.
As expected, in my opinion now the point is just about understanding why the engine detected your host with the wrong CPU features set. To be fully honest, as you can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/READM... , we already have a variable (he_cluster_cpu_type) to force a cluster CPU type from the ansible role but I don't think is exposed in the interactive installer.
I'm also able to connect to its console from web admin gui
The command line generated now is:
qemu 29450 1 43 23:38 ? 00:03:09 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T21:38:21,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-b7696ce2-fd8c-4856-8c38-197fc520271b,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=43:44,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=45:46 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=47,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=50,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -device usb-tablet,id=input0,bus=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67.0,port=1 -vnc 172.19.0.224:3,password -k en-us -spice port=5904,tls-port=5905,addr=172.19.0.224,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
Only aftermath is that in web admin gui I still see the temporary engine marked as "external-HostedEngineLocal" and I'm quite afraid of deleting it and having some effects on real engine one...
That one is just a leftover from the install process. It's normally automatically cleaned up as one of the latest actions in the ansible role used for the deployment. I suspect that, due to the wrongly detected CPU type, in your case something failed really close to the end of the deployment and so the leftover: you can safely manually delete it.
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VTMEQVPPUCVLQK...
On Fri, May 29, 2020 at 9:34 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Thu, May 28, 2020 at 11:56 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type.
As expected, in my opinion now the point is just about understanding why the engine detected your host with the wrong CPU features set.
To be fully honest, as you can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/READM... , we already have a variable (he_cluster_cpu_type) to force a cluster CPU type from the ansible role but I don't think is exposed in the interactive installer.
Can I artificially set it into a playbook, just to verify correct completion of setup workflow or do you think that it will be any way overwritten at run time by what detected? It is not clear in my opinion what does it mean the sentence: "cluster CPU type to be used in hosted-engine cluster (the same as HE host or lower)" With "as HE host" does it mean what gotten from vdsm capabilities or what?
That one is just a leftover from the install process. It's normally automatically cleaned up as one of the latest actions in the ansible role used for the deployment. I suspect that, due to the wrongly detected CPU type, in your case something failed really close to the end of the deployment and so the leftover: you can safely manually delete it.
Yes, the deploy failed because it was not anle to detected final engine as up....
As asked by Lucia, the Origin of the VM was "External" The VM had no disks and no network interfaces. I was able to remove it without problems at the moment. Thanks, Gianluca
Hi, I think I found the problem - our definition of CPU types in the database is not correct. We do autodetection of the CPU type based on the CPU flags but they are not in sync with what we send to the VDSM. https://github.com/oVirt/ovirt-engine/blob/874e390a40ee2f23ea108955e2946c7c4... And yes, it happened for both your servers, Secure Intel Cascadelake Server Family and Secure AMD EPYC. As for the he_cluster_cpu_type option, I think those are taken from the DB configuration as well. In your case it should be "Intel Cascadelake Server Family". Regards, Lucia On Fri, May 29, 2020 at 11:39 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Fri, May 29, 2020 at 9:34 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Thu, May 28, 2020 at 11:56 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type.
As expected, in my opinion now the point is just about understanding why the engine detected your host with the wrong CPU features set.
To be fully honest, as you can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/READM... , we already have a variable (he_cluster_cpu_type) to force a cluster CPU type from the ansible role but I don't think is exposed in the interactive installer.
Can I artificially set it into a playbook, just to verify correct completion of setup workflow or do you think that it will be any way overwritten at run time by what detected? It is not clear in my opinion what does it mean the sentence: "cluster CPU type to be used in hosted-engine cluster (the same as HE host or lower)" With "as HE host" does it mean what gotten from vdsm capabilities or what?
That one is just a leftover from the install process. It's normally automatically cleaned up as one of the latest actions in the ansible role used for the deployment. I suspect that, due to the wrongly detected CPU type, in your case something failed really close to the end of the deployment and so the leftover: you can safely manually delete it.
Yes, the deploy failed because it was not anle to detected final engine as up....
As asked by Lucia, the Origin of the VM was "External" The VM had no disks and no network interfaces. I was able to remove it without problems at the moment. Thanks, Gianluca
Thank you Lucia, that's good news to hear. Is this something that might be fixed up in the 4.4.1 release whenever it comes along, and/or something simple to patch to get a clean deploy using 4.4.0? I have the servers and time to experiment with for now. If I can somehow edit that '4:Secure AMD EPYC:svm,nx,ibpb,ssbd,model_EPYC:EPYC,+ibpb,+virt-ssbd:x86_64; ' line to instead use amd-ssbd before it's added to the vdc_options table, and thus used for the HostedEngine config, I think it'd work. I could be wildly wrong, I'm basing that guess on the fact that all I have to do is swap 'virt-ssbd' with 'amd-ssbd' in the xml and can start the failed-to-deploy HostedEngine VM right up. Thank you! Mark
On Fri, May 29, 2020 at 11:39 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Fri, May 29, 2020 at 9:34 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Thu, May 28, 2020 at 11:56 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type.
As expected, in my opinion now the point is just about understanding why the engine detected your host with the wrong CPU features set.
To be fully honest, as you can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/READM... , we already have a variable (he_cluster_cpu_type) to force a cluster CPU type from the ansible role but I don't think is exposed in the interactive installer.
Can I artificially set it into a playbook, just to verify correct completion of setup workflow or do you think that it will be any way overwritten at run time by what detected?
The interactive installer is not passing it and the default behaviour is to omit the parameter if the variable is unset to let the engine wisely detect and choose. https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/tasks... So yes, you can simply inject a custom value It is not clear in my opinion what does it mean the sentence: "cluster CPU
type to be used in hosted-engine cluster (the same as HE host or lower)" With "as HE host" does it mean what gotten from vdsm capabilities or what?
Read this as "as first HE host". This parameter can be useful if you plan to add older hosts in the future and you prefer to start from the beginning with a CPU type for the cluster that's less required than what can be detected from the first one. I tend to think that in the past the set of CPU features was monotonically increasing between a CPU family and a newer one. This is less easy now with all the different security patches.
That one is just a leftover from the install process. It's normally automatically cleaned up as one of the latest actions in the ansible role used for the deployment. I suspect that, due to the wrongly detected CPU type, in your case something failed really close to the end of the deployment and so the leftover: you can safely manually delete it.
Yes, the deploy failed because it was not anle to detected final engine as up....
As asked by Lucia, the Origin of the VM was "External" The VM had no disks and no network interfaces. I was able to remove it without problems at the moment. Thanks, Gianluca
I believe your problem is similar to Mark’s, you need qemu-kvm’s virtualized tsx-ctrl that was added to qemu 4.2 upstream and (I think) also the corresponding kernel-4.18.0-147.4.1.el8_1 can you confirm your versions? all of these issues should be addressed in 8.2, we’re just still waiting for that:/
On 28 May 2020, at 23:52, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Thu, May 28, 2020 at 3:09 PM Gianluca Cecchi <gianluca.cecchi@gmail.com <mailto:gianluca.cecchi@gmail.com>> wrote:
[snip]
for the cluster type in the mean time I was able to change it to "Intel Cascadelake Server Family" from web admin gui and now I have to try these steps and see if engine starts automatically without manual operations
1) set global maintenance 2) shutdown engine 3) exit maintenance 4) see if the engine vm starts without the cpu flag....
I confirm that point 4) was successful and engine vm was able to autostart, after changing cluster type. I'm also able to connect to its console from web admin gui
The command line generated now is:
qemu 29450 1 43 23:38 ? 00:03:09 /usr/libexec/qemu-kvm -name guest=HostedEngine,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-HostedEngine/master-key.aes -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Cascadelake-Server,hle=off,rtm=off,arch-capabilities=on -m size=16777216k,slots=16,maxmem=67108864k -overcommit mem-lock=off -smp 2,maxcpus=32,sockets=16,cores=2,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-31,mem=16384 -uuid b572d924-b278-41c7-a9da-52c4f590aac1 -smbios type=1,manufacturer=oVirt,product=RHEL,version=8-1.1911.0.9.el8,serial=d584e962-5461-4fa5-affa-db413e17590c,uuid=b572d924-b278-41c7-a9da-52c4f590aac1,family=oVirt -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,fd=40,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2020-05-28T21:38:21,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 -device pcie-root-port,port=0x20,chassis=17,id=pci.17,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.18,bus=pci.1,addr=0x0 -device qemu-xhci,p2=8,p3=8,id=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67,bus=pci.4,addr=0x0 -device virtio-scsi-pci,iothread=iothread1,id=ua-b7696ce2-fd8c-4856-8c38-197fc520271b,bus=pci.5,addr=0x0 -device virtio-serial-pci,id=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad,max_ports=16,bus=pci.3,addr=0x0 -drive if=none,id=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,readonly=on -device ide-cd,bus=ide.2,drive=drive-ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,id=ua-fa671f6c-dc42-4c59-a66d-ccfa3d5d422b,werror=report,rerror=report -drive file=/var/run/vdsm/storage/3df8f6d4-d572-4d2b-9ab2-8abc456a396f/df02bff9-2c4b-4e14-a0a3-591a84ccaed9/bf435645-2999-4fb2-8d0e-5becab5cf389,format=raw,if=none,id=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,cache=none,aio=threads -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.6,addr=0x0,drive=drive-ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,id=ua-df02bff9-2c4b-4e14-a0a3-591a84ccaed9,bootindex=1,write-cache=on,serial=df02bff9-2c4b-4e14-a0a3-591a84ccaed9,werror=stop,rerror=stop -netdev tap,fds=43:44,id=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,vhost=on,vhostfds=45:46 -device virtio-net-pci,mq=on,vectors=6,host_mtu=1500,netdev=hostua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,id=ua-b29ca99f-a53e-4de7-8655-b65ef4ba5dc4,mac=00:16:3e:0a:96:80,bus=pci.2,addr=0x0 -chardev socket,id=charserial0,fd=47,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=48,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=49,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,fd=50,server,nowait -device virtserialport,bus=ua-608f9599-30b2-4ee6-a0d3-d5fb588583ad.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -device usb-tablet,id=input0,bus=ua-b630a65c-8156-4542-b8e8-98b4d2c48f67.0,port=1 -vnc 172.19.0.224:3 <http://172.19.0.224:3/>,password -k en-us -spice port=5904,tls-port=5905,addr=172.19.0.224,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=ua-ac7aed4d-d824-40f9-aaa8-1c0be702e38c,ram_size=67108864,vram_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device intel-hda,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854,bus=pci.18,addr=0x1 -device hda-duplex,id=ua-c7078e28-5585-4866-bdbd-528ebddd8854-codec0,bus=ua-c7078e28-5585-4866-bdbd-528ebddd8854.0,cad=0 -device virtio-balloon-pci,id=ua-fc4f6b20-0b17-4198-b059-b5753893584d,bus=pci.7,addr=0x0 -object rng-random,id=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,filename=/dev/urandom -device virtio-rng-pci,rng=objua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,id=ua-c4c3e5e7-1c19-4582-a87c-4f3fee4a0ee5,bus=pci.8,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
Only aftermath is that in web admin gui I still see the temporary engine marked as "external-HostedEngineLocal" and I'm quite afraid of deleting it and having some effects on real engine one...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VTMEQVPPUCVLQK...
Hi Lucia,
the cluster is set to use the secure variant of the CPU type but your host does not support all the necessary flags.
For my deployments the system is auto-detecting and using the 'Secure AMD EPYC' CPU type. Note that the HostedEngineLocal VM does run with 'amd-ssbd' and works fine (amd-ssbd is supposed to be faster/preferred over virt-ssbd according to another thread I found with a comment from Tom Landsky at AMD). So the CPUs can definitely run in a a secure configuration, they just need the 'amd-ssbd' flag instead of 'virt-ssbd'. I don't know why the local HE VM runs with the correct flag, but then the final one does not... If I can figure that out I'll be much further along towards a resolution. Thanks! Mark
Oof, I butchered his last name, but the quote I found was this: "Details: Tom Lendacky from AMD writes[4] -- "The idea behind 'virt-ssbd' was to provide an architectural method for a guest to do SSBD when 'amd-ssbd' isn't present. The 'amd-ssbd' feature will use SPEC_CTRL which is intended to not be intercepted and will be fast. The use of 'virt-ssbd' will always be intercepted and therefore will not be as fast. So a guest should be presented with 'amd-ssbd', if available, in preference to 'virt-ssbd'. It seems for these EPYC CPUs with qemu on 8.1, virt-ssbd isn't an option anymore and it has to be amd-ssbd for the VM to work?
I should have also mentioned that I found an existing report for the issue I'm having on RedHat's Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1783180 Mark
On 28 May 2020, at 16:26, Mark R <ovirtlist@beanz.33mail.com> wrote:
I should have also mentioned that I found an existing report for the issue I'm having on RedHat's Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1783180
And that is the one behind the problem In el8 this has been fixed by https://bugzilla.redhat.com/show_bug.cgi?id=1797092 kernel-4.18.0-147.8.el8. That is only in 8.2 (or probably CentOS Stream, but ew don’t have a CI or build system set up for it) It’s probably easier to just update only the kernel from CentOS Stream and it may work good enough We still use virt-ssbd in Secure type because it’s supposed to give you a mitigation in any case. You can always choose the insecure EPYC variant which doesn’t enable any of that. But you have to choose that explicitly Thanks, michal
Mark _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IDIH7L3QLX6TPE...
participants (7)
-
Gianluca Cecchi -
Lucia Jelinkova -
Mark R -
Michal Skrivanek -
Ryan Bullock -
Simone Tiraboschi -
Strahil Nikolov