- Users - oVirt List Archives

Can't find storage server connection
by francesco＠shellrent.com 23 Nov '20

23 Nov '20

Hi all, I'm using oVirt SDK python for retrieving info about storage domain, in several hosts (centos7/ovirt4.3 and centos8/ovirt4.4), but the script exits with the following error in some of them: Traceback (most recent call last): File "get_uuid.py", line 70, in <module> storage_domain = sds_service.list(search='name=data-foo')[0] File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/services.py", line 26296, in list return self._internal_get(headers, query, wait) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '92444a95-0be7-4589-ac46-1ed6dfe7ed4c'.". HTTP response code is 500. The portion of the script that search for the storage domain is the following: sds_service = connection.system_service().storage_domains_service() storage_domain = sds_service.list(search='name={}'.format(storage_domain_name))[0] Now: I have no real clue on which ID "92444a95-0be7-4589-ac46-1ed6dfe7ed4c'" but digging in the engine logs it refers to StorageServerConnections ID: [root@ovirt-engine ovirt-engine]# zgrep 92444a95-0be7-4589-ac46-1ed6dfe7ed4c *.gz engine.log-20201108.gz:2020-11-07 06:05:54,352+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) [3ffb810c] START, ConnectStorageServerVDSCommand(HostName = another-server.foo.com, StorageServerConnectionManagementVDSParameters:{hostId='7d202bc7-002b-4426-8446-99b6b346874e', storagePoolId='82d0b3de-0334-451c-8321-c3533de9a894', storageType='LOCALFS', connectionList='[StorageServerConnections:{id='92444a95-0be7-4589-ac46-1ed6dfe7ed4c', connection='/data', iqn='null', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 1509fe17 As I said, I tried to execute the script from several hosts: some of them with oVirt 4.3 and other with oVirt 4.4 but it can run, or not, on both of versions. When I try to manage the storage domain via ovirt-engine GUI on the hosts that the script exit with the mentioned error, I recieve the following error: Uncaught exception occurred. Please try reloading the page. Details: (TypeError) : Cannot read property 'a' of null Please have your administrator check the UI logs Any slightest idea on what is going on? Thank you for your time and help, Francesco

2 3

Best and clean way to have oVirt qemu and libvirt versions on CentOS 8
by Gianluca Cecchi 22 Nov '20

22 Nov '20

Hello, normally with current CentOS 8.2 I get qemu-kvm-core-15:2.12.0-99.module_el8.2.0+524+f765f7e0.4.x86_64 libvirt-daemon-4.5.0-42.module_el8.2.0+320+13f867d7.x86_64 With oVirt 4.4 having qemu-kvm-core-4.2.0-29.el8.3.x86_64 libvirt-daemon-6.0.0-25.2.el8.x86_64 and in current Fedora 32 updates: qemu-kvm-core-4.2.1-1.fc32.x86_64.rpm libvirt-daemon-6.1.0-4.fc32.x86_64.rpm I see on CentOS these groups somehow related to Virtualization: Virtualization Host Virtualization Client Virtualization Hypervisor Virtualization Platform Virtualization Tools What could be the best and less intrusive repo to enable on plain CentOS 8.2 to get qemu-kvm and libvirt versions quite similar to oVirt shipped ones, or fedora ones, without enabling all the ovirt 4.4 repos? Thanks, Gianluca

2 2

ovirt-imageio-proxy not working after updating SSL certificates with a wildcard cert issued by AlphaSSL (intermediate)
by Lynn Dixon 22 Nov '20

22 Nov '20

All, I recently bought a wildcard certificate for my lab domain (shadowman.dev) and I replaced all the certs on my RHV4.3 machine per our documentation. The WebUI presents the certs successfully and without any issues, and everything seemed to be fine, until I tried to upload a disk image (or an ISO) to my storage domain. I get this error in the events tab: https://share.getcloudapp.com/p9uPvegx [image: image.png] I also see that the disk is showing up in my storage domain, but its showing "Paused by System" and I can't do anything with it. I cant even delete it! I have tried following this document to fix the issue, but it didn't work: https://access.redhat.com/solutions/4148361 I am seeing this error pop into my engine.log: https://pastebin.com/kDLSEq1A And I see this error in my image-proxy.log: WARNING 2020-07-24 15:26:34,802 web:137:web:(log_error) ERROR [172.17.0.30] PUT /tickets/ [403] Error verifying signed ticket: Invalid ovirt ticket (data='------my_ticket_data-----', reason=Untrusted certificate) [request=0.002946/1] Now, when I bought my wildcard, I was given a root certificate for the CA, as well as a separate intermediate CA certificate from the provider. Likewise, they gave me a certificate and a private key of course. The root and intermediate CA's certificates have been added to /etc/pki/ca-trust/source/anchors/ and I did an update-ca-trust. I also started experiencing issues with the ovpn network provider at the same time I replaced the SSL certs, but I disregarded it at the time, but now I am thinking its related. Any advice on what to look for to fix the ovirt-imageio-proxy? Thanks! *Lynn Dixon* | Red Hat Certified Architect #100-006-188 *Solutions Architect* | NA Commercial Google Voice: 423-618-1414 Cell/Text: 423-774-3188 Click here to view my Certification Portfolio <http://red.ht/1XMX2Mi>

6 8

Re: Fix corrupt self-hosted engine
by Alex K 22 Nov '20

22 Nov '20

For the records, After having fixed the major fs issues with guestfish and since the DB was not starting up, I removed everything from DB data dir and recreated it as below: rm -rf /var/opt/rh/rh-postgresql10/lib/pgsql/data/* /opt/rh/rh-postgresql10/root/usr/bin/postgresql-setup --initdb systemctl restart rh-postgresql10-postgresql.service Then proceeded with the restoration, where I requested to provision all missing databases: engine-backup --mode=restore --file=engine-backup.gz --provision-all-databases \ --log=restore.log --restore-permissions Following this, ran engine-setup, as instructed from the restore operation. Gained engine web access and saw the same running VMs were shown as up without issues. I only observed one VM not able to start due to illegal volume, but that's another story. On Thu, Nov 19, 2020 at 9:42 PM Alex K <rightkicktech(a)gmail.com> wrote: > > > On Thu, Nov 19, 2020 at 5:31 PM Alex K <rightkicktech(a)gmail.com> wrote: > >> Hi Didi, >> >> On Thu, Nov 19, 2020 at 5:13 PM Yedidyah Bar David <didi(a)redhat.com> >> wrote: >> >>> On Thu, Nov 19, 2020 at 4:37 PM Alex K <rightkicktech(a)gmail.com> wrote: >>> >>>> Hi all, >>>> >>>> I have a corrupt self-hosted engine (with several file system errors, >>>> postgres not able to start) and thus it does not give access to the web UI. >>>> This happened following an unlucky split brain resolution (I am running 2 >>>> nodes). The two hosts are running VMs also which I would like to keep >>>> running as they are needed. >>>> >>>> When trying to boot into rescue mode (using >>>> systemd.unit=emergency.target boot parameter) I get a cursor and nothing >>>> else. >>>> >>> >>> This means that more than just the DB is corrupt... >>> >>> >>>> >>>> I have backups of engine files with scope all (using the engine-backup >>>> tool). >>>> What is the best approach to try and fix the engine or redeploy. >>>> >>> >>> If you are careful, and know what you are doing, you can try something >>> like the following. I am not giving many details, hopefully you can find on >>> the net tutorials about how to use the things I suggest: >>> >>> 1. Move to global maintenance >>> >>> 2. Stop the current dead vm (if needed) >>> >>> 3. Find current vm conf, edit it to boot from a rescue iso image of your >>> preference or from net/PXE etc., and start the vm with '--vm-conf' pointing >>> to your edited file. >>> >>> 4. Connect a console (hosted-engine --console, or 'virsh console', or >>> use '--add-console-password' and remote viewer, if needed) >>> >>> 5. Clean the disk and install the OS, oVirt, etc. >>> >>> 6. Copy your backup into the vm and restore with engine-backup >>> >>> 7. Then cleanly stop the machine, exit global maint, and let HA start it >>> (or start it yourself with --vm-start). >>> >>> At the time, we had a bug [1] to document this. The result is [2]. It >>> does not detail how to boot/reinstall os/etc., only restore (if e.g. db is >>> dead but fs is ok). >>> For something somewhat similar to what you want, see also [3], which >>> uses guestfish. Might be useful, depending on how badly your disk is >>> corrupted. >>> >> I went with the guestfish approach. It has fixed some fs issues and now >> the yum etc seem fine apart from postgres. >> I had tried previously to uninstall/install packages so I ended >> installing them again with yum install ovirt\*setup\*. >> Now I think I have to run engine-setup but I get the error: >> >> Failed to execute stage 'Environment setup': Cannot connect to Engine >> database using existing credentials: engine@localhost:5432 >> > Seems that I need to have psql running to be able to run engine-backup > --mode=restore. Are there any steps how one could manually prepare pgsql > for ovirt so as to attempt restoration? > >> >> So I guess I need to follow [2]. What do you think? >> >> >>> How did you run into a split brain? There is a lock on the shared >>> storage that should prevent this. >>> >>> Good luck and best regards, >>> >>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1482710 >>> [2] >>> https://www.ovirt.org/documentation/administration_guide/#Overwriting_a_Sel… >>> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1569827#c4 >>> -- >>> Didi >>> >>

2 1

User portal and DNS entries removal
by Nathanaël Blanchet 21 Nov '20

21 Nov '20

2 1

Can't delete a disk after a failed upload
by lorenzobarbati02＠gmail.com 21 Nov '20

21 Nov '20

Hi, After trying to upload an ISO via the web interface, the upload remains in the "paused by system" state. When I click "cancel" in the upload menu the status stops at "finalizing cleanup". Is there an alternative way to delete this disk?

1 0

Found a host rebooting - ways to watch?
by Chris Adams 20 Nov '20

20 Nov '20

I just noticed that one of my oVirt physical hosts has been rebooting due to an apparent hardware voltage fault. It's a Dell, and I've got their tools installed and am monitoring status, but the issue clears itself. It has apparently been doing this for a bit now, and we didn't catch it because (a) there weren't any VMs on it (probably were the first time but they were restarted elsewhere fast enough that it wasn't noticed) and (b) it reboots fast enough that at most it pops up in our monitoring system for one pass and then clears so our NOC either didn't see it or assumed it was okay since it cleared. oVirt has been logging alerts when it happens, but seeing that requires someone to log in and check the logs (and we've got a bunch of different systems to manage, including multiple oVirt clusters, so nobody is doing that on a regular basis). We monitor most things with SNMP and/or CLI checks (we have PRTG, Nagios, and LibreNMS for various different things). What are people doing to monitor the health of their oVirt systems? Is it possible to get alerts emailed to admins? Is there any SNMP support in oVirt to allow external systems to monitor its health? This setup is on 4.3.10 if that matters. -- Chris Adams <cma(a)cmadams.net>

2 2

Re: EPYC CPU not being detected correctly on cluster
by Lucia Jelinkova 20 Nov '20

20 Nov '20

Hi, oVirt CPU detection depends on libvirt (and that depends on qemu) CPU models. Could you please run the following command to see what libvirt reports? virsh domcapabilities That should give you the list of CPUs known to libvirt with a usability flag for each CPU. If you find out that the CPU is not usable by libvirt, you might want to dig deeper by querying quemu directly. Locate any VM running on the system by sudo virsh list --all Use the name of a VM in the following command: sudo virsh qemu-monitor-command [your-vm's-name] --pretty '{"execute":"query-cpu-definitions"}' That would give you the list of all CPUs supported by qemu and it will list all cpu's features that are not available on your system. Regards, Lucia On Thu, Nov 19, 2020 at 9:38 PM Vinícius Ferrão via Users <users(a)ovirt.org> wrote: > Hi > > > > I’ve an strange issue with two hosts (not using the hypervisor image) with > EPYC CPUs, on the engine I got this message: > > > > The host CPU does not match the Cluster CPU Type and is running in a > degraded mode. It is missing the following CPU flags: model_EPYC. Please > update the host CPU microcode or change the Cluster CPU Type. > > > > But it is an EPYC CPU, the firmware is updated to the latest versions, but > for some reason oVirt does not like it. > > > > Here’s the relevant output from VDSM: > > "cpuCores": "128", > > "cpuFlags": > "ibs,vme,abm,sep,ssse3,perfctr_core,sse4_2,skip-l1dfl-vmentry,cx16,pae,misalignsse,avx2,smap,movbe,vgif,rdctl-no,extapic,clflushopt,de,sse4_1,xsaveerptr,perfctr_llc,fma,mca,sse,rdtscp,monitor,umip,mwaitx,cr8_legacy,mtrr,stibp,bmi2,pclmulqdq,amd-ssbd,lbrv,pdpe1gb,constant_tsc,vmmcall,f16c,ibrs,fsgsbase,invtsc,nopl,lm,3dnowprefetch,smca,ht,tsc_adjust,popcnt,cpb,bmi1,mmx,arat,aperfmperf,bpext,cqm_occup_llc,virt-ssbd,tce,pse,xsave,xgetbv1,topoext,sha_ni,amd_ppin,rdrand,cpuid,tsc_scale,extd_apicid,cqm,rep_good,tsc,sse4a,flushbyasid,pschange-mc-no,mds-no,ibpb,smep,clflush,tsc-deadline,fxsr,pat,avx,pfthreshold,v_vmsave_vmload,osvw,xsavec,cdp_l3,clzero,svm_lock,nonstop_tsc,adx,hw_pstate,spec-ctrl,arch-capabilities,xsaveopt,skinit,rdt_a,svm,rdpid,lahf_lm,fpu,rdseed,fxsr_opt,sse2,nrip_save,vmcb_clean,sme,cat_l3,cqm_mbm_local,irperf,overflow_recov,avic,mce,mmxext,msr,cx8,hypervisor,wdt,mba,nx,decodeassists,cmp_legacy,x2apic,perfctr_nb,succor,pni,xsaves,clwb,cqm_llc,syscall,apic,pge,npt,pse36,cmov,ssbd,pausefilter,sev,aes,wbnoinvd,cqm_mbm_total,spec_ctrl,model_qemu32,model_Opteron_G3,model_Nehalem-IBRS,model_qemu64,model_Conroe,model_kvm64,model_Penryn,model_SandyBridge,model_pentium,model_pentium2,model_kvm32,model_Nehalem,model_Opteron_G2,model_pentium3,model_Opteron_G1,model_SandyBridge-IBRS,model_486,model_Westmere-IBRS,model_Westmere", > > "cpuModel": "AMD EPYC 7H12 64-Core Processor", > > "cpuSockets": "2", > > "cpuSpeed": "3293.405", > > "cpuThreads": "256", > > > > Any ideia on why ou what to do to fix it? > > > > Thanks, > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/WP6XL6ODTLJVB… >

2 1

Upgrade hyperconverged self-hosted ovirt from 4.3 to 4.4
by ralf＠os-s.de 20 Nov '20

20 Nov '20

Hi, has anyone attempted an upgrade from 4.3 to 4.4 in a hyperconverged self-hosted setup? The posted guidelines seem a bit contradictive and not complete. Has anyone tried it and could share his experiences? I am currently having problems when deploying the hosted engine and restoring. The host becomes unresponsive and has hung tasks. Kind regards, Ralf

3 6

change domain name from export domain
by jb 20 Nov '20

20 Nov '20

Hello, I needed to rename all domain names from my NFS storage. VM data and seconds data domains was no problem. But when I try to rename the domain from export domain, a red popup on top opens with this message: Uncaught exception occurred. Please try reloading the page. Details: (TypeError) : ZOj(...) is null Please have your administrator check the UI logs I also notice that, when I rename the path an hit enter, it doesn't write anything to the engine.log Is this a known issue? Best regards Jonathan

1 0