- Users - oVirt List Archives

Failed to connect to the hypervisor
by Eugène Ngontang 28 Aug '20

28 Aug '20

Hi, I'm facing a strange issue with ovirt host. I can't list nodes status running virsh, vdsdm seems bad. I can't even delete the hosted engine, nor ping it, but I can still access my infra vm inside it[image: 😉] My outputs : > [root@moe ~]# /usr/sbin/ovirt-hosted-engine-cleanup > This will de-configure the host to run ovirt-hosted-engine-setup from > scratch. > Caution, this operation should be used with care. > Are you sure you want to proceed? [y/n] > y > -=== Destroy hosted-engine VM ===- > You must run deploy first > setlocale: No such file or directory > error: failed to connect to the hypervisor > error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No > such file or directory > -=== Stop HA services ===- > -=== Shutdown sanlock ===- > shutdown force 1 wait 0 > shutdown done -111 > -=== Disconnecting the hosted-engine storage domain ===- > You must run deploy first > -=== De-configure VDSM networks ===- > ovirtmgmt > ovirtmgmt > A previously configured management bridge has been found on the system, > this will try to de-configure it. Under certain circumstances you can loose > network connection. > Caution, this operation should be used with care. > Are you sure you want to proceed? [y/n] > y > -=== Stop other services ===- > -=== De-configure external daemons ===- > -=== Removing configuration files ===- > ? /etc/init/libvirtd.conf already missing > ? /etc/libvirt/nwfilter/vdsm-no-mac-spoofing.xml already missing > ? /etc/ovirt-hosted-engine/answers.conf already missing > ? /etc/ovirt-hosted-engine/hosted-engine.conf already missing > ? /etc/vdsm/vdsm.conf already missing > ? /etc/pki/vdsm/*/*.pem already missing > ? /etc/pki/CA/cacert.pem already missing > ? /etc/pki/libvirt/*.pem already missing > ? /etc/pki/libvirt/private/*.pem already missing > ? /etc/pki/ovirt-vmconsole/*.pem already missing > ? /var/cache/libvirt/* already missing > ? /var/run/ovirt-hosted-engine-ha/* already missing > [root@moe ~]# ping 192.168.33.143 > PING 192.168.33.143 (192.168.33.143) 56(84) bytes of data. > 64 bytes from 192.168.33.143: icmp_seq=1 ttl=64 time=0.132 ms > 64 bytes from 192.168.33.143: icmp_seq=2 ttl=64 time=0.082 ms > 64 bytes from 192.168.33.143: icmp_seq=3 ttl=64 time=0.072 ms > 64 bytes from 192.168.33.143: icmp_seq=4 ttl=64 time=0.083 ms > 64 bytes from 192.168.33.143: icmp_seq=5 ttl=64 time=0.075 ms > 64 bytes from 192.168.33.143: icmp_seq=6 ttl=64 time=0.082 ms > ^C > --- 192.168.33.143 ping statistics --- > 6 packets transmitted, 6 received, 0% packet loss, time 4999ms > rtt min/avg/max/mdev = 0.072/0.087/0.132/0.022 ms > [root@moe ~]# nodectl check > Status: WARN > Bootloader ... OK > Layer boot entries ... OK > Valid boot entries ... OK > Mount points ... OK > Separate /var ... OK > Discard is used ... OK > Basic storage ... OK > Initialized VG ... OK > Initialized Thin Pool ... OK > Initialized LVs ... OK > Thin storage ... OK > Checking available space in thinpool ... OK > Checking thinpool auto-extend ... OK > vdsmd ... BAD > [root@moe ~]# virsh -r list > setlocale: No such file or directory > error: failed to connect to the hypervisor > error: Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No > such file or directory > [root@moe ~]# Have you guys ever faced this issue? How can I access VM inside my virtual data center while the VM Manager is broken? Any solution to work around this please? Regards, Eugène NG -- LesCDN <http://lescdn.com> engontang(a)lescdn.com ------------------------------------------------------------ *Aux hommes il faut un chef, et au* * chef il faut des hommes!L'habit ne fait pas le moine, mais lorsqu'on te voit on te juge!*

1 0

Compatibilty issue
by carl langlois 28 Aug '20

28 Aug '20

Hi, I have upgraded ovirt to 4.3 and now I have some vm that I can start because of the cluster compatibility version. Those VM where in save state when I updated the cluster compatibility version. Now they are in a down state but if i try to run it I get this message. Cannot run VM. The Custom Compatibility Version of VM vmfpgatest (4.2) is not supported in Data Center compatibility version 4.3. any suggestions? Regards Carl

1 0

TroubleshootingoVirt Node deploy FQDN not reachable
by David White 27 Aug '20

27 Aug '20

In a recent thread, Roberto mentioned seeing the error message "FQDN Not Reachable" when trying to deploy oVirt Node 4.4.1, but was able to get past that error when using ovirt-node-ng-installer-4.4.2-2020080612.el8.iso. I experienced the same problems on oVirt Node install 4.4.1, so I tried the latest release of 4.4.2. When that failed, I went back and installed from the exact same image as Roberto said worked on the 4.4.2 branch: ovirt-node-ng-installer-4.4.2-2020080612.el8.iso Unfortunately, that's still not working for me - so that tells me I'm probably doing something wrong. Given the following facts: [root@dev1-centos ~]# hostname dev1-centos.office.barredowlweb.com [root@dev1-centos ~]# host dev1-centos.office.barredowlweb.com dev1-centos.office.barredowlweb.com has address 192.168.2.96 I am trying to install oVirt using the Hyperconverged Gluster Wizard for a Single Node. In the "Host1" box, I enter the full hostname: dev1-centos.office.barredowlweb.com And that's when I get the FQDN error message. Am I missing something here? Sent with ProtonMail Secure Email.

4 5

Is the Hosted Engine setup finished + Can't connect vdsm storage
by Bielej SRE 27 Aug '20

27 Aug '20

Hi, We have 6 node oVirt setup with Compatibility version 4.3 running on CentOS 7.6.1810 Recently We have found out some very interesting log entries on a few of our nodes. /var/log/ovirt-hosted-engine-ha/broker.log: MainThread::INFO::2020-08-27 12:51:50,279::storage_backends::345::ovirt_hosted_engine_ha.lib.storage_backends::(connect) Connecting the storage MainThread::INFO::2020-08-27 12:51:50,280::storage_server::349::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::warning::2020-08-27 12:51:50,284::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'NoneType' object has no attribute 'startswith' /var/log/messages Aug 27 12:52:11 ildeco25 vdsm[6909]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? We have inherited that setup from the previous team but I'm fairly certain the the Hosted Engine setup is finished. Also, this line is not present on all hosts. Should We worry about ovirt-ha-broker service not running on some hosts? host24 Active: inactive (dead) host25 Active: failed (Result: start-limit) since Thu 2020-08-27 12:56:13 CEST; 2s ago host26 Active: active (running) since Sun 2020-08-23 22:55:01 CEST; 3 days ago host27 Active: inactive (dead) host28 Active: failed (Result: start-limit) since Thu 2020-08-27 12:56:18 CEST; 4s ago host29 Active: inactive (dead) Currently the hosted engine VM is running on host26. I'd be very grateful for any help. Kind regards, Michal Bielejewski

2 1

Impossible to install Windows VMs on 4.4.1
by fgarat＠gmail.com 27 Aug '20

27 Aug '20

Hi, I'm having problem after I upgraded to 4.4.1 with Windows machines. The installation sees no disk. Even IDE disk doesn't get detected and installation won't move forward no matter what driver i use for the disk. Any one else having this issue?. Regards, Facundo

3 4

Host has no default route
by Wesley Stewart 27 Aug '20

27 Aug '20

I am trying to add a host to a single host cluster. Everything has gone pretty well so far, except when the host gets to the end of the installation process, I see an exclamation mark indicating "Host has no default route". The installation "finishes" and the host goes into a non-responsive state. I am not really sure where to check. I get a: host *name* installation has failed. Network error during communication with host. Running 4.3.6 Thanks!

2 1

[ANN] oVirt 4.4.2 Fifth Release Candidate is now available for testing
by Lev Veyde 27 Aug '20

27 Aug '20

oVirt 4.4.2 Fifth Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.4.2 Fifth Release Candidate for testing, as of August 27th, 2020. This update is the second in a series of stabilization updates to the 4.4 series. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer * oVirt Node 4.4 based on CentOS Linux 8.2 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 Additional Resources: * Read more about the oVirt 4.4.2 release highlights: http://www.ovirt.org/release/4.4.2/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.2/ [2] http://resources.ovirt.org/pub/ovirt-4.4-pre/iso/ -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

3 3

LDAP/AD issue
by kim.kargaard＠noroff.no 26 Aug '20

26 Aug '20

Hi all, We have had our ovirt instance connected to our internal AD for users to log into the VM portal for the last year, linked to studentdomene.noroff.no. This has been working without any problems. We had it set up and the DNS server had a forward record to the DC's. All good. Then, of course, the institution decided to introduce student emails and they decided to add the domain stud.noroff.no for student emails and made this the primary domain in the AD. The problem is that when this is changed, students can no longer log into the engine. I have of course changed the ldap settings and added a forward record on the DNS to the new domain. However, it seems that the domain is studentdomene.noroff.no, but with an added UPN suffix with stud.noroff.no When students try to log in, with the config changes, they get this error in the browser: server_error: An error occurred while attempting to query DNS in order to retrieve SRV records with name '_ldap._tcp.stud.noroff.no': NameNotFoundException(DNS name not found [response code 3]), ldapSDKVersion=4.0.7, revision=b28fb50058dfe2864171df2448ad2ad2b4c2ad58 Any ideas on how to solve this issue? My config looks like this: sudo cat /etc/ovirt-engine/aaa/Students.properties [sudo] password for noroffadmin: include = <ad.properties> vars.domain = studentdomene.noroff.no vars.user = CN=ovirt auth,CN=Users,DC=stud,DC=noroff,DC=no vars.password = PASSWORD pool.default.auth.simple.bindDN = ${global:vars.user} pool.default.auth.simple.password = ${global:vars.password} pool.default.serverset.type = srvrecord pool.default.serverset.srvrecord.domain = ${global:vars.domain} my forward on the DNS server looks like this: sudo cat /etc/named/named.conf.local [sudo] password for noroffadmin: zone "platform.noroff.no"{ type master; file "/etc/named/zones/db.platform.noroff.no"; # zone file path }; zone "stud.noroff.no" { type forward; forward only; forwarders { 172.24.111.20; 172.27.111.20; 172.21.111.20; 172.16.111.20; }; }; zone "studentdomene.noroff.no" { type forward; forward only; forwarders { 172.24.111.20; 172.27.111.20; 172.21.111.20; 172.16.111.20; }; }; zone "122.16.172.in-addr.arpa" { type master; file "/etc/named/zones/db.122.16.172"; # 172.16.122.0/24 subnet }; Any pointers would be greatly appreciated :) Kim

2 1

oVirt SHE smart-automated backup
by Vrgotic, Marko 26 Aug '20

26 Aug '20

Dear oVirt, For a while now I have been running backup of oVirt engine via ansible. And that works just fine. However, its still recommended, in SHE HA environment, especially for Restore purpose to have one of HA nodes in Maintenance mode, clean from Virtual load. Has anyone so far been able to automate oVirt engine backup in following or similar way: 1. Register on which HA node is SHE atm 2. Pick the other HA node to go into maintenance mode – in order to have one of HA nodes without virtual load 3. Verify it’s in maintenance mode 4. Execute engine-backup 5. Remove the maintenance mode of the node from step 2 Even if you are half way through, I would be happy to get involved and contribute. Kindly awaiting your reply. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

2 4

oVirt HCI 4.4.1 DR cannot import target volume after disaster
by wodel youchi 25 Aug '20

25 Aug '20

Hi, I have installed two HCI platforms. Then I set up the disaster-recovery configuration On source platform I did : # gluster volume set all cluster.enable-shared-storage enable # gluster system:: execute gsec_create # gluster volume geo-replication *source* gnodesen2-01.example.com::dest create push-pem # gluster volume geo-replication *source* gnodesen2-01.example.com::dest config use_meta_volume true On target platform I did : # gluster volume set *dest* features.shard enable I then realized these two tests : *1 - Sync data using the schedule_georep.py* a - Use the : "python3 /usr/share/glusterfs/scripts/schedule_georep.py source gnodesen2-01.example.com dest " command to sync data between the volumes b - After the sync is complete, execute "gluster volume dest set features.read-only off" c - Execute *import* (not create) *dest* volume on the target platform manually through the Admin-UI, Result : The domain could not be added and I get this error on vdsm log : " 2020-08-24 10:18:05,644+0100 ERROR (jsonrpc/5) [storage.HSM] Unexpected error (hsm:2843) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 456, in getVersion version = self.getMetaParam(DMDK_VERSION) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 452, in getMetaParam return self._metadata[key] File "/usr/lib/python3.6/site-packages/vdsm/storage/persistent.py", line 114, in __getitem__ return dec(self._dict[key]) File "/usr/lib/python3.6/site-packages/vdsm/storage/persistent.py", line 225, in __getitem__ return self._metadata[key] KeyError: 'VERSION' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2829, in getStorageDomainsList dom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/glusterSD.py", line 62, in findDomain return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 385, in __init__ manifest = self.manifestClass(domainPath) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 162, in __init__ sd.StorageDomainManifest.__init__(self, sdUUID, domaindir, metadata) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 352, in __init__ version = self.getVersion() * File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 458, in getVersion raise se.MetaDataKeyNotFoundError("key={}".format(DMDK_VERSION))vdsm.storage.exception.MetaDataKeyNotFoundError: Meta Data key not found error: ('key=VERSION',)* 2020-08-24 10:18:05,644+0100 INFO (jsonrpc/5) [vdsm.api] FINISH getStorageDomainsList return={'domlist': []} from=::ffff:10.80.101.27,48282, flow_id=5209d576-ee09-4de3-bc46-0c186c6ad52a, task_id=bc93aaae-5449-4d0d-9884-f47ea542a963 (api:54) 2020-08-24 10:18:05,645+0100 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageDomains succeeded in 3.01 seconds (__init__:312) 2020-08-24 10:18:05,754+0100 INFO (jsonrpc/4) [vdsm.api] START disconnectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'glusterfs', 'port': '', 'mnt_options': 'backup-volfile-servers=10.90.101.23:10.90.101.25', 'iqn': '', 'connection': '10.90.101.21:/dest', 'ipv6_enabled': 'false', 'id': 'cc24df41-19b0-41ed-ae85-e091e40cc612', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:10.80.101.27,48282, flow_id=5fcaf7d7-d3b6-4430-b5f9-28e6bd95656a, task_id=73654a43-c86b-4250-95d7-3e3399e5bc72 (api:48) 2020-08-24 10:18:05,754+0100 INFO (jsonrpc/4) [storage.Mount] unmounting /rhev/data-center/mnt/glusterSD/10.90.101.21:_dest (mount:215) " *Note :* I had to modify the line 105 from /usr/share/glusterfs/scripts/schedule_georep.py to be able to use it with python3 : - ey = "_".join([func.func_name] + list(args)) + key = "_".join([func.__name__] + list(args)) *2 - Sync data using the adminUI :* a - Create a schedule on the adminUI and wait until the sync is complete. b - Switch off read-only on target volume : gluster volume set dest features.read-only off (on the target volume) c - Execute ansible failover scripts : ansible-playbook dr-rhv-failover.yml --tags="fail_over" command (on the ansible machine) Result : 020-08-24 15:51:58,874+0100 INFO (jsonrpc/0) [IOProcessClient] (3a5d378b-8043-4072-9c58-e838343d6028) Starting client (init:308) 2020-08-24 15:51:58,886+0100 INFO (ioprocess/140212) [IOProcess] (3a5d378b-8043-4072-9c58-e838343d6028) Starting ioprocess (init:434) 2020-08-24 15:51:58,888+0100 INFO (jsonrpc/0)** [vdsm.api] FINISH createStorageDomain error=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) from=::ffff:10.80.101.27,45118, flow_id=765c5009, task_id=ed3a34f9-5e81-40f4-a592-78e585c1f937 (api:52) 2020-08-24 15:51:58,888+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='ed3a34f9-5e81-40f4-a592-78e585c1f937') Unexpected error (task:880) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 887, in _run return fn(args, *kargs) File "<decorator-gen-121>", line 2, in createStorageDomain File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(args, *kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2644, in createStorageDomain max_hosts=max_hosts) File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 95, in create version) **File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 60, in _preCreateValidation *raise se.StorageDomainNotEmpty(typeSpecificArg)vdsm.storage.exception.StorageDomainNotEmpty: Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',)2020-08-24 15:51:58,888+0100 INFO (jsonrpc/0) [storage.TaskManager.Task] (Task='ed3a34f9-5e81-40f4-a592-78e585c1f937') aborting: Task is aborted: "value=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) abortedcode=361" (task:1190)* 2020-08-24 15:51:58,888+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH createStorageDomain error=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) (dispatcher:83) 2020-08-24 15:51:58,889+0100 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC* call StorageDomain.create failed *(error 361) in 0.20 seconds (init:312) 2020-08-24 15:51:58,934+0100 INFO (jsonrpc/2) [vdsm.api] START disconnectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '****', 'vfs_type': 'glusterfs', 'port': '', 'iqn': '', 'connection': 'gnodesen2-01.example.com:/dest', 'ipv6_enabled': 'false', 'id': 'd104177f-f391-45bc-a8a7-905da90d4e7d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:10.80.101.27,45118, flow_id=f8c31837-fd32-48f2-bed9-6138ea4c6bb0, task_id=badf6391-06cd-4e11-ade5-95dd10b86af5 (api:48) 2020-08-24 15:51:58,934+0100 INFO (jsonrpc/2) [storage.Mount] unmounting /rhev/data-center/mnt/glusterSD/gnodesen2-01.example.com:_dest (mount:215) Any ideas? Regards.

1 0