- Users - oVirt List Archives

Re: Gluster volume not responding
by Strahil Nikolov 09 Oct '20

09 Oct '20

Hi Simon, I doubt the system needs tuning from network perspective. I guess you can run some 'screen'-s which a pinging another system and logging everything to a file. Best Regards, Strahil Nikolov В петък, 9 октомври 2020 г., 01:05:22 Гринуич+3, Simon Scott <simon(a)justconnect.ie> написа: Thanks Strahil. I have found between 1 & 4 Gluster peer rpc-clnt-ping timer expired messages in the rhev-data-center-mnt-glusterSD-hostname-strg:_pltfm_data01.log on the storage network IP. Of the 6 Hosts only 1 does not have these timeouts. Fencing has been disabled but can you identify which logs are key to identifying the cause please. It's a bonded (bond1) 10GB ovirt-mgmt logical network and Prod VM VLAN interface AND a bonded (bond2) 10GB Gluster storage network. Dropped packets are seen incrementing in the vdsm.log but neither ethtool -S or kernel logs are showing dropped packets. I am wondering if they are being dropped due to the ring buffers being small. Kind Regards Shimme ________________________________ From: Strahil Nikolov <hunter86_bg(a)yahoo.com> Sent: Thursday 8 October 2020 20:40 To: users(a)ovirt.org <users(a)ovirt.org>; Simon Scott <simon(a)justconnect.ie> Subject: Re: [ovirt-users] Gluster volume not responding >Every Monday and Wednesday morning there are gluster connectivity timeouts >but all checks of the network and network configs are ok. Based on this one I make the following conclusions: 1. Issue is reoccuring 2. You most probably have a network issue Have you checked the following: - are there any ping timeouts between fuse clients and gluster nodes - Have you tried to disable fencing and check the logs after the issue reoccurs - Are you sharing Blackup and Prod networks ? Is it possible some backup/other production load in your environment to "black-out" your oVirt ? - Have you check the gluster cluster's logs for anything meaningful ? Best Regards, Strahil Nikolov

1 0

Moving VM disks from one storage domain to another. Automate?
by Green, Jacob Allen /C 08 Oct '20

08 Oct '20

I am looking for an automated way, via Ansible to move a VM disk from one storage domain to another. I found the following, https://docs.ansible.com/ansible/latest/modules/ovirt_disk_module.html and while it mentions copying a VM disk image from one domain to another it does not mention a live storage migration. Which is what I am looking to do. I want to take roughly 100 VMs and move their disk images from one domain to another that is available to the datacenter in some automated/scripted fashion. I am just curious if anyone out there has had to do this and how they tackled it. Or perhaps I am missing some easy obvious way, other than clicking all the disks and clicking move. However from the looks of it, if I did click all the disk and selected move, it appears RHV tries to do them all at once, which is probably not ideal, I would like it to move the disks in a serial One after another fashion, to conserve throughput and IO. I also did not see anything on Ansible galaxy or the ovirt github that would do this. Thank you.

3 2

Gluster volume not responding
by simon＠justconnect.ie 08 Oct '20

08 Oct '20

Hi, Every Monday and Wednesday morning there are gluster connectivity timeouts but all checks of the network and network configs are ok. Description of problem: The following entries were found in the engine.log following VMs becoming unresponsive and hosts fencing. This issue has been causing issues since the beginning of September and no amount of reading logs is helping. This issue occurs every Wednesday morning at exactly the same time. WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-974045) [] domain 'bc482086-598b-46b1-9189-0146fa03447c:pltfm_data03' in problem 'PROBLEMATIC'. vds: 'bdtpltfmovt02' WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-974069) [] domain 'bf807836-b64e-4913-ab41-cfe04ca9abab:pltfm_data01' in problem 'PROBLEMATIC'. vds: 'bdtpltfmovt02' WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-974121) [] domain 'bc482086-598b-46b1-9189-0146fa03447c:pltfm_data03' in problem 'PROBLEMATIC'. vds: 'bdtpltfmovt03' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM '00a082de-c827-4c97-9846-ec32d1ddbfa6'(bdtfmnpproddb03) moved from 'Up' --> 'NotResponding' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM 'f5457f04-054e-4684-9702-40ed4a3e4bdb'(bdtk8shaproxy02) moved from 'Up' --> 'NotResponding' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM 'bea85b27-18e7-4936-9871-cdb987baebdd'(bdtdepjump) moved from 'Up' --> 'NotResponding' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM 'ba1d4fe2-97e7-491a-9485-8319281e7784'(bdtcmgmtnfs01) moved from 'Up' --> 'NotResponding' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM '9e253848-7153-43e8-8126-dba2d7f2d214'(bdtdepnfs01) moved from 'Up' --> 'NotResponding' INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [] VM '1e58d106-4b65-4296-8d11-2142abb7808e'(bdtionjump) moved from 'Up' --> 'NotResponding' VDSM Log from one of the Gluster Peers: [2020-10-05 03:03:25.038883] W [MSGID: 114031] [client-rpc-fops_v2.c:2634:client4_0_lookup_cbk] 0-pltfm_data02-client-0: remote operation failed. Path: / (00000000-0000-0000-0000-000000000001) [Transport endpoint is not connected] [2020-10-07 05:37:39.582138] C [rpc-clnt-ping.c:162:rpc_clnt_ping_timer_expired] 0-pltfm_data02-client-0: server x.x.x.x:49153 has not responded in the last 30 seconds, disconnecting. [2020-10-07 05:37:39.583217] I [MSGID: 114018] [client.c:2288:client_rpc_notify] 0-pltfm_data02-client-0: disconnected from pltfm_data02-client-0. Client process will keep trying to connect to glusterd until brick's port is available [2020-10-07 05:37:39.584213] E [rpc-clnt.c:346:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fe83aa96fbb] (--> /lib64/libgfrpc.so.0(+0xce11)[0x7fe83a85fe11] (--> /lib64/libgfrpc.so.0(+0xcf2e)[0x7fe83a85ff2e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x91)[0x7fe83a861521] (--> /lib64/libgfrpc.so.0(+0xf0c8)[0x7fe83a8620c8] ))))) 0-pltfm_data02-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(LOOKUP(27)) called at 2020-10-07 05:37:09.003907 (xid=0x7e6a8c) Current Version: 4.3.4.3-1.el7 - Although we are keen to upgrade, we need stability for this production environment before doing so. This Data Center has 2 x 3 Node clusters (Admin & Platform) which each have a 3 Replica Gluster configuration which is not managed by the self hosted ovirt engine. Any assistance is appreciated. Regards Shimme

3 3

user role
by kim.kargaard＠noroff.no 08 Oct '20

08 Oct '20

Hi all, For a group of users added to ovirt through AD, which role should one add to an ISO for the group to be able to select an ISO when creating VM's in the VM portal? Thanks. Kim

2 1

[ANN] oVirt 4.4.3 Fourth Release Candidate is now available for testing
by Lev Veyde 08 Oct '20

08 Oct '20

oVirt 4.4.3 Fourth Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.4.3 Fourth Release Candidate for testing, as of October 8th, 2020. This update is the third in a series of stabilization updates to the 4.4 series. How to prevent hosts entering emergency mode after upgrade from oVirt 4.4.1 Note: Upgrading from 4.4.2 GA should not require re-doing these steps, if already performed while upgrading from 4.4.1 to 4.4.2 GA. These are only required to be done once. Due to Bug 1837864 <https://bugzilla.redhat.com/show_bug.cgi?id=1837864> - Host enter emergency mode after upgrading to latest build If you have your root file system on a multipath device on your hosts you should be aware that after upgrading from 4.4.1 to 4.4.3 you may get your host entering emergency mode. In order to prevent this be sure to upgrade oVirt Engine first, then on your hosts: 1. Remove the current lvm filter while still on 4.4.1, or in emergency mode (if rebooted). 2. Reboot. 3. Upgrade to 4.4.3 (redeploy in case of already being on 4.4.3). 4. Run vdsm-tool config-lvm-filter to confirm there is a new filter in place. 5. Only if not using oVirt Node: - run "dracut --force --add multipath” to rebuild initramfs with the correct filter configuration 6. Reboot. Documentation - If you want to try oVirt as quickly as possible, follow the instructions on the Download <https://ovirt.org/download/> page. - For complete installation, administration, and usage instructions, see the oVirt Documentation <https://ovirt.org/documentation/>. - For upgrading from a previous version, see the oVirt Upgrade Guide <https://ovirt.org/documentation/upgrade_guide/>. - For a general overview of oVirt, see About oVirt <https://ovirt.org/community/about.html>. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer * oVirt Node 4.4 based on CentOS Linux 8.2 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 Additional Resources: * Read more about the oVirt 4.4.3 release highlights: http://www.ovirt.org/release/4.4.3/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.3/ [2] http://resources.ovirt.org/pub/ovirt-4.4-pre/iso/ -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Re: Is it possible to backup stopped vm? (ovirt 4.4)
by Eyal Shenitzky 08 Oct '20

08 Oct '20

Hi Lukaz, Backup using the new incremental backup API is supported only for running VMs. On Thu, 8 Oct 2020 at 12:36, Łukasz Kołaciński <l.kolacinski(a)storware.eu> wrote: > Hello, > While trying to backup stopped VM (with POST on > /ovirt-engine/api/vms/b79b34c0-d8db-43e5-916e-5528ff7bcfbe/backups) I got > the response: > > <?xml version="1.0" encoding="UTF-8" standalone="yes"?> > <fault> > <detail>[Cannot backup VM. The Virtual Machine should be in Up status.]</ > detail> > <reason>Operation Failed</reason> > </fault> > > I found here: > https://www.ovirt.org/develop/release-management/features/storage/increment… > that it should be possible to back up a virtual machine that is not running. > *"If the VM is not running, the system will create a paused, stripped-down > version of the VM, with only backup disks attached, and use libvirt API to > start and stop the backup."* > If you will read until the end of the paragraph you will see that the support for this deferred at this time- We considered alternative solution using qemu-nbd, but According to Eric Blake qemu-nbd does not support yet exposing bitmap info, so we would not be able to provide the change block list. Since creating special paused VM for backing up non-running VM is a lot of work, we may defer support for backing up non-running VMs. > > But it doesn't seem to work. > > Regards, > > Łukasz Kołaciński > > Junior Java Developer > > e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu> > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/QGUVJW5HWQG6W… > -- Regards, Eyal Shenitzky

1 0

oVirt documentation updates
by Sandro Bonazzola 08 Oct '20

08 Oct '20

The oVirt documentation team updated the following guides: - Administration Guide <https://ovirt.org/documentation/administration_guide/> - Data Warehouse Guide <https://ovirt.org/documentation/data_warehouse_guide/> - Disaster Recovery Guide <https://ovirt.org/documentation/disaster_recovery_guide/> - Virtual Machine Management Guide <https://ovirt.org/documentation/virtual_machine_management_guide/> Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* * <https://www.redhat.com/it/forums/emea/italy-track>*

1 0

Re: Upgrade Host Compatibility Version
by jb 07 Oct '20

07 Oct '20

Sorry, last time I didn't answer to the mailing list... Here are the output: *CentOS Linux release 8.2.2004 (Core)* I use the kernel-plus, because of an old raid controller. Can this be a problem? ** Jonathan * * * * Am 07.10.20 um 13:21 schrieb Dana Elfassy: > Thanks Jonathan > Can you tell me what is the OS version of the host that you're using? > (cat /etc/redhat-release) > > Dana > > On Mon, Oct 5, 2020 at 4:31 PM jb <jonbae77(a)gmail.com > <mailto:jonbae77@gmail.com>> wrote: > > Hello Dana, > > I added a second host, and tried again, without success. > > The error is unspecific, it just says: *Upgrade of cluster default > failed.* > > Here the engine.log > > > Jonathan > > > > * > * > > Am 05.10.20 um 11:00 schrieb Dana Elfassy: >> Hi Jonathan, >> What is the error you're getting when you're trying to upgrade >> using the upgrade guide? >> Can you also attach the engine log? >> (/var/log/ovirt-engine/engine.log) >> Thanks, >> Dana >> >> On Sun, Oct 4, 2020 at 6:54 PM jb <jonbae77(a)gmail.com >> <mailto:jonbae77@gmail.com>> wrote: >> >> Hello everybody, >> >> for some days I upgrade our environment from ovirt 4.3 to >> 4.4.2 and now >> I stuck on upgrading the host compatibility version. It show >> there a >> compatibility until version 4.4. The VMs have a limit until >> version 4.5 >> and the cluster shows to that I can upgrade them until 4.5. >> >> On the cluster page is also a Upgrade guide, but this fails >> when I try >> that one. At the moment I have only one host to upgrade (no >> hosted >> engine)... >> >> Thanks for helping! >> >> Jonathan >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> >> To unsubscribe send an email to users-leave(a)ovirt.org >> <mailto:users-leave@ovirt.org> >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> <https://www.ovirt.org/privacy-policy.html> >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> <https://www.ovirt.org/community/about/community-guidelines/> >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/OULJ2FCU3CLFQ… >> <https://lists.ovirt.org/archives/list/users@ovirt.org/message/OULJ2FCU3CLFQ…> >>

2 2

oVirt Survey Autumn 2020
by Sandro Bonazzola 06 Oct '20

06 Oct '20

As we continue to develop oVirt 4.4, the Development and Integration teams at Red Hat would value insights on how you are deploying the oVirt environment. Please help us to hit the mark by completing this short survey. The survey will close on October 18th 2020. If you're managing multiple oVirt deployments with very different use cases or very different deployments you can consider answering this survey multiple times. *Please note the answers to this survey will be publicly accessible*. This survey is under oVirt Privacy Policy available at https://www.ovirt.org/site/privacy-policy.html . The survey is available https://forms.gle/bPvEAdRyUcyCbgEc7 -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

3 4

ldap auth problem after upgrade from 4.4.1 to 4.4.2
by Jiří Sléžka 06 Oct '20

06 Oct '20

Hi, I just upgraded my HE to 4.4.2 but now I cannot login using my ldap aaa profile anymore. We are using Novell/NetIQ E-directory (load ballanced by haproxy, probably not important...) In 4.4.1 I was hit by removed TLSv1 (which is the newest protocol supported by our edir) from default crypto policies but I was able revert it by update-crypto-policies --set LEGACY after upgrade to 4.4.2 the error is server_error: An error occurred while attempting to connect to server ldap1.slu.cz:389: IOException(LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server ldap1.slu.cz/193.84.206.212:389: SocketException(Network is unreachable (connect failed)), ldapSDKVersion=4.0.14, revision=c0fb784eebf9d36a67c736d0428fb3577f2e25bb')) but our ldap server is reachable from ovirt, I tested it via (also ldaps and startls variants are working) ldapsearch -H ldap://ldap1.slu.cz -x -D cn=*****,ou=******,o=su -w '************' -b 'o=su' As a workaround I tried to set plain ldap protocol in profile cat /etc/ovirt-engine/aaa/CRO.properties include = <rfc2307-edir.properties> vars.server = ldap1.slu.cz vars.port = 389 vars.user = cn=*****,ou=******,o=su vars.password = ************** pool.default.serverset.single.server = ${global:vars.server} pool.default.serverset.single.port = ${global:vars.port} pool.default.auth.simple.bindDN = ${global:vars.user} pool.default.auth.simple.password = ${global:vars.password} pool.default.ssl.startTLS = false pool.default.ssl.enable = false #pool.default.ssl.protocol = TLSv1 #pool.default.ssl.startTLSProtocol = TLSv1 #pool.default.ssl.insecure = true sequence-init.init.100-my-edir-init-vars = my-edir-init-vars sequence.my-edir-init-vars.010.description = set baseDN sequence.my-edir-init-vars.010.type = var-set sequence.my-edir-init-vars.010.var-set.variable = simple_baseDN sequence.my-edir-init-vars.010.var-set.value = o=su #search.default.search-request.derefPolicy = ALWAYS but the error is the same... ovirt-engine-extensions-tool aaa login-user --profile=CRO --user-name=my_user .... WARNING: [ovirt-engine-extension-aaa-ldap.authn::SU-LDAP-authentication] TLS/SSL insecure mode ... WARNING: [ovirt-engine-extension-aaa-ldap.authn::auth.CRO.slu.cz] Cannot initialize LDAP framework, deferring initialization. Error: An error occurred while attempting to connect to server ldap1.slu.cz:389: IOException(LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server ldap1.slu.cz/193.84.206.212:389: SocketException(Network is unreachable (connect failed)), ldapSDKVersion=4.0.14, revision=c0fb784eebf9d36a67c736d0428fb3577f2e25bb')) ... INFO: API: -->Authn.InvokeCommands.AUTHENTICATE_CREDENTIALS profile='CRO' user='my_user' Password: ... WARNING: [ovirt-engine-extension-aaa-ldap.authn::auth.CRO.slu.cz] Cannot initialize LDAP framework, deferring initialization. Error: An error occurred while attempting to connect to server ldap1.slu.cz:389: IOException(LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server ldap1.slu.cz/193.84.206.212:389: SocketException(Network is unreachable (connect failed)), ldapSDKVersion=4.0.14, revision=c0fb784eebf9d36a67c736d0428fb3577f2e25bb')) Oct 01, 2020 10:57:37 AM org.ovirt.engine.exttool.core.ExtensionsToolExecutor main SEVERE: An error occurred while attempting to connect to server ldap1.slu.cz:389: IOException(LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server ldap1.slu.cz/193.84.206.212:389: SocketException(Network is unreachable (connect failed)), ldapSDKVersion=4.0.14, revision=c0fb784eebf9d36a67c736d0428fb3577f2e25bb')) debug with tcpdump reveals only that connection is made and there are only "bindRequest" and "bindResponse success" messages visible (with correct tcp handshake and close) and nothing more any help would be appreciated Cheers, Jiri

4 8