- Users - oVirt List Archives

Ovirt 4.3.10 Glusterfs SSD slow performance over 10GE
by jury cat 07 Jul '20

07 Jul '20

Hello all, I am using Ovirt 4.3.10 on Centos 7.8 with glusterfs 6.9 . My Gluster setup is of 3 hosts in replica 3 (2 hosts + 1 arbiter). All the 3 hosts are Dell R720 with Perc Raid Controller H710 mini(that has maximim throughtout 6Gbs) and with 2×1TB samsumg SSD in RAID 0. The volume is partitioned using LVM thin provision and formated XFS. The hosts have separate 10GE network cards for storage traffic. The Gluster Network is connected to this 10GE network cards and is mounted using Fuse Glusterfs(NFS is disabled).Also Migration Network is activated on the same storage network. The problem is that the 10GE network is not used at full potential by the Gluster. If i do live Migration of Vms i can see speeds of 7GB/s ~ 9GB/s. The same network tests using iperf3 reported 9.9GB/s , these exluding the network setup as a bottleneck(i will not paste all the iperf3 tests here for now). I did not enable all the Volume options from "Optimize for Virt Store", because of the bug that cant set volume cluster.granural-heal to enable(this was fixed in vdsm-4 40, but that is working only on Centos 8 with ovirt 4.4 ) . i whould be happy to know what are all these "Optimize for Virt Store" options, so i can set them manually. The speed on the disk inside the host using dd is b etween 1GB/s to 700Mbs. [root@host1 ~]# dd if=/dev/zero of=test bs=100M count=40 cou nt=80 status=progress 8074035200 bytes (8.1 GB) copied, 11.059372 s, 730 MB/s 80+0 records in 80+0 records out 8388608000 bytes (8.4 GB) copied, 11.9928 s, 699 MB/s The dd write test on the gluster volme inside the host is poor only ~ 120MB/s . During the dd test, if i look at Networks->Gluster network ->Hosts at Tx and Rx the network speed barerly reaches over 1Gbs (~1073 Mbs) out of maximum of 10000 Mbs. dd if=/dev/zero of=/rhev/data-center/mnt/glu sterSD/gluster1.domain.local\:_data/test bs=100M count=80 status=progress 8283750400 bytes (8.3 GB) copied, 71.297942 s, 116 MB/s 80+0 records in 80+0 records out 8388608000 bytes (8.4 GB) copied, 71.9545 s, 117 MB/s I have attached my Gluster volume settings and mount options. Thanks, Emy

4 10

Running Self Hosted-Engine on the same iSCSI target as other Storage Domains
by Erez Zarum 07 Jul '20

07 Jul '20

We are using Dell SC (Storage) with iSCSI with oVirt, it is impossible to create a new Target Portal with a specific LUN so it's impossible to isolate the SE LUN from other LUNs that are in use by other Storage Domains. According to the documentations this is not a best practice, while searching i ran into that specific issue but it seems to be solved. Is it still by best practice when using oVirt 4.4 to use a whole different target for the SE? If it is not possible giving the current envrionment restrictions is the only option for us is to attach a few disks to each servers and create a small gluster cluster (volume) just for the SE?

2 1

Re: broker.log not rotating
by Strahil Nikolov 07 Jul '20

07 Jul '20

Yes, ovirt-ha-broker and ovirt-ha-agent take care of HostedEngine to be up and running and in case something goes bad - to be migrated away. Best Regards, Strahil Nikolov На 3 юли 2020 г. 7:53:14 GMT+03:00, Anton Louw via Users <users(a)ovirt.org> написа: >Hi Everybody, > >Thanks for all the responses. So I managed to truncate the log. I just >want to make sure, the ‘ovirt-ha-agent’ service is only for the hosted >engine correct? > >I can still migrate machines between the hosts, so it should then not >impact the VMs? > >Thanks > > >Anton Louw >Cloud Engineer: Storage and Virtualization >______________________________________ >D: 087 805 1572 | M: N/A >A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg >anton.louw(a)voxtelecom.co.za > >www.vox.co.za > > > >From: Darrell Budic <budic(a)onholyground.com> >Sent: 02 July 2020 19:56 >To: Anton Louw <Anton.Louw(a)voxtelecom.co.za> >Cc: users(a)ovirt.org >Subject: Re: [ovirt-users] broker.log not rotating > >They’re just log files, so generally safe to delete. You may want to >take a look at the huge one though, see what’s up. I had a similar >problem that turned out to be a broken HA agent install, cleaned and >reinstalled and it went back to the same volume of logs as the others. > >Now I need to check if they added logrotate.d config files for it to >clean those up, hum. > > -Darrell > > >On Jul 2, 2020, at 12:00 AM, Anton Louw via Users ><users(a)ovirt.org<mailto:users@ovirt.org>> wrote: > > > >Hi All, > >I had a space alert on one of my nodes this morning, and when I looked >around, I saw that var/log/ovirt-hosted-engine-ha/broker.log was >sitting at around 30GB. Does anybody know if it is safe to delete the >log file? Or is there another process that I should follow? > >I had a look at my other nodes, and the broker.log file does not exceed >4GB. > >Thank you > >Anton Louw >Cloud Engineer: Storage and Virtualization at Vox >________________________________ >T: 087 805 0000 | D: 087 805 1572 >M: N/A >E: anton.louw(a)voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> >A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg >www.vox.co.za<http://www.vox.co.za/> > >[F]<https://www.facebook.com/voxtelecomZA> > >[T]<https://www.twitter.com/voxtelecom> > >[I]<https://www.instagram.com/voxtelecomza/> > >[L]<https://www.linkedin.com/company/voxtelecom> > >[Y]<https://www.youtube.com/user/VoxTelecom> > > >[#VoxBrand]<https://www.vox.co.za/fibre/fibre-to-the-home/?prod=HOME> > >Disclaimer >The contents of this email are confidential to the sender and the >intended recipient. Unless the contents are clearly and entirely of a >personal nature, they are subject to copyright in favour of the holding >company of the Vox group of companies. Any recipient who receives this >email in error should immediately report the error to the sender and >permanently delete this email from all storage devices. > >This email has been scanned for viruses and malware, and may have been >automatically archived by Mimecast Ltd, an innovator in Software as a >Service (SaaS) for business. Providing a safer and more useful place >for your human generated data. Specializing in; Security, archiving and >compliance. To find out more Click >Here<https://www.voxtelecom.co.za/security/mimecast/?prod=Enterprise>. > > >_______________________________________________ >Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> >To unsubscribe send an email to >users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> >Privacy Statement: >https://www.ovirt.org/privacy-policy.html<https://www.ovirt.org/privacy-policy.html> >oVirt Code of Conduct: >https://www.ovirt.org/community/about/community-guidelines/<https://www.ovirt.org/community/about/community-guidelines/> >List Archives: >https://lists.ovirt.org/archives/list/users@ovirt.org/message/4L6OOVVW6KOSB…<https://lists.ovirt.org/archives/list/users@ovirt.org/message/4L6OOVVW6KOSB…>

1 0

oVirt 4.4.0, slave interfaces are down after a reboot
by Erez Zarum 07 Jul '20

07 Jul '20

Fresh installation of oVirt 4.4.0 (using the ISO), after restarting an host, the host comes up but the slave interfaces of a bond remains in down state. The only way to bring it up is to login via IPMI and restart the "NetworkManager" service. Weird thing, after this, the host says that it's network is out of sync, this happens on a network that is marked as "migration network", what is out of sync is the gateway. I have restart the host again (this happens on all hosts) and i can see that after a restart the NetworkManager state (nmcli c show bond0.421 as example) does not list the routes, after syncing the routes are there.

2 2

iSCSI array "Storage domain does not exist" error after array reboot
by Kevin Doyle 07 Jul '20

07 Jul '20

I am running a hosted engine using iscsi. I had to reboot the array, I took down all the VM's and put the VM data storage into maintenance mode. I also put the hosted engine "hosted-engine --set-maintenance --mode =global" and shut down the engine "hosted-engine --vm-shutdown" I also stopped the broker and agent systemctl stop ovirt-ha-broker' ; 'systemctl stop ovirt-ha-agent' The array was rebooted. I started the broker and agent systemctl start ovirt-ha-broker' ; 'systemctl start ovirt-ha-agent' However I now see errors in the vdsm log INFO (jsonrpc/4) [storage.TaskManager.Task] (Task='76547cc5-e4f0-454c-ac46-c341e7f6d54c') aborting: Task is aborted: "Storage domain does not exist: (u'5626eba7-ebd8-4578-baf0-ca4f4af5889b',)" - code 358 (task:1181) 2020-07-07 14:01:25,098+0100 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH prepareImage error=Storage domain does not exist: (u'5626eba7-ebd8-4578-baf0-ca4f4af5889b',) (dispatcher:83) 2020-07-07 14:01:25,098+0100 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Image.prepare failed (error 358) in 0.07 seconds (__init__:312) INFO (vmrecovery) [vdsm.api] START getConnectedStoragePoolsList(options=None) from=internal, task_id=93d7e2d6-17ee-4370-b8c4-fb32a3887021 (api:48) 2020-07-07 14:01:25,592+0100 INFO (vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return={'poollist': []} from=internal, task_id=93d7e2d6-17ee-4370-b8c4-fb32a3887021 (api:54) 2020-07-07 14:01:25,592+0100 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:709) I have 6 hosts. I have tried rebooting them all, I have tried just running 1 host that was originally hosting the engine, but I cannot bring up the hosted engine. Has anyone any ideas ? how to reconnect the storage domain The hosted engine had its own iscsi disk separate from the VM's thanks Kevin

2 1

How to use sysprep to create Windows templates
by wodel youchi 07 Jul '20

07 Jul '20

Hi, I need to create some Windows Templates for my client, and I am not quite familiar with the Windows sysprep installations. I read the "Sealing a Windows Virtual Machine for Deployment as a Template" on the Virtual Machine Management Guide. What I don't understand is the process of sealing a Windows VM. When should I use the /usr/share/ovirt-engine/conf/sysprep/sysprep.* files? at the creation of the VM that will be a template at the end? or at the creation of new VMs based on the template? When should I use the /etc/ovirt-engine/osinfo.conf.d/xx.files.properties? at the creation of the VM that will be a template at the end? or at the creation of new VMs based on the template? What is the difference between : "Use Cloud-Init/Sysprep" in the VM edit window and the RUN Once window? For me it is not clear, could someone help me to understand. Regards.

2 2

Re: Migration of self Hosted Engine from iSCSI to Gluster/NFS
by houwei.chen＠vinchin.com 07 Jul '20

07 Jul '20

Hi Erez, would you like to use the backup tool to backup your self Hosted Engine from iSCSI, then restore to the Gluster of NFS storage? In this way, you can migration as you want. If you are looking forward to a backup tool, maybe you can try Vinchin Backup & Recovery, which provides the solution. Hope you can solve your question soon. Thanks

1 0

Using ovirt imageio
by Łukasz Kołaciński 06 Jul '20

06 Jul '20

Dear ovirt community, I am trying to use ovirt imageio api to receive changed blocks (dirty bitmap) on ovirt 4.4. Could anyone tell me how to get them step by step? On the documentation I saw endpoint "GET /images/ticket-uuid/map". I don't know what ticket-uuid is and how to generate it. I also need to know how to use this api because I can't reach it via /ovirt-engine/api/ I am asking about this endpoint: https://www.ovirt.org/documentation/incremental-backup-guide/incremental-ba… Best regards, Łukasz Kołaciński Junior Java Developer e-mail: l.kolacinski(a)storware.eu<mailto:l.kolacinski@storware.eu> <mailto:m.helbert@storware.eu> [STORWARE]<http://www.storware.eu/> ul. Leszno 8/44 01-192 Warszawa www.storware.eu <https://www.storware.eu/> [facebook]<https://www.facebook.com/storware> [twitter]<https://twitter.com/storware> [linkedin]<https://www.linkedin.com/company/storware> [Storware_Stopka_09]<https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa 000510131 , NIP 5213672602. Wiadomość ta jest przeznaczona jedynie dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów. This message is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you have received this message in error, please contact the sender and remove the material from all of your computer systems.

1 0

Re: Problems with setting up ovirt 4.4 for incremental backups
by Eyal Shenitzky 05 Jul '20

05 Jul '20

Hi Lukasz, Thank you for testing the new Incrementa backup feature. For those kinds of questions, it is better to send the mail to +users <users(a)ovirt.org> list so other people can learn/answer the question. Please see my comments inline. On Fri, 3 Jul 2020 at 15:55, Łukasz Kołaciński <l.kolacinski(a)storware.eu> wrote: > Dear Eyal Shenitzky > > Could you help me with setting up ovirt 4.4? I am trying to implement > incremental backups to our application and I have some questions. > > 1. First of all I am using mostly this documentation: > https://www.ovirt.org/develop/release-management/features/storage/increment… > and I saw session on youtube. Are there any other sources of knowledge? > Currently no, those are the two best resources for this feature at this moment. Note that we updated https://www.ovirt.org/develop/release-management/features/storage/increment… recently so now it contains up-to-date information about all the operations that need to be done with the Engine in order to use this feature. > 2. I can't change fields on diskattachment with PUT Method (to set > incremental in backup field). I think I tried in every way and I am getting > 405 Method Not Allowed. > Please try the following request - PUT path-to-engine /ovirt-engine/api/vms/vm-uuid/diskattachments/disk-attachment-uuid <disk_attachment> <disk> <name>disk-uuid</name> <backup>incremental</backup> </disk> </disk_attachment> > 3. When I am using REST API to get info about checkpoints with "GET > /vms/vm-uuid/checkpoints/" I am getting response: "Could not find resource > for full path". So this looks like something is not enabled or missing. > There is no 'checkpoints' service under any path at this moment, so the path you are using is not valid. There is a backups service that you can find under - GET path-to-engine/ovirt-engine/api/vms/vm-uuid/backups To get a single backup you can use GET path-to-engine/ovirt-engine/api/vms/vm-uuid/backups/backup-uuid We are now working on exposing also the checkpoints service to allow removal of checkpoints, but this isn't ready yet. > 4. I don't understand what ticket-uuid in imageio API is. How can I get > dirty bitmaps after full backup? And how to use it because there is not > "images" resource on /ovirt-engine/api/ > Please look and the example script we developed in order to use this feature - https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/backup_v… Using this script you will be able to do all the needed operations in order to back up a VM. There is a very detailed manual for it (when using --help / -h). This script can interact with ImageIO in order to download the backup disks using the new 'client' library in ImageIO. You can use it as a reference on how to interact with the Engine API and ImageIO. > On my environment I have: > CentOS 8.2 > ovirt 4.4.0.3-1 > libvirt 6.0.0-17 > qemu-kvm 4.2.0 > kvm 4.2.0-19 > vdsm 4.40.16-1 > It looks like you have the correct versions in order to use Incremental backup, but remember that the Engine blocks incremental backup operations and allows only full backup by default. In order to allow incremental backup in the engine, you need to do the following commands - # engine-config -s "IsIncrementalBackupSupported=true" # systemctl restart ovirt-engine > I look forward to hearing from you. > > Best regards, > > Łukasz Kołaciński > > Junior Java Developer > > e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu> > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > -- Regards, Eyal Shenitzky

1 0

oVirt-node 4.4.0 - Hosted engine deployment fails when host is unable to download updates
by Marco Fais 04 Jul '20

04 Jul '20

Hi, fresh installation of oVirt-node 4.4.0 on a cluster -- the hosted-engine --deploy command fails if DNF is unable to download updates. This cluster is not connected to the public network at the moment. If I use a proxy (setting the relevant env. variables) it fails at a later stage (I think the engine VM is trying to download updates as well, but encounters the same issue and doesn't seem to use the proxy). With oVirt-node 4.3.x I didn't have this issue -- any suggestions? [~]# hosted-engine --deploy [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200604214638-vae2wf.log Version: otopi-1.9.1 (otopi-1.9.1-1.el8) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded Extra Packages for Enterprise Linux 8 - x86_64 [ * ERROR ] DNF Failed to download metadata for repo 'ovirt-4.4-epel'[ ERROR ] DNF Failed to download metadata for repo 'ovirt-4.4-epel'[ ERROR ] Failed to execute stage 'Environment setup': Failed to download metadata for repo 'ovirt-4.4-epel'* [ INFO ] Stage: Clean up [...] Thanks, Marco

2 2