- Users - oVirt List Archives

Re: Hosted Engine can't migrate
by Yedidyah Bar David 18 Nov '20

18 Nov '20

Hi, On Wed, Nov 18, 2020 at 8:08 AM McNamara, Bradley <Bradley.McNamara(a)seattle.gov> wrote: > > I'm new to oVirt, but not new to virtualization technologies. I've run into a wall, but the solution has gotta be simple. I've seen other threads with the same issue, but a solution is never posted. > > I've followed the guide perfectly. I get the self-hosted engine up and running with no issues. I add hosts to the cluster, but it won't let me migrate the HE because of Hosted Engine HA score of "N/A" on every host except the one it was initially deployed on. The physical servers are exactly the same: same networks, storage, OS, patches, etc. The additional hosts are added thru the HE GUI and are added without issue. Is there a secret button I gotta push? > > What am I missing? This is with 4.3 on CentOS 7.9. Can other VMs migrate (who use the same storage server, so you know it's not a storage/network issue)? Please check/share /var/log/ovirt-hosted-engine-ha/* on all hosts, as well as output of 'hosted-engine --vm-status' on all of them. Good luck and best regards, -- Didi

1 0

Hosted Engine can't migrate
by McNamara, Bradley 18 Nov '20

18 Nov '20

I'm new to oVirt, but not new to virtualization technologies. I've run into a wall, but the solution has gotta be simple. I've seen other threads with the same issue, but a solution is never posted. I've followed the guide perfectly. I get the self-hosted engine up and running with no issues. I add hosts to the cluster, but it won't let me migrate the HE because of Hosted Engine HA score of "N/A" on every host except the one it was initially deployed on. The physical servers are exactly the same: same networks, storage, OS, patches, etc. The additional hosts are added thru the HE GUI and are added without issue. Is there a secret button I gotta push? What am I missing? This is with 4.3 on CentOS 7.9. HELP! Thanks. :-)

1 0

Upgrading engine from 4.3 -> 4.4
by David Metcalf 17 Nov '20

17 Nov '20

Followed this for standalone hosts: https://ovirt.org/documentation/upgrade_guide/#Upgrading_from_4-3 but get this message.. engine-backup --mode=restore --file=engine-backup-2020-11-16.bak --provision-all-databases ... FATAL: Backup was created by version '4.3.9.4' and can not be restored using the installed version 4.4.3.11

2 1

ovirt-shell SSL Error
by FRIES, Dirk, SHS Informatik 17 Nov '20

17 Nov '20

Hello there, I am trying to connect via ovirt-shell to my ovirt manager. To do this I've downloaded the CA File (https://127.0.0.1/ovirt-engine/services/pki-resource?resource=ca-certificat…) to /tmp and set up my config in .ovirtshellrc : [cli] autoconnect = True autopage = True [ovirt-shell] username = adminuser renew_session = False timeout = None extended_prompt = False url = https://127.0.0.1/api insecure = False kerberos = False filter = False session_timeout = None ca_file = /tmp/pki-resource?resource=ca-certificate dont_validate_cert_chain = True key_file = None password = somepassword cert_file = None So when I am trying to connect I get the following error: (60, "Peer's certificate issuer has been marked as not trusted by the user.") While researching I also found a pretty similar issue here https://lists.ovirt.org/archives/list/users@ovirt.org/thread/IYG6AQPII25366… but unfortunally there is no solution posted. Does anyone have a hint for me what could cause this problem or where to look else ? Thanks a lot ! Bye, Dirk

2 1

Re: Really slow manager node
by Nicolás 17 Nov '20

17 Nov '20

Hi Toni, (re-adding the users(a)ovirt.org list) El 16/11/20 a las 23:37, tferic(a)swissonline.ch escribió: > > Hi, > > Chances are, that this performance issue may be unrelated to oVirt. > > The first thing that comes to my mind is that the system may have a > bottleneck on the I/O. > > > Please have a look at: > > iostat -x 5 > > The system may have problems with disk performance, if %util > 95 and > avgqu-sz > 3 over longer periods of time. > > Actually %util never goes beyond the 85%, being the average on 70%; for the avgqu-sz, the average value is about 25 (with peaks of 160 as far as I could see). > If that's not the case, you may next want to have a look at limits > (/etc/security/limits.conf). > > There may also be default limits which may be too low. (nofile, nproc > come to my mind) > > This file seems not to have been touched (all the content is commented out). > Are there any errors or warnings in the messages or audit logs? > > Nothing special, I've checked for the last couple days and everything seems ok. > And also htop, iotop, vmstat may help to get a quick overview. > > Yep, I've used most of these commands to check, but I've been unable to see something clarifying. In my opinion the slowness could be caused for not using SSD disks and because of a low polling interval by the DWHd, but that's just my assumption and I've been unable to get some data that would confirm this. Thanks for the help! Nico > Hope this helps, > > Toni > > > On 16.11.20 14:08, Nicolás wrote: >> Hi, >> >> We're running oVirt 4.3.8 and even if this is a problem we've had >> since a lot of time (I would say since 4.0.x), I decided to look for >> help in case anything can be done. >> >> Our environment is heavily used by users in our University (about >> 3000 users), and currently our oVirt infrastructure has 1928 virtual >> machines, being 882 of them currently running. We have a separate >> physical machine for the manager node, and the problem is that this >> machine is very, very, very slow, despite it has (from my point of >> view) enough physical resources to run efficiently. >> >> By slow I mean even the SSH access to it takes about 10 seconds, not >> just the admin/user portals. Any operation takes enough time to make >> the experience not comfortable to our users (enter the VM portal, >> start a VM, open a console...). >> >> Node machine's parameters are: >> 18GB of RAM memory >> 1 processor with 12CPUs >> 300GB SCSI Disk, local storage. No Storage Domain is stored in this >> node machine. >> >> Currently, most consuming processes are: >> >> 28802 ovirt 20 0 3302876 1,5g 25988 S 1,0 8,7 22:46.36 >> ovirt-engine -server -XX:+TieredCompilation -Xms1024M -Xmx1024M >> -Xss1M -Djava.aw+ >> 28701 ovirt 20 0 5465432 807704 13140 S 5,9 4,4 3:15.53 >> ovirt-engine-dwhd >> -Dorg.ovirt.engine.dwh.settings=/tmp/tmp8wtnTA/settings.proper+ >> >> # free -m >> total used free shared buff/cache >> available >> Mem: 17886 6669 2734 255 8482 10625 >> Swap: 6143 1002 5141 >> >> There are also a lot of postgresql processes: >> >> postgres 2186 0.0 0.0 261812 4136 ? Ss jun05 37:39 >> /opt/rh/rh-postgresql10/root/usr/bin/postmaster -D >> /var/opt/rh/rh-postgresql10/lib/pgsql/data >> postgres 3176 0.0 0.0 216084 656 ? Ss jun05 0:00 >> postgres: logger process >> postgres 3290 0.0 0.2 262204 37476 ? Ss jun05 77:31 >> postgres: checkpointer process >> postgres 3291 0.3 0.2 262052 36960 ? Ss jun05 754:02 >> postgres: writer process >> postgres 3292 0.0 0.0 261812 1988 ? Ss jun05 100:51 >> postgres: wal writer process >> postgres 3293 0.0 0.2 262380 36748 ? Ss jun05 21:58 >> postgres: autovacuum launcher process >> postgres 3294 0.1 0.0 219216 1412 ? Ds jun05 335:41 >> postgres: stats collector process >> postgres 3295 0.0 0.0 262220 1460 ? Ss jun05 0:11 >> postgres: bgworker: logical replication launcher >> postgres 3393 0.0 0.2 265792 40452 ? Ss jun05 15:36 >> postgres: engine engine ::1(51664) idle >> postgres 6105 0.3 0.0 271532 15976 ? Ds 13:01 0:00 >> postgres: autovacuum worker process ovirt_engine_history >> postgres 6216 0.2 0.0 263864 11440 ? Ss 13:02 0:00 >> postgres: autovacuum worker process engine >> postgres 6245 0.0 0.0 262888 6212 ? Ss 13:02 0:00 >> postgres: engine engine 127.0.0.1(42400) idle >> postgres 6246 0.0 0.0 262844 3256 ? Ss 13:02 0:00 >> postgres: autovacuum worker process template1 >> postgres 18815 0.0 0.0 262912 5852 ? Ss nov01 0:00 >> postgres: django django 127.0.0.1(59564) idle >> postgres 23148 0.0 0.2 266052 43024 ? Ss oct28 9:01 >> postgres: engine engine 127.0.0.1(59714) idle >> postgres 23149 0.0 0.0 262980 6820 ? Ss oct28 0:00 >> postgres: engine engine 127.0.0.1(59716) idle >> postgres 28784 0.0 0.0 262816 3492 ? Ss 12:02 0:00 >> postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39470) >> idle >> postgres 28785 0.0 0.0 262816 3496 ? Ss 12:02 0:00 >> postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39472) >> idle >> postgres 28921 0.8 0.7 375152 146452 ? Ss 12:03 0:30 >> postgres: engine engine 127.0.0.1(39484) idle >> postgres 29007 3.3 0.7 369348 142184 ? Ds 12:03 1:58 >> postgres: engine engine 127.0.0.1(39498) SELECT >> postgres 29009 5.0 0.3 294736 72776 ? Ss 12:03 2:58 >> postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39500) >> idle >> postgres 29048 0.0 0.0 263664 12176 ? Ss 12:03 0:00 >> postgres: engine engine 127.0.0.1(39530) idle >> postgres 29064 0.6 0.8 384936 157852 ? Ss 12:03 0:24 >> postgres: engine engine 127.0.0.1(39532) idle >> postgres 29065 1.2 0.7 358944 137028 ? Ss 12:03 0:43 >> postgres: engine engine 127.0.0.1(39534) idle >> postgres 29066 1.9 0.7 355800 132800 ? Ss 12:03 1:08 >> postgres: engine engine 127.0.0.1(39536) idle >> postgres 29067 1.0 0.7 370688 140504 ? Ss 12:03 0:37 >> postgres: engine engine 127.0.0.1(39538) idle >> postgres 29068 1.7 0.7 360984 139584 ? Ss 12:03 1:02 >> postgres: engine engine 127.0.0.1(39540) idle >> postgres 29069 1.3 0.7 358140 136268 ? Ss 12:03 0:48 >> postgres: engine engine 127.0.0.1(39542) idle >> postgres 29070 3.7 0.7 372064 141724 ? Ss 12:03 2:13 >> postgres: engine engine 127.0.0.1(39544) idle >> postgres 29071 0.8 0.6 337224 114132 ? Ss 12:03 0:31 >> postgres: engine engine 127.0.0.1(39546) idle >> postgres 29072 0.5 0.6 336616 114276 ? Ss 12:03 0:18 >> postgres: engine engine 127.0.0.1(39548) idle >> postgres 29073 1.7 0.7 357872 134540 ? Ss 12:03 1:02 >> postgres: engine engine 127.0.0.1(39550) idle >> postgres 29139 3.5 0.7 361244 134176 ? Ss 12:03 2:07 >> postgres: engine engine 127.0.0.1(39572) idle >> postgres 29140 3.0 0.7 367884 138372 ? Ss 12:03 1:46 >> postgres: engine engine 127.0.0.1(39570) idle >> postgres 29141 1.0 0.9 391984 169204 ? Ss 12:03 0:38 >> postgres: engine engine 127.0.0.1(39574) idle >> postgres 29142 1.3 0.7 358820 136140 ? Ss 12:03 0:48 >> postgres: engine engine 127.0.0.1(39576) idle >> postgres 29143 1.6 0.8 385000 156592 ? Ss 12:03 0:58 >> postgres: engine engine 127.0.0.1(39578) idle >> postgres 29144 3.6 0.9 403740 174552 ? Ss 12:03 2:09 >> postgres: engine engine 127.0.0.1(39580) idle >> postgres 29145 3.4 0.7 355800 128876 ? Ss 12:03 2:01 >> postgres: engine engine 127.0.0.1(39582) idle >> postgres 29146 1.8 0.8 380884 157828 ? Ss 12:03 1:06 >> postgres: engine engine 127.0.0.1(39586) idle >> postgres 29147 3.5 0.6 353228 123200 ? Ss 12:03 2:05 >> postgres: engine engine 127.0.0.1(39584) idle >> postgres 29148 0.9 0.8 383452 156268 ? Ss 12:03 0:34 >> postgres: engine engine 127.0.0.1(39588) idle >> postgres 29149 1.3 0.7 365552 139612 ? Ss 12:03 0:47 >> postgres: engine engine 127.0.0.1(39590) idle >> postgres 29150 1.2 0.7 363448 136564 ? Ss 12:03 0:43 >> postgres: engine engine 127.0.0.1(39592) SELECT >> postgres 29151 1.4 0.7 365940 139184 ? Ss 12:03 0:51 >> postgres: engine engine 127.0.0.1(39594) idle >> postgres 29152 1.2 0.7 358328 134268 ? Ss 12:03 0:44 >> postgres: engine engine 127.0.0.1(39596) idle >> postgres 29402 1.3 0.7 362044 139812 ? Ss 12:04 0:47 >> postgres: engine engine 127.0.0.1(39698) idle >> postgres 29403 2.1 0.8 377760 154556 ? Ss 12:04 1:15 >> postgres: engine engine 127.0.0.1(39700) idle >> postgres 29404 2.2 0.7 369160 144600 ? Ss 12:04 1:18 >> postgres: engine engine 127.0.0.1(39702) idle >> postgres 32230 0.1 0.5 336252 101912 ? Ss 12:19 0:04 >> postgres: engine engine 127.0.0.1(40564) idle >> postgres 32232 3.4 0.6 357644 127732 ? Ss 12:19 1:29 >> postgres: engine engine 127.0.0.1(40566) idle >> postgres 32234 0.0 0.1 270924 19836 ? Ss 12:19 0:00 >> postgres: engine engine 127.0.0.1(40568) idle >> >> Actually, I'm not really sure why is this machine so slow. In terms >> of RAM memory and CPU, it still has a bunch of free resources. >> >> Could someone help track this down and point some advices on how to >> optimize the user experience in this case? >> >> Thanks! >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/HYVADDU3XJXYK…

1 0

Cannot copy or move disks
by suporte＠logicworks.pt 17 Nov '20

17 Nov '20

Hello, I just update to Version 4.4.3.11-1.el8. Engine and host and now I cannot copy or move disks. Storage domains are glusterfs # gluster --version glusterfs 7.8 Here is what I found on vdsm.log 2020-11-14 14:08:16,917+0000 INFO (tasks/5) [storage.SANLock] Releasing Lease(name='01178644-2ad6-4d37-8657-f33f547bee6b', path='/rhev/data-center/mnt/glusterSD/node1-teste.aclou d.pt:_data2/83f8bbfd-cfa3-46d9-a823-c36054826d13/images/97977cbf-eecc-4476-a11f-7798425d40c4/01178644-2ad6-4d37-8657-f33f547bee6b.lease', offset=0) (clusterlock:530) 2020-11-14 14:08:17,015+0000 INFO (tasks/5) [storage.SANLock] Successfully released Lease(name='01178644-2ad6-4d37-8657-f33f547bee6b', path='/rhev/data-center/mnt/glusterSD/node1 -teste.acloud.pt:_data2/83f8bbfd-cfa3-46d9-a823-c36054826d13/images/97977cbf-eecc-4476-a11f-7798425d40c4/01178644-2ad6-4d37-8657-f33f547bee6b.lease', offset=0) (clusterlock:540) 2020-11-14 14:08:17,016+0000 ERROR (tasks/5) [root] Job '8cd732fc-d69b-4c32-8b35-e4a8e47396fb' failed (jobs:223) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/jobs.py", line 159, in run self._run() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdm/api/copy_data.py", line 110, in _run self._operation.run() File "/usr/lib/python3.6/site-packages/vdsm/storage/qemuimg.py", line 374, in run for data in self._operation.watch(): File "/usr/lib/python3.6/site-packages/vdsm/storage/operation.py", line 106, in watch self._finalize(b"", err) File "/usr/lib/python3.6/site-packages/vdsm/storage/operation.py", line 179, in _finalize raise cmdutils.Error(self._cmd, rc, out, err) vdsm.common.cmdutils.Error: Command ['/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f', 'raw', '-O', 'raw', '/rhev/data-center/mnt/glusterSD/node1-teste.aclou d.pt:_data2/83f8bbfd-cfa3-46d9-a823-c36054826d13/images/789f6e50-b954-4dda-a6d5-077fdfb357d2/d95a3e83-74d2-40a6-9f8f-e6ae68794051', '/rhev/data-center/mnt/glusterSD/node1-teste.ac loud.pt:_data2/83f8bbfd-cfa3-46d9-a823-c36054826d13/images/97977cbf-eecc-4476-a11f-7798425d40c4/01178644-2ad6-4d37-8657-f33f547bee6b'] failed with rc=1 out=b'' err=bytearray(b'qem u-img: error while reading sector 260177858: No such file or directory\n') 2020-11-14 14:08:17,017+0000 INFO (tasks/5) [root] Job '8cd732fc-d69b-4c32-8b35-e4a8e47396fb' will be deleted in 3600 seconds (jobs:251) 2020-11-14 14:08:17,017+0000 INFO (tasks/5) [storage.ThreadPool.WorkerThread] FINISH task 6cb1d496-d1ca-40b5-a488-a72982738bab (threadPool:151) 2020-11-14 14:08:17,316+0000 INFO (jsonrpc/2) [api.host] START getJobs(job_type='storage', job_ids=['8cd732fc-d69b-4c32-8b35-e4a8e47396fb']) from=::ffff:192.168.5.165,36616, flow _id=49320e0a-14fb-4cbb-bdfd-b2546c260bf7 (api:48) -- Jose Ferradeira http://www.logicworks.pt

5 15

Re: VM with illegal snapshots
by Florian Schmid 17 Nov '20

17 Nov '20

Hi, I have also sometimes the issue with snapshots or disk move, which is also related to snapshots, when the engine process is running for a longer time. After restarting the engine service, then it is working again for several weeks, until it gets too long and issues starting again to happen. I had never the time to dig in deeper here and I'm never on a version, which is still supported, but it would be nice to here, if you maybe have the same issue? How long was your engine process running, until the issues have started? BR Florian Von: "Magnus Isaksson" <magnus(a)vmar.se> An: "Alex K" <rightkicktech(a)gmail.com>, "Giorgio Biacchi" <giorgio(a)di.unimi.it> CC: "users" <users(a)ovirt.org> Gesendet: Dienstag, 17. November 2020 10:37:24 Betreff: [ovirt-users] Re: VM with illegal snapshots Hello We have the same issue, and not just one VM, we have about 10-15 VMs that have either snapshots that are illegal or snapshots that is not possible to remove. This have been an issue for quite some time, all since beginning of 4.3 we get "vm with illegal snapshots" almost every week, and most often we can remove the snapshot and all is ok, but now this have escalated quite much and leaves us with customer VM that we may not be able to reboot or that the disk gets corrupted, that have happed a few times. This is the VM with most snapshots that we are unable to remove any of them. Engine is on 4.3.10 and hosts on 4.3.9 How can i safely remove these snapshots? (I am not super comfortable around database "hacking" so some informative description would be much appreciated.) And how can we eliminate that these things happen? Cheers Magnus From: Alex K <rightkicktech(a)gmail.com> Sent: 10 November 2020 05:05 To: Giorgio Biacchi <giorgio(a)di.unimi.it> Cc: users <users(a)ovirt.org> Subject: [ovirt-users] Re: VM with illegal snapshots On Fri, Oct 9, 2020, 12:59 Giorgio Biacchi < [ mailto:giorgio@di.unimi.it | giorgio(a)di.unimi.it ] > wrote: Hi, due to a bug in our Ovirt integrated backup system now we have some VMs with snapshots in illegal state. It seems that there's an inconsistency between the db and the real status of images on disk. Let me show an example: engine=# select image_guid,parentid,imagestatus,vm_snapshot_id,volume_type,volume_format,active from images where image_group_id='e34f77cb-54d5-40d0-b539-e0a5fd512d2d'; image_guid | parentid | imagestatus | vm_snapshot_id | volume_type | volume_format | active --------------------------------------+--------------------------------------+-------------+--------------------------------------+-------------+---------------+-------- a107b6c4-842e-4b40-9215-c965431a0c0f | 00000000-0000-0000-0000-000000000000 | 4 | d19d6ca3-1989-4c67-8ee7-c0c43b3e6d74 | 2 | 4 | f a4c86a68-9123-454c-b417-1b15038a4bf2 | a107b6c4-842e-4b40-9215-c965431a0c0f | 1 | e7a405ee-8fd4-4733-ae9c-5252bf07c9d3 | 2 | 4 | f f6a61f2e-26bd-4b63-97c6-d66913ce48c5 | a4c86a68-9123-454c-b417-1b15038a4bf2 | 1 | 9d0958b9-4995-4e11-a027-a32d4bac52e4 | 2 | 4 | t (3 rows) [root@host02 ~]# lvs -o+lv_tags |grep e34f77cb-54d5-40d0-b539-e0a5fd512d2d a107b6c4-842e-4b40-9215-c965431a0c0f 459011cf-ebb6-46ff-831d-8ccfafd82c8a -wi------- 149.50g IU_e34f77cb-54d5-40d0-b539-e0a5fd512d2d,MD_68,PU_00000000-0000-0000-0000-000000000000 f6a61f2e-26bd-4b63-97c6-d66913ce48c5 459011cf-ebb6-46ff-831d-8ccfafd82c8a -wi------- 10.00g IU_e34f77cb-54d5-40d0-b539-e0a5fd512d2d,MD_348,PU_a107b6c4-842e-4b40-9215-c965431a0c0f so image guid a4c86a68-9123-454c-b417-1b15038a4bf2 is not present on disk, i think that the image was correctly merged but not removed from the database. Any suggestion on how to fix the database to reflect the real situation on disk?? In those cases I delete the entry from engine DB to reflect the status of the image chain. BQ_BEGIN TIA -- gb PGP Key: [ http://pgp.mit.edu/ | http://pgp.mit.edu/ ] Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34 _______________________________________________ Users mailing list -- [ mailto:users@ovirt.org | users(a)ovirt.org ] To unsubscribe send an email to [ mailto:users-leave@ovirt.org | users-leave(a)ovirt.org ] Privacy Statement: [ https://www.ovirt.org/privacy-policy.html | https://www.ovirt.org/privacy-policy.html ] oVirt Code of Conduct: [ https://www.ovirt.org/community/about/community-guidelines/ | https://www.ovirt.org/community/about/community-guidelines/ ] List Archives: [ https://lists.ovirt.org/archives/list/users@ovirt.org/message/OF4NTAC6BPGRP… | https://lists.ovirt.org/archives/list/users@ovirt.org/message/OF4NTAC6BPGRP… ] BQ_END _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FRAXF7POUODBM…

1 0

VM with illegal snapshots
by Giorgio Biacchi 17 Nov '20

17 Nov '20

Hi, due to a bug in our Ovirt integrated backup system now we have some VMs with snapshots in illegal state. It seems that there's an inconsistency between the db and the real status of images on disk. Let me show an example: engine=# select image_guid,parentid,imagestatus,vm_snapshot_id,volume_type,volume_format,active from images where image_group_id='e34f77cb-54d5-40d0-b539-e0a5fd512d2d'; image_guid | parentid | imagestatus | vm_snapshot_id | volume_type | volume_format | active --------------------------------------+--------------------------------------+-------------+--------------------------------------+-------------+---------------+-------- a107b6c4-842e-4b40-9215-c965431a0c0f | 00000000-0000-0000-0000-000000000000 | 4 | d19d6ca3-1989-4c67-8ee7-c0c43b3e6d74 | 2 | 4 | f a4c86a68-9123-454c-b417-1b15038a4bf2 | a107b6c4-842e-4b40-9215-c965431a0c0f | 1 | e7a405ee-8fd4-4733-ae9c-5252bf07c9d3 | 2 | 4 | f f6a61f2e-26bd-4b63-97c6-d66913ce48c5 | a4c86a68-9123-454c-b417-1b15038a4bf2 | 1 | 9d0958b9-4995-4e11-a027-a32d4bac52e4 | 2 | 4 | t (3 rows) [root@host02 ~]# lvs -o+lv_tags |grep e34f77cb-54d5-40d0-b539-e0a5fd512d2d a107b6c4-842e-4b40-9215-c965431a0c0f 459011cf-ebb6-46ff-831d-8ccfafd82c8a -wi------- 149.50g IU_e34f77cb-54d5-40d0-b539-e0a5fd512d2d,MD_68,PU_00000000-0000-0000-0000-000000000000 f6a61f2e-26bd-4b63-97c6-d66913ce48c5 459011cf-ebb6-46ff-831d-8ccfafd82c8a -wi------- 10.00g IU_e34f77cb-54d5-40d0-b539-e0a5fd512d2d,MD_348,PU_a107b6c4-842e-4b40-9215-c965431a0c0f so image guid a4c86a68-9123-454c-b417-1b15038a4bf2 is not present on disk, i think that the image was correctly merged but not removed from the database. Any suggestion on how to fix the database to reflect the real situation on disk?? TIA -- gb PGP Key: http://pgp.mit.edu/ Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34

3 3

Really slow manager node
by Nicolás 17 Nov '20

17 Nov '20

Hi, We're running oVirt 4.3.8 and even if this is a problem we've had since a lot of time (I would say since 4.0.x), I decided to look for help in case anything can be done. Our environment is heavily used by users in our University (about 3000 users), and currently our oVirt infrastructure has 1928 virtual machines, being 882 of them currently running. We have a separate physical machine for the manager node, and the problem is that this machine is very, very, very slow, despite it has (from my point of view) enough physical resources to run efficiently. By slow I mean even the SSH access to it takes about 10 seconds, not just the admin/user portals. Any operation takes enough time to make the experience not comfortable to our users (enter the VM portal, start a VM, open a console...). Node machine's parameters are: 18GB of RAM memory 1 processor with 12CPUs 300GB SCSI Disk, local storage. No Storage Domain is stored in this node machine. Currently, most consuming processes are: 28802 ovirt 20 0 3302876 1,5g 25988 S 1,0 8,7 22:46.36 ovirt-engine -server -XX:+TieredCompilation -Xms1024M -Xmx1024M -Xss1M -Djava.aw+ 28701 ovirt 20 0 5465432 807704 13140 S 5,9 4,4 3:15.53 ovirt-engine-dwhd -Dorg.ovirt.engine.dwh.settings=/tmp/tmp8wtnTA/settings.proper+ # free -m total used free shared buff/cache available Mem: 17886 6669 2734 255 8482 10625 Swap: 6143 1002 5141 There are also a lot of postgresql processes: postgres 2186 0.0 0.0 261812 4136 ? Ss jun05 37:39 /opt/rh/rh-postgresql10/root/usr/bin/postmaster -D /var/opt/rh/rh-postgresql10/lib/pgsql/data postgres 3176 0.0 0.0 216084 656 ? Ss jun05 0:00 postgres: logger process postgres 3290 0.0 0.2 262204 37476 ? Ss jun05 77:31 postgres: checkpointer process postgres 3291 0.3 0.2 262052 36960 ? Ss jun05 754:02 postgres: writer process postgres 3292 0.0 0.0 261812 1988 ? Ss jun05 100:51 postgres: wal writer process postgres 3293 0.0 0.2 262380 36748 ? Ss jun05 21:58 postgres: autovacuum launcher process postgres 3294 0.1 0.0 219216 1412 ? Ds jun05 335:41 postgres: stats collector process postgres 3295 0.0 0.0 262220 1460 ? Ss jun05 0:11 postgres: bgworker: logical replication launcher postgres 3393 0.0 0.2 265792 40452 ? Ss jun05 15:36 postgres: engine engine ::1(51664) idle postgres 6105 0.3 0.0 271532 15976 ? Ds 13:01 0:00 postgres: autovacuum worker process ovirt_engine_history postgres 6216 0.2 0.0 263864 11440 ? Ss 13:02 0:00 postgres: autovacuum worker process engine postgres 6245 0.0 0.0 262888 6212 ? Ss 13:02 0:00 postgres: engine engine 127.0.0.1(42400) idle postgres 6246 0.0 0.0 262844 3256 ? Ss 13:02 0:00 postgres: autovacuum worker process template1 postgres 18815 0.0 0.0 262912 5852 ? Ss nov01 0:00 postgres: django django 127.0.0.1(59564) idle postgres 23148 0.0 0.2 266052 43024 ? Ss oct28 9:01 postgres: engine engine 127.0.0.1(59714) idle postgres 23149 0.0 0.0 262980 6820 ? Ss oct28 0:00 postgres: engine engine 127.0.0.1(59716) idle postgres 28784 0.0 0.0 262816 3492 ? Ss 12:02 0:00 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39470) idle postgres 28785 0.0 0.0 262816 3496 ? Ss 12:02 0:00 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39472) idle postgres 28921 0.8 0.7 375152 146452 ? Ss 12:03 0:30 postgres: engine engine 127.0.0.1(39484) idle postgres 29007 3.3 0.7 369348 142184 ? Ds 12:03 1:58 postgres: engine engine 127.0.0.1(39498) SELECT postgres 29009 5.0 0.3 294736 72776 ? Ss 12:03 2:58 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(39500) idle postgres 29048 0.0 0.0 263664 12176 ? Ss 12:03 0:00 postgres: engine engine 127.0.0.1(39530) idle postgres 29064 0.6 0.8 384936 157852 ? Ss 12:03 0:24 postgres: engine engine 127.0.0.1(39532) idle postgres 29065 1.2 0.7 358944 137028 ? Ss 12:03 0:43 postgres: engine engine 127.0.0.1(39534) idle postgres 29066 1.9 0.7 355800 132800 ? Ss 12:03 1:08 postgres: engine engine 127.0.0.1(39536) idle postgres 29067 1.0 0.7 370688 140504 ? Ss 12:03 0:37 postgres: engine engine 127.0.0.1(39538) idle postgres 29068 1.7 0.7 360984 139584 ? Ss 12:03 1:02 postgres: engine engine 127.0.0.1(39540) idle postgres 29069 1.3 0.7 358140 136268 ? Ss 12:03 0:48 postgres: engine engine 127.0.0.1(39542) idle postgres 29070 3.7 0.7 372064 141724 ? Ss 12:03 2:13 postgres: engine engine 127.0.0.1(39544) idle postgres 29071 0.8 0.6 337224 114132 ? Ss 12:03 0:31 postgres: engine engine 127.0.0.1(39546) idle postgres 29072 0.5 0.6 336616 114276 ? Ss 12:03 0:18 postgres: engine engine 127.0.0.1(39548) idle postgres 29073 1.7 0.7 357872 134540 ? Ss 12:03 1:02 postgres: engine engine 127.0.0.1(39550) idle postgres 29139 3.5 0.7 361244 134176 ? Ss 12:03 2:07 postgres: engine engine 127.0.0.1(39572) idle postgres 29140 3.0 0.7 367884 138372 ? Ss 12:03 1:46 postgres: engine engine 127.0.0.1(39570) idle postgres 29141 1.0 0.9 391984 169204 ? Ss 12:03 0:38 postgres: engine engine 127.0.0.1(39574) idle postgres 29142 1.3 0.7 358820 136140 ? Ss 12:03 0:48 postgres: engine engine 127.0.0.1(39576) idle postgres 29143 1.6 0.8 385000 156592 ? Ss 12:03 0:58 postgres: engine engine 127.0.0.1(39578) idle postgres 29144 3.6 0.9 403740 174552 ? Ss 12:03 2:09 postgres: engine engine 127.0.0.1(39580) idle postgres 29145 3.4 0.7 355800 128876 ? Ss 12:03 2:01 postgres: engine engine 127.0.0.1(39582) idle postgres 29146 1.8 0.8 380884 157828 ? Ss 12:03 1:06 postgres: engine engine 127.0.0.1(39586) idle postgres 29147 3.5 0.6 353228 123200 ? Ss 12:03 2:05 postgres: engine engine 127.0.0.1(39584) idle postgres 29148 0.9 0.8 383452 156268 ? Ss 12:03 0:34 postgres: engine engine 127.0.0.1(39588) idle postgres 29149 1.3 0.7 365552 139612 ? Ss 12:03 0:47 postgres: engine engine 127.0.0.1(39590) idle postgres 29150 1.2 0.7 363448 136564 ? Ss 12:03 0:43 postgres: engine engine 127.0.0.1(39592) SELECT postgres 29151 1.4 0.7 365940 139184 ? Ss 12:03 0:51 postgres: engine engine 127.0.0.1(39594) idle postgres 29152 1.2 0.7 358328 134268 ? Ss 12:03 0:44 postgres: engine engine 127.0.0.1(39596) idle postgres 29402 1.3 0.7 362044 139812 ? Ss 12:04 0:47 postgres: engine engine 127.0.0.1(39698) idle postgres 29403 2.1 0.8 377760 154556 ? Ss 12:04 1:15 postgres: engine engine 127.0.0.1(39700) idle postgres 29404 2.2 0.7 369160 144600 ? Ss 12:04 1:18 postgres: engine engine 127.0.0.1(39702) idle postgres 32230 0.1 0.5 336252 101912 ? Ss 12:19 0:04 postgres: engine engine 127.0.0.1(40564) idle postgres 32232 3.4 0.6 357644 127732 ? Ss 12:19 1:29 postgres: engine engine 127.0.0.1(40566) idle postgres 32234 0.0 0.1 270924 19836 ? Ss 12:19 0:00 postgres: engine engine 127.0.0.1(40568) idle Actually, I'm not really sure why is this machine so slow. In terms of RAM memory and CPU, it still has a bunch of free resources. Could someone help track this down and point some advices on how to optimize the user experience in this case? Thanks!

2 1

easy way to migrate to new engine
by jb 17 Nov '20

17 Nov '20

Hello everybody! At the moment I have here a standalone engine and two nodes. Storage type is NFS. Now I want to setup a 3 node self hosted cluster with Glusterfs and import my VMs from the old engine. I know, there is a guide for migration from standalone to self-hosted engine, but there is so many changes (new Subnet/DNS names, different type of storage, new CPU type) that I would like to skip this migration guide. Is there a different way to migrate my VMs? Mounting my old storage and moving the disks to the Gluster would be nice, but I guess I can not just mount a storage with existing VM images. Best regards Jonathan

3 2