Engine backup / restore
by duparchy＠esrf.fr
Hi,
I'm testing Backup / Restore of the engine.
Once the engine is restored ; how to resync the engine database with the "reality" of the Storage Domains ? (I'm talking about anything that could have been done after the engine backup : Deleted / Created VMs ; Snapshots taken since the backup, etc..
I'm on 4.3, Oracle LVM context.
My Tests :
- Create a snapshot of some random VM. Say VM-1
- Enable that VM preview and start it on a host.
- Stop OLVM engine server.
- Start a full backup copy of that OLVM engine (That backup being a snapshot taken, say, yesterday. Our test OLVM engine being a VM running in another separate virltualization infrastucture.)
The restored OLVM does not show the existence of the VM-1 snapshot. It only shows the VM running on the active instance. No snapshot.
I tried Storage Domain / Scan disks. No refresh to be seen.
3 years, 1 month
Re: Ceph integration
by Sandro Bonazzola
Il giorno mer 20 ott 2021 alle ore 10:09 samuel.xhu(a)horebdata.cn <
samuel.xhu(a)horebdata.cn> ha scritto:
> Great, sounds very interesting. I will give a try.
>
> How about the stability of this approach to use Ceph RBD? Can we call it
> prodouction-ready or just experimental?
>
I'll let the storage team to reply to this, but it seems stable enough for
production to me.
+Eyal Shenitzky <eshenitz(a)redhat.com> ?
>
> ------------------------------
> Do Right Thing (做正确的事) / Pursue Excellence (追求卓越) / Help Others Succeed
> (成就他人)
>
>
> *From:* Sandro Bonazzola <sbonazzo(a)redhat.com>
> *Date:* 2021-10-20 10:04
> *To:* samuel.xhu(a)horebdata.cn
> *CC:* users <users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: Ovirt VM started with the most recent
> snapshot with the current disk missing
>
>
> Il giorno mer 20 ott 2021 alle ore 09:59 samuel.xhu(a)horebdata.cn <
> samuel.xhu(a)horebdata.cn> ha scritto:
>
>> Thanks for the tip. I will then upgrade to 4.4.
>>
>> By the way, how about the progress of the support of Ceph storage for
>> Ovirt? Proxmox has a very good successful story of integrating Ceph, and we
>> hope Ovirt would soon catch up.
>>
>
> I wrote a blog about it here:
> https://blogs.ovirt.org/2021/07/using-ceph-only-storage-for-ovirt-datacen...
>
>
>
>
>>
>> best regards,
>>
>> samuel
>>
>> ------------------------------
>> Do Right Thing (做正确的事) / Pursue Excellence (追求卓越) / Help Others Succeed
>> (成就他人)
>>
>>
>> *From:* Sandro Bonazzola <sbonazzo(a)redhat.com>
>> *Date:* 2021-10-20 09:16
>> *To:* Xiaoyu Hu <samuel.xhu(a)gmail.com>
>> *CC:* oVirt Users <users(a)ovirt.org>
>> *Subject:* [ovirt-users] Re: Ovirt VM started with the most recent
>> snapshot with the current disk missing
>>
>>
>> Il giorno dom 17 ott 2021 alle ore 13:57 Xiaoyu Hu <samuel.xhu(a)gmail.com>
>> ha scritto:
>>
>>> Dear Ovirt folks,
>>>
>>> I am faced with a very strange situation with Ovirt 4.3. A virtual
>>> machine has two snapshots and a working disk, with the former two snapshots
>>> being read only and the working one read-write. After an unplanned power
>>> outage, a virtual machine can only be started with the recent snapshot, and
>>> the actual working disk is missing in the Ovirt UI. I can see the disk
>>> still exists in the backend storage, but it seems not Ovirt does not know
>>> it. Thus the VM lose the recent data on the missing disk.
>>>
>>> Does any one know whether this is a known bug in Ovirt 4.3, or is there
>>> a method to resotre the missing working disk back to the virtuanl machine?
>>> Otherwise i am faced with a terrible data loss issue on Ovirt
>>>
>> Hi, oVirt 4.3 reached the end of life a long time ago. I would recommend
>> upgrading to oVirt 4.4 as soon as practical.
>>
>>
>>
>>> best regards,
>>>
>>> Samuel
>>> _______________________________________________
>>> Users mailing list -- users(a)ovirt.org
>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/N7JVXAU5MLA...
>>>
>>
>>
>> --
>>
>> Sandro Bonazzola
>>
>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>>
>> Red Hat EMEA <https://www.redhat.com/>
>>
>> sbonazzo(a)redhat.com
>> <https://www.redhat.com/>
>>
>> *Red Hat respects your work life balance. Therefore there is no need to
>> answer this email out of your office hours.*
>>
>>
>>
>
> --
>
> Sandro Bonazzola
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>
> Red Hat EMEA <https://www.redhat.com/>
>
> sbonazzo(a)redhat.com
> <https://www.redhat.com/>
>
> *Red Hat respects your work life balance. Therefore there is no need to
> answer this email out of your office hours.*
>
>
>
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://www.redhat.com/>
*Red Hat respects your work life balance. Therefore there is no need to
answer this email out of your office hours.*
3 years, 1 month
Re: Ovirt VM started with the most recent snapshot with the current disk missing
by Sandro Bonazzola
Il giorno mer 20 ott 2021 alle ore 09:59 samuel.xhu(a)horebdata.cn <
samuel.xhu(a)horebdata.cn> ha scritto:
> Thanks for the tip. I will then upgrade to 4.4.
>
> By the way, how about the progress of the support of Ceph storage for
> Ovirt? Proxmox has a very good successful story of integrating Ceph, and we
> hope Ovirt would soon catch up.
>
I wrote a blog about it here:
https://blogs.ovirt.org/2021/07/using-ceph-only-storage-for-ovirt-datacen...
>
> best regards,
>
> samuel
>
> ------------------------------
> Do Right Thing (做正确的事) / Pursue Excellence (追求卓越) / Help Others Succeed
> (成就他人)
>
>
> *From:* Sandro Bonazzola <sbonazzo(a)redhat.com>
> *Date:* 2021-10-20 09:16
> *To:* Xiaoyu Hu <samuel.xhu(a)gmail.com>
> *CC:* oVirt Users <users(a)ovirt.org>
> *Subject:* [ovirt-users] Re: Ovirt VM started with the most recent
> snapshot with the current disk missing
>
>
> Il giorno dom 17 ott 2021 alle ore 13:57 Xiaoyu Hu <samuel.xhu(a)gmail.com>
> ha scritto:
>
>> Dear Ovirt folks,
>>
>> I am faced with a very strange situation with Ovirt 4.3. A virtual
>> machine has two snapshots and a working disk, with the former two snapshots
>> being read only and the working one read-write. After an unplanned power
>> outage, a virtual machine can only be started with the recent snapshot, and
>> the actual working disk is missing in the Ovirt UI. I can see the disk
>> still exists in the backend storage, but it seems not Ovirt does not know
>> it. Thus the VM lose the recent data on the missing disk.
>>
>> Does any one know whether this is a known bug in Ovirt 4.3, or is there a
>> method to resotre the missing working disk back to the virtuanl machine?
>> Otherwise i am faced with a terrible data loss issue on Ovirt
>>
> Hi, oVirt 4.3 reached the end of life a long time ago. I would recommend
> upgrading to oVirt 4.4 as soon as practical.
>
>
>
>> best regards,
>>
>> Samuel
>> _______________________________________________
>> Users mailing list -- users(a)ovirt.org
>> To unsubscribe send an email to users-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/N7JVXAU5MLA...
>>
>
>
> --
>
> Sandro Bonazzola
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>
> Red Hat EMEA <https://www.redhat.com/>
>
> sbonazzo(a)redhat.com
> <https://www.redhat.com/>
>
> *Red Hat respects your work life balance. Therefore there is no need to
> answer this email out of your office hours.*
>
>
>
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://www.redhat.com/>
*Red Hat respects your work life balance. Therefore there is no need to
answer this email out of your office hours.*
3 years, 1 month
prod network glitch after reboot lab server
by Dominique D
Hi,
I have two ovirt environments on the same network.
2 switches (10G switch for storage and 1G switch for data / management)
1) Prod environment:
3 oVirt 4.4.6 nodes (1 port 10G per node for storage gluster and 1 port 1G for data / management)
2) Lab environment
2 oVirt nodes 4.4.6 (1 Port 10G per node for storage gluster and 1 port 1G for data / management)
The storage is on the same vlan and subnet for both environments.
When I restart one of my lab servers, I have communication cuts/glitch on my production ovirt clusters. I even had server reboots.
Do you know why ?
Here somes log :
VDSM ovnode1.telecom.lan command SpmStatusVDS failed: Connection timeout for host 'ovnode1.telecom.lan', last response arrived 42479 ms ago.
Host ovnode1.telecom.lan is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued.
Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host ovnode1.telecom.lan, Error: Network error during communication with the Host.).
Host ovnode1.telecom.lan cannot access the Storage Domain(s) datassd attached to the Data Center Default. Setting Host state to Non-Operational.
3 years, 1 month
Ovirt VM started with the most recent snapshot with the current disk missing
by Xiaoyu Hu
Dear Ovirt folks,
I am faced with a very strange situation with Ovirt 4.3. A virtual machine has two snapshots and a working disk, with the former two snapshots being read only and the working one read-write. After an unplanned power outage, a virtual machine can only be started with the recent snapshot, and the actual working disk is missing in the Ovirt UI. I can see the disk still exists in the backend storage, but it seems not Ovirt does not know it. Thus the VM lose the recent data on the missing disk.
Does any one know whether this is a known bug in Ovirt 4.3, or is there a method to resotre the missing working disk back to the virtuanl machine? Otherwise i am faced with a terrible data loss issue on Ovirt
best regards,
Samuel
3 years, 1 month
Can't access ovirtengine
by mohammed.abuhajar＠edugrade.se
Hi
I have installed oVirt 4.4.8 och hosted-engine. The installation har done without error, but I am not able to access ovirtengine. When I check the status of engine it seems everything is ok and up. Any idea?
//Mohammed
3 years, 1 month
Q: oVirt guest agent + spice-vdagent on Debian 11 Bullseye
by Andrei Verovski
Hi,
Anyone have compiled these deb packages for Debian 11 Bullseye?
oVirt guest agent + spice-vdagent
Packages from Buster can’t be installed on Bullseye because of broken libnl dependencies.
Thanks in advance.
Andrei
3 years, 1 month
GPG key <infra@ovirt.org> expired since April ?
by Fredrik Arneving
Hi,
According to output pasted below it seems that the GPG key used to sign installation media has expired 2021-04-03. Why is new installation ISO signed 7 days ago with a key that has been expired for almost 6 months? Is this correct?
My main question though is if this iso is authentic?
$ ll -h ovirt-node-ng-installer-4.4.8-2021090310.el8.iso*
-rw-r--r--. 1 fredde fredde 1.9G Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
-rw-r--r--. 1 fredde fredde 32 Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum
-rw-r--r--. 1 fredde fredde 490 Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum.sig
$ gpg --list-keys oVirt
pub rsa2048 2014-03-30 [SC] [expired: 2021-04-03]
31A5D7837FAD7CB286CD3469AB8C4F9DFE590CB7
uid [ expired] oVirt <infra(a)ovirt.org>
$ gpg --verify-files *.sig
gpg: assuming signed data in 'ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum'
gpg: Signature made Thu 23 Sep 2021 02:41:24 PM CEST
gpg: using RSA key AB8C4F9DFE590CB7
gpg: Good signature from "oVirt <infra(a)ovirt.org>" [expired]
gpg: Note: This key has expired!
Primary key fingerprint: 31A5 D783 7FAD 7CB2 86CD 3469 AB8C 4F9D FE59 0CB7
$ cat *.md5sum
e75ac6f671c666140a205e6eab3d0c4a
$ md5sum ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
e75ac6f671c666140a205e6eab3d0c4a ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
BR
/F
3 years, 1 month
UEFI Guest can only be started on UEFI host (4.4)
by nroach44＠nroach44.id.au
Hi All,
A problem I've just "dealt with" over the past months is that the two UEFI VMs I have installed (One Windows 10, one RHEL8) will only start on the oVirt Node (4.4.x, still an issue on 4.4.8) hosts that have been installed using UEFI.
In the case of both guests, they will "start" but get stuck on a small 640x480-ish black screen, with no CPU or disk activity. It looks as if the VM has been started with "Start paused" enabled, but the VM is not paused. I've noticed that this matches the normal startup of the guest, although it only spends a second or two like that before TianoCore takes over.
Occasionally, I'm able to migrate the VM to a BIOS host. When it fails, the following is seen on the /sending/ host:
2021-09-21 20:09:42,915+0800 ERROR (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') internal error: qemu unexpectedly closed the monitor: 2021-09-21T12:08:57.355188Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 1 should be encrypted
2021-09-21T12:08:57.393585Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 3 should be encrypted
2021-09-21T12:08:57.393805Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 4 should be encrypted
2021-09-21T12:08:57.393960Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 2 should be encrypted
2021-09-21T12:09:40.799119Z qemu-kvm: warning: TSC frequency mismatch between VM (3099980 kHz) and host (3392282 kHz), and TSC scaling unavailable
2021-09-21T12:09:40.799228Z qemu-kvm: error: failed to set MSR 0x204 to 0x1000000000
qemu-kvm: ../target/i386/kvm/kvm.c:2778: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed. (migration:331)
2021-09-21 20:09:42,938+0800 INFO (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Switching from State.STARTED to State.FAILED (migration:234)
2021-09-21 20:09:42,938+0800 ERROR (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Failed to migrate (migration:503)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 477, in _regular_run
time.time(), machineParams
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 578, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 667, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 596, in _perform_migration
self._migration_flags)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 159, in call
return getattr(self._vm._dom, name)(*a, **kw)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 101, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2126, in migrateToURI3
raise libvirtError('virDomainMigrateToURI3() failed')
libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2021-09-21T12:08:57.355188Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 1 should be encrypted
2021-09-21T12:08:57.393585Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 3 should be encrypted
2021-09-21T12:08:57.393805Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 4 should be encrypted
2021-09-21T12:08:57.393960Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 2 should be encrypted
2021-09-21T12:09:40.799119Z qemu-kvm: warning: TSC frequency mismatch between VM (3099980 kHz) and host (3392282 kHz), and TSC scaling unavailable
2021-09-21T12:09:40.799228Z qemu-kvm: error: failed to set MSR 0x204 to 0x1000000000
qemu-kvm: ../target/i386/kvm/kvm.c:2778: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
The receiving host simply sees
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') underlying process disconnected (vm:1135)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Release VM resources (vm:5325)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [vdsm.api] START teardownImage(sdUUID='3f46f0f3-1cbb-4154-8af5-dcc3a09c6177', spUUID='924e5fbe-beba-11ea-b679-00163e03ad3e', imgUUID='d91282d3-2552-44d3-aa0f-84f7330be4ce', volUUID=None) from=internal, task_id=51eb32fc-1167-4c4c-bea8-4664c92d15e9 (api:48)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [storage.StorageDomain] Removing image rundir link '/run/vdsm/storage/3f46f0f3-1cbb-4154-8af5-dcc3a09c6177/d91282d3-2552-44d3-aa0f-84f7330be4ce' (fileSD:601)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [vdsm.api] FINISH teardownImage return=None from=internal, task_id=51eb32fc-1167-4c4c-bea8-4664c92d15e9 (api:54)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [vdsm.api] START inappropriateDevices(thiefId='86df93bc-3304-4002-8939-cbefdea4cc60') from=internal, task_id=1e3aafc2-62c7-4fe5-a807-69942709e936 (api:48)
2021-09-21 20:09:42,842+0800 INFO (libvirt/events) [vdsm.api] FINISH inappropriateDevices return=None from=internal, task_id=1e3aafc2-62c7-4fe5-a807-69942709e936 (api:54)
2021-09-21 20:09:42,847+0800 WARN (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Couldn't destroy incoming VM: Domain not found: no domain with matching uuid '86df93bc-3304-4002-8939-cbefdea4cc60' (vm:4073)
2021-09-21 20:09:42,847+0800 INFO (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Changed state to Down: VM destroyed during the startup (code=10) (vm:1921)
2021-09-21 20:09:42,849+0800 INFO (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,856+0800 INFO (jsonrpc/3) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.1.2.30,59424, flow_id=47e0a91b, vmId=86df93bc-3304-4002-8939-cbefdea4cc60 (api:48)
2021-09-21 20:09:42,917+0800 INFO (jsonrpc/5) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.1.2.7,50798, vmId=86df93bc-3304-4002-8939-cbefdea4cc60 (api:48)
The Data center is configured with BIOS as a default.
As an aside, *all* hosts have the following cmdline set: (to allow nested virt)
intel_iommu=on kvm-intel.nested=1 kvm.ignore_msrs=1
Any suggestions?
3 years, 1 month
Power Saving schedule, hosts not shutting down
by Maton, Brett
Hi,
I'm having trouble with the power_saving Scheduling Policy not shutting
down idle hosts
Policy is more or less default, I added 'HostsInReserve 0' too see if
that would help, and then 24hrs later I bumped
CpuOverCommitDurationMinutes to 15 that didn't make a difference either.
(not unexpected as the CPU is only being tickled by two small VMs at the
moment).
3x Dell Hosts with iDRAC 8 management cards, power management configured
and functional.
oVrit 4.4.8.6-1.el8
Thanks in advance for any help
Brett
3 years, 1 month
