prod network glitch after reboot lab server
by Dominique D
Hi,
I have two ovirt environments on the same network.
2 switches (10G switch for storage and 1G switch for data / management)
1) Prod environment:
3 oVirt 4.4.6 nodes (1 port 10G per node for storage gluster and 1 port 1G for data / management)
2) Lab environment
2 oVirt nodes 4.4.6 (1 Port 10G per node for storage gluster and 1 port 1G for data / management)
The storage is on the same vlan and subnet for both environments.
When I restart one of my lab servers, I have communication cuts/glitch on my production ovirt clusters. I even had server reboots.
Do you know why ?
Here somes log :
VDSM ovnode1.telecom.lan command SpmStatusVDS failed: Connection timeout for host 'ovnode1.telecom.lan', last response arrived 42479 ms ago.
Host ovnode1.telecom.lan is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued.
Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host ovnode1.telecom.lan, Error: Network error during communication with the Host.).
Host ovnode1.telecom.lan cannot access the Storage Domain(s) datassd attached to the Data Center Default. Setting Host state to Non-Operational.
2 years, 5 months
Ovirt VM started with the most recent snapshot with the current disk missing
by Xiaoyu Hu
Dear Ovirt folks,
I am faced with a very strange situation with Ovirt 4.3. A virtual machine has two snapshots and a working disk, with the former two snapshots being read only and the working one read-write. After an unplanned power outage, a virtual machine can only be started with the recent snapshot, and the actual working disk is missing in the Ovirt UI. I can see the disk still exists in the backend storage, but it seems not Ovirt does not know it. Thus the VM lose the recent data on the missing disk.
Does any one know whether this is a known bug in Ovirt 4.3, or is there a method to resotre the missing working disk back to the virtuanl machine? Otherwise i am faced with a terrible data loss issue on Ovirt
best regards,
Samuel
2 years, 5 months
Can't access ovirtengine
by mohammed.abuhajar@edugrade.se
Hi
I have installed oVirt 4.4.8 och hosted-engine. The installation har done without error, but I am not able to access ovirtengine. When I check the status of engine it seems everything is ok and up. Any idea?
//Mohammed
2 years, 5 months
Q: oVirt guest agent + spice-vdagent on Debian 11 Bullseye
by Andrei Verovski
Hi,
Anyone have compiled these deb packages for Debian 11 Bullseye?
oVirt guest agent + spice-vdagent
Packages from Buster can’t be installed on Bullseye because of broken libnl dependencies.
Thanks in advance.
Andrei
2 years, 5 months
GPG key <infra@ovirt.org> expired since April ?
by Fredrik Arneving
Hi,
According to output pasted below it seems that the GPG key used to sign installation media has expired 2021-04-03. Why is new installation ISO signed 7 days ago with a key that has been expired for almost 6 months? Is this correct?
My main question though is if this iso is authentic?
$ ll -h ovirt-node-ng-installer-4.4.8-2021090310.el8.iso*
-rw-r--r--. 1 fredde fredde 1.9G Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
-rw-r--r--. 1 fredde fredde 32 Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum
-rw-r--r--. 1 fredde fredde 490 Oct 1 14:48 ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum.sig
$ gpg --list-keys oVirt
pub rsa2048 2014-03-30 [SC] [expired: 2021-04-03]
31A5D7837FAD7CB286CD3469AB8C4F9DFE590CB7
uid [ expired] oVirt <infra(a)ovirt.org>
$ gpg --verify-files *.sig
gpg: assuming signed data in 'ovirt-node-ng-installer-4.4.8-2021090310.el8.iso.md5sum'
gpg: Signature made Thu 23 Sep 2021 02:41:24 PM CEST
gpg: using RSA key AB8C4F9DFE590CB7
gpg: Good signature from "oVirt <infra(a)ovirt.org>" [expired]
gpg: Note: This key has expired!
Primary key fingerprint: 31A5 D783 7FAD 7CB2 86CD 3469 AB8C 4F9D FE59 0CB7
$ cat *.md5sum
e75ac6f671c666140a205e6eab3d0c4a
$ md5sum ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
e75ac6f671c666140a205e6eab3d0c4a ovirt-node-ng-installer-4.4.8-2021090310.el8.iso
BR
/F
2 years, 5 months
UEFI Guest can only be started on UEFI host (4.4)
by nroach44@nroach44.id.au
Hi All,
A problem I've just "dealt with" over the past months is that the two UEFI VMs I have installed (One Windows 10, one RHEL8) will only start on the oVirt Node (4.4.x, still an issue on 4.4.8) hosts that have been installed using UEFI.
In the case of both guests, they will "start" but get stuck on a small 640x480-ish black screen, with no CPU or disk activity. It looks as if the VM has been started with "Start paused" enabled, but the VM is not paused. I've noticed that this matches the normal startup of the guest, although it only spends a second or two like that before TianoCore takes over.
Occasionally, I'm able to migrate the VM to a BIOS host. When it fails, the following is seen on the /sending/ host:
2021-09-21 20:09:42,915+0800 ERROR (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') internal error: qemu unexpectedly closed the monitor: 2021-09-21T12:08:57.355188Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 1 should be encrypted
2021-09-21T12:08:57.393585Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 3 should be encrypted
2021-09-21T12:08:57.393805Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 4 should be encrypted
2021-09-21T12:08:57.393960Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 2 should be encrypted
2021-09-21T12:09:40.799119Z qemu-kvm: warning: TSC frequency mismatch between VM (3099980 kHz) and host (3392282 kHz), and TSC scaling unavailable
2021-09-21T12:09:40.799228Z qemu-kvm: error: failed to set MSR 0x204 to 0x1000000000
qemu-kvm: ../target/i386/kvm/kvm.c:2778: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed. (migration:331)
2021-09-21 20:09:42,938+0800 INFO (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Switching from State.STARTED to State.FAILED (migration:234)
2021-09-21 20:09:42,938+0800 ERROR (migsrc/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Failed to migrate (migration:503)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 477, in _regular_run
time.time(), machineParams
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 578, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 667, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 596, in _perform_migration
self._migration_flags)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 159, in call
return getattr(self._vm._dom, name)(*a, **kw)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 101, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2126, in migrateToURI3
raise libvirtError('virDomainMigrateToURI3() failed')
libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2021-09-21T12:08:57.355188Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 1 should be encrypted
2021-09-21T12:08:57.393585Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 3 should be encrypted
2021-09-21T12:08:57.393805Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 4 should be encrypted
2021-09-21T12:08:57.393960Z qemu-kvm: warning: Spice: reds.c:2305:reds_handle_read_link_done: spice channels 2 should be encrypted
2021-09-21T12:09:40.799119Z qemu-kvm: warning: TSC frequency mismatch between VM (3099980 kHz) and host (3392282 kHz), and TSC scaling unavailable
2021-09-21T12:09:40.799228Z qemu-kvm: error: failed to set MSR 0x204 to 0x1000000000
qemu-kvm: ../target/i386/kvm/kvm.c:2778: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
The receiving host simply sees
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') underlying process disconnected (vm:1135)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Release VM resources (vm:5325)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,840+0800 INFO (libvirt/events) [vdsm.api] START teardownImage(sdUUID='3f46f0f3-1cbb-4154-8af5-dcc3a09c6177', spUUID='924e5fbe-beba-11ea-b679-00163e03ad3e', imgUUID='d91282d3-2552-44d3-aa0f-84f7330be4ce', volUUID=None) from=internal, task_id=51eb32fc-1167-4c4c-bea8-4664c92d15e9 (api:48)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [storage.StorageDomain] Removing image rundir link '/run/vdsm/storage/3f46f0f3-1cbb-4154-8af5-dcc3a09c6177/d91282d3-2552-44d3-aa0f-84f7330be4ce' (fileSD:601)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [vdsm.api] FINISH teardownImage return=None from=internal, task_id=51eb32fc-1167-4c4c-bea8-4664c92d15e9 (api:54)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,841+0800 INFO (libvirt/events) [vdsm.api] START inappropriateDevices(thiefId='86df93bc-3304-4002-8939-cbefdea4cc60') from=internal, task_id=1e3aafc2-62c7-4fe5-a807-69942709e936 (api:48)
2021-09-21 20:09:42,842+0800 INFO (libvirt/events) [vdsm.api] FINISH inappropriateDevices return=None from=internal, task_id=1e3aafc2-62c7-4fe5-a807-69942709e936 (api:54)
2021-09-21 20:09:42,847+0800 WARN (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Couldn't destroy incoming VM: Domain not found: no domain with matching uuid '86df93bc-3304-4002-8939-cbefdea4cc60' (vm:4073)
2021-09-21 20:09:42,847+0800 INFO (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Changed state to Down: VM destroyed during the startup (code=10) (vm:1921)
2021-09-21 20:09:42,849+0800 INFO (vm/86df93bc) [virt.vm] (vmId='86df93bc-3304-4002-8939-cbefdea4cc60') Stopping connection (guestagent:438)
2021-09-21 20:09:42,856+0800 INFO (jsonrpc/3) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.1.2.30,59424, flow_id=47e0a91b, vmId=86df93bc-3304-4002-8939-cbefdea4cc60 (api:48)
2021-09-21 20:09:42,917+0800 INFO (jsonrpc/5) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.1.2.7,50798, vmId=86df93bc-3304-4002-8939-cbefdea4cc60 (api:48)
The Data center is configured with BIOS as a default.
As an aside, *all* hosts have the following cmdline set: (to allow nested virt)
intel_iommu=on kvm-intel.nested=1 kvm.ignore_msrs=1
Any suggestions?
2 years, 5 months
Power Saving schedule, hosts not shutting down
by Maton, Brett
Hi,
I'm having trouble with the power_saving Scheduling Policy not shutting
down idle hosts
Policy is more or less default, I added 'HostsInReserve 0' too see if
that would help, and then 24hrs later I bumped
CpuOverCommitDurationMinutes to 15 that didn't make a difference either.
(not unexpected as the CPU is only being tickled by two small VMs at the
moment).
3x Dell Hosts with iDRAC 8 management cards, power management configured
and functional.
oVrit 4.4.8.6-1.el8
Thanks in advance for any help
Brett
2 years, 5 months
Assigning public IPs to a VM
by admin@foundryserver.com
My provider will give me 20 public IP's, and attach them to each of my hosts. So a total of 60. The question I have, is I have two nics on the host, public and private. Is assigning the public IP as simple as selecting the public network and typing in one of the public IP's? I am still trying to get my hosted engine running, so at this point can't try this. Just thought I would ask in advance.
Again Thank you for any help.
Brad
2 years, 5 months
Understanding Cluster Networking between Hosts
by admin@foundryserver.com
Hello everyone. I am looking to setup a ovrit cluster. I am struggling with the network side of things. I have watched some videos about Ovirt and kvm/libvirt. I understand the bridge and NAT networks. The part I am struggling with is the host network. Here is my setup.
I have two bare metal servers, that have one physical nic with a public IP. There is no private physical network on these boxes virtual or physical.
I want to host an application on a single vm for each customer. the customer access the application via username.domain.com.
So the question is. I have to have a layer 7 load balancer to route the url to an IP. This load balancer is haproxy on a separate bare metal box. (it could be in a vm on one of the hosts) So the load balancer has to be on the same network as all the vms. I just don't know to setup the networking between the LB on one box and the vm's on potentially may bare metal hosts.
It really feels like I am missing something really obvious. All the videos I have watched all assume the host networking is already in place. Any help or resources I can read/watch would really be helpful.
Brad
2 years, 5 months
not able to upload disks, iso - paused by the system error -- Version 4.4.6.7-1.el8
by dhanaraj.ramesh@yahoo.com
Hi Team
in one of the cluster infra, we are unable to upload the images or disks via gui. up on checking the /var/log/ovirt-imageio/daemon.log found that throwing ssl connection failure, help us to check what are we missing..
We are using thirdparty CA approved SSL for web GUI..
2021-10-11 22:45:42,812 INFO (Thread-6) [http] OPEN connection=6 client=127.0.0.1
2021-10-11 22:45:42,812 INFO (Thread-6) [tickets] [127.0.0.1] REMOVE ticket=f18cff91-1fc4-43b6-91ea-ca2a11d409a6
2021-10-11 22:45:42,813 INFO (Thread-6) [http] CLOSE connection=6 client=127.0.0.1 [connection 1 ops, 0.000539 s] [dispatch 1 ops, 0.000216 s]
2021-10-11 22:45:43,621 INFO (Thread-4) [images] [::ffff:10.12.23.212] OPTIONS ticket=53ff98f9-f429-4880-abe6-06c6c01473de
2021-10-11 22:45:43,621 INFO (Thread-4) [backends.http] Open backend netloc='renlovkvma01.test.lab:54322' path='/images/53ff98f9-f429-4880-abe6-06c6c01473de' cafile='/etc/pki/ovirt-engine/ca.pem' secure=True
2021-10-11 22:45:43,626 ERROR (Thread-4) [http] Server error
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/__init__.py", line 66, in get
return ticket.get_context(req.connection_id)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/auth.py", line 146, in get_context
return self._connections[con_id]
KeyError: 4
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 774, in __call__
self.dispatch(req, resp)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 819, in dispatch
return method(req, resp, *match.groups())
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/cors.py", line 84, in wrapper
return func(self, req, resp, *args)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/images.py", line 246, in options
ctx = backends.get(req, ticket, self.config)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/__init__.py", line 85, in get
cafile=ca_file)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 48, in open
return Backend(url, **options)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 76, in __init__
self._connect()
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 117, in _connect
self._con = self._create_tcp_connection()
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 379, in _create_tcp_connection
con.connect()
File "/usr/lib64/python3.6/http/client.py", line 1437, in connect
server_hostname=server_hostname)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 776, in __init__
self.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 1036, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 648, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
2 years, 5 months