- Users - oVirt List Archives

oVirt/Ceph iSCSI Issues
by Matthew J Black 05 Dec '22

05 Dec '22

Hi All, I've got some issues with connecting my oVirt Cluster to my Ceph Cluster via iSCSI. There are two issues, and I don't know if one is causing the other, if they are related at all, or if they are two separate, unrelated issues. Let me explain. The Situation ------------- - I have a working three node Ceph Cluster (Ceph Quincy on Rocky Linux 8.6) - The Ceph Cluster has four Storage Pools of between 4 and 8 TB each - The Ceph Cluster has three iSCSI Gateways - There is a single iSCSI Target on the Ceph Cluster - The iSCSI Target has all three iSCSI Gateways attached - The iSCSI Target has all four Storage Pools attached - The four Storage Pools have been assigned LUNs 0-3 - I have set up (Discovery) CHAP Authorisation on the iSCSI Target - I have a working three node self-hosted oVirt Cluster (oVirt v4.5.3 on Rocky Linux 8.6) - The oVirt Cluster has (in addition to the hosted_storage Storage Domain) three GlusterFS Storage Domains - I can ping all three Ceph Cluster Nodes to/from all three oVirt Hosts - The iSCSI Target on the Ceph Cluster has all three oVirt Hosts Initiators attached - Each Initiator has all four Ceph Storage Pools attached - I have set up CHAP Authorisation on the iSCSI Target's Initiators - The Ceph Cluster Admin Portal reports that all three Initiators are "logged_in" - I have previous connected Ceph iSCSI LUNs to the oVirt Cluster successfully (as an experiment), but had to remove and re-instate them for the "final" version(?). - The oVirt Admin Portal (ie HostedEngine) reports that Initiators are 1 & 2 (ie oVirt Hosts 1 & 2) are "logged_in" to all three iSCSI Gateways - The oVirt Admin Portal reports that Initiator 3 (ie oVirt Host 3) is "logged_in" to iSCSI Gateways 1 & 2 - I can "force" Initiator 3 to become "logged_in" to iSCSI Gateway 3, but when I do this it is *not* persistent - oVirt Hosts 1 & 2 can/have discovered all three iSCSI Gateways - oVirt Hosts 1 & 2 can/have discovered all four LUNs/Targets on all three iSCSI Gateways - oVirt Host 3 can only discover 2 of the iSCSI Gateways - For Target/LUN 0 oVirt Host 3 can only "see" the LUN provided by iSCSI Gateway 1 - For Targets/LUNs 1-3 oVirt Host 3 can only "see" the LUNs provided by iSCSI Gateways 1 & 2 - oVirt Host 3 can *not* "see" any of the Targets/LUNs provided by iSCSI Gateway 3 - When I create a new oVirt Storage Domain for any of the four LUNs: - I am presented with a message saying "The following LUNs are already in use..." - I am asked to "Approve operation" via a checkbox, which I do - As I watch the oVirt Admin Portal I can see the new iSCSI Storage Domain appear in the Storage Domain list, and then after a few minutes it is removed - After those few minutes I am presented with this failure message: "Error while executing action New SAN Storage Domain: Network error during communication with the Host." - I have looked in the engine.log and all I could find that was relevant (as far as I know) was this: ~~~ 2022-11-28 19:59:20,506+11 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-1) [77b0c12d] Command 'CreateStorageDomainVDSCommand(HostName = ovirt_node_1.mynet.local, CreateStorageDomainVDSCommandParameters:{hostId='967301de-be9f-472a-8e66-03c24f01fa71', storageDomain='StorageDomainStatic:{name='data', id='2a14e4bd-c273-40a0-9791-6d683d145558'}', args='s0OGKR-80PH-KVPX-Fi1q-M3e4-Jsh7-gv337P'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2022-11-28 19:59:20,507+11 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-1) [77b0c12d] Command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues (Failed with error VDS_NETWORK_ERROR and code 5022) ~~~ I cannot see/detect any "communication issue" - but then again I'm not 100% sure what I should be looking for I have looked on-line for an answer, and apart from not being able to get past Red Hat's "wall" to see the solutions that they have, all I could find that was relevant was this: https://lists.ovirt.org/archives/list/devel@ovirt.org/thread/AVLORQNOLJHRWM… . If this *is* relevant then there is not enough context here for me to proceed (ie/eg *where* (which host/vm) should that command be run?). I also found (for a previous version of oVirt) notes about modifying the Postgres DB manual to resolve a similar issue. While I am more than comfortable doing this (I've been an SQL DBA for well over 20 years) this seems like asking for trouble - at least until I hear back from the oVirt Devs that this is OK to do - and of course, I'll need the relevant commands / locations / authorisations to get into the DB. Questions --------- - Are the two issues (oVirt Host 3 not having a full picture of the Ceph iSCSI environment and the oVirt iSCSI Storage Domain creation failure) related? - Do I need to "refresh" the iSCSI info on the oVirt Hosts, and if so, how do I do this? - Do I need to "flush" the old LUNs from the oVirt Cluster, and if so, how do I do this? - Where else should I be looking for info in the logs (& which logs)? - Does *anyone* have any other ideas how to resolve the situation - especially when using the Ceph iSCII Gateways? Thanks in advance Cheers Dulux-Oz

3 7

Import VM via KVM. Can't see vm's.
by piotret＠wp.pl 05 Dec '22

05 Dec '22

Hi I am trying to migrate vm's from oVirt 4.3.10.4-1.el7 to oVirt 4.5.3.2-1.el8. I use Provider KVM (via libvirt) My problem is that I can't see vm's from old oVirt when they are shutdown. When they are running I see but can't import because "All chosen VMs are running in the external system and therefore have been filtered. Please see log for details." Thank you for help. Regards

2 2

How does ovirt handle disks across multiple iscsi LUNs
by peterd＠mdg-it.com 05 Dec '22

05 Dec '22

A possibly obvious question I can't find the answer to anywhere—how does ovirt allocate VM disk images when a storage domain has multiple LUNs? Are these allocated one per LUN, so if e.g. a LUN runs out of space the disks on that LUN (only) will be unable to write? Or are these distributed across LUNs, so if a LUN fails due to storage failure etc the entire storage domain can be affected? Many thanks in advance, Peter

3 7

Warning alert: Failed to parse server certificates
by me＠alexsmirnov.us 02 Dec '22

02 Dec '22

After an update 2 days ago we no longer able to log into the oVirt console. The certificate was expiring, so we ran the system update and the ovirt-engine update as well. It went fine and re-issued (upgraded) certificates. After the work was done, we no longer able to log into the WEB UI. The error we get on the screen is 'Failed to parse server certificates'. After an attempt to login as local administrator, it reverts back to landing page with the same error. Any help is appreciated at this time.

2 1

confuse about vm's memory stat on the engine web pannel
by coolloves＠gmail.com 30 Nov '22

30 Nov '22

version 4.4.10 vm's mem stat on the engine web pannel indicate 98% mem usage (img below), https://i.328888.xyz/img/2022/12/01/ibMPN.png while I log into the vm terminal,check the mem status below. total used free shared buff/cache available Mem: 7820 2925 127 192 4767 4368 Swap: 1639 17 1622 i am counfused about the difference,plz help.thanks!

2 2

How to remove cloud-init check on vm using ansible module or rest api
by kishorekumar.goli＠gmail.com 29 Nov '22

29 Nov '22

Hi All, I have a requirement to automate template creation using ansible module or apis. I need to remove cloud-init check box from vm before creating the template. Below are the steps I follow from gui 1. Shutdown the vm. 2. click edit vm > initial run> uncheck cloud-init box 3. create template. I cannot automate second step as there is no option to remove cloud-init option. Could anyone please help me if there is a way to uncheck cloud-init?? BRs Kishore

3 4

2nd Self-Hosted Engine Node Not Attached To "ovirtmgmt" Network
by Matthew J Black 29 Nov '22

29 Nov '22

Hi All, So, I've got the Self-Hosted Engine up and running on the first Host, and everything *seems* to be working OK. I'm now attempting to use the Web UI to add a new ie 2nd (Self-Hosted Engine) Host to the cluster. Everything *seems* to go OK except the 2nd Host is left in a Non-Operational state (which I believe is "normal") awaiting the set up of the Host Networks. Its here where things go wrong: the 2nd Host does not have the ovirtmgmt Network attached (nor any of the others, for that matter). When I drag the ovirtmgmt Network to the 2nd Host's pre-existing (and working) bond interface the Engine works away for a while and then reports "Error while executing action HostSetupNetworks: Unexpected exception". I have located (but have not yet read) these logs from the 2nd Host: - agent.log - broker.log I have located (but have not yet read) these logs from the Engine: - engine.log - ovirt-host-deploy-ansible-20221124150742-ovirt_node_1.mynet.local-c26ca3fc-3c3f-4ee0-9562-fd7fd5066f8b.log - ovirt-host-deploy-ansible-20221124150742-ovirt_node_2.mynet.local-c26ca3fc-3c3f-4ee0-9562-fd7fd5066f8b.log Just to make things clear: - Both the Hosts are physically the same. Same brand/model of M/Board, NICS, HDDs, ect. Same layout, etc - The Web UI says that the 2nd Host's Bond is working AOK (as is the 1st Host) - I can ssh into the 2nd Host fine. So my questions are: 1) Which other logs should I be looking at? 2) Has anyone else struck this issue before (I had a look through the mail list archives, etc, and couldn't really find anything relevant - but as always, I may be mistaken)? Any and all help greately appreciated Cheers Dulux-Oz

2 2

Is it possible to auto-start VMs on single LocalStorage host without the engine?
by ernest.beinrohr＠axonpro.sk 29 Nov '22

29 Nov '22

I currently use KVM/virsh for my DNS. I would like it to ovirt, but I need DNS started for the engine to work. So I need to start the DNS vm before the engine. Is that possible with ovirt? I was thinking I could use the same mechanism as hosted engine, as that would autostart. In my current KVM setup I needed only to symlink /etc/libvirtd/qemu/dns.xml to /etc/libvirtd/qemu/autostart/ to get it to run.

3 2

Re: How does ovirt handle disks across multiple iscsi LUNs
by Peter Denison 28 Nov '22

28 Nov '22

Thanks Jean-Louis, The system we’re working with definitely has multiple LUNs in the one storage domain, hence the question (see attached). Regards, Peter

1 0

Re: 4.5.2 Create Additional Gluster Logical Volumes fails
by Simon Scott 28 Nov '22

28 Nov '22

Many thanks Gobinda

1 0