- Users - oVirt List Archives

Issue deploying self hosted engine on new install
by Robert Webb 06 Dec '19

06 Dec '19

Hi all, I am trying to deploy the self hosted engine via cockpit and it is failing at the very end of deployment when it is checking the VM Health. I have done multiple clean installs and run this and it fails every time. I have also tried from the cli via ssh with the same outcome. After the failure, if I look at the oVirt Machines section of the node, the VM is there but will not start. Anyone else having this issue?

1 0

[ANN] oVirt 4.3.8 First Release Candidate is now available for testing
by Sandro Bonazzola 06 Dec '19

06 Dec '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.8 First Release Candidate for testing, as of December 6th, 2019. This update is a release candidate of the eighth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Release See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available Additional Resources: * Read more about the oVirt 4.3.8 release highlights: http://www.ovirt.org/release/4.3.8/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.8/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Gluster FS & hosted engine fails to set up
by rob.downer＠orbitalsystems.co.uk 05 Dec '19

05 Dec '19

I have set up 3 new servers and as you can see Gluster is working well, however the hosted engine deployment fails can anyone suggest a reason ? I have wiped and set up all three servers again and set up Gluster first. This is the gluster congig I have used for the setup. Please review the configuration. Once you click the 'Finish Deployment' button, the management VM will be transferred to the configured storage and the configuration of your hosted engine cluster will be finalized. You will be able to use your hosted engine once this step finishes. * StorageStorage Type:glusterfs Storage Domain Connection:gfs3.gluster.private:/engine Mount Options:backup-volfile-servers=gfs2.gluster.private:gfs1.gluster.private Disk Size (GiB):58 [ INFO ] TASK [ovirt.hosted_engine_setup : Add glusterfs storage domain] [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[Unexpected exception]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Unexpected exception]\". HTTP response code is 400."} root@ovirt3 ~]# gluster volume status Status of volume: data Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gfs3.gluster.private:/gluster_bricks/ data/data 49152 0 Y 3756 Brick gfs2.gluster.private:/gluster_bricks/ data/data 49153 0 Y 3181 Brick gfs1.gluster.private:/gluster_bricks/ data/data 49152 0 Y 15548 Self-heal Daemon on localhost N/A N/A Y 17602 Self-heal Daemon on gfs1.gluster.private N/A N/A Y 15706 Self-heal Daemon on gfs2.gluster.private N/A N/A Y 3348 Task Status of Volume data ------------------------------------------------------------------------------ There are no active volume tasks Status of volume: engine Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gfs3.gluster.private:/gluster_bricks/ engine/engine 49153 0 Y 3769 Brick gfs2.gluster.private:/gluster_bricks/ engine/engine 49154 0 Y 3194 Brick gfs1.gluster.private:/gluster_bricks/ engine/engine 49153 0 Y 15559 Self-heal Daemon on localhost N/A N/A Y 17602 Self-heal Daemon on gfs1.gluster.private N/A N/A Y 15706 Self-heal Daemon on gfs2.gluster.private N/A N/A Y 3348 Task Status of Volume engine ------------------------------------------------------------------------------ There are no active volume tasks Status of volume: vmstore Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gfs3.gluster.private:/gluster_bricks/ vmstore/vmstore 49154 0 Y 3786 Brick gfs2.gluster.private:/gluster_bricks/ vmstore/vmstore 49152 0 Y 2901 Brick gfs1.gluster.private:/gluster_bricks/ vmstore/vmstore 49154 0 Y 15568 Self-heal Daemon on localhost N/A N/A Y 17602 Self-heal Daemon on gfs1.gluster.private N/A N/A Y 15706 Self-heal Daemon on gfs2.gluster.private N/A N/A Y 3348 Task Status of Volume vmstore ------------------------------------------------------------------------------ There are no active volume tasks

1 0

Re: Importing a Virtual Machine from a KVM Host
by Dirk Streubel 05 Dec '19

05 Dec '19

Hello Paul, thanks a lot for your answer. If i have any other questions during the import maybe i can asked again? And i let you know the result :) Regards Dirk Am 05.12.19 um 13:22 schrieb Staniforth, Paul: > Hello Dirk, > The link you are using is old developer docs, try the following. > https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html > > You need to setup an external provider and use a proxy host in the datacentre (not the engine host) you want to import into, this needs to be on the same network as your KVM host and have the virt-v2v package installed. > > Regards, > Paul S. > > -----Original Message----- > From: Dirk Streubel <dirk.streubel(a)posteo.de> > Sent: 05 December 2019 11:10 > To: users <users(a)ovirt.org> > Subject: [ovirt-users] Importing a Virtual Machine from a KVM Host > > Hello everbody, > > i am just a little bit confused about importing Qemu Images from my KVM Host. > > At home, and at work i am using a ovirt engine in a VM. The Version is 4.3.7 and on bare metal my host. > > Everything works fine but i have the same problem at home and on work. I want to import my qcow2 Images from a KVM Host but i > > will not work. > > The command virsh -r -c 'qemu+tcp://root@host1.example.org/system' list --all shows me akk my VMs. > > But when i want to load the images in the GUI, nothing happens. I follow this instruction: > > https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt… > > After that, i take a look on the official 4.3 Rhev Documentation from Red Hat. There stand this: > > Log in to the proxy host and generate SSH keys for the vdsm user. > > sudo -u vdsm ssh-keygen > > So, i have no proxy host. So i want to do this on my ovirt-engine. Is this right? > > Next thing: the user vdsm is locked on the ovirt-engine machine and the host. > > So, do i have to unlocked the account of the user vdsm or what is the best way. > > My Problem is i found nothing that would help me to solve this problem. > > Maybe somebody here do me a favour and help me to solve this "challenge" > > Regards > > Dirk > > > > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt… > oVirt Code of Conduct: https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt… > List Archives: https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ovi… > To view the terms under which this email is distributed, please go to:- > http://leedsbeckett.ac.uk/disclaimer/email/

1 0

Possible sources of cpu steal and countermeasures
by klaasdemter＠gmail.com 05 Dec '19

05 Dec '19

Hi, I'm having performance issues with a ovirt installation. It is showing high steal (5-10%) for a cpu intensive VM. The hypervisor however has more than 65% of his resources idle while the steal is seen inside of the VM. Even when placing only a single VM on a hypervisor it still receives steal (0-2%), even though the hypervisor is not overcommited. Hypervisor: 2 Socket system in total 2*28(56HT) cores VM: 30vCPUs (ovirt seems to think its a good idea to make that 15 sockets * 2 cores) My questions are: a) Could it be that the hypervisor is trying to schedule all 30 cores on a single numa node, ie using the HT cores instead of "real" ones and this shows up as steal? b) Do I need to make VMs this big numa-aware and spread the vm over both numa nodes? c) Would using the High Performance VM type help in this kind of situation? d) General advise: how do I reduce steal in an environment where the hypervisor has idle resources Any advise would be appreciated. Greetings Klaas

3 2

Re: Unable to attach ISO domain to Datacenter
by Strahil 04 Dec '19

04 Dec '19

Hi Ivan, Which file is this ? Contact me in private and I will send you my copy for comparison. Best Regards, Strahil NikolovOn Dec 4, 2019 18:55, Ivan Apolonio <ivan(a)apolonio.com.br> wrote: > > I've tried to commenting out this line on /etc/pam.d/password-auth: > > auth requisite pam_succeed_if.so uid >= 1000 quiet_success > > However, although it has stopped complaining about "requirement uid >= 1000 not met by user vdsm" on /var/log/secure , the error trying to attach ISO Domain to Datacenter still persists, showing the same error message, so it looks that this was not the cause of the problem :-( > > Going back to stage zero. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TTD4PH2FIUK23…

1 0

Ovirt-OSN integration
by siovelrm＠gmail.com 04 Dec '19

04 Dec '19

Hi, I want to use External Logical Networks Openstack Neutron. For this it is necessary to have all Openstack installed or is it possible to have only Neutron and used as a service? Sorry if the question is very obvious but I am starting now with this topic.

2 2

Re: 3node HCI fails when HostedEngineLocal is trying to add additional Gluster members
by Strahil 04 Dec '19

04 Dec '19

If you manage to get your VM on the gluster - you are almost done. I had similar situation and using virsh/hosted-engine , I have managed to reach the GUI. From there we can have a Clue what is going on. Usually there is a dependency: - Master storage domain should be UP -This allows the DC to become UP -Then a host in the cluster can become SPM (control storage) -And only then you can start your other gluster volumes. So I can only recommend you to try to reach GUI and check the situation. Then it will be moderately easy to bring everything up. Best Regards, Strahil NikolovOn Dec 4, 2019 17:06, thomas(a)hoberg.net wrote: > > Thanks Strahil, for your suggestions. > > Actually, I was far beyond the pick-up point you describe, as the Gluster had all been prepared and was operable, even the local VM was already running and accessible via the GUI. > > But I picked up your hint to try to continue with the scripted variant, and found that it allowed me much better insight into what was going on. > > I am a little worried, though, that it actually works a little differently from the GUI-wizard variant, in any case the failures don't seem identical. > That would have implications in terms of test-automation, which I'd rather not have to worry about. > > In a separate test for the same operation on a separate set of hardware the installation got significantly further, up to the point where the local VM had actually been moved onto the Gluster storage and into the Cluster, but then failed a validation step at the very end (while the VM is actually up and running, albeit only withe primary host and that listed as "Unresponsive" while it's hosting the VM....) > > I have opened a separate tickt for that... > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FTCFIP7Q5KWXT…

1 0

Cannot activate/deactivate storage domain
by Albl, Oliver 04 Dec '19

04 Dec '19

Hi all, I run an oVirt 4.3.6.7-1.el7 installation (50+ hosts, 40+ FC storage domains on two all-flash arrays) and experienced a problem accessing single storage domains. As a result, hosts were taken "not operational" because they could not see all storage domains, SPM started to move around the hosts. oVirt messages start with: 2019-11-04 15:10:22.739+01 | VDSM HOST082 command SpmStatusVDS failed: (-202, 'Sanlock resource read failure', 'IO timeout') 2019-11-04 15:10:22.781+01 | Invalid status on Data Center <name>. Setting Data Center status to Non Responsive (On host HOST82, Error: General Exception). ... 2019-11-04 15:13:58.836+01 | Host HOST017 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.85+01 | Host HOST005 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.85+01 | Host HOST012 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.851+01 | Host HOST002 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.851+01 | Host HOST010 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.851+01 | Host HOST011 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:58.852+01 | Host HOST004 cannot access the Storage Domain(s) HOST_LUN_204 attached to the Data Center <name>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.011+01 | Host HOST017 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.238+01 | Host HOST004 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.249+01 | Host HOST005 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.255+01 | Host HOST012 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.273+01 | Host HOST002 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.279+01 | Host HOST010 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:13:59.386+01 | Host HOST011 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center <UNKNOWN>. Setting Host state to Non-Operational. 2019-11-04 15:15:14.145+01 | Storage domain HOST_LUN_221 experienced a high latency of 9.60953 seconds from host HOST038. This may cause performance and functional issues. Please consult your Storage Administrator. The problem mainly affected two storage domains (on the same array) but I also saw single messages for other storage domains (one the other array as well). Storage domains stayed available to the hosts, all VMs continued to run. When constantly reading from the storage domains (/bin/dd iflag=direct if=<metadata> bs=4096 count=1 of=/dev/null) we got expected 20+ MBytes/s on all but some storage domains. One of them showed "transfer rates" around 200 Bytes/s, but went up to normal performance from time to time. Transfer rate to this domain was different between the hosts. /var/log/messages contain qla2xxx abort messages on almost all hosts. There are no errors on SAN switches or storage array (but vendor is still investigating). I did not see high load on the storage array. The system seemed to stabilize when I stopped all VMs on the affected storage domain and this storage domain became "inactive". Currently, this storage domain still is inactive and we cannot place it in maintenance mode ("Failed to deactivate Storage Domain") nor activate it. OVF Metadata seems to be corrupt as well (failed to update OVF disks <id>, OVF data isn't updated on those OVF stores). The first six 512 byte blocks of /dev/<id>/metadata seem to contain only zeros. Any advice on how to proceed here? Is there a way to recover this storage domain? All the best, Oliver

4 13

3node HCI fails when HostedEngineLocal is trying to add additional Gluster members
by thomas＠hoberg.net 04 Dec '19

04 Dec '19

Three Gen8 HP360 recalled from retirement with single 1TB TLC SATA SSD for boot and oVirt /engine and 7x4TB HDD RAID6 for /vmstore and /data, 10Gbit NICs and network. All CentOS 7.7 updated daily. These machines may not be used exclusively for oVirt so I don't want to re-install the OS, when an oVirt setup fails: Instead I try my best to clean up the nodes when doing another oVirt installation run. They ran oVirt for a week or two using a completely distinct set of storage, so they are fundamentally sound, but we wanted higher storage capacity so I swapped everything and re-installed CentOS very much the same way as before. The first oVirt setup went smoothly but the cluster crumbled without much usage. I won't go into details here, because I didn't want to investigate for now, instead I focussed on redoing the installation and cleaning up the old setup. I know the docs actually recommend starting with wiped hardware, but operationally that would be a show-stopper for the intended use case. So I cleaned up the best I can (ovirt-hosted-engine-cleanup with and without redoing the whole Gluster storage setup, where apart from SSD caching not working, I don't have issues). Undoing the network changes in such a way that the oVirt HCI wizard ceases complaining is a bit more involved. I typically run: - vdsm-tool ovn-unconfigure - vdsm-tool clear-nets (now need to switch to the console) - vdsm-tool remove-config and then I still need to edit /etc/sysconfig/network-scripts/ifcfg-<ethernet-device> to bring the physical adapter back to life. Sometimes I still need to remove the ovirtmgmnt bridge manually etc. Whether I remove and redo the Gluster as a bit of an effect in re-installation, but it doesn't make a difference in what follows. So here is where I am currently getting stuck consistently: The wizard is gone through preparing the Gluster storage (which is completely functional at that point), has created the local VM on the installation node, installed the Postgres database, filled it etc. basically has oVirt up and running with the primary Gluster node and now would like to add the second and third nodes. At that point I get "Connection lost" in the Web-Wizard, evidently as a consequence of Ansible fiddling around heavily to set up the local bridge for the VM. I remember that for the scripted variant of the setup it is recommended to run the script behind 'screen' or 'tmux' in order to ensure its execution isn't interrupted by that. But for the GUI variant, evidently there *should* be some other type of potection, perhaps via the re-connecting nature of HTTP... Pushing the "Reconnect" button on the GUI at that point doesn't return you to the point of the setup, but only offers to redeploy, while the HostedEngineLocal is still there and running. I ssh'd into the machine and started looking for errors and warning and saw that the installation had gone rather far without incidence. OTOPI had completely finished the WildFly server is up and running the Postgres database fully installed and running smoothly, the only thing I can find is that it's trying to add the additional gluster nodes, but complains that these nodes (quotes gluster-UUIDs) are not part of the "cluster". An investigation into the Postgres database shows, that the 'gluster_server' table indeed only has the primary node in it. I don't know what part of the process should have added the other two nodes, but there seems to be no *remaining* connectivity issue with the Gluster members. I installed gscli and connected to all three nodes and volumes without issue. I am guessing at this point, that the complex rewiring of the software defined network is causing a temporary issue and a race condition that I don't know how to recover from. Since the oVirt management GUI is actually fully operational and can be reached from the primary node via the temporary bridge, I went into the GUI and even managed to add the additional two nodes without any problems. Their installation went through without any issues, they showed up in the gluster_servers table on Postgress and basically the installation could have proceeded from that point, except... that I don't know how to restart the process from that point: It still has to 'beam' the local VM into the Gluster storage and restart it there. I have gone through the process three times now, with absolutely identical results. I could use some help how to recover from that situation, which looks like a race condition, nothing a re-installation of everything would really resolve. In the mean-time, I'll try the scripted variant on 'screen' to see if that fares better.

2 3