December 2016 - Users - oVirt List Archives

[HEADS UP] CentOS 7.3 is rolling out, need qemu-kvm-ev 2.6
by Sandro Bonazzola 02 Jan '17

02 Jan '17

Hi, as you probably noticed, CentOS 7.3 is rolling out right now and will be announced pretty soon. Please note that due to new libvirt shipped within CentOS 7.3, it is required to update qemu-kvm-ev to 2.6 at the same time. If you're using CentOS Virt SIG rpms / repos for oVirt this should happen automatically. If you're using only ovirt repositories be sure to have qemu-kvm-ev 2.6 if you update to CentOS 7.3. In terms of ovirt repositories, qemu-kvm-ev 2.6 is available right now in ovirt-master-snapshot-static, ovirt-4.0-snapshot-static, and ovirt-4.0-pre (contains 4.0.6 RC4 rpms going to be announced in a few minutes.) Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

10 32

Using zRam with oVirt Nodes
by Fernando Frediani 02 Jan '17

02 Jan '17

Hello folks. On simple libvirt/KVM hosts in order to improve RAM usage and avoid swap to disk I use zRam with swap to RAM technique. So create half of amount of host memory in zRam disk divided by the number of CPU cores. Works pretty well. Has anyone tried it with oVirt Nodes or even has it been considered as a feature to simply turn on/off ? Thanks Fernando

3 4

New oVirt user
by Jim Kusznir 01 Jan '17

01 Jan '17

Hello: I've been involved in virtualization from its very early days, and been running linux virtualization solutions off and on for a decade. Previously, I was always frustrated with the long feature list offered by many linux virtualization systems but with no reasonable way to manage that. It seemed that I had to spend an inordinate amount of time doing everything by hand. Thus, when I found oVirt, I was ecstatic! Unfortunately, at that time I changed employment (or rather left employment and became self-employed), and didn't have any reason to build my own virt cluster..until now! So I'm back with oVirt, and actually deploying a small 3-node cluster. I intend to run on it: VoIP Server Web Server Business backend server UniFi management server Monitoring server (zabbix) Not a heavy load, and 3 servers is probably overkill, but I need this to work, and it sounds like 3 is the magic entry level for all the cluster/failover stuff to work. For now, my intent is to use a single SSD on each node with gluster for the storage backend. I figure if all the failover stuff actually working, if I loose a node due to disk failure, its not the end of the world. I can rebuild it, reconnect gluster, and restart everything. As this is for a startup business, funds are thin at the moment, so I'm trying to cut a couple corners that don't affect overall reliability. If this side of the business grows more, I would likely invest in some dedicated servers. So far, I've based my efforts around this guide on oVirt's website: http://www.ovirt.org/blog/2016/08/up-and-running-with-ovirt-4-0-and-gluster… My cluster is currently functioning, but not entirely correctly. Some of it is gut feel, some of it is specific test cases (more to follow). First, some areas that lacked clarity and the choices I made in them: Early on, Jason talks about using a dedicated gluster network for the gluster storage sync'ing. I liked that idea, and as I had 4 nics on each machine, I thought dedicating one or two to gluster would be fine. So, on my clean, bare machines, I setup another network with private NiCs and put it on a standalone switch. I added hostnames with a designator (-g on the end) for the IPs for all three nodes into /etc/hosts on all three nodes so now each node can resolve itself and the other nodes on the -g name (and private IP) as well as their main host name and "more public" (but not public) IP. Then, for gdeploy, I put the hostnames in as the -g hostnames, as I didn't see anywhere to tell gluster to use the private network. I think this is a place I went wrong, but didn't realize it until the end.... I set up the gdeploy script (it took a few times, and a few OS rebuilds to get it just right...), and ran it, and it was successful! When complete, I had a working gluster cluster and the right software installed on each node! I set up the engine on node1, and that worked, and I was able to log in to the web gui. I mistakenly skipped the web gui enable gluster service before doing the engine vm reboot to complete the engine setup process, but I did go back in after the reboot and do that. After doing that, I was notified in the gui that there were additional nodes, did I want to add them. Initially, I skipped that and went back to the command line as Jason suggests. Unfortunately, it could not find any other nodes through his method, and it didn't work. Combine that with the warnings that I should not be using the command line method, and it would be removed in the next release, I went back to the gui and attempted to add the nodes that way. Here's where things appeared to go wrong...It showed me two additional nodes, but ONLY by their -g (private gluster) hostname. And the ssh fingerprints were not populated, so it would not let me proceed. After messing with this for a bit, I realized that the engine cannot get to the nodes via the gluster interface (and as far as I knew, it shouldn't). Working late at night, I let myself "hack it up" a bit, and on the engine VM, I added /etc/hosts entries for the -g hostnames pointing to the main IPs. It then populated the ssh host keys and let me add them in. Ok, so things appear to be working..kinda. I noticed at this point that ALL aspects of the gui became VERY slow. Clicking in and typing in any field felt like I was on ssh over a satellite link. Everything felt a bit worse than the early days of vSphere....Painfully slow. but it was still working, so I pressed on. I configured gluster storage. Eventually I was successful, but initially it would only let me add a "Data" storage domain, the drop-down menu did NOT contain iso, export, or anything else... Somehow, on its own, after leaving and re-entering that tab a few times, iso and export materialized on their own in the menu, so I was able to finish that setup. Ok, all looks good. I wanted to try out his little tip on adding a VM, too. I saw "ovirt-imiage-repository" in the "external providers" section, but he mentioned it in the storage section. It wasn't there on mine, and in external providers, I couldn't find anyway to do anything useful. I tried and fumbled with this, and still, I have not figured out how to use this feature. It would be nice.... Anyway, I moved on for now. As I was skeptical that things were set up correctly, i tried putting node 1 (which was running my engine, and was NOT set up with the -g hostname) into maintence mode, to see if it really did smoothly failover. It failed to go into maintence mode (left it for 12 hours, too!). I suspect its because of the hostnames/networks in use. Oh, I forgot to mention...I did follow the instructions in Jason's guide to set up the gluster network in ovirt and map that to the right physical interface on all 3 nodes. I also moved migration from the main network to the gluster network as Jason had suggested. So...How badly did I do? How do I fix the issues? (I'm not opposed to starting from scratch again, either...I've already done that 3-4 times in the early phases of getting the gdeploy script down, and I already have kickstart files setup with a network environment...I was rebuilding that often! I just need to know how to fix my setup this time....) I do greatly appreciate others' help and insight. I am in the IRC channel under kusznir currently, too. --Jim

2 1

Current status of 4.0.6 | EL7.3?
by Derek Atkins 01 Jan '17

01 Jan '17

Hi, What is the current status of Ovirt 4.0.6 and EL7.3? From previous threads it seemed to me that there was a potential compatibility issue with the 7.3 kernel and an updated version of vdsm or qemu? I just want to ensure any potential issues have been cleared up before I upgrade my systems. Thanks, -derek -- Derek Atkins 617-623-3745 derek(a)ihtfp.com www.ihtfp.com Computer and Internet Security Consultant

3 5

Re: [ovirt-users] hosted_storage locked
by Evgenia Tokar 31 Dec '16

31 Dec '16

It seems that the import of the storage domain failed since it was inaccessible: 2016-12-28 07:06:25,406 ERROR [org.ovirt.engine.core.bll.storage.domain.GetExistingStorageDomainListQuery] (org.ovirt.thread.pool-6-thread-46) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HSMGetStorageDomainInfoVDS, error = Domain is either partially accessible or entirely inaccessible: (u'247b250c-ccd3-4056-853f-f03a05369da9',), code = 379 (Failed with error StorageDomainAccessError and code 379) .... 2016-12-28 07:06:25,413 ERROR [org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-46) [] Failed query for all Storage Domains. The import command can not proceed without this info Once that happened the command was logged in the db as failed, and every engine restart the engine tries to run it again (compensate mechanism). However, since the import storage domain command is not "compensatable" this fails and the domain remains locked. 2016-12-28 07:09:02,643 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (ServerService Thread Pool -- 56) [] CommandsFactory : Failed to get type information using reflection for Class 'org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand', Command Id '2ab30734-bbe4-4f50-897f-f1b393a31852': org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand.<init>(org.ovirt.engine.core.compat.Guid) 2016-12-28 07:09:02,643 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (ServerService Thread Pool -- 56) [] Exception: java.lang.NoSuchMethodException: org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand.<init>(org.ovirt.engine.core.compat.Guid) .... 2016-12-28 07:09:02,646 ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 56) [] Failed to run compensation on startup for Command 'org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand', Command Id '2ab30734-bbe4-4f50-897f-f1b393a31852' I would suggest: 1. Verify the you can access the storage domain. 2. If you can access the domain, you can try removing it from the ui (I think the destroy option should be available). After the removal the engine will try to reimport it, and hopefully will succeed. Removing the storage domain may cause the engine and the ha-agent to restart. Jenny On Wed, Dec 28, 2016 at 6:29 PM, Gary Pedretty <gary(a)ravnalaska.net> wrote: > Here is the engine log from the engine vm. > > Gary > > > > > ------------------------------------------------------------------------ > Gary Pedretty gary(a)ravnalaska.net > <gary(a)eraalaska.net> > Systems Manager www.flyravn.com > Ravn Alaska /\ 907-450-7251 > <(907)%20450-7251> > 5245 Airport Industrial Road / \/\ 907-450-7238 > <(907)%20450-7238> fax > Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment > Serving All of Alaska / \/ /\ \ \/\ “Love your neighbor as > Really loving the record green up date! Summmer!! yourself” Matt 22:39 > ------------------------------------------------------------------------ > > > > > > > > > > > > > On Dec 28, 2016, at 1:12 AM, Evgenia Tokar <etokar(a)redhat.com> wrote: > > Hi! > > Can you attach the engine log from the hosted engine vm? > > Thanks, > Jenny > > > On Wed, Dec 28, 2016 at 10:56 AM, Gary Pedretty <gary(a)ravnalaska.net> > wrote: > >> Just setup a new hosted-engine Data Center. With 3 hosts and gluster >> based storage domains. Used the latest installers. Everything went well, >> until I got the engine deployed and each host setup and went to the web >> interface for the first time. >> >> After adding the master storage domain, the hosted_storage added itself >> like usual, but remained locked. Everything is working just fine with the >> exception of no listing of the engine as a VM under that tab or the icon on >> the current host. The Volumes tab does not show anything on the >> hosted_storage, but all the other volumes including the master data and a >> second data all show normal status for bricks. Error log is not reporting >> anything unusual either. All the gluster volumes including the engine >> (hosted_storage) show good status via command line in the hosts. >> >> Going to let it run for a day or so before I add any VMs. >> >> Any ideas? >> >> Gary >> >> ------------------------------------------------------------------------ >> Gary Pedretty gary(a)ravnalaska.net >> <gary(a)eraalaska.net> >> Systems Manager www.flyravn.com >> Ravn Alaska /\ 907-450-7251 >> <(907)%20450-7251> >> 5245 Airport Industrial Road / \/\ 907-450-7238 >> <(907)%20450-7238> fax >> Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment >> Serving All of Alaska / \/ /\ \ \/\ “Love your neighbor as >> Really loving the record green up date! Summmer!! yourself” Matt 22:39 >> ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> >> > > >

2 3

How to track RHEV/RHV releases/bugfixes/advisories
by Giuseppe Ragusa 31 Dec '16

31 Dec '16

Hi all, Sorry if the question has already been answered or is prominently explained on some publicly available forum/page (I haven't been able to find it anywhere nor through Google neither on Red Hat website). Since some customers could opt to follow the testing phase with oVirt by going in production with RHEV/RHV, I would like to know whether there is some dedicated mailing list / web page to check for announces (plus release planning / roadmap etc.). Many thanks in advance. Best regards (and Happy New Year! ;-) ) Giuseppe

1 0

Exit message: Bad volume specification
by Rodrick Brown 30 Dec '16

30 Dec '16

One of our VM's went today and after further investigation it said it was paused because of storage I/O error I immediately powered off the VM and tried starting it again and this is what I get when it tries to come up before it fails. What are my options? I have snapshots but when I try to clone a new VM from a snapshot it fails saying unable to create VMVolume or something. Their is nothing wrong with the storage volume other VM's are fine. I'm desperate here and need to recover this VM. VM nyc01ipa01 is down with error. Exit message: Bad volume specification {'index': 0, 'iface': 'virtio', 'type': 'disk', 'format': 'cow', 'bootOrder': '3', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': '0e29650b-6168-4ab6-a402-f6b425423680', 'apparentsize': '197632', 'imageID': '16ad7903-f3d3-4839-a6d0-c4278f67de28', 'specParams': {}, 'readonly': 'false', 'domainID': '7f971af9-6dfa-421f-a5c5-f500d9745869', 'reqsize': '0', 'deviceId': '16ad7903-f3d3-4839-a6d0-c4278f67de28', 'truesize': '33280', 'poolID': 'bff80e3e-140e-464f-adc9-53664b1723c0', 'device': 'disk', 'shared': 'false', 'propagateErrors': 'off', 'optional': 'false'}. -- *Rodrick Brown */ *Site Reliability Engineer *(917) 445 - 6839 / *rbrown(a)marketfactory.com <rbrown(a)marketfactory.com>**425 Broadway #3, New York, NY 10013*

1 0

Issues with RHV 4.1
by knarra 30 Dec '16

30 Dec '16

Hi, I have latest RHV 4.1 installed on HC stack. My app vms runs on Host1 and HE vm runs on Host2. I have a test case where i bring down glusternw on Host1 and i expect all app vms to be migrated to another node. But when i am running the above said test case i run into following issues. 1) I expect HE vm not to go down since everything works fine on the host where HE vm is running. But i see that HE vm goes down and comes up back. Why does this happen? 2) Some times i see that Host where HE vm runs restarts and I am not sure why does it reboot. I checked /var/log/messages and i see the errors below but still unable to figure out why the system restarts. Due to this i see that HE VM is unavailable for some time. https://paste.fedoraproject.org/514976/83092116/ Any idea why the host system reboots here? 3) I see another issue being logged there related to ovirt-imageio-daemon thought not relevant to the above test case. https://paste.fedoraproject.org/514978/83092677/ why does it throw IO error and why is this Traceback logged ? Thanks kasturi.

1 1

HA Score
by knarra 29 Dec '16

29 Dec '16

Hi, I have latest 4.1 installed and i see that HA score on hosts keeps going to 0 and comes back to 3400. This behavior is something which i am observing with 4.1 and i see that it takes considerable amount of time to get back to normal state. Any reason why it takes such a long time? Thanks kasturi.

1 0

Re: [ovirt-users] oVirt 3.6.7 upgrade to 4.0.5 and CentOS 7.3
by Robert Story 29 Dec '16

29 Dec '16

--Sig_/mYv1WOnJrWi76oeMjypGz3f Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 28 Dec 2016 09:39:16 +0200 Yaniv wrote: YK> On Dec 27, 2016 10:09 PM, "Robert Story" <rstory(a)tislabs.com> wrote: YK> >Just as a counterpoint, I had 2 issues, but both were related to chang= es YK> >I'd made.. YK> > YK> >- I have ipv6 disabled. with CentOS 7.3 this can cause rpcbind to chok= e, YK> > causing cascading dependency failures. I had to manually tweak syste= md YK> > config. YK>=20 YK> Did you open a bug about it, on rpcbind? There was an existing bug. Unfortunately it's another case of people just accepting that systemd can break what ever it wants and folks just have to 'get over it'. https://bugzilla.redhat.com/show_bug.cgi?id=3D1402961 YK> - I had tweaked vdsm.conf for debugging a long time ago, so the new ver= sion YK> installed it's conf file as vsdm.conf.rpmnew. New vdsm uses a differe= nt YK> logger, and wouldn't start with the old logger config. No error messa= ge, YK> 0 byte log file. Copying rpmnew conf file over existing log files fix= ed YK> this one. YK>=20 YK> That is somewhat expected. It is a little distressing that it fails with no indication of why it's failing. It would be nice if something caught stderr to be able to log it. I also forgot about my 3rd issue - the GUI 'broke' because I had a CNAME for ovirt.local to ovirt-engine.local. Luckily I don't have a lot of users to retrain to use a different domain name. Robert --=20 Senior Software Engineer @ Parsons --Sig_/mYv1WOnJrWi76oeMjypGz3f Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlhlKFkACgkQ7/fVLLY1mni9VgCeIBNJTsVH57uK0mtk3eLzj3Go Ai8AoJM5EGCN0R7NTEuWq5qaI8UcGW8O =PA53 -----END PGP SIGNATURE----- --Sig_/mYv1WOnJrWi76oeMjypGz3f--

1 0