- Users - oVirt List Archives

4.3 hosted-engine setup & yum-utils RPM installation
by Simon Coter 31 May '19

31 May '19

Hi, is there any particular reason to get “yum-utils” (and its dependency) RPM installed during the hosted-engine deployment ? I mean, why don’t we get yum-utils RPM part of the hosted-engine image ? This “yum” process, executed during the deployment, could fail (or wait forever) if the host/engine is behind a proxy — while trying to install the RPMs. I see two options: get all the required RPMs part of the hosted-engine image add the option to supply a proxy for yum during the hosted-engine setup Could this be a request for enhancement ? Thanks Simon

2 1

Re: Install fresh 4.3 fails with mounting shared storage
by Vrgotic, Marko 30 May '19

30 May '19

Hi Roy, I will run all those tests tomorrow morning (Amsterdam TimeZone) and reply back with results. Regarding NetApp documentation you mentioned below, I assume it should be enough to just “google” for it. Thank you very much for jumping in, we really appreciate it. Kind regards, Marko Vrgotic From: "Morris, Roy" <roy.morris(a)ventura.org> Date: Thursday, 30 May 2019 at 18:46 To: "Vrgotic, Marko" <M.Vrgotic(a)activevideo.com>, "users(a)ovirt.org" <users(a)ovirt.org> Cc: "users-request(a)ovirt.org" <users-request(a)ovirt.org>, "Stojchev, Darko" <D.Stojchev(a)activevideo.com> Subject: RE: Install fresh 4.3 fails with mounting shared storage Marko, Can you try disabling NFSv4 on the NetApp side for testing and rerun the installer? I don’t advise leaving it at NFSv3 but just for testing we can try it out. Also, there is some documentation on NetApp support regarding manually mounting the NFS share to change permissions then unmount. It has to be done once but after that the mounting should be fine. Do you have root squash set on NetApp? Best regards, Roy Morris GSA Virtualization Systems Analyst County of Ventura (805) 654-3625 (805) 603-9403 [cid:7c03dd9d67a9cfb78447b56087323d91a66d7c29.camel@ventura.org] From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com> Sent: Thursday, May 30, 2019 1:34 AM To: Morris, Roy <roy.morris(a)ventura.org>; users(a)ovirt.org Cc: users-request(a)ovirt.org; Stojchev, Darko <D.Stojchev(a)activevideo.com> Subject: [External] Re: Install fresh 4.3 fails with mounting shared storage Hi Roy, Sure, here is the output: Last login: Wed May 29 17:25:30 2019 from ovirt-engine.avinity.tv [root@ovirt-hv-03 ~]# showmount -e 172.17.28.5 Export list for 172.17.28.5: / (everyone) [root@ovirt-hv-03 ~]# ls -la /rhev/data-center/mnt/ total 0 drwxr-xr-x. 2 vdsm kvm 6 May 29 17:14 . drwxr-xr-x. 3 vdsm kvm 17 May 29 17:11 .. [root@ovirt-hv-03 ~]# In addition, if it helps, here is the list of shares/mount points from Netapp side, behind the 172.17.28.5 IP: [cid:image003.png@01D516CC.8548AE10] Kind regards Marko Vrgotic From: "Morris, Roy" <roy.morris(a)ventura.org<mailto:roy.morris@ventura.org>> Date: Thursday, 30 May 2019 at 00:57 To: "Vrgotic, Marko" <M.Vrgotic(a)activevideo.com<mailto:M.Vrgotic@activevideo.com>>, "users(a)ovirt.org<mailto:users@ovirt.org>" <users(a)ovirt.org<mailto:users@ovirt.org>> Cc: "users-request(a)ovirt.org<mailto:users-request@ovirt.org>" <users-request(a)ovirt.org<mailto:users-request@ovirt.org>> Subject: RE: Install fresh 4.3 fails with mounting shared storage Marko, Can you run the following commands and let us know the results. showmount -e 172.17.28.5 ls -la /rhev/data-center/mnt/ Best regards, Roy Morris From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com<mailto:M.Vrgotic@activevideo.com>> Sent: Wednesday, May 29, 2019 4:07 AM To: users(a)ovirt.org<mailto:users@ovirt.org> Cc: users-request(a)ovirt.org<mailto:users-request@ovirt.org> Subject: [External] [ovirt-users] Install fresh 4.3 fails with mounting shared storage CAUTION: This email contains links. If it looks suspicious or is not expected, DO NOT click and immediately forward to Spam.Manager(a)ventura.org<mailto:Spam.Manager@ventura.org>. Dear oVIrt, We are trying to deploy new setup with Hosted-Engine , oVirt version 4.3. Volume is on the Netapp, protocol NFS v4. Upon populating shared storage information and path: Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]: nfs Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: auto Please specify the full shared storage connection path to use (example: host:/path): 172.17.28.5:/ovirt_hosted_engine Following is displayed on the screen: [ INFO ] Creating Storage Domain [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Check local VM dir stat] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Enforce local VM dir existence] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Add NFS storage domain] [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[General Exception]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[General Exception]\". HTTP response code is 400."} Even with this error – storage gets mounted on the Host: 172.17.28.5:/ovirt_hosted_engine on /rhev/data-center/mnt/172.17.28.5:_ovirt__hosted__engine type nfs4 (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=172.17.28.11,local_lock=none,addr=172.17.28.5) But playbook execution fails and we can not proceed with install. Please advise. Kindly awaiting your reply. Marko Vrgotic

2 1

Re: Install fresh 4.3 fails with mounting shared storage
by Vrgotic, Marko 30 May '19

30 May '19

Hi Roy, Sure, here is the output: Last login: Wed May 29 17:25:30 2019 from ovirt-engine.avinity.tv [root@ovirt-hv-03 ~]# showmount -e 172.17.28.5 Export list for 172.17.28.5: / (everyone) [root@ovirt-hv-03 ~]# ls -la /rhev/data-center/mnt/ total 0 drwxr-xr-x. 2 vdsm kvm 6 May 29 17:14 . drwxr-xr-x. 3 vdsm kvm 17 May 29 17:11 .. [root@ovirt-hv-03 ~]# In addition, if it helps, here is the list of shares/mount points from Netapp side, behind the 172.17.28.5 IP: [cid:image001.png@01D516D3.205660E0] Kind regards Marko Vrgotic From: "Morris, Roy" <roy.morris(a)ventura.org> Date: Thursday, 30 May 2019 at 00:57 To: "Vrgotic, Marko" <M.Vrgotic(a)activevideo.com>, "users(a)ovirt.org" <users(a)ovirt.org> Cc: "users-request(a)ovirt.org" <users-request(a)ovirt.org> Subject: RE: Install fresh 4.3 fails with mounting shared storage Marko, Can you run the following commands and let us know the results. showmount -e 172.17.28.5 ls -la /rhev/data-center/mnt/ Best regards, Roy Morris From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com> Sent: Wednesday, May 29, 2019 4:07 AM To: users(a)ovirt.org Cc: users-request(a)ovirt.org Subject: [External] [ovirt-users] Install fresh 4.3 fails with mounting shared storage CAUTION: This email contains links. If it looks suspicious or is not expected, DO NOT click and immediately forward to Spam.Manager(a)ventura.org<mailto:Spam.Manager@ventura.org>. Dear oVIrt, We are trying to deploy new setup with Hosted-Engine , oVirt version 4.3. Volume is on the Netapp, protocol NFS v4. Upon populating shared storage information and path: Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]: nfs Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: auto Please specify the full shared storage connection path to use (example: host:/path): 172.17.28.5:/ovirt_hosted_engine Following is displayed on the screen: [ INFO ] Creating Storage Domain [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Check local VM dir stat] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Enforce local VM dir existence] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Add NFS storage domain] [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[General Exception]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[General Exception]\". HTTP response code is 400."} Even with this error – storage gets mounted on the Host: 172.17.28.5:/ovirt_hosted_engine on /rhev/data-center/mnt/172.17.28.5:_ovirt__hosted__engine type nfs4 (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=172.17.28.11,local_lock=none,addr=172.17.28.5) But playbook execution fails and we can not proceed with install. Please advise. Kindly awaiting your reply. Marko Vrgotic

2 2

Re: change he vm memoy size
by Strahil 30 May '19

30 May '19

Actually, you need to untar the OVF from the shared storage and check the configuration from the tar. Just keep it like that (running ) and tomorrow power down and then up the HostedEngine. Best Regards, Strahil NikolovOn May 30, 2019 12:06, "Valkov, Alexey" <Valkov.Alexey(a)knauf.ru> wrote: > > Hello, Strahil. I've just tried with engine-config -s OvfUpdateIntervalInMinutes=1 systemctl restart ovirt-engine.service After that, i changed Memory Size in manager UI. And waited about 30 minutes. Then checked memSize in /var/run/ovirt-hosted-engine-ha/vm.conf (which if i right understand syncronized with ovf every minute) and saw memSize have not been changed. And Memory Size property (in manager UI) also remains initial. Thus i think that ovf dont changes. I return OvfUpdateIntervalInMinutes=60 and will wait till tomorrow, may be the setting will be magically aplyed. > > -- > Best regards > Alexey > >> Hi Alexey, >> How much time did you check before rebooting. >> I have noticed ,that despite the default OVF update interval of 1 hour, it takes 5-6 hours for the engine to update the OVF. >> >> Best Regards, >> Strahil Nikolov >> >> On May 30, 2019 10:30, "Valkov, Alexey" <Valkov.Alexey(a)knauf.ru> wrote: >>> >>> I try to increase memory of HE VM (oVirt 4.2.8). If i do it from manager UI, i see that hot plug works - new memory devices appear and corresponding memory increase appeares inside engine guest. But 'Memory Size' property of hosted engine (in manager UI) don't reflect that new amount of memory. Also after reboot of engine vm, memory size changes back to initial value. Is it possible to change memory size of HE vm ( as far as i know the settings stored in ovf on HE domain) and how i can make this change to be persistent. >>> >>> -- >>> Best regards >>> Alexey >>> >> _______________________________________________ >> >> Users mailing list -- >> >> users(a)ovirt.org >> >> >> To unsubscribe send an email to >> >> users-leave(a)ovirt.org >> >> >> Privacy Statement: >> >> https://www.ovirt.org/site/privacy-policy/ >> >> >> oVirt Code of Conduct: >> >> https://www.ovirt.org/community/about/community-guidelines/ >> >> >> List Archives: >> >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/IKID3B2TH3VR2… >> >>

4 3

[ANN] oVirt 4.3.4 Third Release Candidate is now available
by Sandro Bonazzola 30 May '19

30 May '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.4 Third Release Candidate, as of May 30th, 2019. This update is a release candidate of the fourth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used inproduction. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later * oVirt Node 4.3 (available for x86_64 only) Experimental tech preview for x86_64 and s390x architectures for Fedora 28 is also included. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] Additional Resources: * Read more about the oVirt 4.3.4 release highlights: http://www.ovirt.org/release/4.3.4/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.4/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig> <https://redhat.com/summit>

2 1

oVirt 3.6 SLA and QoS algorithm
by m.isaiev＠kwebbl.com 30 May '19

30 May '19

Hello! We are using oVirt 3.6 in our infrastructure and storage QoS in passive state (they are configured but not applie) for some of the servers. Last time we needed to activate QoS on high load servers and it caused storage load decrease. And this situation has raised some questions: 1. What happened to overlimit storage traffic, when QoS applying on high load virtual machine: a) All over traffic is pulling into some queue, where all disk transactions can be successfully finished with QoS throughput? b) All over-limitation data is dropped and opened transactions failed? c) All active transactions finishes on speed beyond QoS, but all new transactions immediately start to work based on configured limits? 2. How QoS mechanism actually works? Does it use some oVirt specific functions or it uses some system features to limit storage, network or CPU usage? If system one, then the QoS are provided by libvirt (using virsh or directly) or system kernel options and which one? Thanks in advance!

2 2

Re: change he vm memoy size
by Strahil 30 May '19

30 May '19

Hi Alexey, How much time did you check before rebooting. I have noticed ,that despite the default OVF update interval of 1 hour, it takes 5-6 hours for the engine to update the OVF. Best Regards, Strahil NikolovOn May 30, 2019 10:30, "Valkov, Alexey" <Valkov.Alexey(a)knauf.ru> wrote: > > I try to increase memory of HE VM (oVirt 4.2.8). If i do it from manager UI, i see that hot plug works - new memory devices appear and corresponding memory increase appeares inside engine guest. But 'Memory Size' property of hosted engine (in manager UI) don't reflect that new amount of memory. Also after reboot of engine vm, memory size changes back to initial value. Is it possible to change memory size of HE vm ( as far as i know the settings stored in ovf on HE domain) and how i can make this change to be persistent. > > -- > Best regards > Alexey >

3 2

change he vm memoy size
by Valkov, Alexey 30 May '19

30 May '19

1 0

Install fresh 4.3 fails with mounting shared storage
by Vrgotic, Marko 29 May '19

29 May '19

Dear oVIrt, We are trying to deploy new setup with Hosted-Engine , oVirt version 4.3. Volume is on the Netapp, protocol NFS v4. Upon populating shared storage information and path: Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]: nfs Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: auto Please specify the full shared storage connection path to use (example: host:/path): 172.17.28.5:/ovirt_hosted_engine Following is displayed on the screen: [ INFO ] Creating Storage Domain [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Check local VM dir stat] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Enforce local VM dir existence] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Add NFS storage domain] [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[General Exception]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[General Exception]\". HTTP response code is 400."} Even with this error – storage gets mounted on the Host: 172.17.28.5:/ovirt_hosted_engine on /rhev/data-center/mnt/172.17.28.5:_ovirt__hosted__engine type nfs4 (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=172.17.28.11,local_lock=none,addr=172.17.28.5) But playbook execution fails and we can not proceed with install. Please advise. Kindly awaiting your reply. Marko Vrgotic

2 1

Re: ovirt metrics ansible error
by Jayme 29 May '19

29 May '19

I just tried again from scratch this time making sure a proper wildcard DNS entry existed and without using the set /etc/hosts option and am still running in to the pods issue. Can anyone confirm if this requires a public external IP to work? I am working on an internal DNS zone here and natted ips. On Tue, May 28, 2019 at 3:28 PM Edward Berger <edwberger(a)gmail.com> wrote: > In my case it was a single bare metal host, so that would be equivalent to > disabling iptables on the master0 VM you're installing to, in your ovirt > scenario. > > On Tue, May 28, 2019 at 1:25 PM Jayme <jaymef(a)gmail.com> wrote: > >> Do you mean the iptables firewall on the server being installed to i.e. >> master0 or the actual oVirt host that the master0 VM is running on? I did >> try flushing iptables rules on master0 VM then ran plays again from >> installer VM but fail at the same point. >> >> Does this log message have anything to do with the issue, /etc/cni >> directory does not even exist on master0 VM. >> >> May 28 17:23:35 master0 origin-node: W0528 17:23:35.012902 10434 >> cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d >> May 28 17:23:35 master0 origin-node: E0528 17:23:35.013398 10434 >> kubelet.go:2101] Container runtime network not ready: NetworkReady=false >> reason:NetworkPluginNotReady message:docker: network plugin is not ready: >> cni config uninitialized >> >> >> >> On Tue, May 28, 2019 at 1:19 PM Edward Berger <edwberger(a)gmail.com> >> wrote: >> >>> > TASK [openshift_control_plane : Wait for control plane pods to appear] >>> ********* >>> > Monday 27 May 2019 13:31:54 +0000 (0:00:00.180) 0:14:33.857 >>> ************ >>> > FAILED - RETRYING: Wait for control plane pods to appear (60 retries >>> left). >>> > FAILED - RETRYING: Wait for control plane pods to appear (59 retries >>> left). >>> > It eventually counts all the way down to zero and fails. >>> >>> This looks a lot like the issues I saw when the host firewall (iptables) >>> was blocking another OKD all-in-one-host install script [1]. >>> Disabling iptables allowed the installation to continue for my proof of >>> concept "cluster". >>> >>> [1]https://github.com/gshipley/installcentos >>> >>> The other error I had with [1] was it was trying to install a couple of >>> packages (zile and python2-pip) from EPEL with the repo disabled. >>> >>> >>> >>> On Tue, May 28, 2019 at 10:41 AM Jayme <jaymef(a)gmail.com> wrote: >>> >>>> Shirly, >>>> >>>> Oh and I should mention that I did verify that NetworkManager was >>>> installed on the master0 VM and enabled/started the second go around. So >>>> that service is there and running. >>>> >>>> # systemctl list-unit-files | grep Network >>>> dbus-org.freedesktop.NetworkManager.service >>>> enabled >>>> NetworkManager-dispatcher.service >>>> enabled >>>> NetworkManager-wait-online.service >>>> enabled >>>> NetworkManager.service >>>> enabled >>>> >>>> On Tue, May 28, 2019 at 11:13 AM Jayme <jaymef(a)gmail.com> wrote: >>>> >>>>> Shirly, >>>>> >>>>> I appreciate the help with this. Unfortunately I am still running in >>>>> to the same problem. So far I've tried to install/enable/start >>>>> NetworkManager on the existing "master0" server and re-ran the plans from >>>>> the installer VM. I ran in to the same problem waiting for control plane >>>>> pods and same errors in syslog. >>>>> >>>>> So I wiped everything out, killed the template along with the >>>>> installer and master VMs. On oVirt engine (I am running 4.3.3.7-1 stable) >>>>> I did have ovirt-engine-metrics-1.3.0x rpm installed, no yum updates >>>>> available on an update check. So I installed >>>>> http://resources.ovirt.org/pub/yum-repo/ovirt-release43-pre.rpm then >>>>> proceeded to install the latest version of ovirt-engine-metrics which gave >>>>> me: ovirt-engine-metrics-1.3.1-1.el7.noarch on hosted engine. >>>>> >>>>> After that package was installed I proceeded to follow steps from >>>>> beginning outlined at: >>>>> https://ovirt.org/documentation/metrics-install-guide/Installing_Metrics_St… -- >>>>> I ran in to the docker check issue again (same as my initial email) so I >>>>> disabled that and again got as far as starting control plane pods before >>>>> failure. >>>>> >>>>> Not sure where to go from here at this point. The only thing I can >>>>> think of that I did differently vs the instructions outlined above is that >>>>> I have not crated the wildcard DNS record, however I did set configs to >>>>> create /etc/hosts entries and they /etc/hosts on the machines have the >>>>> proper IPs assigned for all hostnames (automatically added by the ansible >>>>> plays). >>>>> >>>>> Any ideas how I can get past the plane pods issue? >>>>> >>>>> Thanks! >>>>> >>>>> On Tue, May 28, 2019 at 4:23 AM Shirly Radco <sradco(a)redhat.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> The latest release of 4.3.z should already include a fix for this >>>>>> issue, ovirt-engine-metrics-1.3.1 rpm. >>>>>> >>>>>> The issue is that it requires the NetworkManagar to be installed, >>>>>> running and enabled for it to work. >>>>>> >>>>>> You can install it manually on the master0 vm , start and enable it >>>>>> or you can also install the updated rpm from the nightly builds if your >>>>>> environment is oVirt 4.2.z: >>>>>> >>>>>> https://resources.ovirt.org/pub/ovirt-4.2-snapshot/rpm/el7/noarch/ovirt-eng… >>>>>> >>>>>> Relevant patches are: >>>>>> https://gerrit.ovirt.org/#/c/99959/ >>>>>> https://gerrit.ovirt.org/#/c/99718/ >>>>>> >>>>>> Best regards, >>>>>> >>>>>> -- >>>>>> >>>>>> Shirly Radco >>>>>> >>>>>> BI Senior Software Engineer >>>>>> >>>>>> Red Hat <https://www.redhat.com/> >>>>>> >>>>>> <https://www.redhat.com/> >>>>>> >>>>>> >>>>>> On Mon, May 27, 2019 at 4:41 PM Jayme <jaymef(a)gmail.com> wrote: >>>>>> >>>>>>> I managed to get past that but am running in to another problem >>>>>>> later in the process on the control plane pods to appear task. I thought >>>>>>> perhaps a glitch in the process from the failed docker step previously so >>>>>>> after a few more runs I tried killing everything and restarting the metrics >>>>>>> process again from the very beginning and end up hitting the same issue >>>>>>> with control plane pods even though all other steps/tasks seem to be >>>>>>> working. >>>>>>> >>>>>>> I'm just getting this: >>>>>>> >>>>>>> TASK [openshift_control_plane : Wait for control plane pods to >>>>>>> appear] ********* >>>>>>> Monday 27 May 2019 13:31:54 +0000 (0:00:00.180) 0:14:33.857 >>>>>>> ************ >>>>>>> FAILED - RETRYING: Wait for control plane pods to appear (60 retries >>>>>>> left). >>>>>>> FAILED - RETRYING: Wait for control plane pods to appear (59 retries >>>>>>> left). >>>>>>> FAILED - RETRYING: Wait for control plane pods to appear (58 retries >>>>>>> left). >>>>>>> FAILED - RETRYING: Wait for control plane pods to appear (57 retries >>>>>>> left). >>>>>>> FAILED - RETRYING: Wait for control plane pods to appear (56 retries >>>>>>> left). >>>>>>> >>>>>>> It eventually counts all the way down to zero and fails. >>>>>>> >>>>>>> In syslog of the master0 server I'm seeing some errors related to >>>>>>> cni config >>>>>>> >>>>>>> May 27 13:39:07 master0 ansible-oc_obj: Invoked with files=None >>>>>>> kind=pod force=False all_namespaces=None field_selector=None >>>>>>> namespace=kube-system delete_after=False >>>>>>> kubeconfig=/etc/origin/master/admin.kubeconfig content=None state=list >>>>>>> debug=False selector=None name=master-api-master0.xxxxxx.com >>>>>>> May 27 13:39:09 master0 origin-node: W0527 13:39:09.064230 20150 >>>>>>> cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d >>>>>>> May 27 13:39:09 master0 origin-node: E0527 13:39:09.064670 20150 >>>>>>> kubelet.go:2101] Container runtime network not ready: NetworkReady=false >>>>>>> reason:NetworkPluginNotReady message:docker: network plugin is not ready: >>>>>>> cni config uninitialized >>>>>>> May 27 13:39:13 master0 ansible-oc_obj: Invoked with files=None >>>>>>> kind=pod force=False all_namespaces=None field_selector=None >>>>>>> namespace=kube-system delete_after=False >>>>>>> kubeconfig=/etc/origin/master/admin.kubeconfig content=None state=list >>>>>>> debug=False selector=None name=master-api-master0.xxxxxx.com >>>>>>> May 27 13:39:14 master0 origin-node: W0527 13:39:14.066911 20150 >>>>>>> cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d >>>>>>> May 27 13:39:14 master0 origin-node: E0527 13:39:14.067321 20150 >>>>>>> kubelet.go:2101] Container runtime network not ready: NetworkReady=false >>>>>>> reason:NetworkPluginNotReady message:docker: network plugin is not ready: >>>>>>> cni config uninitialized >>>>>>> May 27 13:39:14 master0 origin-node: E0527 13:39:14.814705 20150 >>>>>>> summary.go:102] Failed to get system container stats for >>>>>>> "/system.slice/origin-node.service": failed to get cgroup stats for >>>>>>> "/system.slice/origin-node.service": failed to get container info for >>>>>>> "/system.slice/origin-node.service": unknown container >>>>>>> "/system.slice/origin-node.service" >>>>>>> May 27 13:39:19 master0 origin-node: W0527 13:39:19.069450 20150 >>>>>>> cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d >>>>>>> May 27 13:39:19 master0 origin-node: E0527 13:39:19.069850 20150 >>>>>>> kubelet.go:2101] Container runtime network not ready: NetworkReady=false >>>>>>> reason:NetworkPluginNotReady message:docker: network plugin is not ready: >>>>>>> cni config uninitialized >>>>>>> >>>>>>> On Mon, May 27, 2019 at 9:35 AM Shirly Radco <sradco(a)redhat.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Jayme, >>>>>>>> >>>>>>>> Thank you for reaching out. >>>>>>>> Please try rerunning the ansible playbook. >>>>>>>> If this doesn't work, try adding to the integ.ini in the metrics vm >>>>>>>> openshift_disable_check=docker_storage >>>>>>>> and rerun the ansible playbook again. >>>>>>>> >>>>>>>> Please update how it goes. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Shirly Radco >>>>>>>> >>>>>>>> BI Senior Software Engineer >>>>>>>> >>>>>>>> Red Hat <https://www.redhat.com/> >>>>>>>> >>>>>>>> <https://www.redhat.com/> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, May 26, 2019 at 9:34 PM Jayme <jaymef(a)gmail.com> wrote: >>>>>>>> >>>>>>>>> I'm running in to this ansible error during oVirt metrics >>>>>>>>> installation (following procedures at: >>>>>>>>> https://ovirt.org/documentation/metrics-install-guide/Installing_Metrics_St… >>>>>>>>> ) >>>>>>>>> >>>>>>>>> This is happening late in the process, after successfully >>>>>>>>> deploying the installation VM and then running second step from the metrics >>>>>>>>> VM. >>>>>>>>> >>>>>>>>> CHECK [memory_availability : master0.xxxxxx.com] >>>>>>>>> ************************************************************************************************* >>>>>>>>> fatal: [master0.xxxxxxx.com] FAILED! => {"changed": true, >>>>>>>>> "checks": {"disk_availability": {}, "docker_image_availability": >>>>>>>>> {"changed": true}, "docker_storage": {"failed": true, "failures": >>>>>>>>> [["OpenShiftCheckException", "Could not find imported module support code >>>>>>>>> for docker_info. Looked for either AnsibleDockerClient.py or >>>>>>>>> docker_common.py\nTraceback (most recent call last):\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", >>>>>>>>> line 225, in run_check\n result = check.run()\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", >>>>>>>>> line 53, in run\n docker_info = self.execute_module(\"docker_info\", >>>>>>>>> {})\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", >>>>>>>>> line 211, in execute_module\n result = self._execute_module(module_name, >>>>>>>>> module_args, self.tmp, self.task_vars)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>>>>>>>> line 809, in _execute_module\n (module_style, shebang, module_data, >>>>>>>>> module_path) = self._configure_module(module_name=module_name, >>>>>>>>> module_args=module_args, task_vars=task_vars)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>>>>>>>> line 203, in _configure_module\n environment=final_environment)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 1023, in modify_module\n environment=environment)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 859, in _find_module_utils\n recursive_finder(module_name, >>>>>>>>> b_module_data, py_module_names, py_module_cache, zf)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 621, in recursive_finder\n raise AnsibleError(' >>>>>>>>> '.join(msg))\nAnsibleError: Could not find imported module support code for >>>>>>>>> docker_info. Looked for either AnsibleDockerClient.py or >>>>>>>>> docker_common.py\n"]], "msg": "Could not find imported module support code >>>>>>>>> for docker_info. Looked for either AnsibleDockerClient.py or >>>>>>>>> docker_common.py\nTraceback (most recent call last):\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", >>>>>>>>> line 225, in run_check\n result = check.run()\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", >>>>>>>>> line 53, in run\n docker_info = self.execute_module(\"docker_info\", >>>>>>>>> {})\n File >>>>>>>>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", >>>>>>>>> line 211, in execute_module\n result = self._execute_module(module_name, >>>>>>>>> module_args, self.tmp, self.task_vars)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>>>>>>>> line 809, in _execute_module\n (module_style, shebang, module_data, >>>>>>>>> module_path) = self._configure_module(module_name=module_name, >>>>>>>>> module_args=module_args, task_vars=task_vars)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>>>>>>>> line 203, in _configure_module\n environment=final_environment)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 1023, in modify_module\n environment=environment)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 859, in _find_module_utils\n recursive_finder(module_name, >>>>>>>>> b_module_data, py_module_names, py_module_cache, zf)\n File >>>>>>>>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>>>>>>>> line 621, in recursive_finder\n raise AnsibleError(' >>>>>>>>> '.join(msg))\nAnsibleError: Could not find imported module support code for >>>>>>>>> docker_info. Looked for either AnsibleDockerClient.py or >>>>>>>>> docker_common.py\n"}, "memory_availability": {}, "package_availability": >>>>>>>>> {"changed": false, "invocation": {"module_args": {"packages": ["PyYAML", >>>>>>>>> "bash-completion", "bind", "ceph-common", "dnsmasq", "docker", "firewalld", >>>>>>>>> "flannel", "glusterfs-fuse", "httpd-tools", "iptables", >>>>>>>>> "iptables-services", "iscsi-initiator-utils", "libselinux-python", >>>>>>>>> "nfs-utils", "ntp", "openssl", "origin", "origin-clients", >>>>>>>>> "origin-hyperkube", "origin-node", "pyparted", "python-httplib2", >>>>>>>>> "yum-utils"]}}}, "package_version": {"changed": false, "invocation": >>>>>>>>> {"module_args": {"package_list": [{"check_multi": false, "name": "origin", >>>>>>>>> "version": ""}, {"check_multi": false, "name": "origin-master", "version": >>>>>>>>> ""}, {"check_multi": false, "name": "origin-node", "version": ""}], >>>>>>>>> "package_mgr": "yum"}}}}, "msg": "One or more checks failed", >>>>>>>>> "playbook_context": "install"} >>>>>>>>> >>>>>>>>> NO MORE HOSTS LEFT >>>>>>>>> ******************************************************************************************************************************************* >>>>>>>>> >>>>>>>>> PLAY RECAP >>>>>>>>> *************************************************************************************************************************************************** >>>>>>>>> localhost : ok=35 changed=1 unreachable=0 >>>>>>>>> failed=0 skipped=16 rescued=0 ignored=0 >>>>>>>>> master0.xxxxxxx.com : ok=96 changed=6 unreachable=0 >>>>>>>>> failed=1 skipped=165 rescued=0 ignored=0 >>>>>>>>> >>>>>>>>> >>>>>>>>> INSTALLER STATUS >>>>>>>>> ********************************************************************************************************************************************* >>>>>>>>> Initialization : Complete (0:00:16) >>>>>>>>> Health Check : In Progress (0:00:36) >>>>>>>>> This phase can be restarted by running: >>>>>>>>> playbooks/openshift-checks/pre-install.yml >>>>>>>>> Sunday 26 May 2019 16:36:25 +0000 (0:00:36.151) 0:01:56.339 >>>>>>>>> ************ >>>>>>>>> >>>>>>>>> =============================================================================== >>>>>>>>> Run health checks (install) - EL >>>>>>>>> --------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 36.15s >>>>>>>>> os_firewall : Ensure iptables services are not enabled >>>>>>>>> ------------------------------------------------------------------------------------------------ >>>>>>>>> 2.74s >>>>>>>>> openshift_repos : Ensure libselinux-python is installed >>>>>>>>> ----------------------------------------------------------------------------------------------- >>>>>>>>> 1.77s >>>>>>>>> openshift_repos : refresh cache >>>>>>>>> ----------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.60s >>>>>>>>> Gather Cluster facts >>>>>>>>> ---------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.51s >>>>>>>>> container_runtime : Fixup SELinux permissions for docker >>>>>>>>> ---------------------------------------------------------------------------------------------- >>>>>>>>> 1.33s >>>>>>>>> container_runtime : Place additional/blocked/insecure registries >>>>>>>>> in /etc/containers/registries.conf >>>>>>>>> --------------------------------------------------- 1.30s >>>>>>>>> Ensure openshift-ansible installer package deps are installed >>>>>>>>> ----------------------------------------------------------------------------------------- >>>>>>>>> 1.29s >>>>>>>>> container_runtime : Install Docker >>>>>>>>> -------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.17s >>>>>>>>> Initialize openshift.node.sdn_mtu >>>>>>>>> --------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.13s >>>>>>>>> os_firewall : Install firewalld packages >>>>>>>>> -------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.13s >>>>>>>>> container_runtime : Set various Docker options >>>>>>>>> -------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.11s >>>>>>>>> install NetworkManager >>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.10s >>>>>>>>> openshift_repos : Configure correct origin release repository >>>>>>>>> ----------------------------------------------------------------------------------------- >>>>>>>>> 1.05s >>>>>>>>> container_runtime : Get current installed Docker version >>>>>>>>> ---------------------------------------------------------------------------------------------- >>>>>>>>> 1.04s >>>>>>>>> openshift_repos : Configure origin gpg keys >>>>>>>>> ----------------------------------------------------------------------------------------------------------- >>>>>>>>> 1.04s >>>>>>>>> openshift_repos : Remove openshift_additional.repo file >>>>>>>>> ----------------------------------------------------------------------------------------------- >>>>>>>>> 0.99s >>>>>>>>> container_runtime : Setup the docker-storage for overlay >>>>>>>>> ---------------------------------------------------------------------------------------------- >>>>>>>>> 0.96s >>>>>>>>> Detecting Operating System from ostree_booted >>>>>>>>> --------------------------------------------------------------------------------------------------------- >>>>>>>>> 0.95s >>>>>>>>> Gather Cluster facts >>>>>>>>> ---------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>> 0.92s >>>>>>>>> >>>>>>>>> >>>>>>>>> Failure summary: >>>>>>>>> >>>>>>>>> >>>>>>>>> 1. Hosts: master0.xxxxxxx.com >>>>>>>>> Play: OpenShift Health Checks >>>>>>>>> Task: Run health checks (install) - EL >>>>>>>>> Message: One or more checks failed >>>>>>>>> Details: check "docker_storage": >>>>>>>>> Could not find imported module support code for >>>>>>>>> docker_info. Looked for either AnsibleDockerClient.py or docker_common.py >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File >>>>>>>>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py", >>>>>>>>> line 225, in run_check >>>>>>>>> result = check.run() >>>>>>>>> File >>>>>>>>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py", >>>>>>>>> line 53, in run >>>>>>>>> docker_info = >>>>>>>>> self.execute_module("docker_info", {}) >>>>>>>>> File >>>>>>>>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py", >>>>>>>>> line 211, in execute_module >>>>>>>>> result = self._execute_module(module_name, >>>>>>>>> module_args, self.tmp, self.task_vars) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line >>>>>>>>> 809, in _execute_module >>>>>>>>> (module_style, shebang, module_data, >>>>>>>>> module_path) = self._configure_module(module_name=module_name, >>>>>>>>> module_args=module_args, task_vars=task_vars) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line >>>>>>>>> 203, in _configure_module >>>>>>>>> environment=final_environment) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>>>>>>>> 1023, in modify_module >>>>>>>>> environment=environment) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>>>>>>>> 859, in _find_module_utils >>>>>>>>> recursive_finder(module_name, b_module_data, >>>>>>>>> py_module_names, py_module_cache, zf) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>>>>>>>> 621, in recursive_finder >>>>>>>>> raise AnsibleError(' '.join(msg)) >>>>>>>>> AnsibleError: Could not find imported module >>>>>>>>> support code for docker_info. Looked for either AnsibleDockerClient.py or >>>>>>>>> docker_common.py >>>>>>>>> >>>>>>>>> >>>>>>>>> The execution of "install_okd.yaml" includes checks designed to >>>>>>>>> fail early if the requirements of the playbook are not met. One or more of >>>>>>>>> these checks failed. To disregard these results,explicitly disable checks >>>>>>>>> by setting an Ansible variable: >>>>>>>>> openshift_disable_check=docker_storage >>>>>>>>> Failing check names are shown in the failure details above. Some >>>>>>>>> checks may be configurable by variables if your requirements are different >>>>>>>>> from the defaults; consult check documentation. >>>>>>>>> Variables can be set in the inventory or passed on the command >>>>>>>>> line using the -e flag to ansible-playbook. >>>>>>>>> _______________________________________________ >>>>>>>>> Users mailing list -- users(a)ovirt.org >>>>>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>> oVirt Code of Conduct: >>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>> List Archives: >>>>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/SEFPOF36T7G4G… >>>>>>>>> >>>>>>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/CN6G76Z7MVNK6… >>>> >>>

2 3