[ovirt-users] Re: ovirt metrics ansible error

Hi, The latest release of 4.3.z should already include a fix for this issue, ovirt-engine-metrics-1.3.1 rpm. The issue is that it requires the NetworkManagar to be installed, running and enabled for it to work. You can install it manually on the master0 vm , start and enable it or you can also install the updated rpm from the nightly builds if your environment is oVirt 4.2.z: https://resources.ovirt.org/pub/ovirt-4.2-snapshot/rpm/el7/noarch/ovirt-e... Relevant patches are: https://gerrit.ovirt.org/#/c/99959/ https://gerrit.ovirt.org/#/c/99718/ Best regards, -- Shirly Radco BI Senior Software Engineer Red Hat <https://www.redhat.com/> <https://www.redhat.com/> On Mon, May 27, 2019 at 4:41 PM Jayme <jaymef(a)gmail.com&gt; wrote: > I managed to get past that but am running in to another problem later in > the process on the control plane pods to appear task. I thought perhaps a > glitch in the process from the failed docker step previously so after a few > more runs I tried killing everything and restarting the metrics process > again from the very beginning and end up hitting the same issue with > control plane pods even though all other steps/tasks seem to be working. > > I'm just getting this: > > TASK [openshift_control_plane : Wait for control plane pods to appear] > ********* > Monday 27 May 2019 13:31:54 +0000 (0:00:00.180) 0:14:33.857 > ************ > FAILED - RETRYING: Wait for control plane pods to appear (60 retries > left). > FAILED - RETRYING: Wait for control plane pods to appear (59 retries > left). > FAILED - RETRYING: Wait for control plane pods to appear (58 retries > left). > FAILED - RETRYING: Wait for control plane pods to appear (57 retries > left). > FAILED - RETRYING: Wait for control plane pods to appear (56 retries > left). > > It eventually counts all the way down to zero and fails. > > In syslog of the master0 server I'm seeing some errors related to cni > config > > May 27 13:39:07 master0 ansible-oc_obj: Invoked with files=None kind=pod > force=False all_namespaces=None field_selector=None namespace=kube-system > delete_after=False kubeconfig=/etc/origin/master/admin.kubeconfig > content=None state=list debug=False selector=None name= > master-api-master0.xxxxxx.com > May 27 13:39:09 master0 origin-node: W0527 13:39:09.064230 20150 > cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d > May 27 13:39:09 master0 origin-node: E0527 13:39:09.064670 20150 > kubelet.go:2101] Container runtime network not ready: NetworkReady=false > reason:NetworkPluginNotReady message:docker: network plugin is not ready: > cni config uninitialized > May 27 13:39:13 master0 ansible-oc_obj: Invoked with files=None kind=pod > force=False all_namespaces=None field_selector=None namespace=kube-system > delete_after=False kubeconfig=/etc/origin/master/admin.kubeconfig > content=None state=list debug=False selector=None name= > master-api-master0.xxxxxx.com > May 27 13:39:14 master0 origin-node: W0527 13:39:14.066911 20150 > cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d > May 27 13:39:14 master0 origin-node: E0527 13:39:14.067321 20150 > kubelet.go:2101] Container runtime network not ready: NetworkReady=false > reason:NetworkPluginNotReady message:docker: network plugin is not ready: > cni config uninitialized > May 27 13:39:14 master0 origin-node: E0527 13:39:14.814705 20150 > summary.go:102] Failed to get system container stats for > "/system.slice/origin-node.service": failed to get cgroup stats for > "/system.slice/origin-node.service": failed to get container info for > "/system.slice/origin-node.service": unknown container > "/system.slice/origin-node.service" > May 27 13:39:19 master0 origin-node: W0527 13:39:19.069450 20150 > cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d > May 27 13:39:19 master0 origin-node: E0527 13:39:19.069850 20150 > kubelet.go:2101] Container runtime network not ready: NetworkReady=false > reason:NetworkPluginNotReady message:docker: network plugin is not ready: > cni config uninitialized > > On Mon, May 27, 2019 at 9:35 AM Shirly Radco <sradco(a)redhat.com&gt; wrote: > >> Hi Jayme, >> >> Thank you for reaching out. >> Please try rerunning the ansible playbook. >> If this doesn't work, try adding to the integ.ini in the metrics vm >> openshift_disable_check=docker_storage >> and rerun the ansible playbook again. >> >> Please update how it goes. >> >> Best regards, >> >> -- >> >> Shirly Radco >> >> BI Senior Software Engineer >> >> Red Hat <https://www.redhat.com/> >> >> <https://www.redhat.com/> >> >> >> On Sun, May 26, 2019 at 9:34 PM Jayme <jaymef(a)gmail.com&gt; wrote: >> >>> I'm running in to this ansible error during oVirt metrics installation >>> (following procedures at: >>> https://ovirt.org/documentation/metrics-install-guide/Installing_Metrics_... >>> ) >>> >>> This is happening late in the process, after successfully deploying the >>> installation VM and then running second step from the metrics VM. >>> >>> CHECK [memory_availability : master0.xxxxxx.com] >>> ************************************************************************************************* >>> fatal: [master0.xxxxxxx.com]: FAILED! => {"changed": true, "checks": >>> {"disk_availability": {}, "docker_image_availability": {"changed": true}, >>> "docker_storage": {"failed": true, "failures": [["OpenShiftCheckException", >>> "Could not find imported module support code for docker_info. Looked for >>> either AnsibleDockerClient.py or docker_common.py\nTraceback (most recent >>> call last):\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", >>> line 225, in run_check\n result = check.run()\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", >>> line 53, in run\n docker_info = self.execute_module(\"docker_info\", >>> {})\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", >>> line 211, in execute_module\n result = self._execute_module(module_name, >>> module_args, self.tmp, self.task_vars)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>> line 809, in _execute_module\n (module_style, shebang, module_data, >>> module_path) = self._configure_module(module_name=module_name, >>> module_args=module_args, task_vars=task_vars)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>> line 203, in _configure_module\n environment=final_environment)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 1023, in modify_module\n environment=environment)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 859, in _find_module_utils\n recursive_finder(module_name, >>> b_module_data, py_module_names, py_module_cache, zf)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 621, in recursive_finder\n raise AnsibleError(' >>> '.join(msg))\nAnsibleError: Could not find imported module support code for >>> docker_info. Looked for either AnsibleDockerClient.py or >>> docker_common.py\n"]], "msg": "Could not find imported module support code >>> for docker_info. Looked for either AnsibleDockerClient.py or >>> docker_common.py\nTraceback (most recent call last):\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", >>> line 225, in run_check\n result = check.run()\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", >>> line 53, in run\n docker_info = self.execute_module(\"docker_info\", >>> {})\n File >>> \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", >>> line 211, in execute_module\n result = self._execute_module(module_name, >>> module_args, self.tmp, self.task_vars)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>> line 809, in _execute_module\n (module_style, shebang, module_data, >>> module_path) = self._configure_module(module_name=module_name, >>> module_args=module_args, task_vars=task_vars)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", >>> line 203, in _configure_module\n environment=final_environment)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 1023, in modify_module\n environment=environment)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 859, in _find_module_utils\n recursive_finder(module_name, >>> b_module_data, py_module_names, py_module_cache, zf)\n File >>> \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", >>> line 621, in recursive_finder\n raise AnsibleError(' >>> '.join(msg))\nAnsibleError: Could not find imported module support code for >>> docker_info. Looked for either AnsibleDockerClient.py or >>> docker_common.py\n"}, "memory_availability": {}, "package_availability": >>> {"changed": false, "invocation": {"module_args": {"packages": ["PyYAML", >>> "bash-completion", "bind", "ceph-common", "dnsmasq", "docker", "firewalld", >>> "flannel", "glusterfs-fuse", "httpd-tools", "iptables", >>> "iptables-services", "iscsi-initiator-utils", "libselinux-python", >>> "nfs-utils", "ntp", "openssl", "origin", "origin-clients", >>> "origin-hyperkube", "origin-node", "pyparted", "python-httplib2", >>> "yum-utils"]}}}, "package_version": {"changed": false, "invocation": >>> {"module_args": {"package_list": [{"check_multi": false, "name": "origin", >>> "version": ""}, {"check_multi": false, "name": "origin-master", "version": >>> ""}, {"check_multi": false, "name": "origin-node", "version": ""}], >>> "package_mgr": "yum"}}}}, "msg": "One or more checks failed", >>> "playbook_context": "install"} >>> >>> NO MORE HOSTS LEFT >>> ******************************************************************************************************************************************* >>> >>> PLAY RECAP >>> *************************************************************************************************************************************************** >>> localhost : ok=35 changed=1 unreachable=0 >>> failed=0 skipped=16 rescued=0 ignored=0 >>> master0.xxxxxxx.com : ok=96 changed=6 unreachable=0 failed=1 >>> skipped=165 rescued=0 ignored=0 >>> >>> >>> INSTALLER STATUS >>> ********************************************************************************************************************************************* >>> Initialization : Complete (0:00:16) >>> Health Check : In Progress (0:00:36) >>> This phase can be restarted by running: >>> playbooks/openshift-checks/pre-install.yml >>> Sunday 26 May 2019 16:36:25 +0000 (0:00:36.151) 0:01:56.339 >>> ************ >>> >>> =============================================================================== >>> Run health checks (install) - EL >>> --------------------------------------------------------------------------------------------------------------------- >>> 36.15s >>> os_firewall : Ensure iptables services are not enabled >>> ------------------------------------------------------------------------------------------------ >>> 2.74s >>> openshift_repos : Ensure libselinux-python is installed >>> ----------------------------------------------------------------------------------------------- >>> 1.77s >>> openshift_repos : refresh cache >>> ----------------------------------------------------------------------------------------------------------------------- >>> 1.60s >>> Gather Cluster facts >>> ---------------------------------------------------------------------------------------------------------------------------------- >>> 1.51s >>> container_runtime : Fixup SELinux permissions for docker >>> ---------------------------------------------------------------------------------------------- >>> 1.33s >>> container_runtime : Place additional/blocked/insecure registries in >>> /etc/containers/registries.conf >>> --------------------------------------------------- 1.30s >>> Ensure openshift-ansible installer package deps are installed >>> ----------------------------------------------------------------------------------------- >>> 1.29s >>> container_runtime : Install Docker >>> -------------------------------------------------------------------------------------------------------------------- >>> 1.17s >>> Initialize openshift.node.sdn_mtu >>> --------------------------------------------------------------------------------------------------------------------- >>> 1.13s >>> os_firewall : Install firewalld packages >>> -------------------------------------------------------------------------------------------------------------- >>> 1.13s >>> container_runtime : Set various Docker options >>> -------------------------------------------------------------------------------------------------------- >>> 1.11s >>> install NetworkManager >>> -------------------------------------------------------------------------------------------------------------------------------- >>> 1.10s >>> openshift_repos : Configure correct origin release repository >>> ----------------------------------------------------------------------------------------- >>> 1.05s >>> container_runtime : Get current installed Docker version >>> ---------------------------------------------------------------------------------------------- >>> 1.04s >>> openshift_repos : Configure origin gpg keys >>> ----------------------------------------------------------------------------------------------------------- >>> 1.04s >>> openshift_repos : Remove openshift_additional.repo file >>> ----------------------------------------------------------------------------------------------- >>> 0.99s >>> container_runtime : Setup the docker-storage for overlay >>> ---------------------------------------------------------------------------------------------- >>> 0.96s >>> Detecting Operating System from ostree_booted >>> --------------------------------------------------------------------------------------------------------- >>> 0.95s >>> Gather Cluster facts >>> ---------------------------------------------------------------------------------------------------------------------------------- >>> 0.92s >>> >>> >>> Failure summary: >>> >>> >>> 1. Hosts: master0.xxxxxxx.com >>> Play: OpenShift Health Checks >>> Task: Run health checks (install) - EL >>> Message: One or more checks failed >>> Details: check "docker_storage": >>> Could not find imported module support code for >>> docker_info. Looked for either AnsibleDockerClient.py or docker_common.py >>> Traceback (most recent call last): >>> File >>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py", >>> line 225, in run_check >>> result = check.run() >>> File >>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py", >>> line 53, in run >>> docker_info = self.execute_module("docker_info", {}) >>> File >>> "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py", >>> line 211, in execute_module >>> result = self._execute_module(module_name, >>> module_args, self.tmp, self.task_vars) >>> File >>> "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line >>> 809, in _execute_module >>> (module_style, shebang, module_data, module_path) = >>> self._configure_module(module_name=module_name, module_args=module_args, >>> task_vars=task_vars) >>> File >>> "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line >>> 203, in _configure_module >>> environment=final_environment) >>> File >>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>> 1023, in modify_module >>> environment=environment) >>> File >>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>> 859, in _find_module_utils >>> recursive_finder(module_name, b_module_data, >>> py_module_names, py_module_cache, zf) >>> File >>> "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line >>> 621, in recursive_finder >>> raise AnsibleError(' '.join(msg)) >>> AnsibleError: Could not find imported module support >>> code for docker_info. Looked for either AnsibleDockerClient.py or >>> docker_common.py >>> >>> >>> The execution of "install_okd.yaml" includes checks designed to fail >>> early if the requirements of the playbook are not met. One or more of these >>> checks failed. To disregard these results,explicitly disable checks by >>> setting an Ansible variable: >>> openshift_disable_check=docker_storage >>> Failing check names are shown in the failure details above. Some checks >>> may be configurable by variables if your requirements are different from >>> the defaults; consult check documentation. >>> Variables can be set in the inventory or passed on the command line >>> using the -e flag to ansible-playbook. >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/SEFPOF36T7G... >>> >>