Hi, The latest release of 4.3.z should already include a fix for this issue, ovirt-engine-metrics-1.3.1 rpm. The issue is that it requires the NetworkManagar to be installed, running and enabled for it to work. You can install it manually on the master0 vm , start and enable it or you can also install the updated rpm from the nightly builds if your environment is oVirt 4.2.z: https://resources.ovirt.org/pub/ovirt-4.2-snapshot/rpm/el7/noarch/ovirt-engi... Relevant patches are: https://gerrit.ovirt.org/#/c/99959/ https://gerrit.ovirt.org/#/c/99718/ Best regards, -- Shirly Radco BI Senior Software Engineer Red Hat <https://www.redhat.com/> <https://www.redhat.com/> On Mon, May 27, 2019 at 4:41 PM Jayme <jaymef@gmail.com> wrote:
I managed to get past that but am running in to another problem later in the process on the control plane pods to appear task. I thought perhaps a glitch in the process from the failed docker step previously so after a few more runs I tried killing everything and restarting the metrics process again from the very beginning and end up hitting the same issue with control plane pods even though all other steps/tasks seem to be working.
I'm just getting this:
TASK [openshift_control_plane : Wait for control plane pods to appear] ********* Monday 27 May 2019 13:31:54 +0000 (0:00:00.180) 0:14:33.857 ************ FAILED - RETRYING: Wait for control plane pods to appear (60 retries left). FAILED - RETRYING: Wait for control plane pods to appear (59 retries left). FAILED - RETRYING: Wait for control plane pods to appear (58 retries left). FAILED - RETRYING: Wait for control plane pods to appear (57 retries left). FAILED - RETRYING: Wait for control plane pods to appear (56 retries left).
It eventually counts all the way down to zero and fails.
In syslog of the master0 server I'm seeing some errors related to cni config
May 27 13:39:07 master0 ansible-oc_obj: Invoked with files=None kind=pod force=False all_namespaces=None field_selector=None namespace=kube-system delete_after=False kubeconfig=/etc/origin/master/admin.kubeconfig content=None state=list debug=False selector=None name= master-api-master0.xxxxxx.com May 27 13:39:09 master0 origin-node: W0527 13:39:09.064230 20150 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d May 27 13:39:09 master0 origin-node: E0527 13:39:09.064670 20150 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized May 27 13:39:13 master0 ansible-oc_obj: Invoked with files=None kind=pod force=False all_namespaces=None field_selector=None namespace=kube-system delete_after=False kubeconfig=/etc/origin/master/admin.kubeconfig content=None state=list debug=False selector=None name= master-api-master0.xxxxxx.com May 27 13:39:14 master0 origin-node: W0527 13:39:14.066911 20150 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d May 27 13:39:14 master0 origin-node: E0527 13:39:14.067321 20150 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized May 27 13:39:14 master0 origin-node: E0527 13:39:14.814705 20150 summary.go:102] Failed to get system container stats for "/system.slice/origin-node.service": failed to get cgroup stats for "/system.slice/origin-node.service": failed to get container info for "/system.slice/origin-node.service": unknown container "/system.slice/origin-node.service" May 27 13:39:19 master0 origin-node: W0527 13:39:19.069450 20150 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d May 27 13:39:19 master0 origin-node: E0527 13:39:19.069850 20150 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
On Mon, May 27, 2019 at 9:35 AM Shirly Radco <sradco@redhat.com> wrote:
Hi Jayme,
Thank you for reaching out. Please try rerunning the ansible playbook. If this doesn't work, try adding to the integ.ini in the metrics vm openshift_disable_check=docker_storage and rerun the ansible playbook again.
Please update how it goes.
Best regards,
--
Shirly Radco
BI Senior Software Engineer
Red Hat <https://www.redhat.com/>
On Sun, May 26, 2019 at 9:34 PM Jayme <jaymef@gmail.com> wrote:
I'm running in to this ansible error during oVirt metrics installation (following procedures at: https://ovirt.org/documentation/metrics-install-guide/Installing_Metrics_Sto... )
This is happening late in the process, after successfully deploying the installation VM and then running second step from the metrics VM.
CHECK [memory_availability : master0.xxxxxx.com] ************************************************************************************************* fatal: [master0.xxxxxxx.com]: FAILED! => {"changed": true, "checks": {"disk_availability": {}, "docker_image_availability": {"changed": true}, "docker_storage": {"failed": true, "failures": [["OpenShiftCheckException", "Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py\nTraceback (most recent call last):\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", line 225, in run_check\n result = check.run()\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", line 53, in run\n docker_info = self.execute_module(\"docker_info\", {})\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", line 211, in execute_module\n result = self._execute_module(module_name, module_args, self.tmp, self.task_vars)\n File \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", line 809, in _execute_module\n (module_style, shebang, module_data, module_path) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)\n File \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", line 203, in _configure_module\n environment=final_environment)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 1023, in modify_module\n environment=environment)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 859, in _find_module_utils\n recursive_finder(module_name, b_module_data, py_module_names, py_module_cache, zf)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 621, in recursive_finder\n raise AnsibleError(' '.join(msg))\nAnsibleError: Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py\n"]], "msg": "Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py\nTraceback (most recent call last):\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py\", line 225, in run_check\n result = check.run()\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py\", line 53, in run\n docker_info = self.execute_module(\"docker_info\", {})\n File \"/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py\", line 211, in execute_module\n result = self._execute_module(module_name, module_args, self.tmp, self.task_vars)\n File \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", line 809, in _execute_module\n (module_style, shebang, module_data, module_path) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)\n File \"/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py\", line 203, in _configure_module\n environment=final_environment)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 1023, in modify_module\n environment=environment)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 859, in _find_module_utils\n recursive_finder(module_name, b_module_data, py_module_names, py_module_cache, zf)\n File \"/usr/lib/python2.7/site-packages/ansible/executor/module_common.py\", line 621, in recursive_finder\n raise AnsibleError(' '.join(msg))\nAnsibleError: Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py\n"}, "memory_availability": {}, "package_availability": {"changed": false, "invocation": {"module_args": {"packages": ["PyYAML", "bash-completion", "bind", "ceph-common", "dnsmasq", "docker", "firewalld", "flannel", "glusterfs-fuse", "httpd-tools", "iptables", "iptables-services", "iscsi-initiator-utils", "libselinux-python", "nfs-utils", "ntp", "openssl", "origin", "origin-clients", "origin-hyperkube", "origin-node", "pyparted", "python-httplib2", "yum-utils"]}}}, "package_version": {"changed": false, "invocation": {"module_args": {"package_list": [{"check_multi": false, "name": "origin", "version": ""}, {"check_multi": false, "name": "origin-master", "version": ""}, {"check_multi": false, "name": "origin-node", "version": ""}], "package_mgr": "yum"}}}}, "msg": "One or more checks failed", "playbook_context": "install"}
NO MORE HOSTS LEFT *******************************************************************************************************************************************
PLAY RECAP *************************************************************************************************************************************************** localhost : ok=35 changed=1 unreachable=0 failed=0 skipped=16 rescued=0 ignored=0 master0.xxxxxxx.com : ok=96 changed=6 unreachable=0 failed=1 skipped=165 rescued=0 ignored=0
INSTALLER STATUS ********************************************************************************************************************************************* Initialization : Complete (0:00:16) Health Check : In Progress (0:00:36) This phase can be restarted by running: playbooks/openshift-checks/pre-install.yml Sunday 26 May 2019 16:36:25 +0000 (0:00:36.151) 0:01:56.339 ************
=============================================================================== Run health checks (install) - EL --------------------------------------------------------------------------------------------------------------------- 36.15s os_firewall : Ensure iptables services are not enabled ------------------------------------------------------------------------------------------------ 2.74s openshift_repos : Ensure libselinux-python is installed ----------------------------------------------------------------------------------------------- 1.77s openshift_repos : refresh cache ----------------------------------------------------------------------------------------------------------------------- 1.60s Gather Cluster facts ---------------------------------------------------------------------------------------------------------------------------------- 1.51s container_runtime : Fixup SELinux permissions for docker ---------------------------------------------------------------------------------------------- 1.33s container_runtime : Place additional/blocked/insecure registries in /etc/containers/registries.conf --------------------------------------------------- 1.30s Ensure openshift-ansible installer package deps are installed ----------------------------------------------------------------------------------------- 1.29s container_runtime : Install Docker -------------------------------------------------------------------------------------------------------------------- 1.17s Initialize openshift.node.sdn_mtu --------------------------------------------------------------------------------------------------------------------- 1.13s os_firewall : Install firewalld packages -------------------------------------------------------------------------------------------------------------- 1.13s container_runtime : Set various Docker options -------------------------------------------------------------------------------------------------------- 1.11s install NetworkManager -------------------------------------------------------------------------------------------------------------------------------- 1.10s openshift_repos : Configure correct origin release repository ----------------------------------------------------------------------------------------- 1.05s container_runtime : Get current installed Docker version ---------------------------------------------------------------------------------------------- 1.04s openshift_repos : Configure origin gpg keys ----------------------------------------------------------------------------------------------------------- 1.04s openshift_repos : Remove openshift_additional.repo file ----------------------------------------------------------------------------------------------- 0.99s container_runtime : Setup the docker-storage for overlay ---------------------------------------------------------------------------------------------- 0.96s Detecting Operating System from ostree_booted --------------------------------------------------------------------------------------------------------- 0.95s Gather Cluster facts ---------------------------------------------------------------------------------------------------------------------------------- 0.92s
Failure summary:
1. Hosts: master0.xxxxxxx.com Play: OpenShift Health Checks Task: Run health checks (install) - EL Message: One or more checks failed Details: check "docker_storage": Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py Traceback (most recent call last): File "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/action_plugins/openshift_health_check.py", line 225, in run_check result = check.run() File "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/docker_storage.py", line 53, in run docker_info = self.execute_module("docker_info", {}) File "/usr/share/ansible/openshift-ansible/roles/openshift_health_checker/openshift_checks/__init__.py", line 211, in execute_module result = self._execute_module(module_name, module_args, self.tmp, self.task_vars) File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 809, in _execute_module (module_style, shebang, module_data, module_path) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars) File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 203, in _configure_module environment=final_environment) File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 1023, in modify_module environment=environment) File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 859, in _find_module_utils recursive_finder(module_name, b_module_data, py_module_names, py_module_cache, zf) File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 621, in recursive_finder raise AnsibleError(' '.join(msg)) AnsibleError: Could not find imported module support code for docker_info. Looked for either AnsibleDockerClient.py or docker_common.py
The execution of "install_okd.yaml" includes checks designed to fail early if the requirements of the playbook are not met. One or more of these checks failed. To disregard these results,explicitly disable checks by setting an Ansible variable: openshift_disable_check=docker_storage Failing check names are shown in the failure details above. Some checks may be configurable by variables if your requirements are different from the defaults; consult check documentation. Variables can be set in the inventory or passed on the command line using the -e flag to ansible-playbook. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/SEFPOF36T7G4GI...