- Users - oVirt List Archives

Re: ovirt 4.2 HCI rollout
by Simone Tiraboschi 24 Jan '19

24 Jan '19

On Thu, Jan 24, 2019 at 3:14 PM Markus Schaufler < markus.schaufler(a)digit-all.at> wrote: > The hosted engine is not running and cannot be started. > > > Do you have on your first host a directory like /var/log/ovirt-hosted-engine-setup/engine-logs-2019-01-21T22:47:03Z with logs from the engine VM? > > ------------------------------ > *Von:* Simone Tiraboschi <stirabos(a)redhat.com> > *Gesendet:* Donnerstag, 24. Jänner 2019 14:45:59 > *An:* Markus Schaufler > *Cc:* Dominik Holler; users(a)ovirt.org > *Betreff:* Re: [ovirt-users] ovirt 4.2 HCI rollout > > > > On Thu, Jan 24, 2019 at 2:21 PM Markus Schaufler < > markus.schaufler(a)digit-all.at> wrote: > > Hi, > > > thanks for the replies. > > > I updated to 4.2.8 and tried again: > > > [ INFO ] TASK [Check engine VM health] > [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": > true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": > "0:00:00.165316", "end": "2019-01-24 14:12:06.899564", "rc": 0, "start": > "2019-01-24 14:12:06.734248", "stderr": "", "stderr_lines": [], "stdout": > "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3049 > (Thu Jan 24 14:11:59 > 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3049 (Thu Jan 24 > 14:11:59 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"0c1a3ddb\", > \"local_conf_timestamp\": 3049, \"host-ts\": 3049}, \"global_maintenance\": > false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, > \"live-data\": true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3049 > (Thu Jan 24 14:11:59 > 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3049 (Thu Jan 24 > 14:11:59 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"0c1a3ddb\", > \"local_conf_timestamp\": 3049, \"host-ts\": 3049}, \"global_maintenance\": > false}"]} > > > > It's still the same issue: the host fail to properly check the status of > the engine over a dedicate health page. > > You should connect to ovirt-hci.res01.ads.ooe.local and check the status > of ovirt-engine service and /var/log/ovirt-engine/engine.log there. > > > > [ INFO ] TASK [Check VM status at virt level] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Fail if engine VM is not running] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Get VDSM's target engine VM stats] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Convert stats to JSON format] > [ INFO ] ok: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address from VDSM stats] > [ INFO ] ok: [localhost] > [ INFO ] TASK [Fail if the Engine has no IP address] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Fail if Engine IP is different from engine's FQDN resolved > IP] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Reconfigure OVN central address] > [ ERROR ] fatal: [localhost]: FAILED! => {"msg": "The task includes an > option with an undefined variable. The error was: 'dict object' has no > attribute 'stdout_lines'\n\nThe error appears to have been in > '/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml': line > 518, column 5, but may\nbe elsewhere in the file depending on the exact > syntax problem.\n\nThe offending line appears to be:\n\n # > https://github.com/oVirt/ovirt-engine/blob/master/packaging/playbooks/roles… > - name: Reconfigure OVN central address\n ^ here\n"} > > > > attached you'll find the setup logs. > > > best regards, > > Markus Schaufler > ------------------------------ > *Von:* Simone Tiraboschi <stirabos(a)redhat.com> > *Gesendet:* Donnerstag, 24. Jänner 2019 11:56:50 > *An:* Dominik Holler > *Cc:* Markus Schaufler; users(a)ovirt.org > *Betreff:* Re: [ovirt-users] ovirt 4.2 HCI rollout > > > > On Thu, Jan 24, 2019 at 9:40 AM Dominik Holler <dholler(a)redhat.com> wrote: > > On Tue, 22 Jan 2019 11:15:12 +0000 > Markus Schaufler <markus.schaufler(a)digit-all.at> wrote: > > > Thanks for your reply, > > > > getent ahosts ovirt-hci.res01.ads.ooe.local | cut -d' ' -f1 | uniq > > 10.1.31.20 > > > > attached you'll find the logs. > > > > Thanks, to my eyes this looks like a bug. > I tried to isolate the relevant lines in the attached playbook. > > Markus, would you be so kind to check if ovirt-4.2.8 is working for you? > > > > OK, understood: the real error was just a few lines before what Dominik > pointed out: > > "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": > true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=5792 > (Mon Jan 21 13:57:45 > 2019)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=5792 (Mon Jan 21 > 13:57:45 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3000, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"ba303717\", > \"local_conf_timestamp\": 5792, \"host-ts\": 5792}, \"global_maintenance\": > false}", > "stdout_lines": [ > "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, > \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=5792 > (Mon Jan 21 13:57:45 > 2019)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=5792 (Mon Jan 21 > 13:57:45 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3000, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"ba303717\", > \"local_conf_timestamp\": 5792, \"host-ts\": 5792}, \"global_maintenance\": > false}" > ] > }" > 2019-01-21 13:57:46,695+0100 ERROR ansible failed {'status': 'FAILED', > 'ansible_type': 'task', 'ansible_task': u'Check engine VM health', > 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, > \'stderr_lines\': [], u\'changed\': True, u\'end\': u\'2019-01-21 > 13:57:46.242423\', \'_ansible_no_log\': False, u\'stdout\': u\'{"1": > {"conf_on_shared_storage": true, "live-data": true, "extra": > "metadata_parse_version=1\\\\nmetadata_feature_version=1\\\\ntimestamp=5792 > (Mon Jan 21 13:57:4', 'ansible_host': u'localhost', 'ansible_playbook': > u'/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml'} > > and in particular it's here: > for some reason we got \"engine-status\": {\"reason\": \"failed > liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": > \"Up\"} > over 120 attempts: we have to check engine.log (it got collected as well > from the engine VM) to understand why the engine was failing to start. > > > > > > ________________________________ > > Von: Dominik Holler <dholler(a)redhat.com> > > Gesendet: Montag, 21. Jänner 2019 17:52:35 > > An: Markus Schaufler > > Cc: users(a)ovirt.org; Simone Tiraboschi > > Betreff: Re: [ovirt-users] ovirt 4.2 HCI rollout > > > > Would you please share the related ovirt-host-deploy-ansible-*.log > > stored on the host in /var/log/ovirt-hosted-engine-setup ? > > > > Would you please also share the output of > > getent ahosts YOUR_HOSED_ENGNE_FQDN | cut -d' ' -f1 | uniq > > if executed on this host? > > > > > > On Mon, 21 Jan 2019 13:37:53 -0000 > > "Markus Schaufler" <markus.schaufler(a)digit-all.at> wrote: > > > > > Hi, > > > > > > I'm trying a (nested) ovirt 4.2.7 HCI rollout on 3 centos VM's by > > > following > > > > https://ovirt.org/documentation/gluster-hyperconverged/chap-Deploying_Hyper… > > > gluster deployment was successful but at HE deployment "stage 5" I > > > got following error: > > > > > > [ INFO ] TASK [Reconfigure OVN central address] > > > [ ERROR ] fatal: [localhost]: FAILED! => {"msg": "The task includes > > > an option with an undefined variable. The error was: 'dict object' > > > has no attribute 'stdout_lines'\n\nThe error appears to have been > > > in > > > '/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml': > > > line 522, column 5, but may\nbe elsewhere in the file depending on > > > the exact syntax problem.\n\nThe offending line appears to be:\n\n > > > # > > > > https://github.com/oVirt/ovirt-engine/blob/master/packaging/playbooks/roles… > > > - name: Reconfigure OVN central address\n ^ here\n"} > > > > > > > > > /var/log/messages: > > > Jan 21 14:09:56 HCI01 journal: ovirt-ha-agent > > > ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR > > > Engine VM stopped on localhost Jan 21 14:10:01 HCI01 systemd: > > > Started Session 22 of user root. Jan 21 14:10:02 HCI01 systemd: > > > Started Session c306 of user root. Jan 21 14:10:03 HCI01 systemd: > > > Started Session c307 of user root. Jan 21 14:10:06 HCI01 > > > vdsm[3650]: WARN executor state: count=5 workers=set([<Worker > > > name=periodic/4 waiting task#=141 at 0x7fd2d4316910>, <Worker > > > name=periodic/1 running <Task discardable <Operation > > > action=<vdsm.virt.sampling.VMBulkstatsMonitor object at > > > 0x7fd2d4679490> at 0x7fd2d4679710> timeout=7.5, duration=7 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> disca rded task#=413 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>, <Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>name=periodic/3 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>waiting task#=414 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>, > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10><Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>name=periodic/5 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>waiting > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>task#=0 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>, > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650><Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>name > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>=periodic/2 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>waiting > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>task#=412 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>]) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>ovirtmgmt: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>port > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>2(vnet0) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>entered > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>disabled > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>state > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>device > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>vnet0 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>left > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>promiscuous > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>mode > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>ovirtmgmt: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>port > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>2(vnet0) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>entered > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>disabled > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>state > > > Jan 21 14:10:06 HCI01 NetworkManager[3666]: <info> > > > [1548076206.9177] device (vnet0): state change: disconnected -> > > > unmanaged (reason 'unmanaged', sys-iface-state: 'remo ved') Jan 21 > > > 14:10:06 HCI01 NetworkManager[3666]: <info> [1548076206.9180] > > > device (vnet0): released from master device ovirtmgmt Jan 21 > > > 14:10:06 HCI01 lldpad: recvfrom(Event interface): No buffer space > > > available Jan 21 14:10:06 HCI01 libvirtd: 2019-01-21 > > > 13:10:06.925+0000: 2651: error : qemuMonitorIORead:609 : Unable to > > > read from monitor: Connection reset by peer Jan 21 14:10:07 HCI01 > > > kvm: 0 guests now active Jan 21 14:10:07 HCI01 systemd-machined: > > > Machine qemu-3-HostedEngine terminated. Jan 21 14:10:07 HCI01 > > > libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : > > > qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 > > > 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: > > > warning : qemuGetProcessInfo:1406 : cannot parse process status > > > data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: > > > 2704: warning : qemuGetProcessInfo:1406 : cannot parse process > > > status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 > > > 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot > > > parse process status data Jan 21 14:10:07 HCI01 libvirtd: > > > 2019-01-21 13:10:07.126+0000: 2704: error : > > > virNetDevTapInterfaceStats:764 : internal error: /proc/net/dev: > > > Interface not found Jan 21 14:10:07 HCI01 firewalld[24040]: > > > WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out > > > -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet 0' > > > failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try > > > `iptables -h' or 'iptables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out > > > vnet0 -g FP-vnet0' failed: iptables v 1.4.21: goto 'FP-vnet0' is > > > not a chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev > > > --physdev-in vnet0 -g FJ-vnet0' failed: iptables v1. 4.21: goto > > > 'FJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' > > > failed: iptable s v1.4.21: goto 'HJ-vnet0' is not a > > > chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FP-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FP-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FJ-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FJ-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HJ-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X HJ-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vne t0' > > > failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a > > > chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: > > > goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or > > > 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: > > > ip6tables v 1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try > > > `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HJ-vnet0' failed: ip6tab les v1.4.21: goto > > > 'HJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -F FP-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FP-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -F FJ-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -X FJ-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HJ-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -X HJ-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' > > > failed: Illegal target name 'libvirt-J-vnet0'. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j > > > libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Jan > > > 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j > > > libvirt-J-vnet0' failed: Illegal targe t name 'libvirt-J-vnet0'. > > > Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j > > > libvirt-P-vnet0' failed: Illegal targ et name 'libvirt-P-vnet0'. > > > Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: > > > Chain 'libvirt-J-vnet0' doesn't exis t. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -L libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -F libvirt-J-vnet0' failed: Chain > > > 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-J-vnet0' failed: Chain > > > 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -F libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out > > > vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not > > > a chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: > > > goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or > > > 'iptables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: > > > iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try > > > `iptables -h' or 'iptables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto > > > 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' > > > failed: iptables: Bad rule (does a matching rule exist in that > > > chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FO-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FI-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X HI-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' > > > failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a > > > chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: > > > goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or > > > 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: > > > ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try > > > `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto > > > 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' > > > failed: ip6tables: Bad rule (does a matching rule exist in that > > > chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D > > > POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target > > > name 'libvirt-O-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: > > > WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L > > > libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Jan > > > 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: > > > Chain 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-O-vnet0' failed: Chain > > > 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 vdsm[3650]: > > > WARN > > > File: > /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.ovirt-guest-agent.0 > > > already removed Jan 21 14:10:07 HCI01 vdsm[3650]: WARN Attempting > > > to remove a non existing network: > > > ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 14:10:07 > > > HCI01 vdsm[3650]: WARN Attempting to remove a non existing net > > > user: ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 > > > 14:10:07 HCI01 vdsm[3650]: WARN > > > File: > /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.org.qemu.guest_agent.0 > > > already removed > > > > > > any ideas on that? > > > _______________________________________________ > > > Users mailing list -- users(a)ovirt.org > > > To unsubscribe send an email to users-leave(a)ovirt.org > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ List > > > Archives: > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/XMMX5CY6VHFWW… > > > >

2 1

Re: ovirt 4.2 HCI rollout
by Simone Tiraboschi 24 Jan '19

24 Jan '19

On Thu, Jan 24, 2019 at 2:21 PM Markus Schaufler < markus.schaufler(a)digit-all.at> wrote: > Hi, > > > thanks for the replies. > > > I updated to 4.2.8 and tried again: > > > [ INFO ] TASK [Check engine VM health] > [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": > true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": > "0:00:00.165316", "end": "2019-01-24 14:12:06.899564", "rc": 0, "start": > "2019-01-24 14:12:06.734248", "stderr": "", "stderr_lines": [], "stdout": > "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3049 > (Thu Jan 24 14:11:59 > 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3049 (Thu Jan 24 > 14:11:59 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"0c1a3ddb\", > \"local_conf_timestamp\": 3049, \"host-ts\": 3049}, \"global_maintenance\": > false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, > \"live-data\": true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3049 > (Thu Jan 24 14:11:59 > 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3049 (Thu Jan 24 > 14:11:59 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"0c1a3ddb\", > \"local_conf_timestamp\": 3049, \"host-ts\": 3049}, \"global_maintenance\": > false}"]} > It's still the same issue: the host fail to properly check the status of the engine over a dedicate health page. You should connect to ovirt-hci.res01.ads.ooe.local and check the status of ovirt-engine service and /var/log/ovirt-engine/engine.log there. > [ INFO ] TASK [Check VM status at virt level] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Fail if engine VM is not running] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Get VDSM's target engine VM stats] > [ INFO ] changed: [localhost] > [ INFO ] TASK [Convert stats to JSON format] > [ INFO ] ok: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address from VDSM stats] > [ INFO ] ok: [localhost] > [ INFO ] TASK [Fail if the Engine has no IP address] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Fail if Engine IP is different from engine's FQDN resolved > IP] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Get target engine VM IPv4 address] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [Reconfigure OVN central address] > [ ERROR ] fatal: [localhost]: FAILED! => {"msg": "The task includes an > option with an undefined variable. The error was: 'dict object' has no > attribute 'stdout_lines'\n\nThe error appears to have been in > '/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml': line > 518, column 5, but may\nbe elsewhere in the file depending on the exact > syntax problem.\n\nThe offending line appears to be:\n\n # > https://github.com/oVirt/ovirt-engine/blob/master/packaging/playbooks/roles… > - name: Reconfigure OVN central address\n ^ here\n"} > > > > attached you'll find the setup logs. > > > best regards, > > Markus Schaufler > ------------------------------ > *Von:* Simone Tiraboschi <stirabos(a)redhat.com> > *Gesendet:* Donnerstag, 24. Jänner 2019 11:56:50 > *An:* Dominik Holler > *Cc:* Markus Schaufler; users(a)ovirt.org > *Betreff:* Re: [ovirt-users] ovirt 4.2 HCI rollout > > > > On Thu, Jan 24, 2019 at 9:40 AM Dominik Holler <dholler(a)redhat.com> wrote: > > On Tue, 22 Jan 2019 11:15:12 +0000 > Markus Schaufler <markus.schaufler(a)digit-all.at> wrote: > > > Thanks for your reply, > > > > getent ahosts ovirt-hci.res01.ads.ooe.local | cut -d' ' -f1 | uniq > > 10.1.31.20 > > > > attached you'll find the logs. > > > > Thanks, to my eyes this looks like a bug. > I tried to isolate the relevant lines in the attached playbook. > > Markus, would you be so kind to check if ovirt-4.2.8 is working for you? > > > > OK, understood: the real error was just a few lines before what Dominik > pointed out: > > "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": > true, \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=5792 > (Mon Jan 21 13:57:45 > 2019)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=5792 (Mon Jan 21 > 13:57:45 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3000, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"ba303717\", > \"local_conf_timestamp\": 5792, \"host-ts\": 5792}, \"global_maintenance\": > false}", > "stdout_lines": [ > "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, > \"extra\": > \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=5792 > (Mon Jan 21 13:57:45 > 2019)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=5792 (Mon Jan 21 > 13:57:45 > 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", > \"hostname\": \"HCI01.res01.ads.ooe.local\", \"host-id\": 1, > \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": > \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3000, \"stopped\": > false, \"maintenance\": false, \"crc32\": \"ba303717\", > \"local_conf_timestamp\": 5792, \"host-ts\": 5792}, \"global_maintenance\": > false}" > ] > }" > 2019-01-21 13:57:46,695+0100 ERROR ansible failed {'status': 'FAILED', > 'ansible_type': 'task', 'ansible_task': u'Check engine VM health', > 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, > \'stderr_lines\': [], u\'changed\': True, u\'end\': u\'2019-01-21 > 13:57:46.242423\', \'_ansible_no_log\': False, u\'stdout\': u\'{"1": > {"conf_on_shared_storage": true, "live-data": true, "extra": > "metadata_parse_version=1\\\\nmetadata_feature_version=1\\\\ntimestamp=5792 > (Mon Jan 21 13:57:4', 'ansible_host': u'localhost', 'ansible_playbook': > u'/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml'} > > and in particular it's here: > for some reason we got \"engine-status\": {\"reason\": \"failed > liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": > \"Up\"} > over 120 attempts: we have to check engine.log (it got collected as well > from the engine VM) to understand why the engine was failing to start. > > > > > > ________________________________ > > Von: Dominik Holler <dholler(a)redhat.com> > > Gesendet: Montag, 21. Jänner 2019 17:52:35 > > An: Markus Schaufler > > Cc: users(a)ovirt.org; Simone Tiraboschi > > Betreff: Re: [ovirt-users] ovirt 4.2 HCI rollout > > > > Would you please share the related ovirt-host-deploy-ansible-*.log > > stored on the host in /var/log/ovirt-hosted-engine-setup ? > > > > Would you please also share the output of > > getent ahosts YOUR_HOSED_ENGNE_FQDN | cut -d' ' -f1 | uniq > > if executed on this host? > > > > > > On Mon, 21 Jan 2019 13:37:53 -0000 > > "Markus Schaufler" <markus.schaufler(a)digit-all.at> wrote: > > > > > Hi, > > > > > > I'm trying a (nested) ovirt 4.2.7 HCI rollout on 3 centos VM's by > > > following > > > > https://ovirt.org/documentation/gluster-hyperconverged/chap-Deploying_Hyper… > > > gluster deployment was successful but at HE deployment "stage 5" I > > > got following error: > > > > > > [ INFO ] TASK [Reconfigure OVN central address] > > > [ ERROR ] fatal: [localhost]: FAILED! => {"msg": "The task includes > > > an option with an undefined variable. The error was: 'dict object' > > > has no attribute 'stdout_lines'\n\nThe error appears to have been > > > in > > > '/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml': > > > line 522, column 5, but may\nbe elsewhere in the file depending on > > > the exact syntax problem.\n\nThe offending line appears to be:\n\n > > > # > > > > https://github.com/oVirt/ovirt-engine/blob/master/packaging/playbooks/roles… > > > - name: Reconfigure OVN central address\n ^ here\n"} > > > > > > > > > /var/log/messages: > > > Jan 21 14:09:56 HCI01 journal: ovirt-ha-agent > > > ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR > > > Engine VM stopped on localhost Jan 21 14:10:01 HCI01 systemd: > > > Started Session 22 of user root. Jan 21 14:10:02 HCI01 systemd: > > > Started Session c306 of user root. Jan 21 14:10:03 HCI01 systemd: > > > Started Session c307 of user root. Jan 21 14:10:06 HCI01 > > > vdsm[3650]: WARN executor state: count=5 workers=set([<Worker > > > name=periodic/4 waiting task#=141 at 0x7fd2d4316910>, <Worker > > > name=periodic/1 running <Task discardable <Operation > > > action=<vdsm.virt.sampling.VMBulkstatsMonitor object at > > > 0x7fd2d4679490> at 0x7fd2d4679710> timeout=7.5, duration=7 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> disca rded task#=413 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>, <Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>name=periodic/3 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>waiting task#=414 at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>, > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10><Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>name=periodic/5 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>waiting > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>task#=0 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> 0x7fd2d5ed0510>0x7fd2d5ed0b10>at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>, > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650><Worker > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>name > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>=periodic/2 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>waiting > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>task#=412 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>at > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>]) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>ovirtmgmt: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>port > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>2(vnet0) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>entered > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>disabled > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>state > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>device > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>vnet0 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>left > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>promiscuous > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>mode > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>Jan > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>21 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>14:10:06 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>HCI01 > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>kernel: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>ovirtmgmt: > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>port > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>2(vnet0) > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>entered > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>disabled > > > 0x7fd2d4679490> 0x7fd33c1e0ed0> > 0x7fd2d5ed0510>0x7fd2d5ed0b10>0x7fd2d425f650>0x7fd2d5ed07d0>state > > > Jan 21 14:10:06 HCI01 NetworkManager[3666]: <info> > > > [1548076206.9177] device (vnet0): state change: disconnected -> > > > unmanaged (reason 'unmanaged', sys-iface-state: 'remo ved') Jan 21 > > > 14:10:06 HCI01 NetworkManager[3666]: <info> [1548076206.9180] > > > device (vnet0): released from master device ovirtmgmt Jan 21 > > > 14:10:06 HCI01 lldpad: recvfrom(Event interface): No buffer space > > > available Jan 21 14:10:06 HCI01 libvirtd: 2019-01-21 > > > 13:10:06.925+0000: 2651: error : qemuMonitorIORead:609 : Unable to > > > read from monitor: Connection reset by peer Jan 21 14:10:07 HCI01 > > > kvm: 0 guests now active Jan 21 14:10:07 HCI01 systemd-machined: > > > Machine qemu-3-HostedEngine terminated. Jan 21 14:10:07 HCI01 > > > libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : > > > qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 > > > 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: > > > warning : qemuGetProcessInfo:1406 : cannot parse process status > > > data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: > > > 2704: warning : qemuGetProcessInfo:1406 : cannot parse process > > > status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 > > > 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot > > > parse process status data Jan 21 14:10:07 HCI01 libvirtd: > > > 2019-01-21 13:10:07.126+0000: 2704: error : > > > virNetDevTapInterfaceStats:764 : internal error: /proc/net/dev: > > > Interface not found Jan 21 14:10:07 HCI01 firewalld[24040]: > > > WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out > > > -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet 0' > > > failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try > > > `iptables -h' or 'iptables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out > > > vnet0 -g FP-vnet0' failed: iptables v 1.4.21: goto 'FP-vnet0' is > > > not a chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev > > > --physdev-in vnet0 -g FJ-vnet0' failed: iptables v1. 4.21: goto > > > 'FJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' > > > failed: iptable s v1.4.21: goto 'HJ-vnet0' is not a > > > chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FP-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FP-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FJ-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FJ-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HJ-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X HJ-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vne t0' > > > failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a > > > chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: > > > goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or > > > 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: > > > ip6tables v 1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try > > > `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HJ-vnet0' failed: ip6tab les v1.4.21: goto > > > 'HJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -F FP-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FP-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -F FJ-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -X FJ-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HJ-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -X HJ-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' > > > failed: Illegal target name 'libvirt-J-vnet0'. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j > > > libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Jan > > > 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j > > > libvirt-J-vnet0' failed: Illegal targe t name 'libvirt-J-vnet0'. > > > Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j > > > libvirt-P-vnet0' failed: Illegal targ et name 'libvirt-P-vnet0'. > > > Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: > > > Chain 'libvirt-J-vnet0' doesn't exis t. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -L libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -F libvirt-J-vnet0' failed: Chain > > > 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-J-vnet0' failed: Chain > > > 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -F libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-P-vnet0' failed: Chain > > > 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out > > > vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not > > > a chain#012#012Try `iptables -h' or 'iptables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: > > > goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or > > > 'iptables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: > > > iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try > > > `iptables -h' or 'iptables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto > > > 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' > > > failed: iptables: Bad rule (does a matching rule exist in that > > > chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FO-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X FI-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: > > > iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 > > > -w -X HI-vnet0' failed: iptables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' > > > failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a > > > chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more > > > information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m > > > physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: > > > goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or > > > 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: > > > ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try > > > `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 > > > 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev > > > --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto > > > 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables > > > --help' for more information. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' > > > failed: ip6tables: Bad rule (does a matching rule exist in that > > > chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: > > > ip6tables: No chain/target/match by that name. Jan 21 14:10:07 > > > HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No > > > chain/target/match by that name. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 > > > -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that > > > name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: > > > COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D > > > POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target > > > name 'libvirt-O-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: > > > WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L > > > libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Jan > > > 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: > > > '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: > > > Chain 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 > > > firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables > > > --concurrent -t nat -X libvirt-O-vnet0' failed: Chain > > > 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 vdsm[3650]: > > > WARN > > > File: > /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.ovirt-guest-agent.0 > > > already removed Jan 21 14:10:07 HCI01 vdsm[3650]: WARN Attempting > > > to remove a non existing network: > > > ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 14:10:07 > > > HCI01 vdsm[3650]: WARN Attempting to remove a non existing net > > > user: ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 > > > 14:10:07 HCI01 vdsm[3650]: WARN > > > File: > /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.org.qemu.guest_agent.0 > > > already removed > > > > > > any ideas on that? > > > _______________________________________________ > > > Users mailing list -- users(a)ovirt.org > > > To unsubscribe send an email to users-leave(a)ovirt.org > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ List > > > Archives: > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/XMMX5CY6VHFWW… > > > >

2 1

lvm problem
by Nyika Csaba 24 Jan '19

24 Jan '19

Hi all, I have an Ovirt 4.2.8 cluster. Nodes are 4.2 ovirt nodes and the volumes (5) attached to nodes by FC. 2 weeks ago i made a small vm (centos 7 based) for me to test (name A). After the test i dropped a vm. Next day a made an another vm (named B) to the developes and tired to add a new disk to the vm (B). Than a original volume group (vg) of the vm (B) missed and i got back the yesterday dropped vm`s (A) vg! I tired to restart a vm, but the vm never started again. I dropped this vm (B) too, and tired to add a new disk an older running vm (C) too, but a volume group changed to that vm`s vg what i dropped before (B). I checked this „error” and i got it ,then i delete or move disks from the end of the FC volume. Have sombody ever seen error like this? Thanks, csaba Ps: in this cluster a managed 120 productive vm, so….

1 0

Lvm problem
by csabany＠freemail.hu 24 Jan '19

24 Jan '19

Hi all, I have an Ovirt 4.2.8 cluster. Nodes are 4.2 ovirt nodes and the volumes (5) attached to nodes by FC. 2 weeks ago i made a small vm (centos 7 based) for me to test (name A). After the test i dropped a vm. Next day a made an another vm (named B) to the developes and tired to add a new disk to the vm (B). Than a original volume group (vg) of the vm (B) missed and i got back the yesterday dropped vm`s (A) vg! I tired to restart a vm, but the vm never started again. I dropped this vm (B) too, and tired to add a new disk an older running vm (C) too, but a volume group changed to that vm`s vg what i dropped before (B). I checked this „error” and i got it ,then i delete or move disks from the end of the FC volume. Have sombody ever seen error like this? Thanks, csaba Ps: in this cluster a managed 120 productive vm, so….

1 0

latest pycurl 7.43 brokes ovirtsdk4
by Nathanaël Blanchet 24 Jan '19

24 Jan '19

Hi all, If anyone uses latest pycurl 7.43 provided by pip or ansible tower/awx, any ovirtsdk4 calling will issue with the log: The full traceback is: WARNING: The below traceback may *not* be related to the actual failure. File "/tmp/ansible_ovirt_auth_payload_L1HK9E/__main__.py", line 202, in <module> import ovirtsdk4 as sdk File "/opt/awx/embedded/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 22, in <module> import pycurl fatal: [localhost]: FAILED! => { "changed": false, "invocation": { "module_args": { "ca_file": null, "compress": true, "headers": null, "hostname": null, "insecure": true, "kerberos": false, "ovirt_auth": null, "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "state": "present", "timeout": 0, "token": null, "url": "https://acore.v100.abes.fr/ovirt-engine/api", "username": "admin@internal" } }, "msg": "ovirtsdk4 version 4.2.4 or higher is required for this module" } The only way is to set the version of pycurl with pip install -U "pycurl == 7.19.0" (Before this, in tower/awx, you should create venv) -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

2 4

oVirt 4.2.8 CPU Compatibility
by Stefano Danzi 24 Jan '19

24 Jan '19

Hello! I'm running oVirt 4.2.7.5-1.el7 on 3 hosts cluster. Cluster CPU Type is "AMD Opteron G3". On default cluster I can see the warning: "Warning: The CPU type 'AMD Opteron G3' will not be supported in the next minor version update'" Is it still supported in version 4.2.8? I can't see references in documentation or changelog.

5 5

ovirt 4.2 HCI rollout
by Markus Schaufler 23 Jan '19

23 Jan '19

Hi, I'm trying a (nested) ovirt 4.2.7 HCI rollout on 3 centos VM's by following https://ovirt.org/documentation/gluster-hyperconverged/chap-Deploying_Hyper… gluster deployment was successful but at HE deployment "stage 5" I got following error: [ INFO ] TASK [Reconfigure OVN central address] [ ERROR ] fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'stdout_lines'\n\nThe error appears to have been in '/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml': line 522, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n # https://github.com/oVirt/ovirt-engine/blob/master/packaging/playbooks/roles… - name: Reconfigure OVN central address\n ^ here\n"} /var/log/messages: Jan 21 14:09:56 HCI01 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Jan 21 14:10:01 HCI01 systemd: Started Session 22 of user root. Jan 21 14:10:02 HCI01 systemd: Started Session c306 of user root. Jan 21 14:10:03 HCI01 systemd: Started Session c307 of user root. Jan 21 14:10:06 HCI01 vdsm[3650]: WARN executor state: count=5 workers=set([<Worker name=periodic/4 waiting task#=141 at 0x7fd2d4316910>, <Worker name=periodic/1 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7fd2d4679490> at 0x7fd2d4679710> timeout=7.5, duration=7 at 0x7fd33c1e0ed0> disca rded task#=413 at 0x7fd2d5ed0510>, <Worker name=periodic/3 waiting task#=414 at 0x7fd2d5ed0b10>, <Worker name=periodic/5 waiting task#=0 at 0x7fd2d425f650>, <Worker name =periodic/2 waiting task#=412 at 0x7fd2d5ed07d0>]) Jan 21 14:10:06 HCI01 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Jan 21 14:10:06 HCI01 kernel: device vnet0 left promiscuous mode Jan 21 14:10:06 HCI01 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Jan 21 14:10:06 HCI01 NetworkManager[3666]: <info> [1548076206.9177] device (vnet0): state change: disconnected -> unmanaged (reason 'unmanaged', sys-iface-state: 'remo ved') Jan 21 14:10:06 HCI01 NetworkManager[3666]: <info> [1548076206.9180] device (vnet0): released from master device ovirtmgmt Jan 21 14:10:06 HCI01 lldpad: recvfrom(Event interface): No buffer space available Jan 21 14:10:06 HCI01 libvirtd: 2019-01-21 13:10:06.925+0000: 2651: error : qemuMonitorIORead:609 : Unable to read from monitor: Connection reset by peer Jan 21 14:10:07 HCI01 kvm: 0 guests now active Jan 21 14:10:07 HCI01 systemd-machined: Machine qemu-3-HostedEngine terminated. Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.125+0000: 2704: warning : qemuGetProcessInfo:1406 : cannot parse process status data Jan 21 14:10:07 HCI01 libvirtd: 2019-01-21 13:10:07.126+0000: 2704: error : virNetDevTapInterfaceStats:764 : internal error: /proc/net/dev: Interface not found Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet 0' failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: iptables v 1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: iptables v1. 4.21: goto 'FJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: iptable s v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FP-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FP-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FJ-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FJ-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HJ-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HJ-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vne t0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: ip6tables v 1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: ip6tab les v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FP-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FP-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' failed: Illegal target name 'libvirt-J-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' failed: Illegal targe t name 'libvirt-J-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-P-vnet0' failed: Illegal targ et name 'libvirt-P-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exis t. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FO-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FI-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HI-vnet0' failed: iptables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: ip6tables: Bad rule (does a matching rule exist in that chain?). Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that name. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target name 'libvirt-O-vnet0'. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 firewalld[24040]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Jan 21 14:10:07 HCI01 vdsm[3650]: WARN File: /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.ovirt-guest-agent.0 already removed Jan 21 14:10:07 HCI01 vdsm[3650]: WARN Attempting to remove a non existing network: ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 14:10:07 HCI01 vdsm[3650]: WARN Attempting to remove a non existing net user: ovirtmgmt/ea1b312c-a462-45a9-ab75-78008bc4c9c9 Jan 21 14:10:07 HCI01 vdsm[3650]: WARN File: /var/lib/libvirt/qemu/channels/ea1b312c-a462-45a9-ab75-78008bc4c9c9.org.qemu.guest_agent.0 already removed any ideas on that?

2 2

Q: Is it safe to execute on node "saslpasswd2 -a libvirt username" ?
by Andrei Verovski 23 Jan '19

23 Jan '19

Hi ! Is it safe to execute on oVirt node this command ? saslpasswd2 -a libvirt username Its a production environment, screwing up anything is not an option. I have no idea how VDSM interacts with libvirt, so not sure about this. Thanks in advance Andrei

2 1

Host non-responsive after yum update CentOS7/Ovirt3.6
by jaherring＠usa.net 23 Jan '19

23 Jan '19

Hi, I working on a CentOS7 based Ovirt 3.6 system (ovirt-engine/db on one machine, two separate ovirt vm hosts) which has been running fine but mostly ignored for 2-3 of years. Recently it was decided to update the OS as it was far behind on security updates, so one host was put into maintenance mode, yum update'd, rebooted, and then it was attempted to take out of maintenance mode but it's "non-responsive" now. If I look in /var/log/ovirt-engine/engine.log on the engine machine I see for this host (vmserver2): "ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) [2bc0978d] Command 'GetCapabilitiesVDSCommand(HostName = vmserver2, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='6725086f-42c0-40eb-91f1-0f2411ea9432', vds='Host[vmserver2,6725086f-42c0-40eb-91f1-0f2411ea9432]'})' execution failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed" and thereafter more errors. This keep repeating in the log. In the Ovirt GUI I see multiple occurrences of log entries for the problem host: "vmserver2...command failed: Vds timeout occurred" "vmserver2...command failed: Heartbeat exceeded" "vmserver2...command failed: internal error: Unknown CPU model Broadwell-noTSX-IBRS" Firewall rules look identical to the host which is working normally but has not been updated. Any thoughts about how to fix or further troubleshoot this?

6 7

Re: Cannot Increase Hosted Engine VM Memory
by Douglas Duckworth 23 Jan '19

23 Jan '19

Hi Simone Can I get help with this issue? Still cannot increase memory for Hosted Engine. Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 8:08 AM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sure, they're attached. In "first attempt" the error seems to be: 2019-01-17 07:49:24,795-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [680f82b3-7612-4d91-afdc-43937aa298a2] EVENT_ID: FAILED_HOT_SET_MEMORY_NOT_DIVIDABLE(2,048), Failed to hot plug memory to VM HostedEngine. Amount of added memory (4000MiB) is not dividable by 256MiB. Followed by: 2019-01-17 07:49:24,814-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-29) [26f5f3ed] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:49:24,815-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-29) [26f5f3ed] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. In "second attempt" I used values that are dividable by 256 MiB so that's no longer present. Though same error: 2019-01-17 07:56:59,795-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] START, SetAmountOfMemoryVDSCommand(HostName = ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu>, Params:{hostId='cdd5ffda-95c7-4ffa-ae40-be66f1d15c30', vmId='adf14389-1563-4b1a-9af6-4b40370a825b', memoryDevice='VmDevice:{id='VmDeviceId:{deviceId='7f7d97cc-c273-4033-af53-bc9033ea3abe', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='memory', type='MEMORY', specParams='[node=0, size=2048]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}', minAllocatedMem='6144'}), log id: 50873daa 2019-01-17 07:56:59,855-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] FINISH, SetAmountOfMemoryVDSCommand, log id: 50873daa 2019-01-17 07:56:59,862-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [7059a48f] EVENT_ID: HOT_SET_MEMORY(2,039), Hotset memory: changed the amount of memory on VM HostedEngine from 4096 to 4096 2019-01-17 07:56:59,881-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-22) [28fd4c82] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:56:59,882-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-22) [28fd4c82] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. This message repeats throughout engine.log: 2019-01-17 07:55:43,270-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [] EVENT_ID: VM_MEMORY_UNDER_GUARANTEED_VALUE(148), VM HostedEngine on host ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu> was guaranteed 8192 MB but currently has 4224 MB As you can see attached the host has plenty of memory. Thank you Simone! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 5:09 AM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 16, 2019 at 8:22 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sorry for accidental send. Anyway I try to increase physical memory however it won't go above 4096MB. The hypervisor has 64GB. Do I need to modify this value with Hosted Engine offline? No, it's not required. Can you please attach your engine.log for the relevant time frame? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 16, 2019 at 1:58 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hello I am trying to increase Hosted Engine physical memory above 4GB Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_site_pri…> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_communit…> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGSXQVVPJJ2CR…<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archiv…>

2 1