Hi all, On Sun, Oct 1, 2017 at 10:15 AM, Code Review <gerrit(a)ovirt.org&gt; wrote: > Jenkins CI posted comments on this change. > > View Change > > Patch set 1:Continuous-Integration -1 > > Build Failed > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > : FAILURE Above was triggered by [1]. It failed with [2]: TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] ********************* fatal: [lago-he-basic-suite-master-host1]: FAILED! => {"changed": true, "cmd": ["vdsm-tool", "ovn-config", "192.168.200.99", "ovirtmgmt"], "delta": "0:00:00.623565", "end": "2017-10-01 03:15:11.179717", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2017-10-01 03:15:10.556152", "stderr": "Traceback (most recent call last):\n File \"/usr/bin/vdsm-tool\", line 219, in main\n return tool_command[cmd][\"command\"](*args)\n File \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line 58, in ovn_config\n ip_address = get_ip_addr(get_network(network_caps(), net_name))\n File \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line 79, in get_network\n raise NetworkNotFoundError(net_name)\nNetworkNotFoundError: ovirtmgmt", "stderr_lines": ["Traceback (most recent call last):", " File \"/usr/bin/vdsm-tool\", line 219, in main", " return tool_command[cmd][\"command\"](*args)", " File \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line 58, in ovn_config", " ip_address = get_ip_addr(get_network(network_caps(), net_name))", " File \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line 79, in get_network", " raise NetworkNotFoundError(net_name)", "NetworkNotFoundError: ovirtmgmt"], "stdout": "", "stdout_lines": []} Meaning, 'ovirtmgmt' is missing. In host-deploy [3] of this host, I see that the engine asked host-deploy to configure ovirtmgmt: 2017-10-01 03:14:36,036-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ### Customization phase, use 'install' to proceed 2017-10-01 03:14:36,036-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ### COMMAND> 2017-10-01 03:14:36,036-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%QHidden: FALSE 2017-10-01 03:14:36,037-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ***Q:STRING CUSTOMIZATION_COMMAND 2017-10-01 03:14:36,037-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%QEnd: CUSTOMIZATION_COMMAND 2017-10-01 03:14:36,038-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:RECEIVE env-query -k HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,038-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%QStart: HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,039-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ### 2017-10-01 03:14:36,039-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ### Please specify value for 'HOSTED_ENGINE_CONFIG/bridge': 2017-10-01 03:14:36,039-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ### Response is VALUE HOSTED_ENGINE_CONFIG/bridge=type:value or ABORT HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,039-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND ***Q:VALUE HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,039-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%QEnd: HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,040-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:RECEIVE VALUE HOSTED_ENGINE_CONFIG/bridge=str:ovirtmgmt 2017-10-01 03:14:36,040-0400 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%QStart: CUSTOMIZATION_COMMAND However, AFAIU this was deprecated quite a long time ago [4], and does not appear in other runs, e.g. the latest succeeded run of the same job [5][6]. Also, I can't see in engine log there [7] how ovirtmgmt was created (if at all). I now see a difference between them: The successful one [7] passed to ansible: --extra-vars=host_deploy_ovn_central=null And the failed one [8] passed: --extra-vars=host_deploy_ovn_central=192.168.200.99 So perhaps this was caused by a recent change related to OVN? Although I looked at engine git log and can't find anything.

Thanks, [1] https://gerrit.ovirt.org/82403 [2] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... [3] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... [4] https://gerrit.ovirt.org/70833 [5] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... [6] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... [7] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... [8] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > To view, visit change 82403. To unsubscribe, visit settings. > > Gerrit-Project: ovirt-system-tests > Gerrit-Branch: master > Gerrit-MessageType: comment > Gerrit-Change-Id: I46f453c7e86ce0e3a7282aca4d56f23363d1683b > Gerrit-Change-Number: 82403 > Gerrit-PatchSet: 1 > Gerrit-Owner: Yedidyah Bar David <didi(a)redhat.com&gt; > Gerrit-Reviewer: Jenkins CI > Gerrit-Comment-Date: Sun, 01 Oct 2017 07:15:31 +0000 > Gerrit-HasComments: No -- Didi

My point if view is the following, please point me to weak points. The error says, that the network ovirtmgmt is expected, but not found on host1. As far as I understand the scenario, this error message is wanted, because ovirtmgmt should available be on host1, because: host1 is in cluster Default: [root@hc-engine ~]# su - postgres -c "psql -U postgres engine -c 'select vds_name,cluster_name,cluster_id from vds;'" vds_name | cluster_name | cluster_id ----------------------------------+--------------+-------------------------------------- lago-hc-basic-suite-master-host2 | Default | 59d1f257-012b-01e6-0019-000000000133 lago-hc-basic-suite-master-host1 | Default | 59d1f257-012b-01e6-0019-000000000133 lago_basic_suite_hc_host0 | Default | 59d1f257-012b-01e6-0019-000000000133 (3 rows) [root@hc-engine ~]# su - postgres -c "psql -U postgres engine -c 'select id,name from network;'" id | name --------------------------------------+----------- 00000000-0000-0000-0000-000000000009 | ovirtmgmt (1 row) ovirtmgmt with id 9 is required and management in cluster Default: [root@hc-engine ~]# su - postgres -c "psql -U postgres engine -c 'select network_id,cluster_id,status,required,management from network_cluster;'" network_id | cluster_id | status | required | management --------------------------------------+--------------------------------------+--------+----------+------------ 00000000-0000-0000-0000-000000000009 | 59d1f257-012b-01e6-0019-000000000133 | 1 | t | t (1 row) but ovirtmgmt is not available on host1: [root@hc-engine ~]# ssh lago-hc-basic-suite-master-host1 "python -c \"from vdsm.network.api import network_caps; print network_caps()['networks']\"" root@lago-hc-basic-suite-master-host1's password: {} If this argumentation would be correct, the next step would be to analyze why ovirtmgmt is not available on host1. On Sun, 1 Oct 2017 16:48:14 +0300 Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > On Sun, Oct 1, 2017 at 12:19 PM, Yedidyah Bar David > <didi(a)redhat.com&gt; wrote: > > Hi all, > > > > On Sun, Oct 1, 2017 at 10:15 AM, Code Review <gerrit(a)ovirt.org&gt; > > wrote: > >> Jenkins CI posted comments on this change. > >> > >> View Change > >> > >> Patch set 1:Continuous-Integration -1 > >> > >> Build Failed > >> > >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > >> : FAILURE > > > > Above was triggered by [1]. > > > > It failed with [2]: > > > > TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] > > ********************* fatal: [lago-he-basic-suite-master-host1]: > > FAILED! => {"changed": true, "cmd": ["vdsm-tool", "ovn-config", > > "192.168.200.99", "ovirtmgmt"], "delta": "0:00:00.623565", "end": > > "2017-10-01 03:15:11.179717", "failed": true, "msg": "non-zero > > return code", "rc": 1, "start": "2017-10-01 03:15:10.556152", > > "stderr": "Traceback (most recent call last):\n File > > \"/usr/bin/vdsm-tool\", line 219, in main\n return > > tool_command[cmd][\"command\"](*args)\n File > > \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line > > 58, in ovn_config\n ip_address = > > get_ip_addr(get_network(network_caps(), net_name))\n File > > \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line > > 79, in get_network\n raise > > NetworkNotFoundError(net_name)\nNetworkNotFoundError: ovirtmgmt", > > "stderr_lines": ["Traceback (most recent call last):", " File > > \"/usr/bin/vdsm-tool\", line 219, in main", " return > > tool_command[cmd][\"command\"](*args)", " File > > \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line > > 58, in ovn_config", " ip_address = > > get_ip_addr(get_network(network_caps(), net_name))", " File > > \"/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py\", line > > 79, in get_network", " raise NetworkNotFoundError(net_name)", > > "NetworkNotFoundError: ovirtmgmt"], "stdout": "", "stdout_lines": > > []} > > > > Meaning, 'ovirtmgmt' is missing. > > In host-deploy [3] of this host, I see that the engine asked > > host-deploy to configure ovirtmgmt: > > > > 2017-10-01 03:14:36,036-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ### Customization phase, use 'install' to > > proceed 2017-10-01 03:14:36,036-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ### COMMAND> 2017-10-01 03:14:36,036-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND **%QHidden: FALSE 2017-10-01 03:14:36,037-0400 > > DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ***Q:STRING CUSTOMIZATION_COMMAND > > 2017-10-01 03:14:36,037-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND **%QEnd: CUSTOMIZATION_COMMAND > > 2017-10-01 03:14:36,038-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:RECEIVE env-query -k HOSTED_ENGINE_CONFIG/bridge > > 2017-10-01 03:14:36,038-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND **%QStart: HOSTED_ENGINE_CONFIG/bridge > > 2017-10-01 03:14:36,039-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ### 2017-10-01 03:14:36,039-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ### Please specify value for > > 'HOSTED_ENGINE_CONFIG/bridge': 2017-10-01 03:14:36,039-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ### Response is VALUE > > HOSTED_ENGINE_CONFIG/bridge=type:value or ABORT > > HOSTED_ENGINE_CONFIG/bridge 2017-10-01 03:14:36,039-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND ***Q:VALUE HOSTED_ENGINE_CONFIG/bridge > > 2017-10-01 03:14:36,039-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND **%QEnd: HOSTED_ENGINE_CONFIG/bridge > > 2017-10-01 03:14:36,040-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:RECEIVE VALUE HOSTED_ENGINE_CONFIG/bridge=str:ovirtmgmt > > 2017-10-01 03:14:36,040-0400 DEBUG > > otopi.plugins.otopi.dialog.machine dialog.__logString:204 > > DIALOG:SEND **%QStart: CUSTOMIZATION_COMMAND > > > > However, AFAIU this was deprecated quite a long time ago [4], and > > does not appear in other runs, e.g. the latest succeeded run of > > the same job [5][6]. Also, I can't see in engine log there [7] how > > ovirtmgmt was created (if at all). > > > > I now see a difference between them: > > > > The successful one [7] passed to ansible: > > > > --extra-vars=host_deploy_ovn_central=null > > > > And the failed one [8] passed: > > > > --extra-vars=host_deploy_ovn_central=192.168.200.99 > > > > So perhaps this was caused by a recent change related to OVN? > > Although I looked at engine git log and can't find anything. > > Many of project:ovirt-engine branch:master > topic:"ovirt-provider-ovn-driver deployment" have been merged have > been merged already (but kept check-patch very green). I hope > Dominik can read more into it. > > > > Thanks, > > > > [1] https://gerrit.ovirt.org/82403 > > > > [2] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > > > [3] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > > > [4] https://gerrit.ovirt.org/70833 > > > > [5] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > > > [6] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > > > [7] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > > > > [8] > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x8... > >> > >> To view, visit change 82403. To unsubscribe, visit settings. > >> > >> Gerrit-Project: ovirt-system-tests > >> Gerrit-Branch: master > >> Gerrit-MessageType: comment > >> Gerrit-Change-Id: I46f453c7e86ce0e3a7282aca4d56f23363d1683b > >> Gerrit-Change-Number: 82403 > >> Gerrit-PatchSet: 1 > >> Gerrit-Owner: Yedidyah Bar David <didi(a)redhat.com&gt; > >> Gerrit-Reviewer: Jenkins CI > >> Gerrit-Comment-Date: Sun, 01 Oct 2017 07:15:31 +0000 > >> Gerrit-HasComments: No > > > > > > > > -- > > Didi