OST HE fails due to empty CPU type (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master - Build # 2038 - Still Failing!)
Hi all, On Tue, Jun 1, 2021 at 5:23 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/
This has been failing for a week now. Not sure about the root cause. From HE deploy code POV: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/... : 2021-06-01 04:22:22,497+0200 DEBUG var changed: host "localhost" var "cluster_facts" type "<class 'dict'>" value: "{ "changed": false, "failed": false, "ovirt_clusters": [ { "affinity_groups": [], "ballooning_enabled": true, "comment": "", "cpu": { "architecture": "undefined", "type": "" }, Meaning, the engine says that cluster Default's cpu type is "". The code uses this value as-is, and a few tasks later fails in: 2021-06-01 04:22:26,815+0200 DEBUG ansible on_any args TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs is_conditional:False 2021-06-01 04:22:26,816+0200 DEBUG ansible on_any args localhost TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,975+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute ''\n\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/create_target_vm/01_create_target_hosted_engine_vm.yml': line 64, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n {{ server_cpu_list['ovirt_system_option']['values'][0]['value'].split('; ')|list|difference(['']) }}\n - name: Convert CPU model name\n ^ here\n" }, "ansible_task": "Convert CPU model name", "ansible_type": "task", "status": "FAILED", "task_duration": 0 } Any ideas? Thanks and best regards, -- Didi
Hi, On 6/1/21 8:25 AM, Yedidyah Bar David wrote:
Hi all,
On Tue, Jun 1, 2021 at 5:23 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/ This has been failing for a week now. Not sure about the root cause. There's a bug for this [1] Yesterday I pushed a workaround to ost-images for this problem [2], so if you update images you should be good.
Regards, Marcin [1] https://bugzilla.redhat.com/show_bug.cgi?id=1961558 [2] https://gerrit.ovirt.org/#/c/ost-images/+/115002/
From HE deploy code POV:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/... :
2021-06-01 04:22:22,497+0200 DEBUG var changed: host "localhost" var "cluster_facts" type "<class 'dict'>" value: "{ "changed": false, "failed": false, "ovirt_clusters": [ { "affinity_groups": [], "ballooning_enabled": true, "comment": "", "cpu": { "architecture": "undefined", "type": "" },
Meaning, the engine says that cluster Default's cpu type is "". The code uses this value as-is, and a few tasks later fails in:
2021-06-01 04:22:26,815+0200 DEBUG ansible on_any args TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs is_conditional:False 2021-06-01 04:22:26,816+0200 DEBUG ansible on_any args localhost TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,975+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute ''\n\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/create_target_vm/01_create_target_hosted_engine_vm.yml': line 64, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n {{ server_cpu_list['ovirt_system_option']['values'][0]['value'].split('; ')|list|difference(['']) }}\n - name: Convert CPU model name\n ^ here\n" }, "ansible_task": "Convert CPU model name", "ansible_type": "task", "status": "FAILED", "task_duration": 0 }
Any ideas?
Thanks and best regards,
On Tue, Jun 1, 2021 at 11:33 AM Marcin Sobczyk <msobczyk@redhat.com> wrote:
Hi,
On 6/1/21 8:25 AM, Yedidyah Bar David wrote:
Hi all,
On Tue, Jun 1, 2021 at 5:23 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/ This has been failing for a week now. Not sure about the root cause. There's a bug for this [1] Yesterday I pushed a workaround to ost-images for this problem [2], so if you update images you should be good.
Can't tell if CI updates them, we do not log their version AFAICT. It still fails, though. 2039 failed IIRC for the same reason, 2040 failed due to some other weird reason (seems to have failed to start host-0), and so I manually rebuilt it and it failed again on empty cpu type. rpm log (on the host) does show that edk2-ovmf was installed, and based on the timestamps of all lines from beginning to edk2-ovmf it seems to have been part of initial creation, not OST updates/installs: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2041/... Best regards,
Regards, Marcin
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1961558 [2] https://gerrit.ovirt.org/#/c/ost-images/+/115002/
From HE deploy code POV:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/2038/... :
2021-06-01 04:22:22,497+0200 DEBUG var changed: host "localhost" var "cluster_facts" type "<class 'dict'>" value: "{ "changed": false, "failed": false, "ovirt_clusters": [ { "affinity_groups": [], "ballooning_enabled": true, "comment": "", "cpu": { "architecture": "undefined", "type": "" },
Meaning, the engine says that cluster Default's cpu type is "". The code uses this value as-is, and a few tasks later fails in:
2021-06-01 04:22:26,815+0200 DEBUG ansible on_any args TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs is_conditional:False 2021-06-01 04:22:26,816+0200 DEBUG ansible on_any args localhost TASK: ovirt.ovirt.hosted_engine_setup : Convert CPU model name kwargs 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,974+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]" 2021-06-01 04:22:26,975+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute ''\n\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/create_target_vm/01_create_target_hosted_engine_vm.yml': line 64, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n {{ server_cpu_list['ovirt_system_option']['values'][0]['value'].split('; ')|list|difference(['']) }}\n - name: Convert CPU model name\n ^ here\n" }, "ansible_task": "Convert CPU model name", "ansible_type": "task", "status": "FAILED", "task_duration": 0 }
Any ideas?
Thanks and best regards,
-- Didi
участники (2)
-
Marcin Sobczyk -
Yedidyah Bar David