On Sun, Jun 28, 2020 at 3:01 PM Yedidyah Bar David <didi@redhat.com> wrote:

On Sun, Jun 28, 2020 at 2:51 PM Nir Soffer <nsoffer@redhat.com> wrote:
>
> On Sun, Jun 28, 2020 at 2:35 PM Yedidyah Bar David <didi@redhat.com> wrote:
> >
> > On Sun, Jun 28, 2020 at 1:37 PM Nir Soffer <nsoffer@redhat.com> wrote:
> > >
> > > On Sun, Jun 28, 2020 at 1:23 PM Yedidyah Bar David <didi@redhat.com> wrote:
> > > >
> > > > On Sun, Jun 28, 2020 at 6:23 AM <jenkins@jenkins.phx.ovirt.org> wrote:
> > > > >
> > > > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/
> > > > > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1655/
> > > >
> > > > This fails for some time now. Checked last one ^^, and it failed in:
> > > >
> > > > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1655/artifact/exported-artifacts/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-create_target_vm-20200627230157-7lqrnp.log
> > > >
> > > > 2020-06-27 23:05:02,253-0400 INFO ansible task start {'status': 'OK',
> > > > 'ansible_type': 'task', 'ansible_playbook':
> > > > '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml',
> > > > 'ansible_task': 'ovirt.hosted_engine_setup : Check OVF_STORE volume
> > > > status'}
> > > > 2020-06-27 23:05:02,253-0400 DEBUG ansible on_any args TASK:
> > > > ovirt.hosted_engine_setup : Check OVF_STORE volume status kwargs
> > > > is_conditional:False
> > > > 2020-06-27 23:05:02,254-0400 DEBUG ansible on_any args localhostTASK:
> > > > ovirt.hosted_engine_setup : Check OVF_STORE volume status kwargs
> > > > 2020-06-27 23:05:03,816-0400 DEBUG ansible on_any args
> > > > <ansible.executor.task_result.TaskResult object at 0x7f454e3d1eb8>
> > > > kwargs
> > > > ...
> > > >
> > > > 2020-06-27 23:09:39,166-0400 DEBUG var changed: host "localhost" var
> > > > "ovf_store_status" type "<class 'dict'>" value: "{
> > > > "changed": true,
> > > > "failed": true,
> > > > "msg": "All items completed",
> > > > "results": [
> > > > {
> > > > "ansible_loop_var": "item",
> > > > "attempts": 12,
> > > >
> > > > Meaning, it ran 12 times the command:
> > > >
> > > > vdsm-client Volume getInfo
> > > > storagepoolID=41c9fdea-b8e9-11ea-ae2a-5452c0a8c863
> > > > storagedomainID=10a69775-8fb6-437d-9e78-2ecfd77c0a45
> > > > imageID=c2ad2065-1c8b-4ec1-afdd-f7cefc708cf9
> > > > volumeID=6b835f55-a512-4f83-9d25-f6837d8b5cb1
> > > >
> > > > and never got a result with output including "Updated". Any idea?
> > >
> > > I don't know what is the meaning of "Updated".
> >
> > Sorry, meant Updated:true. This is a snippet from above
> > ansible-create_target_vm log:
> >
> > "cmd": [
> > "vdsm-client",
> > "Volume",
> > "getInfo",
> > "storagepoolID=41c9fdea-b8e9-11ea-ae2a-5452c0a8c863",
> > "storagedomainID=10a69775-8fb6-437d-9e78-2ecfd77c0a45",
> > "imageID=c2ad2065-1c8b-4ec1-afdd-f7cefc708cf9",
> > "volumeID=6b835f55-a512-4f83-9d25-f6837d8b5cb1"
> > ],
> > ...
> >
> > "stdout_lines": [
> > "{",
> > " \"apparentsize\": \"134217728\",",
> > " \"capacity\": \"134217728\",",
> > " \"children\": [],",
> > " \"ctime\": \"1593313323\",",
> > " \"description\":
> > \"{\\\"Updated\\\":false,\\\"Last Updated\\\":null,\\\"Storage
> > Domains\\\":[{\\\"uuid\\\":\\\"10a69775-8fb6-437d-9e78-2ecfd77c0a45\\\"}],\\\"Disk
> > Description\\\":\\\"OVF_STORE\\\"}\",",
> > " \"disktype\": \"OVFS\",",
> > " \"domain\": \"10a69775-8fb6-437d-9e78-2ecfd77c0a45\",",
> > " \"format\": \"RAW\",",
> > " \"generation\": 0,",
> > " \"image\": \"c2ad2065-1c8b-4ec1-afdd-f7cefc708cf9\",",
> > " \"lease\": {",
> > " \"offset\": 0,",
> > " \"owners\": [],",
> > " \"path\":
> > \"/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/10a69775-8fb6-437d-9e78-2ecfd77c0a45/images/c2ad2065-1c8b-4ec1-afdd-f7cefc708cf9/6b835f55-a512-4f83-9d25-f6837d8b5cb1.lease\",",
> > " \"version\": null",
> > " },",
> > " \"legality\": \"LEGAL\",",
> > " \"mtime\": \"0\",",
> > " \"parent\": \"00000000-0000-0000-0000-000000000000\",",
> > " \"pool\": \"\",",
> > " \"status\": \"OK\",",
> > " \"truesize\": \"134217728\",",
> > " \"type\": \"PREALLOCATED\",",
> > " \"uuid\": \"6b835f55-a512-4f83-9d25-f6837d8b5cb1\",",
> > " \"voltype\": \"LEAF\"",
> > "}"
> > ]
> >
> > As you can see, it includes: " \"description\": \"{\\\"Updated\\\":false,
> >
> > Compare with a successful run:
> >
> > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1632/
> >
> > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1632/artifact/exported-artifacts/test_logs/he-basic-suite-master/post-012_local_maintenance_sdk.py/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-create_target_vm-20200610225033-lfebht.log
> >
> > "stdout_lines": [
> > "{",
> > " \"apparentsize\": \"23552\",",
> > " \"capacity\": \"134217728\",",
> > " \"children\": [],",
> > " \"ctime\": \"1591843833\",",
> > " \"description\":
> > \"{\\\"Updated\\\":true,\\\"Size\\\":23552,\\\"Last
> > Updated\\\":\\\"Wed Jun 10 22:53:27 EDT 2020\\\",\\\"Storage
> > Domains\\\":[{\\\"uuid\\\":\\\"adf90c1d-52b1-4440-8523-4759498fb055\\\"}],\\\"Disk
> > Description\\\":\\\"OVF_STORE\\\"}\",",
> > " \"disktype\": \"OVFS\",",
> > " \"domain\": \"adf90c1d-52b1-4440-8523-4759498fb055\",",
> > " \"format\": \"RAW\",",
> > " \"generation\": 0,",
> > " \"image\": \"a9982674-a297-4a40-8649-5539b675cb86\",",
> > " \"lease\": {",
> > " \"offset\": 0,",
> > " \"owners\": [],",
> > " \"path\":
> > \"/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/adf90c1d-52b1-4440-8523-4759498fb055/images/a9982674-a297-4a40-8649-5539b675cb86/47fca7a4-f5bc-4873-8660-430449004b8f.lease\",",
> > " \"version\": null",
> > " },",
> > " \"legality\": \"LEGAL\",",
> > " \"mtime\": \"0\",",
> > " \"parent\": \"00000000-0000-0000-0000-000000000000\",",
> > " \"pool\": \"\",",
> > " \"status\": \"OK\",",
> > " \"truesize\": \"24576\",",
> > " \"type\": \"PREALLOCATED\",",
> > " \"uuid\": \"47fca7a4-f5bc-4873-8660-430449004b8f\",",
> > " \"voltype\": \"LEAF\"",
> > "}"
> > ]
> >
> > >
> > > When you call the vdsm API you should get a response. If you don't get
> > > any response maybe vdsm is blocked on inaccessible storage.
> >
> > I do get a response, but it never "Finishes", or at least according to the
> > login we have for testing this - we require both 'status=OK' and 'Updated:true'.
> >
> > >
> > > Or maybe the OVF volume was not created yet and you get an error
> > > that the volume does not exist.
> >
> > It does exist. We also get e.g.: status:OK.
> >
> > What is the meaning of: Updated: False, Last Updated: null?
> >
> > Is it wrong for checking these?
>
> I don't know, this info is generated by engine. Vdsm does not care about the
> content of the description field.

For reference, it was added here:

https://gerrit.ovirt.org/88650

>
> This is also not a public API, so if you check this you depend on engine private
> implementation details.
>
> Generally using HTTP APIs in vdsm for reading OVF data is abuse of private
> APIs that were added for engine and should be used only by engine. We are
> missing a real API for passing configuration updates from hosted
> engine to hosted
> engine agent running on the host. Instead we read OVF_STORE volume or
> engine configuration volume to extract data.
>
> Eyal, did we change anything related to updating OVF_STORE recently?
>
> > > > Perhaps it's just a bug in the test, and we should test something else
> > > > other than "Updated"?
> > > >
> > > > Or perhaps there was some other problem but I failed to find it?
> > > >
> > > > Thanks,
> > > >
> > > > > Build Number: 1655
> > > > > Build Status: Still Failing
> > > > > Triggered By: Started by timer
> > > > >
> > > > > -------------------------------------
> > > > > Changes Since Last Success:
> > > > > -------------------------------------
> > > > > Changes for Build #1643
> > > > > [Galit] Add missing repo which has collectd-write_syslog
> > > > >
> > > > > [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes"
> > > > >
> > > > >
> > > > > Changes for Build #1644
> > > > > [Michal Skrivanek] make GLANCE failures fatal
> > > > >
> > > > >
> > > > > Changes for Build #1645
> > > > > [Michal Skrivanek] use // in glance URL
> > > > >
> > > > >
> > > > > Changes for Build #1646
> > > > > [Galit] Fix: No module named ost_utils.memoized
> > > > >
> > > > >
> > > > > Changes for Build #1647
> > > > > [Martin Necas] ansible: fix deploy scripts to el8
> > > > >
> > > > > [arachmani] Add arachman@redhat.com to jenkins recipient lists
> > > > >
> > > > >
> > > > > Changes for Build #1648
> > > > > [Martin Necas] ansible: fix deploy scripts to el8
> > > > >
> > > > > [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29
> > > > >
> > > > >
> > > > > Changes for Build #1649
> > > > > [Martin Necas] ansible: fix deploy scripts to el8
> > > > >
> > > > >
> > > > > Changes for Build #1650
> > > > > [Martin Necas] ansible: fix deploy scripts to el8
> > > > >
> > > > >
> > > > > Changes for Build #1651
> > > > > [Galit] Add centos8.2 official to templates
> > > > >
> > > > >
> > > > > Changes for Build #1652
> > > > > [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run
> > > > >
> > > > > [Gal Ben Haim] Add gcc to fcraw
> > > > >
> > > > > [Sandro Bonazzola] pipelines: add ovirt-dependencies
> > > > >
> > > > >
> > > > > Changes for Build #1653
> > > > > [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run
> > > > >
> > > > >
> > > > > Changes for Build #1654
> > > > > [Galit] Add centos 8.2 image: el8.2-base
> > > > >
> > > > > [Evgheni Dereveanchin] do not expand values in run_oc_playbook
> > > > >
> > > > >
> > > > > Changes for Build #1655
> > > > > [Galit] Add centos 8.2 image: el8.2-base
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -----------------
> > > > > Failed Tests:
> > > > > -----------------
> > > > > No tests ran.
> > > >
> > > >
> > > >
> > > > --
> > > > Didi
> > > >
> > >
> >
> >
> > --
> > Didi
> >
>

--
Didi
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WJW2GDRUERSXLD3XSOKN6WRV7MXC2SKU/