Hi,

my bet is on the issue with ansible-runner please try to install new runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and restart service (`systemctl restart ansible-runner-service.service`).
if it won't help please use `journalctl -u ansible-runner-service.service -f` to gather info from the runner.
The log which is mentioned in engine.log is not created in dev env everything is logged to journal.

Martin Necas 

On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina@redhat.com> wrote:


On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
HI Martin,
Below error after rebasing you patch
2020-03-27 18:34:24,996+05 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) [] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
2020-03-27 18:34:25,771+05 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute call to start playbook.
2020-03-27 18:34:25,771+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create brick role. Please check logs for more details: /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
2020-03-27 18:34:25,772+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed: EngineException: Failed to execute Ansible create brick role: 'Failed to execute call to start playbook. '. Please check logs for more details: /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log (Failed with error GeneralException and code 100)
2020-03-27 18:34:25,852+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27 18:34:25,863+05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID: CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host 10.70.35.17 of cluster Default.

Dana/Martin, any idea about the issue?

On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina@redhat.com> wrote:


On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
Hi,
Let me know if you require something else
journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB

Hmm, could you please try to reproduce with below patch, maybe there will be more information in stderr:


Thanks,
Martin

On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina@redhat.com> wrote:
Could you please share journal and engine.log? We need to correlate and find out which error in journal is correlated to this issue?

Thanks,
Martin


On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
Hi Artur,
I changed the files mentioned in the mail 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6UYOYRHNLKIUQG54DPNPJ2D/

but unfortunately it is not the same error as described in the mail and changing it gives me the same error.

On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha@redhat.com> wrote:
Hi Kaustav,
You might be hitting a known issue.
Please check the solution posted by Dana on this list. Look for title: '[ovirt-devel] workaround for host deploy failure (get_events)'
Artur


On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <kmajumde@redhat.com> wrote:
Hi all,
I am trying to run create-brick playbook from ansible-runner integrated inside ovirt-engine but it throws errors as
```
Running command: CreateBrickCommand internal: false. Entities affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create brick role. Please check logs for more details: /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log 2020-03-27 15:43:07,519+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed: EngineException: Failed to execute Ansible create brick role. Please check logs for more details: /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log (Failed with error GeneralException and code 100) 2020-03-27 15:43:07,526+05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27 15:43:07,530+05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID: CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host 10.70.35.17 of cluster Default. ```
The corresponding log is not created
kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log /home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log: No such file or directory
checking ansible-runner status gives
[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ± systemctl status ansible-runner-service.service
● ansible-runner-service.service - Ansible Runner Service
   Loaded: loaded (/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2020-03-27 15:42:56 IST; 8min ago
 Main PID: 30097 (gunicorn-3)
    Tasks: 3 (limit: 4915)
   Memory: 62.1M
   CGroup: /system.slice/ansible-runner-service.service
           ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application
           ├─30112 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application
           └─30123 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application

Mar 27 15:42:58 localhost.localdomain gunicorn-3[30097]: Request received, content-type :None
Mar 27 15:42:58 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET /api/v1/hosts/10.70.35.1
Mar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: Request received, content-type :None
Mar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET /api/v1/hosts/10.70.35.1
Mar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: Request received, content-type :None
Mar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET /api/v1/hosts/10.70.35.1
Mar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: Request received, content-type :None
Mar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET /api/v1/hosts/10.70.35.17
Mar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: Request received, content-type :application/json; charset=UTF-8
Mar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml

Is there a way to find the root cause for this?
--

Thanks,

Kaustav Majumder

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBSTFAHZ55ZV3TA7YALWBNNTA/


--

Artur Socha

Senior Software Engineer, RHV

Red Hat



--

Thanks,

Kaustav Majumder

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NMB5MK3QEZEU5BQ2MBQVESP/


--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.


--

Thanks,

Kaustav Majumder



--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.


--

Thanks,

Kaustav Majumder



--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.