5:29 a.m.
Hi all,
I am trying to run create-brick playbook from ansible-runner integrated
inside ovirt-engine but it throws errors as
```
Running command: CreateBrickCommand internal: false. Entities affected :
ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
brick role. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
2020-03-27 15:43:07,519+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
EngineException: Failed to execute Ansible create brick role. Please check
logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
(Failed with error GeneralException and code 100) 2020-03-27
15:43:07,526+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
15:43:07,530+05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
10.70.35.17 of cluster Default. ```
*The corresponding log is not created*
kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
No such file or directory
*checking ansible-runner status gives*
*[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
systemctl status ansible-runner-service.service ●
ansible-runner-service.service - Ansible Runner Service Loaded: loaded
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
preset: disabled) Active: active (running) since Fri 2020-03-27 15:42:56
IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:application ├─30112 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:application └─30123 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
/api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
*Is there a way to find the root cause for this?*
--
Thanks,
Kaustav Majumder
6:15 a.m.
Hi Kaustav,
You might be hitting a known issue.
Please check the solution posted by Dana on this list. Look for title:
'[ovirt-devel] workaround for host deploy failure (get_events)'
Artur
On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
Hi all,
I am trying to run create-brick playbook from ansible-runner integrated
inside ovirt-engine but it throws errors as
```
Running command: CreateBrickCommand internal: false. Entities affected :
ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
brick role. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
2020-03-27 15:43:07,519+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
EngineException: Failed to execute Ansible create brick role. Please check
logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
(Failed with error GeneralException and code 100) 2020-03-27
15:43:07,526+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
15:43:07,530+05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
10.70.35.17 of cluster Default. ```
*The corresponding log is not created*
kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
No such file or directory
*checking ansible-runner status gives*
*[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
systemctl status ansible-runner-service.service ●
ansible-runner-service.service - Ansible Runner Service Loaded: loaded
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
preset: disabled) Active: active (running) since Fri 2020-03-27 15:42:56
IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:application ├─30112 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:application └─30123 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
/api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
localhost.localdomain gunicorn-3[30097]: Request received, content-type
:application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
*Is there a way to find the root cause for this?*
--
Thanks,
Kaustav Majumder
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
--
Artur Socha
Senior Software Engineer, RHV
Red Hat <https://www.redhat.com>
<https://www.redhat.com>
6:31 a.m.
Hi Artur,
I changed the files mentioned in the mail
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
but unfortunately it is not the same error as described in the mail and
changing it gives me the same error.
On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com> wrote:
Hi Kaustav,
You might be hitting a known issue.
Please check the solution posted by Dana on this list. Look for title:
'[ovirt-devel] workaround for host deploy failure (get_events)'
Artur
On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
> Hi all,
> I am trying to run create-brick playbook from ansible-runner integrated
> inside ovirt-engine but it throws errors as
> ```
> Running command: CreateBrickCommand internal: false. Entities affected :
> ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
> brick role. Please check logs for more details:
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
> 2020-03-27 15:43:07,519+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
> EngineException: Failed to execute Ansible create brick role. Please check
> logs for more details:
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
> (Failed with error GeneralException and code 100) 2020-03-27
> 15:43:07,526+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command
> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
> 15:43:07,530+05 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
> 10.70.35.17 of cluster Default. ```
> *The corresponding log is not created*
> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
> No such file or directory
> *checking ansible-runner status gives*
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
> systemctl status ansible-runner-service.service ●
> ansible-runner-service.service - Ansible Runner Service Loaded: loaded
> (/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
> preset: disabled) Active: active (running) since Fri 2020-03-27 15:42:56
> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
> Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2
> runner_service.wsgi:application ├─30112 /usr/bin/python3
> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
> runner_service.wsgi:application └─30123 /usr/bin/python3
> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
> runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
> gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
> localhost.localdomain gunicorn-3[30097]: Request received, content-type
> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
> localhost.localdomain gunicorn-3[30097]: Request received, content-type
> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
> localhost.localdomain gunicorn-3[30097]: Request received, content-type
> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
> localhost.localdomain gunicorn-3[30097]: Request received, content-type
> :application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
> gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
>
> *Is there a way to find the root cause for this?*
> --
>
> Thanks,
>
> Kaustav Majumder
> _______________________________________________
> Devel mailing list -- devel(a)ovirt.org
> To unsubscribe send an email to devel-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>
--
Artur Socha
Senior Software Engineer, RHV
Red Hat <https://www.redhat.com>
<https://www.redhat.com>
--
Thanks,
Kaustav Majumder
6:36 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Could you please share journal and engine.log? We need to correlate and
find out which error in journal is correlated to this issue?
Thanks,
Martin
On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
Hi Artur,
I changed the files mentioned in the mail
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
but unfortunately it is not the same error as described in the mail and
changing it gives me the same error.
On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com> wrote:
> Hi Kaustav,
> You might be hitting a known issue.
> Please check the solution posted by Dana on this list. Look for title:
> '[ovirt-devel] workaround for host deploy failure (get_events)'
> Artur
>
> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <kmajumde(a)redhat.com>
> wrote:
>
>> Hi all,
>> I am trying to run create-brick playbook from ansible-runner integrated
>> inside ovirt-engine but it throws errors as
>> ```
>> Running command: CreateBrickCommand internal: false. Entities affected :
>> ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
>> brick role. Please check logs for more details:
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>> 2020-03-27 15:43:07,519+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>> EngineException: Failed to execute Ansible create brick role. Please check
>> logs for more details:
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>> (Failed with error GeneralException and code 100) 2020-03-27
>> 15:43:07,526+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command
>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>> 15:43:07,530+05 ERROR
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
>> 10.70.35.17 of cluster Default. ```
>> *The corresponding log is not created*
>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>> No such file or directory
>> *checking ansible-runner status gives*
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
>> systemctl status ansible-runner-service.service ●
>> ansible-runner-service.service - Ansible Runner Service Loaded: loaded
>> (/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>> preset: disabled) Active: active (running) since Fri 2020-03-27 15:42:56
>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
>> Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>> runner_service.wsgi:application ├─30112 /usr/bin/python3
>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>> runner_service.wsgi:application └─30123 /usr/bin/python3
>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>> runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
>> gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>> :application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
>> gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
>>
>> *Is there a way to find the root cause for this?*
>> --
>>
>> Thanks,
>>
>> Kaustav Majumder
>> _______________________________________________
>> Devel mailing list -- devel(a)ovirt.org
>> To unsubscribe send an email to devel-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>
>
>
> --
>
> Artur Socha
>
> Senior Software Engineer, RHV
>
> Red Hat <https://www.redhat.com>
> <https://www.redhat.com>
>
--
Thanks,
Kaustav Majumder
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
6:44 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Hi,
Let me know if you require something else
journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
engine.log - > https://pastebin.com/bVv7FxLB
On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com> wrote:
Could you please share journal and engine.log? We need to correlate
and
find out which error in journal is correlated to this issue?
Thanks,
Martin
On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
> Hi Artur,
> I changed the files mentioned in the mail
>
>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>
> but unfortunately it is not the same error as described in the mail and
> changing it gives me the same error.
>
> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com> wrote:
>
>> Hi Kaustav,
>> You might be hitting a known issue.
>> Please check the solution posted by Dana on this list. Look for title:
>> '[ovirt-devel] workaround for host deploy failure (get_events)'
>> Artur
>>
>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <kmajumde(a)redhat.com>
>> wrote:
>>
>>> Hi all,
>>> I am trying to run create-brick playbook from ansible-runner integrated
>>> inside ovirt-engine but it throws errors as
>>> ```
>>> Running command: CreateBrickCommand internal: false. Entities affected
>>> : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>>> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>>> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
>>> brick role. Please check logs for more details:
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>> 2020-03-27 15:43:07,519+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>> EngineException: Failed to execute Ansible create brick role. Please check
>>> logs for more details:
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>> (Failed with error GeneralException and code 100) 2020-03-27
>>> 15:43:07,526+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for command
>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>> 15:43:07,530+05 ERROR
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
>>> 10.70.35.17 of cluster Default. ```
>>> *The corresponding log is not created*
>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>> No such file or directory
>>> *checking ansible-runner status gives*
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
>>> systemctl status ansible-runner-service.service ●
>>> ansible-runner-service.service - Ansible Runner Service Loaded: loaded
>>> (/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>>> preset: disabled) Active: active (running) since Fri 2020-03-27 15:42:56
>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
>>> Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>> runner_service.wsgi:application ├─30112 /usr/bin/python3
>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>> runner_service.wsgi:application └─30123 /usr/bin/python3
>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>> runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
>>> gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>> :application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
>>> gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
>>>
>>> *Is there a way to find the root cause for this?*
>>> --
>>>
>>> Thanks,
>>>
>>> Kaustav Majumder
>>> _______________________________________________
>>> Devel mailing list -- devel(a)ovirt.org
>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>
>>
>>
>> --
>>
>> Artur Socha
>>
>> Senior Software Engineer, RHV
>>
>> Red Hat <https://www.redhat.com>
>> <https://www.redhat.com>
>>
>
>
> --
>
> Thanks,
>
> Kaustav Majumder
> _______________________________________________
> Devel mailing list -- devel(a)ovirt.org
> To unsubscribe send an email to devel-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
--
Thanks,
Kaustav Majumder
7:38 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
Hi,
Let me know if you require something else
journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
engine.log - > https://pastebin.com/bVv7FxLB
Hmm, could you please try to reproduce with below patch, maybe there will
be more information in stderr:
https://gerrit.ovirt.org/108012
Thanks,
Martin
On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com> wrote:
> Could you please share journal and engine.log? We need to correlate and
> find out which error in journal is correlated to this issue?
>
> Thanks,
> Martin
>
>
> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <kmajumde(a)redhat.com>
> wrote:
>
>> Hi Artur,
>> I changed the files mentioned in the mail
>>
>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>
>> but unfortunately it is not the same error as described in the mail and
>> changing it gives me the same error.
>>
>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com> wrote:
>>
>>> Hi Kaustav,
>>> You might be hitting a known issue.
>>> Please check the solution posted by Dana on this list. Look for title:
>>> '[ovirt-devel] workaround for host deploy failure (get_events)'
>>> Artur
>>>
>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder
<kmajumde(a)redhat.com>
>>> wrote:
>>>
>>>> Hi all,
>>>> I am trying to run create-brick playbook from ansible-runner
>>>> integrated inside ovirt-engine but it throws errors as
>>>> ```
>>>> Running command: CreateBrickCommand internal: false. Entities affected
>>>> : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05 ERROR
>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>>>> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible create
>>>> brick role. Please check logs for more details:
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>> EngineException: Failed to execute Ansible create brick role. Please
check
>>>> logs for more details:
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>> (Failed with error GeneralException and code 100) 2020-03-27
>>>> 15:43:07,526+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for
command
>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27
>>>> 15:43:07,530+05 ERROR
>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on
host
>>>> 10.70.35.17 of cluster Default. ```
>>>> *The corresponding log is not created*
>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>> No such file or directory
>>>> *checking ansible-runner status gives*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
>>>> systemctl status ansible-runner-service.service ●
>>>> ansible-runner-service.service - Ansible Runner Service Loaded: loaded
>>>> (/usr/lib/systemd/system/ansible-runner-service.service; disabled;
vendor
>>>> preset: disabled) Active: active (running) since Fri 2020-03-27
15:42:56
>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
>>>> Memory: 62.1M CGroup: /system.slice/ansible-runner-service.service
>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w
2
>>>> runner_service.wsgi:application ├─30112 /usr/bin/python3
>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>> runner_service.wsgi:application └─30123 /usr/bin/python3
>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>> runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
>>>> gunicorn-3[30097]: Request received, content-type :NoneMar 27 15:42:58
>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
>>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]: 127.0.0.1
-
>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:03
>>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]: 127.0.0.1
-
>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:07
>>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]: 127.0.0.1
-
>>>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27 15:43:07
>>>> localhost.localdomain gunicorn-3[30097]: Request received, content-type
>>>> :application/json; charset=UTF-8Mar 27 15:43:07 localhost.localdomain
>>>> gunicorn-3[30097]: 127.0.0.1 - POST /api/v1/playbooks/create-brick.yml*
>>>>
>>>> *Is there a way to find the root cause for this?*
>>>> --
>>>>
>>>> Thanks,
>>>>
>>>> Kaustav Majumder
>>>> _______________________________________________
>>>> Devel mailing list -- devel(a)ovirt.org
>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives:
>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>
>>>
>>>
>>> --
>>>
>>> Artur Socha
>>>
>>> Senior Software Engineer, RHV
>>>
>>> Red Hat <https://www.redhat.com>
>>> <https://www.redhat.com>
>>>
>>
>>
>> --
>>
>> Thanks,
>>
>> Kaustav Majumder
>> _______________________________________________
>> Devel mailing list -- devel(a)ovirt.org
>> To unsubscribe send an email to devel-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>
>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
--
Thanks,
Kaustav Majumder
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
8:06 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
HI Martin,
Below error after rebasing you patch
2020-03-27 18:34:24,996+05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
2020-03-27 18:34:25,771+05 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute
call to start playbook.
2020-03-27 18:34:25,771+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
brick role. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
2020-03-27 18:34:25,772+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
EngineException: Failed to execute Ansible create brick role: 'Failed to
execute call to start playbook. '. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
(Failed with error GeneralException and code 100)
2020-03-27 18:34:25,852+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27 18:34:25,863+05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
10.70.35.17 of cluster Default.
On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com> wrote:
On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
> Hi,
> Let me know if you require something else
> journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
> engine.log - > https://pastebin.com/bVv7FxLB
>
Hmm, could you please try to reproduce with below patch, maybe there will
be more information in stderr:
https://gerrit.ovirt.org/108012
Thanks,
Martin
>
> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>> Could you please share journal and engine.log? We need to correlate and
>> find out which error in journal is correlated to this issue?
>>
>> Thanks,
>> Martin
>>
>>
>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <kmajumde(a)redhat.com>
>> wrote:
>>
>>> Hi Artur,
>>> I changed the files mentioned in the mail
>>>
>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>
>>> but unfortunately it is not the same error as described in the mail and
>>> changing it gives me the same error.
>>>
>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com> wrote:
>>>
>>>> Hi Kaustav,
>>>> You might be hitting a known issue.
>>>> Please check the solution posted by Dana on this list. Look for title:
>>>> '[ovirt-devel] workaround for host deploy failure (get_events)'
>>>> Artur
>>>>
>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder
<kmajumde(a)redhat.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>> I am trying to run create-brick playbook from ansible-runner
>>>>> integrated inside ovirt-engine but it throws errors as
>>>>> ```
>>>>> Running command: CreateBrickCommand internal: false. Entities
>>>>> affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction
group
>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05
ERROR
>>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(default
>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>>>>> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible
create
>>>>> brick role. Please check logs for more details:
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'
failed:
>>>>> EngineException: Failed to execute Ansible create brick role. Please
check
>>>>> logs for more details:
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>> (Failed with error GeneralException and code 100) 2020-03-27
>>>>> 15:43:07,526+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back for
command
>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27
>>>>> 15:43:07,530+05 ERROR
>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on
host
>>>>> 10.70.35.17 of cluster Default. ```
>>>>> *The corresponding log is not created*
>>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>> No such file or directory
>>>>> *checking ansible-runner status gives*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130 ±
>>>>> systemctl status ansible-runner-service.service ●
>>>>> ansible-runner-service.service - Ansible Runner Service Loaded:
loaded
>>>>> (/usr/lib/systemd/system/ansible-runner-service.service; disabled;
vendor
>>>>> preset: disabled) Active: active (running) since Fri 2020-03-27
15:42:56
>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit: 4915)
>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001
-w 2
>>>>> runner_service.wsgi:application ├─30112 /usr/bin/python3
>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>> runner_service.wsgi:application └─30123 /usr/bin/python3
>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>> runner_service.wsgi:applicationMar 27 15:42:58 localhost.localdomain
>>>>> gunicorn-3[30097]: Request received, content-type :NoneMar 27
15:42:58
>>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27 15:43:01
>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:03
>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:07
>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27
15:43:07
>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>
>>>>> *Is there a way to find the root cause for this?*
>>>>> --
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Kaustav Majumder
>>>>> _______________________________________________
>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Artur Socha
>>>>
>>>> Senior Software Engineer, RHV
>>>>
>>>> Red Hat <https://www.redhat.com>
>>>> <https://www.redhat.com>
>>>>
>>>
>>>
>>> --
>>>
>>> Thanks,
>>>
>>> Kaustav Majumder
>>> _______________________________________________
>>> Devel mailing list -- devel(a)ovirt.org
>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>
>>
>>
>> --
>> Martin Perina
>> Manager, Software Engineering
>> Red Hat Czech s.r.o.
>>
>
>
> --
>
> Thanks,
>
> Kaustav Majumder
>
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
--
Thanks,
Kaustav Majumder
9:35 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
HI Martin,
Below error after rebasing you patch
2020-03-27 18:34:24,996+05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
2020-03-27 18:34:25,771+05 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute
call to start playbook.
2020-03-27 18:34:25,771+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
brick role. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
2020-03-27 18:34:25,772+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
EngineException: Failed to execute Ansible create brick role: 'Failed to
execute call to start playbook. '. Please check logs for more details:
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
(Failed with error GeneralException and code 100)
2020-03-27 18:34:25,852+05 ERROR
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
[49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27 18:34:25,863+05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
10.70.35.17 of cluster Default.
Dana/Martin, any idea about the issue?
On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>
> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <kmajumde(a)redhat.com>
> wrote:
>
>> Hi,
>> Let me know if you require something else
>> journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
>> engine.log - > https://pastebin.com/bVv7FxLB
>>
>
> Hmm, could you please try to reproduce with below patch, maybe there will
> be more information in stderr:
>
> https://gerrit.ovirt.org/108012
>
> Thanks,
> Martin
>
>>
>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com>
>> wrote:
>>
>>> Could you please share journal and engine.log? We need to correlate and
>>> find out which error in journal is correlated to this issue?
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>> wrote:
>>>
>>>> Hi Artur,
>>>> I changed the files mentioned in the mail
>>>>
>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>
>>>> but unfortunately it is not the same error as described in the mail
>>>> and changing it gives me the same error.
>>>>
>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha <asocha(a)redhat.com>
wrote:
>>>>
>>>>> Hi Kaustav,
>>>>> You might be hitting a known issue.
>>>>> Please check the solution posted by Dana on this list. Look for
>>>>> title: '[ovirt-devel] workaround for host deploy failure
(get_events)'
>>>>> Artur
>>>>>
>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <
>>>>> kmajumde(a)redhat.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> I am trying to run create-brick playbook from ansible-runner
>>>>>> integrated inside ovirt-engine but it throws errors as
>>>>>> ```
>>>>>> Running command: CreateBrickCommand internal: false. Entities
>>>>>> affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type:
VDSAction group
>>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27 15:43:07,518+05
ERROR
>>>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(default
>>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed
to
>>>>>> execute call to start playbook. 2020-03-27 15:43:07,519+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute Ansible
create
>>>>>> brick role. Please check logs for more details:
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'
failed:
>>>>>> EngineException: Failed to execute Ansible create brick role.
Please check
>>>>>> logs for more details:
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>> (Failed with error GeneralException and code 100) 2020-03-27
>>>>>> 15:43:07,526+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-29)
>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction rolled-back
for command
>>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2020-03-27
>>>>>> 15:43:07,530+05 ERROR
>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
EVENT_ID:
>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick
brick-1 on host
>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>> *The corresponding log is not created*
>>>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ± less
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>> No such file or directory
>>>>>> *checking ansible-runner status gives*
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s 130
±
>>>>>> systemctl status ansible-runner-service.service ●
>>>>>> ansible-runner-service.service - Ansible Runner Service Loaded:
loaded
>>>>>> (/usr/lib/systemd/system/ansible-runner-service.service;
disabled; vendor
>>>>>> preset: disabled) Active: active (running) since Fri 2020-03-27
15:42:56
>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3 (limit:
4915)
>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b
localhost:50001 -w 2
>>>>>> runner_service.wsgi:application ├─30112
/usr/bin/python3
>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>> runner_service.wsgi:application └─30123
/usr/bin/python3
>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>> runner_service.wsgi:applicationMar 27 15:42:58
localhost.localdomain
>>>>>> gunicorn-3[30097]: Request received, content-type :NoneMar 27
15:42:58
>>>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:01
>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:03
>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:07
>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar 27
15:43:07
>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>
>>>>>> *Is there a way to find the root cause for this?*
>>>>>> --
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kaustav Majumder
>>>>>> _______________________________________________
>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>> oVirt Code of Conduct:
>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> List Archives:
>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Artur Socha
>>>>>
>>>>> Senior Software Engineer, RHV
>>>>>
>>>>> Red Hat <https://www.redhat.com>
>>>>> <https://www.redhat.com>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Thanks,
>>>>
>>>> Kaustav Majumder
>>>> _______________________________________________
>>>> Devel mailing list -- devel(a)ovirt.org
>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives:
>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>
>>>
>>>
>>> --
>>> Martin Perina
>>> Manager, Software Engineering
>>> Red Hat Czech s.r.o.
>>>
>>
>>
>> --
>>
>> Thanks,
>>
>> Kaustav Majumder
>>
>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
--
Thanks,
Kaustav Majumder
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
10:14 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Hi,
my bet is on the issue with ansible-runner please try to install new runner
(https://cbs.centos.org/koji/packageinfo?packageID=7439) and restart
service (`systemctl restart ansible-runner-service.service`).
if it won't help please use `journalctl -u ansible-runner-service.service
-f` to gather info from the runner.
The log which is mentioned in engine.log is not created in dev env
everything is logged to journal.
Martin Necas
On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com> wrote:
On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
> HI Martin,
> Below error after rebasing you patch
> 2020-03-27 18:34:24,996+05 INFO
> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
> FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
> 2020-03-27 18:34:25,771+05 ERROR
> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute
> call to start playbook.
> 2020-03-27 18:34:25,771+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
> brick role. Please check logs for more details:
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
> 2020-03-27 18:34:25,772+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
> EngineException: Failed to execute Ansible create brick role: 'Failed to
> execute call to start playbook. '. Please check logs for more details:
>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
> (Failed with error GeneralException and code 100)
> 2020-03-27 18:34:25,852+05 ERROR
> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command
> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
> 2020-03-27 18:34:25,863+05 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
> 10.70.35.17 of cluster Default.
>
Dana/Martin, any idea about the issue?
>
> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>>
>>
>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <kmajumde(a)redhat.com>
>> wrote:
>>
>>> Hi,
>>> Let me know if you require something else
>>> journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>
>>
>> Hmm, could you please try to reproduce with below patch, maybe there
>> will be more information in stderr:
>>
>> https://gerrit.ovirt.org/108012
>>
>> Thanks,
>> Martin
>>
>>>
>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com>
>>> wrote:
>>>
>>>> Could you please share journal and engine.log? We need to correlate
>>>> and find out which error in journal is correlated to this issue?
>>>>
>>>> Thanks,
>>>> Martin
>>>>
>>>>
>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>>> wrote:
>>>>
>>>>> Hi Artur,
>>>>> I changed the files mentioned in the mail
>>>>>
>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>
>>>>> but unfortunately it is not the same error as described in the mail
>>>>> and changing it gives me the same error.
>>>>>
>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Kaustav,
>>>>>> You might be hitting a known issue.
>>>>>> Please check the solution posted by Dana on this list. Look for
>>>>>> title: '[ovirt-devel] workaround for host deploy failure
(get_events)'
>>>>>> Artur
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <
>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> I am trying to run create-brick playbook from ansible-runner
>>>>>>> integrated inside ovirt-engine but it throws errors as
>>>>>>> ```
>>>>>>> Running command: CreateBrickCommand internal: false.
Entities
>>>>>>> affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type:
VDSAction group
>>>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27
15:43:07,518+05 ERROR
>>>>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(default
>>>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception:
Failed to
>>>>>>> execute call to start playbook. 2020-03-27 15:43:07,519+05
ERROR
>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute
Ansible create
>>>>>>> brick role. Please check logs for more details:
>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>> EngineException: Failed to execute Ansible create brick role.
Please check
>>>>>>> logs for more details:
>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>> (Failed with error GeneralException and code 100) 2020-03-27
>>>>>>> 15:43:07,526+05 ERROR
>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction
rolled-back for command
>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
EVENT_ID:
>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick
brick-1 on host
>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>> *The corresponding log is not created*
>>>>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s ±
less
>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>> No such file or directory
>>>>>>> *checking ansible-runner status gives*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+ 1m1s
130 ±
>>>>>>> systemctl status ansible-runner-service.service ●
>>>>>>> ansible-runner-service.service - Ansible Runner Service
Loaded: loaded
>>>>>>> (/usr/lib/systemd/system/ansible-runner-service.service;
disabled; vendor
>>>>>>> preset: disabled) Active: active (running) since Fri
2020-03-27 15:42:56
>>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3
(limit: 4915)
>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b
localhost:50001 -w 2
>>>>>>> runner_service.wsgi:application ├─30112
/usr/bin/python3
>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>> runner_service.wsgi:application └─30123
/usr/bin/python3
>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>> runner_service.wsgi:applicationMar 27 15:42:58
localhost.localdomain
>>>>>>> gunicorn-3[30097]: Request received, content-type :NoneMar 27
15:42:58
>>>>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>>>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:01
>>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>>> :NoneMar 27 15:43:01 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:03
>>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>>> :NoneMar 27 15:43:03 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:07
>>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>>> :NoneMar 27 15:43:07 localhost.localdomain gunicorn-3[30097]:
127.0.0.1 -
>>>>>>> GET /api/v1/hosts/10.70.35.17 <http://10.70.35.17>Mar
27 15:43:07
>>>>>>> localhost.localdomain gunicorn-3[30097]: Request received,
content-type
>>>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>
>>>>>>> *Is there a way to find the root cause for this?*
>>>>>>> --
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Kaustav Majumder
>>>>>>> _______________________________________________
>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>>> oVirt Code of Conduct:
>>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>>> List Archives:
>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Artur Socha
>>>>>>
>>>>>> Senior Software Engineer, RHV
>>>>>>
>>>>>> Red Hat <https://www.redhat.com>
>>>>>> <https://www.redhat.com>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Kaustav Majumder
>>>>> _______________________________________________
>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>
>>>>
>>>>
>>>> --
>>>> Martin Perina
>>>> Manager, Software Engineering
>>>> Red Hat Czech s.r.o.
>>>>
>>>
>>>
>>> --
>>>
>>> Thanks,
>>>
>>> Kaustav Majumder
>>>
>>
>>
>> --
>> Martin Perina
>> Manager, Software Engineering
>> Red Hat Czech s.r.o.
>>
>
>
> --
>
> Thanks,
>
> Kaustav Majumder
>
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
10:17 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Sorry did not notice that link to the journal.
On Fri, Mar 27, 2020 at 4:13 PM Martin Necas <mnecas(a)redhat.com> wrote:
Hi,
my bet is on the issue with ansible-runner please try to install new
runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and
restart service (`systemctl restart ansible-runner-service.service`).
if it won't help please use `journalctl -u ansible-runner-service.service
-f` to gather info from the runner.
The log which is mentioned in engine.log is not created in dev env
everything is logged to journal.
Martin Necas
On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>
> On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde(a)redhat.com>
> wrote:
>
>> HI Martin,
>> Below error after rebasing you patch
>> 2020-03-27 18:34:24,996+05 INFO
>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
>> FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
>> 2020-03-27 18:34:25,771+05 ERROR
>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute
>> call to start playbook.
>> 2020-03-27 18:34:25,771+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
>> brick role. Please check logs for more details:
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>> 2020-03-27 18:34:25,772+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>> EngineException: Failed to execute Ansible create brick role: 'Failed to
>> execute call to start playbook. '. Please check logs for more details:
>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>> (Failed with error GeneralException and code 100)
>> 2020-03-27 18:34:25,852+05 ERROR
>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command
>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
>> 2020-03-27 18:34:25,863+05 ERROR
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
>> 10.70.35.17 of cluster Default.
>>
>
> Dana/Martin, any idea about the issue?
>
>>
>> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>> wrote:
>>>
>>>> Hi,
>>>> Let me know if you require something else
>>>> journalctl -u ansible-runner-service -> https://pastebin.com/e6tnrzSB
>>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>>
>>>
>>> Hmm, could you please try to reproduce with below patch, maybe there
>>> will be more information in stderr:
>>>
>>> https://gerrit.ovirt.org/108012
>>>
>>> Thanks,
>>> Martin
>>>
>>>>
>>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina <mperina(a)redhat.com>
>>>> wrote:
>>>>
>>>>> Could you please share journal and engine.log? We need to correlate
>>>>> and find out which error in journal is correlated to this issue?
>>>>>
>>>>> Thanks,
>>>>> Martin
>>>>>
>>>>>
>>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <
>>>>> kmajumde(a)redhat.com> wrote:
>>>>>
>>>>>> Hi Artur,
>>>>>> I changed the files mentioned in the mail
>>>>>>
>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>>
>>>>>> but unfortunately it is not the same error as described in the
mail
>>>>>> and changing it gives me the same error.
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Kaustav,
>>>>>>> You might be hitting a known issue.
>>>>>>> Please check the solution posted by Dana on this list. Look
for
>>>>>>> title: '[ovirt-devel] workaround for host deploy failure
(get_events)'
>>>>>>> Artur
>>>>>>>
>>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <
>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>> I am trying to run create-brick playbook from
ansible-runner
>>>>>>>> integrated inside ovirt-engine but it throws errors as
>>>>>>>> ```
>>>>>>>> Running command: CreateBrickCommand internal: false.
Entities
>>>>>>>> affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type:
VDSAction group
>>>>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27
15:43:07,518+05 ERROR
>>>>>>>>
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Exception: Failed to
>>>>>>>> execute call to start playbook. 2020-03-27
15:43:07,519+05 ERROR
>>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to execute
Ansible create
>>>>>>>> brick role. Please check logs for more details:
>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>>> EngineException: Failed to execute Ansible create brick
role. Please check
>>>>>>>> logs for more details:
>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>> (Failed with error GeneralException and code 100)
2020-03-27
>>>>>>>> 15:43:07,526+05 ERROR
>>>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand]
(default task-29)
>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction
rolled-back for command
>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>> (default task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
EVENT_ID:
>>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create
brick brick-1 on host
>>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>>> *The corresponding log is not created*
>>>>>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+ 7s
± less
>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>>> No such file or directory
>>>>>>>> *checking ansible-runner status gives*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *[kmajumde:~/work/git/ovirt-engine] global_vol_options+
1m1s 130 ±
>>>>>>>> systemctl status ansible-runner-service.service ●
>>>>>>>> ansible-runner-service.service - Ansible Runner Service
Loaded: loaded
>>>>>>>> (/usr/lib/systemd/system/ansible-runner-service.service;
disabled; vendor
>>>>>>>> preset: disabled) Active: active (running) since Fri
2020-03-27 15:42:56
>>>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks: 3
(limit: 4915)
>>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b
localhost:50001 -w 2
>>>>>>>> runner_service.wsgi:application ├─30112
/usr/bin/python3
>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>> runner_service.wsgi:application └─30123
/usr/bin/python3
>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>> runner_service.wsgi:applicationMar 27 15:42:58
localhost.localdomain
>>>>>>>> gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:42:58
>>>>>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 - GET
>>>>>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar 27
15:43:01
>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>> :NoneMar 27 15:43:01 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar
27 15:43:03
>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>> :NoneMar 27 15:43:03 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>> GET /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar
27 15:43:07
>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>> :NoneMar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>> GET /api/v1/hosts/10.70.35.17
<http://10.70.35.17>Mar 27 15:43:07
>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>>
>>>>>>>> *Is there a way to find the root cause for this?*
>>>>>>>> --
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Kaustav Majumder
>>>>>>>> _______________________________________________
>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>> oVirt Code of Conduct:
>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>> List Archives:
>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Artur Socha
>>>>>>>
>>>>>>> Senior Software Engineer, RHV
>>>>>>>
>>>>>>> Red Hat <https://www.redhat.com>
>>>>>>> <https://www.redhat.com>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kaustav Majumder
>>>>>> _______________________________________________
>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>> oVirt Code of Conduct:
>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> List Archives:
>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Martin Perina
>>>>> Manager, Software Engineering
>>>>> Red Hat Czech s.r.o.
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Thanks,
>>>>
>>>> Kaustav Majumder
>>>>
>>>
>>>
>>> --
>>> Martin Perina
>>> Manager, Software Engineering
>>> Red Hat Czech s.r.o.
>>>
>>
>>
>> --
>>
>> Thanks,
>>
>> Kaustav Majumder
>>
>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
10:37 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
The journal looks okay to me...
Maybe the playbook failed please go
to ovirt-engine/share/ovirt-engine/ansible-runner-service-project/artifacts/
and find the newest folder and check stdout file.
On Fri, Mar 27, 2020 at 4:16 PM Martin Necas <mnecas(a)redhat.com> wrote:
Sorry did not notice that link to the journal.
On Fri, Mar 27, 2020 at 4:13 PM Martin Necas <mnecas(a)redhat.com> wrote:
> Hi,
>
> my bet is on the issue with ansible-runner please try to install new
> runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and
> restart service (`systemctl restart ansible-runner-service.service`).
> if it won't help please use `journalctl -u ansible-runner-service.service
> -f` to gather info from the runner.
> The log which is mentioned in engine.log is not created in dev env
> everything is logged to journal.
>
> Martin Necas
>
> On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>>
>>
>> On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde(a)redhat.com>
>> wrote:
>>
>>> HI Martin,
>>> Below error after rebasing you patch
>>> 2020-03-27 18:34:24,996+05 INFO
>>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
>>> FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
>>> 2020-03-27 18:34:25,771+05 ERROR
>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to execute
>>> call to start playbook.
>>> 2020-03-27 18:34:25,771+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
>>> brick role. Please check logs for more details:
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>> 2020-03-27 18:34:25,772+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>> EngineException: Failed to execute Ansible create brick role: 'Failed to
>>> execute call to start playbook. '. Please check logs for more details:
>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>> (Failed with error GeneralException and code 100)
>>> 2020-03-27 18:34:25,852+05 ERROR
>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for command
>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
>>> 2020-03-27 18:34:25,863+05 ERROR
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on host
>>> 10.70.35.17 of cluster Default.
>>>
>>
>> Dana/Martin, any idea about the issue?
>>
>>>
>>> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> Let me know if you require something else
>>>>> journalctl -u ansible-runner-service ->
https://pastebin.com/e6tnrzSB
>>>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>>>
>>>>
>>>> Hmm, could you please try to reproduce with below patch, maybe there
>>>> will be more information in stderr:
>>>>
>>>> https://gerrit.ovirt.org/108012
>>>>
>>>> Thanks,
>>>> Martin
>>>>
>>>>>
>>>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina
<mperina(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Could you please share journal and engine.log? We need to
correlate
>>>>>> and find out which error in journal is correlated to this issue?
>>>>>>
>>>>>> Thanks,
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <
>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>
>>>>>>> Hi Artur,
>>>>>>> I changed the files mentioned in the mail
>>>>>>>
>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>>>
>>>>>>> but unfortunately it is not the same error as described in
the mail
>>>>>>> and changing it gives me the same error.
>>>>>>>
>>>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Kaustav,
>>>>>>>> You might be hitting a known issue.
>>>>>>>> Please check the solution posted by Dana on this list.
Look for
>>>>>>>> title: '[ovirt-devel] workaround for host deploy
failure (get_events)'
>>>>>>>> Artur
>>>>>>>>
>>>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder <
>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>> I am trying to run create-brick playbook from
ansible-runner
>>>>>>>>> integrated inside ovirt-engine but it throws errors
as
>>>>>>>>> ```
>>>>>>>>> Running command: CreateBrickCommand internal: false.
Entities
>>>>>>>>> affected : ID: 45b7ce67-9939-4d5f-a1ba-92411356b7e6
Type: VDSAction group
>>>>>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27
15:43:07,518+05 ERROR
>>>>>>>>>
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>>>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Exception: Failed to
>>>>>>>>> execute call to start playbook. 2020-03-27
15:43:07,519+05 ERROR
>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to
execute Ansible create
>>>>>>>>> brick role. Please check logs for more details:
>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>>>> EngineException: Failed to execute Ansible create
brick role. Please check
>>>>>>>>> logs for more details:
>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>> (Failed with error GeneralException and code 100)
2020-03-27
>>>>>>>>> 15:43:07,526+05 ERROR
>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Transaction
rolled-back for command
>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>>> (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create
brick brick-1 on host
>>>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>>>> *The corresponding log is not created*
>>>>>>>>> kmajumde:~/work/git/ovirt-engine] global_vol_options+
7s ± less
>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>>>> No such file or directory
>>>>>>>>> *checking ansible-runner status gives*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *[kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 1m1s 130
>>>>>>>>> ± systemctl status ansible-runner-service.service ●
>>>>>>>>> ansible-runner-service.service - Ansible Runner
Service Loaded: loaded
>>>>>>>>>
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>>>>>>>>> preset: disabled) Active: active (running) since
Fri 2020-03-27 15:42:56
>>>>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3) Tasks:
3 (limit: 4915)
>>>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3 -b
localhost:50001 -w 2
>>>>>>>>> runner_service.wsgi:application ├─30112
/usr/bin/python3
>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>> runner_service.wsgi:application └─30123
/usr/bin/python3
>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>> runner_service.wsgi:applicationMar 27 15:42:58
localhost.localdomain
>>>>>>>>> gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:42:58
>>>>>>>>> localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
GET
>>>>>>>>> /api/v1/hosts/10.70.35.1 <http://10.70.35.1>Mar
27 15:43:01
>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>> :NoneMar 27 15:43:01 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:03
>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>> :NoneMar 27 15:43:03 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:07
>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>> :NoneMar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>> GET /api/v1/hosts/10.70.35.17
<http://10.70.35.17>Mar 27 15:43:07
>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>>>
>>>>>>>>> *Is there a way to find the root cause for this?*
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Kaustav Majumder
>>>>>>>>> _______________________________________________
>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>> List Archives:
>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Artur Socha
>>>>>>>>
>>>>>>>> Senior Software Engineer, RHV
>>>>>>>>
>>>>>>>> Red Hat <https://www.redhat.com>
>>>>>>>> <https://www.redhat.com>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Kaustav Majumder
>>>>>>> _______________________________________________
>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>>> oVirt Code of Conduct:
>>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>>> List Archives:
>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Martin Perina
>>>>>> Manager, Software Engineering
>>>>>> Red Hat Czech s.r.o.
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Kaustav Majumder
>>>>>
>>>>
>>>>
>>>> --
>>>> Martin Perina
>>>> Manager, Software Engineering
>>>> Red Hat Czech s.r.o.
>>>>
>>>
>>>
>>> --
>>>
>>> Thanks,
>>>
>>> Kaustav Majumder
>>>
>>
>>
>> --
>> Martin Perina
>> Manager, Software Engineering
>> Red Hat Czech s.r.o.
>>
>
10:53 a.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Unfortunately no artifact stdout files point to the playbook being run.
There are others like when adding a new host and checking for host updates
but none pertaining to create-brick . The latest one is this pointing to (i
guess) host updates checker
[root@localhost artifacts] # less
83029f4c-702d-11ea-b308-68ecc5af4344/stdout
PLAY [all]
*********************************************************************
TASK [Gathering Facts]
*********************************************************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Check for system updates (remove empty
lines)] ***
this message.
changed: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Remove 'Obsoleting' title from file]
**********
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Get yum updates file]
*************************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Get yum updates content]
**********************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Delete yum_updates file from host]
************
changed: [10.70.35.88]
PLAY RECAP
*********************************************************************
10.70.35.88 : ok=6 changed=2 unreachable=0 failed=0
skipped=0 rescued=0 ignored=0
On Fri, Mar 27, 2020 at 9:07 PM Martin Necas <mnecas(a)redhat.com> wrote:
The journal looks okay to me...
Maybe the playbook failed please go
to ovirt-engine/share/ovirt-engine/ansible-runner-service-project/artifacts/
and find the newest folder and check stdout file.
On Fri, Mar 27, 2020 at 4:16 PM Martin Necas <mnecas(a)redhat.com> wrote:
> Sorry did not notice that link to the journal.
>
> On Fri, Mar 27, 2020 at 4:13 PM Martin Necas <mnecas(a)redhat.com> wrote:
>
>> Hi,
>>
>> my bet is on the issue with ansible-runner please try to install new
>> runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and
>> restart service (`systemctl restart ansible-runner-service.service`).
>> if it won't help please use `journalctl -u
>> ansible-runner-service.service -f` to gather info from the runner.
>> The log which is mentioned in engine.log is not created in dev env
>> everything is logged to journal.
>>
>> Martin Necas
>>
>> On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder <kmajumde(a)redhat.com>
>>> wrote:
>>>
>>>> HI Martin,
>>>> Below error after rebasing you patch
>>>> 2020-03-27 18:34:24,996+05 INFO
>>>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
>>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98)
[]
>>>> FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to
execute
>>>> call to start playbook.
>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible create
>>>> brick role. Please check logs for more details:
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>> 2020-03-27 18:34:25,772+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>> EngineException: Failed to execute Ansible create brick role: 'Failed
to
>>>> execute call to start playbook. '. Please check logs for more
details:
>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>> (Failed with error GeneralException and code 100)
>>>> 2020-03-27 18:34:25,852+05 ERROR
>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-1)
>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for
command
>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
>>>> 2020-03-27 18:34:25,863+05 ERROR
>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on
host
>>>> 10.70.35.17 of cluster Default.
>>>>
>>>
>>> Dana/Martin, any idea about the issue?
>>>
>>>>
>>>> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina <mperina(a)redhat.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <
>>>>> kmajumde(a)redhat.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Let me know if you require something else
>>>>>> journalctl -u ansible-runner-service ->
>>>>>> https://pastebin.com/e6tnrzSB
>>>>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>>>>
>>>>>
>>>>> Hmm, could you please try to reproduce with below patch, maybe there
>>>>> will be more information in stderr:
>>>>>
>>>>> https://gerrit.ovirt.org/108012
>>>>>
>>>>> Thanks,
>>>>> Martin
>>>>>
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina
<mperina(a)redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Could you please share journal and engine.log? We need to
correlate
>>>>>>> and find out which error in journal is correlated to this
issue?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Martin
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <
>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>
>>>>>>>> Hi Artur,
>>>>>>>> I changed the files mentioned in the mail
>>>>>>>>
>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>>>>
>>>>>>>> but unfortunately it is not the same error as described
in the
>>>>>>>> mail and changing it gives me the same error.
>>>>>>>>
>>>>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Kaustav,
>>>>>>>>> You might be hitting a known issue.
>>>>>>>>> Please check the solution posted by Dana on this
list. Look for
>>>>>>>>> title: '[ovirt-devel] workaround for host deploy
failure (get_events)'
>>>>>>>>> Artur
>>>>>>>>>
>>>>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder
<
>>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>> I am trying to run create-brick playbook from
ansible-runner
>>>>>>>>>> integrated inside ovirt-engine but it throws
errors as
>>>>>>>>>> ```
>>>>>>>>>> Running command: CreateBrickCommand internal:
false. Entities
>>>>>>>>>> affected : ID:
45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>>>>>>>>>> MANIPULATE_HOST with role type ADMIN 2020-03-27
15:43:07,518+05 ERROR
>>>>>>>>>>
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>>>>>>>> task-29) [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Exception: Failed to
>>>>>>>>>> execute call to start playbook. 2020-03-27
15:43:07,519+05 ERROR
>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed to
execute Ansible create
>>>>>>>>>> brick role. Please check logs for more details:
>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Command
>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>>>>> EngineException: Failed to execute Ansible create
brick role. Please check
>>>>>>>>>> logs for more details:
>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>> (Failed with error GeneralException and code 100)
2020-03-27
>>>>>>>>>> 15:43:07,526+05 ERROR
>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Transaction rolled-back for command
>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>>>> (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to
create brick brick-1 on host
>>>>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>>>>> *The corresponding log is not created*
>>>>>>>>>> kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 7s ± less
>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>>>>> No such file or directory
>>>>>>>>>> *checking ansible-runner status gives*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *[kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 1m1s 130
>>>>>>>>>> ± systemctl status ansible-runner-service.service
●
>>>>>>>>>> ansible-runner-service.service - Ansible Runner
Service Loaded: loaded
>>>>>>>>>>
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>>>>>>>>>> preset: disabled) Active: active (running)
since Fri 2020-03-27 15:42:56
>>>>>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3)
Tasks: 3 (limit: 4915)
>>>>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>>>>> ├─30097 /usr/bin/python3 /usr/bin/gunicorn-3
-b localhost:50001 -w 2
>>>>>>>>>> runner_service.wsgi:application ├─30112
/usr/bin/python3
>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>> runner_service.wsgi:application └─30123
/usr/bin/python3
>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>> runner_service.wsgi:applicationMar 27 15:42:58
localhost.localdomain
>>>>>>>>>> gunicorn-3[30097]: Request received, content-type
:NoneMar 27 15:42:58
>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
127.0.0.1 - GET
>>>>>>>>>> /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:01
>>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>>> :NoneMar 27 15:43:01 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:03
>>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>>> :NoneMar 27 15:43:03 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:07
>>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>>> :NoneMar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>> GET /api/v1/hosts/10.70.35.17
<http://10.70.35.17>Mar 27 15:43:07
>>>>>>>>>> localhost.localdomain gunicorn-3[30097]: Request
received, content-type
>>>>>>>>>> :application/json; charset=UTF-8Mar 27 15:43:07
localhost.localdomain
>>>>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>>>>
>>>>>>>>>> *Is there a way to find the root cause for
this?*
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Kaustav Majumder
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>>> List Archives:
>>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Artur Socha
>>>>>>>>>
>>>>>>>>> Senior Software Engineer, RHV
>>>>>>>>>
>>>>>>>>> Red Hat <https://www.redhat.com>
>>>>>>>>> <https://www.redhat.com>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Kaustav Majumder
>>>>>>>> _______________________________________________
>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org
>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>> oVirt Code of Conduct:
>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>> List Archives:
>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Martin Perina
>>>>>>> Manager, Software Engineering
>>>>>>> Red Hat Czech s.r.o.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kaustav Majumder
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Martin Perina
>>>>> Manager, Software Engineering
>>>>> Red Hat Czech s.r.o.
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Thanks,
>>>>
>>>> Kaustav Majumder
>>>>
>>>
>>>
>>> --
>>> Martin Perina
>>> Manager, Software Engineering
>>> Red Hat Czech s.r.o.
>>>
>>
--
Thanks,
Kaustav Majumder
1:53 p.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Just update for others I and Kaustav were doing more investigation and
found an issue which was introduced by https://gerrit.ovirt.org/#/c/107683/,
because of removing over escaped parameters.
Right now working on the fix.
On Fri, Mar 27, 2020 at 4:54 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
Unfortunately no artifact stdout files point to the playbook being
run.
There are others like when adding a new host and checking for host updates
but none pertaining to create-brick . The latest one is this pointing to (i
guess) host updates checker
[root@localhost artifacts] # less
83029f4c-702d-11ea-b308-68ecc5af4344/stdout
PLAY [all]
*********************************************************************
TASK [Gathering Facts]
*********************************************************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Check for system updates (remove empty
lines)] ***
this message.
changed: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Remove 'Obsoleting' title from file]
**********
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Get yum updates file]
*************************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Get yum updates content]
**********************
ok: [10.70.35.88]
TASK [ovirt-host-check-upgrade : Delete yum_updates file from host]
************
changed: [10.70.35.88]
PLAY RECAP
*********************************************************************
10.70.35.88 : ok=6 changed=2 unreachable=0
failed=0 skipped=0 rescued=0 ignored=0
On Fri, Mar 27, 2020 at 9:07 PM Martin Necas <mnecas(a)redhat.com> wrote:
> The journal looks okay to me...
> Maybe the playbook failed please go
> to ovirt-engine/share/ovirt-engine/ansible-runner-service-project/artifacts/
> and find the newest folder and check stdout file.
>
> On Fri, Mar 27, 2020 at 4:16 PM Martin Necas <mnecas(a)redhat.com> wrote:
>
>> Sorry did not notice that link to the journal.
>>
>> On Fri, Mar 27, 2020 at 4:13 PM Martin Necas <mnecas(a)redhat.com> wrote:
>>
>>> Hi,
>>>
>>> my bet is on the issue with ansible-runner please try to install new
>>> runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and
>>> restart service (`systemctl restart ansible-runner-service.service`).
>>> if it won't help please use `journalctl -u
>>> ansible-runner-service.service -f` to gather info from the runner.
>>> The log which is mentioned in engine.log is not created in dev env
>>> everything is logged to journal.
>>>
>>> Martin Necas
>>>
>>> On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>>> wrote:
>>>>
>>>>> HI Martin,
>>>>> Below error after rebasing you patch
>>>>> 2020-03-27 18:34:24,996+05 INFO
>>>>>
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
>>>>>
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
>>>>> FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7fa278fe
>>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(default
>>>>> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed to
execute
>>>>> call to start playbook.
>>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible
create
>>>>> brick role. Please check logs for more details:
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>>> 2020-03-27 18:34:25,772+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'
failed:
>>>>> EngineException: Failed to execute Ansible create brick role:
'Failed to
>>>>> execute call to start playbook. '. Please check logs for more
details:
>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>>> (Failed with error GeneralException and code 100)
>>>>> 2020-03-27 18:34:25,852+05 ERROR
>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back for
command
>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
>>>>> 2020-03-27 18:34:25,863+05 ERROR
>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] EVENT_ID:
>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick-1 on
host
>>>>> 10.70.35.17 of cluster Default.
>>>>>
>>>>
>>>> Dana/Martin, any idea about the issue?
>>>>
>>>>>
>>>>> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina
<mperina(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <
>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> Let me know if you require something else
>>>>>>> journalctl -u ansible-runner-service ->
>>>>>>> https://pastebin.com/e6tnrzSB
>>>>>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>>>>>
>>>>>>
>>>>>> Hmm, could you please try to reproduce with below patch, maybe
there
>>>>>> will be more information in stderr:
>>>>>>
>>>>>> https://gerrit.ovirt.org/108012
>>>>>>
>>>>>> Thanks,
>>>>>> Martin
>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina
<mperina(a)redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Could you please share journal and engine.log? We need
to
>>>>>>>> correlate and find out which error in journal is
correlated to this issue?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder <
>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Artur,
>>>>>>>>> I changed the files mentioned in the mail
>>>>>>>>>
>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>>>>>
>>>>>>>>> but unfortunately it is not the same error as
described in the
>>>>>>>>> mail and changing it gives me the same error.
>>>>>>>>>
>>>>>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Kaustav,
>>>>>>>>>> You might be hitting a known issue.
>>>>>>>>>> Please check the solution posted by Dana on this
list. Look for
>>>>>>>>>> title: '[ovirt-devel] workaround for host
deploy failure (get_events)'
>>>>>>>>>> Artur
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav Majumder
<
>>>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>> I am trying to run create-brick playbook from
ansible-runner
>>>>>>>>>>> integrated inside ovirt-engine but it throws
errors as
>>>>>>>>>>> ```
>>>>>>>>>>> Running command: CreateBrickCommand internal:
false. Entities
>>>>>>>>>>> affected : ID:
45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>>>>>>>>>>> MANIPULATE_HOST with role type ADMIN
2020-03-27 15:43:07,518+05 ERROR
>>>>>>>>>>>
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>>>>>>>>> task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>>>>>>>>>>> execute call to start playbook. 2020-03-27
15:43:07,519+05 ERROR
>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3] Failed
to execute Ansible create
>>>>>>>>>>> brick role. Please check logs for more
details:
>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Command
>>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>>>>>> EngineException: Failed to execute Ansible
create brick role. Please check
>>>>>>>>>>> logs for more details:
>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>> (Failed with error GeneralException and code
100) 2020-03-27
>>>>>>>>>>> 15:43:07,526+05 ERROR
>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Transaction rolled-back for command
>>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>>>>> (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>>>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to
create brick brick-1 on host
>>>>>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>>>>>> *The corresponding log is not created*
>>>>>>>>>>> kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 7s ± less
>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>>>>>> No such file or directory
>>>>>>>>>>> *checking ansible-runner status gives*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *[kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 1m1s
>>>>>>>>>>> 130 ± systemctl status
ansible-runner-service.service ●
>>>>>>>>>>> ansible-runner-service.service - Ansible
Runner Service Loaded: loaded
>>>>>>>>>>>
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>>>>>>>>>>> preset: disabled) Active: active (running)
since Fri 2020-03-27 15:42:56
>>>>>>>>>>> IST; 8min ago Main PID: 30097 (gunicorn-3)
Tasks: 3 (limit: 4915)
>>>>>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>>>>>> ├─30097 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>>> runner_service.wsgi:application
├─30112 /usr/bin/python3
>>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>>> runner_service.wsgi:application
└─30123 /usr/bin/python3
>>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>>> runner_service.wsgi:applicationMar 27
15:42:58 localhost.localdomain
>>>>>>>>>>> gunicorn-3[30097]: Request received,
content-type :NoneMar 27 15:42:58
>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
127.0.0.1 - GET
>>>>>>>>>>> /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:01
>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>> :NoneMar 27 15:43:01 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:03
>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>> :NoneMar 27 15:43:03 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:07
>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>> :NoneMar 27 15:43:07 localhost.localdomain
gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>> GET /api/v1/hosts/10.70.35.17
<http://10.70.35.17>Mar 27 15:43:07
>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>> :application/json; charset=UTF-8Mar 27
15:43:07 localhost.localdomain
>>>>>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>>>>>
>>>>>>>>>>> *Is there a way to find the root cause for
this?*
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Kaustav Majumder
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>>>> List Archives:
>>>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Artur Socha
>>>>>>>>>>
>>>>>>>>>> Senior Software Engineer, RHV
>>>>>>>>>>
>>>>>>>>>> Red Hat <https://www.redhat.com>
>>>>>>>>>> <https://www.redhat.com>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Kaustav Majumder
>>>>>>>>> _______________________________________________
>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>> List Archives:
>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Martin Perina
>>>>>>>> Manager, Software Engineering
>>>>>>>> Red Hat Czech s.r.o.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Kaustav Majumder
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Martin Perina
>>>>>> Manager, Software Engineering
>>>>>> Red Hat Czech s.r.o.
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Kaustav Majumder
>>>>>
>>>>
>>>>
>>>> --
>>>> Martin Perina
>>>> Manager, Software Engineering
>>>> Red Hat Czech s.r.o.
>>>>
>>>
--
Thanks,
Kaustav Majumder
2:26 p.m.
New subject: Unable to run create-brick.yml from ansible runner dev
Hi,
Thanks to Martin Necas we have fixed the issue in CreateBrickCommand.
Posted patch below. Let's get this reviewed and merged soon.
https://gerrit.ovirt.org/#/c/108021/
On Sat, Mar 28, 2020 at 12:23 AM Martin Necas <mnecas(a)redhat.com> wrote:
Just update for others I and Kaustav were doing more investigation
and
found an issue which was introduced by
https://gerrit.ovirt.org/#/c/107683/, because of removing over escaped
parameters.
Right now working on the fix.
On Fri, Mar 27, 2020 at 4:54 PM Kaustav Majumder <kmajumde(a)redhat.com>
wrote:
> Unfortunately no artifact stdout files point to the playbook being run.
> There are others like when adding a new host and checking for host updates
> but none pertaining to create-brick . The latest one is this pointing to (i
> guess) host updates checker
>
> [root@localhost artifacts] # less
> 83029f4c-702d-11ea-b308-68ecc5af4344/stdout
> PLAY [all]
> *********************************************************************
>
> TASK [Gathering Facts]
> *********************************************************
> ok: [10.70.35.88]
>
> TASK [ovirt-host-check-upgrade : Check for system updates (remove empty
> lines)] ***
> this message.
> changed: [10.70.35.88]
>
> TASK [ovirt-host-check-upgrade : Remove 'Obsoleting' title from file]
> **********
> ok: [10.70.35.88]
>
> TASK [ovirt-host-check-upgrade : Get yum updates file]
> *************************
> ok: [10.70.35.88]
>
> TASK [ovirt-host-check-upgrade : Get yum updates content]
> **********************
> ok: [10.70.35.88]
>
> TASK [ovirt-host-check-upgrade : Delete yum_updates file from host]
> ************
> changed: [10.70.35.88]
>
> PLAY RECAP
> *********************************************************************
> 10.70.35.88 : ok=6 changed=2 unreachable=0
> failed=0 skipped=0 rescued=0 ignored=0
>
> On Fri, Mar 27, 2020 at 9:07 PM Martin Necas <mnecas(a)redhat.com> wrote:
>
>> The journal looks okay to me...
>> Maybe the playbook failed please go
>> to ovirt-engine/share/ovirt-engine/ansible-runner-service-project/artifacts/
>> and find the newest folder and check stdout file.
>>
>> On Fri, Mar 27, 2020 at 4:16 PM Martin Necas <mnecas(a)redhat.com> wrote:
>>
>>> Sorry did not notice that link to the journal.
>>>
>>> On Fri, Mar 27, 2020 at 4:13 PM Martin Necas <mnecas(a)redhat.com>
wrote:
>>>
>>>> Hi,
>>>>
>>>> my bet is on the issue with ansible-runner please try to install new
>>>> runner (https://cbs.centos.org/koji/packageinfo?packageID=7439) and
>>>> restart service (`systemctl restart ansible-runner-service.service`).
>>>> if it won't help please use `journalctl -u
>>>> ansible-runner-service.service -f` to gather info from the runner.
>>>> The log which is mentioned in engine.log is not created in dev env
>>>> everything is logged to journal.
>>>>
>>>> Martin Necas
>>>>
>>>> On Fri, Mar 27, 2020 at 3:35 PM Martin Perina <mperina(a)redhat.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Mar 27, 2020 at 2:07 PM Kaustav Majumder
<kmajumde(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>> HI Martin,
>>>>>> Below error after rebasing you patch
>>>>>> 2020-03-27 18:34:24,996+05 INFO
>>>>>>
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
>>>>>>
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-98) []
>>>>>> FINISH, GlusterVolumesListVDSCommand, return: {}, log id:
7fa278fe
>>>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>>>> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(default
>>>>>> task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3] Exception: Failed
to execute
>>>>>> call to start playbook.
>>>>>> 2020-03-27 18:34:25,771+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Failed to execute Ansible
create
>>>>>> brick role. Please check logs for more details:
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>>>> 2020-03-27 18:34:25,772+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Command
>>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'
failed:
>>>>>> EngineException: Failed to execute Ansible create brick role:
'Failed to
>>>>>> execute call to start playbook. '. Please check logs for more
details:
>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327183424-10.70.35.17-49b9e523-bdc1-4065-98f2-f8581a1119b3.log
>>>>>> (Failed with error GeneralException and code 100)
>>>>>> 2020-03-27 18:34:25,852+05 ERROR
>>>>>> [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default
task-1)
>>>>>> [49b9e523-bdc1-4065-98f2-f8581a1119b3] Transaction rolled-back
for command
>>>>>> 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
>>>>>> 2020-03-27 18:34:25,863+05 ERROR
>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>> (default task-1) [49b9e523-bdc1-4065-98f2-f8581a1119b3]
EVENT_ID:
>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick
brick-1 on host
>>>>>> 10.70.35.17 of cluster Default.
>>>>>>
>>>>>
>>>>> Dana/Martin, any idea about the issue?
>>>>>
>>>>>>
>>>>>> On Fri, Mar 27, 2020 at 6:08 PM Martin Perina
<mperina(a)redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 27, 2020 at 12:45 PM Kaustav Majumder <
>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> Let me know if you require something else
>>>>>>>> journalctl -u ansible-runner-service ->
>>>>>>>> https://pastebin.com/e6tnrzSB
>>>>>>>> engine.log - > https://pastebin.com/bVv7FxLB
>>>>>>>>
>>>>>>>
>>>>>>> Hmm, could you please try to reproduce with below patch,
maybe
>>>>>>> there will be more information in stderr:
>>>>>>>
>>>>>>> https://gerrit.ovirt.org/108012
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Martin
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 27, 2020 at 5:06 PM Martin Perina
<mperina(a)redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Could you please share journal and engine.log? We
need to
>>>>>>>>> correlate and find out which error in journal is
correlated to this issue?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Martin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Mar 27, 2020 at 12:33 PM Kaustav Majumder
<
>>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Artur,
>>>>>>>>>> I changed the files mentioned in the mail
>>>>>>>>>>
>>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MXPFVUAVE6U...
>>>>>>>>>>
>>>>>>>>>> but unfortunately it is not the same error as
described in the
>>>>>>>>>> mail and changing it gives me the same error.
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 27, 2020 at 4:45 PM Artur Socha
<asocha(a)redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Kaustav,
>>>>>>>>>>> You might be hitting a known issue.
>>>>>>>>>>> Please check the solution posted by Dana on
this list. Look for
>>>>>>>>>>> title: '[ovirt-devel] workaround for host
deploy failure (get_events)'
>>>>>>>>>>> Artur
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 27, 2020 at 11:37 AM Kaustav
Majumder <
>>>>>>>>>>> kmajumde(a)redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>> I am trying to run create-brick playbook
from ansible-runner
>>>>>>>>>>>> integrated inside ovirt-engine but it
throws errors as
>>>>>>>>>>>> ```
>>>>>>>>>>>> Running command: CreateBrickCommand
internal: false. Entities
>>>>>>>>>>>> affected : ID:
45b7ce67-9939-4d5f-a1ba-92411356b7e6 Type: VDSAction group
>>>>>>>>>>>> MANIPULATE_HOST with role type ADMIN
2020-03-27 15:43:07,518+05 ERROR
>>>>>>>>>>>>
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default
>>>>>>>>>>>> task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] Exception: Failed to
>>>>>>>>>>>> execute call to start playbook.
2020-03-27 15:43:07,519+05 ERROR
>>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Failed to execute Ansible create
>>>>>>>>>>>> brick role. Please check logs for more
details:
>>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>>> 2020-03-27 15:43:07,519+05 ERROR
>>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Command
>>>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed:
>>>>>>>>>>>> EngineException: Failed to execute
Ansible create brick role. Please check
>>>>>>>>>>>> logs for more details:
>>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>>> (Failed with error GeneralException and
code 100) 2020-03-27
>>>>>>>>>>>> 15:43:07,526+05 ERROR
>>>>>>>>>>>>
[org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-29)
>>>>>>>>>>>> [5ff56203-33ca-424b-9137-491cf1ef2cd3]
Transaction rolled-back for command
>>>>>>>>>>>>
'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-03-27
>>>>>>>>>>>> 15:43:07,530+05 ERROR
>>>>>>>>>>>>
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>>>>>> (default task-29)
[5ff56203-33ca-424b-9137-491cf1ef2cd3] EVENT_ID:
>>>>>>>>>>>> CREATE_GLUSTER_BRICK_FAILED(4,139),
Failed to create brick brick-1 on host
>>>>>>>>>>>> 10.70.35.17 of cluster Default. ```
>>>>>>>>>>>> *The corresponding log is not created*
>>>>>>>>>>>> kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 7s ±
>>>>>>>>>>>> less
>>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log
>>>>>>>>>>>>
/home/kmajumde/work/ovirt-engine-builds/27-03-global-vol-options/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20200327154307-10.70.35.17-5ff56203-33ca-424b-9137-491cf1ef2cd3.log:
>>>>>>>>>>>> No such file or directory
>>>>>>>>>>>> *checking ansible-runner status gives*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *[kmajumde:~/work/git/ovirt-engine]
global_vol_options+ 1m1s
>>>>>>>>>>>> 130 ± systemctl status
ansible-runner-service.service ●
>>>>>>>>>>>> ansible-runner-service.service - Ansible
Runner Service Loaded: loaded
>>>>>>>>>>>>
(/usr/lib/systemd/system/ansible-runner-service.service; disabled; vendor
>>>>>>>>>>>> preset: disabled) Active: active
(running) since Fri 2020-03-27 15:42:56
>>>>>>>>>>>> IST; 8min ago Main PID: 30097
(gunicorn-3) Tasks: 3 (limit: 4915)
>>>>>>>>>>>> Memory: 62.1M CGroup:
/system.slice/ansible-runner-service.service
>>>>>>>>>>>> ├─30097 /usr/bin/python3
/usr/bin/gunicorn-3 -b localhost:50001 -w 2
>>>>>>>>>>>> runner_service.wsgi:application
├─30112 /usr/bin/python3
>>>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w
2
>>>>>>>>>>>> runner_service.wsgi:application
└─30123 /usr/bin/python3
>>>>>>>>>>>> /usr/bin/gunicorn-3 -b localhost:50001 -w
2
>>>>>>>>>>>> runner_service.wsgi:applicationMar 27
15:42:58 localhost.localdomain
>>>>>>>>>>>> gunicorn-3[30097]: Request received,
content-type :NoneMar 27 15:42:58
>>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
127.0.0.1 - GET
>>>>>>>>>>>> /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:01
>>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>>> :NoneMar 27 15:43:01
localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:03
>>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>>> :NoneMar 27 15:43:03
localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>>> GET /api/v1/hosts/10.70.35.1
<http://10.70.35.1>Mar 27 15:43:07
>>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>>> :NoneMar 27 15:43:07
localhost.localdomain gunicorn-3[30097]: 127.0.0.1 -
>>>>>>>>>>>> GET /api/v1/hosts/10.70.35.17
<http://10.70.35.17>Mar 27 15:43:07
>>>>>>>>>>>> localhost.localdomain gunicorn-3[30097]:
Request received, content-type
>>>>>>>>>>>> :application/json; charset=UTF-8Mar 27
15:43:07 localhost.localdomain
>>>>>>>>>>>> gunicorn-3[30097]: 127.0.0.1 - POST
/api/v1/playbooks/create-brick.yml*
>>>>>>>>>>>>
>>>>>>>>>>>> *Is there a way to find the root cause
for this?*
>>>>>>>>>>>> --
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Kaustav Majumder
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>>>>> List Archives:
>>>>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P2GWVT2HBST...
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> Artur Socha
>>>>>>>>>>>
>>>>>>>>>>> Senior Software Engineer, RHV
>>>>>>>>>>>
>>>>>>>>>>> Red Hat <https://www.redhat.com>
>>>>>>>>>>> <https://www.redhat.com>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Kaustav Majumder
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Devel mailing list -- devel(a)ovirt.org
>>>>>>>>>> To unsubscribe send an email to
devel-leave(a)ovirt.org
>>>>>>>>>> Privacy Statement:
https://www.ovirt.org/privacy-policy.html
>>>>>>>>>> oVirt Code of Conduct:
>>>>>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>>>> List Archives:
>>>>>>>>>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULE6X7ZF6NM...
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Martin Perina
>>>>>>>>> Manager, Software Engineering
>>>>>>>>> Red Hat Czech s.r.o.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Kaustav Majumder
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Martin Perina
>>>>>>> Manager, Software Engineering
>>>>>>> Red Hat Czech s.r.o.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kaustav Majumder
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Martin Perina
>>>>> Manager, Software Engineering
>>>>> Red Hat Czech s.r.o.
>>>>>
>>>>
>
> --
>
> Thanks,
>
> Kaustav Majumder
>
--
Thanks,
Kaustav Majumder
1834
days inactive
1834
days old
13 comments
4 participants
participants (4)
-
Artur Socha -
Kaustav Majumder -
Martin Necas -
Martin Perina