Hey steven im facing the same issue, _____ 2020-01-09 17:22:26,901+05 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] Opening ssh-copy-id session on host 10.70.42.92 2020-01-09 17:22:27,086+05 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] Executing ssh-copy-id command on host 10.70.42.92 2020-01-09 17:22:27,300+05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] EVENT_ID: VDS_ANSIBLE_INSTALL_STARTED(560), Ansible host-deploy playbook execution has started on host prajith. 2020-01-09 17:22:27,321+05 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] Exception: Internal server error, please check Ansible runner error log. 2020-01-09 17:22:27,322+05 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] Host installation failed for host '71155740-8f27-473c-8435-2bc9e25938f1', 'prajith': Failed to execute Ansible host-deploy role: Internal server error, please check Ansible runner error log.. Please check logs for more details: /home/prajip/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20200109172227-10.70.42.92-66f0615c.log 2020-01-09 17:22:27,326+05 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] START, SetVdsStatusVDSCommand(HostName = prajith, SetVdsStatusVDSCommandParameters:{hostId='71155740-8f27-473c-8435-2bc9e25938f1', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 158fa975 2020-01-09 17:22:27,334+05 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] FINISH, SetVdsStatusVDSCommand, return: , log id: 158fa975 2020-01-09 17:22:27,345+05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] EVENT_ID: VDS_INSTALL_FAILED(505), Host prajith installation failed. Failed to execute Ansible host-deploy role: Internal server error, please check Ansible runner error log.. Please check logs for more details: /home/prajip/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20200109172227-10.70.42.92-66f0615c.log. 2020-01-09 17:22:27,355+05 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-16) [66f0615c] Lock freed to object 'EngineLock:{exclusiveLocks='[71155740-8f27-473c-8435-2bc9e25938f1=VDS]', sharedLocks=''}' ________ *My ansible runner version:-* [prajip@localhost ovirt-engine]$ ansible-runner --version 1.3.4 *Since im unable to find any logs after i get failure in the engine tailf a status ansible-runner gives me the following traceback* ● ansible-runner-service.service - Ansible Runner Service Loaded: loaded (/usr/lib/systemd/system/ansible-runner-service.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2020-01-09 17:20:47 IST; 10min ago Main PID: 19681 (gunicorn-3) Tasks: 3 (limit: 4915) Memory: 67.6M CGroup: /system.slice/ansible-runner-service.service ├─19681 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application ├─19690 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application └─19692 /usr/bin/python3 /usr/bin/gunicorn-3 -b localhost:50001 -w 2 runner_service.wsgi:application Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: return f(*args, **kwargs) Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: File "/usr/lib/python3.7/site-packages/runner_service/controllers/hosts.py", line 85, in get Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: response = get_host_membership(host_name) Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: File "/usr/lib/python3.7/site-packages/runner_service/services/hosts.py", line 98, in get_host_membership Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: inventory = AnsibleInventory() Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: File "/usr/lib/python3.7/site-packages/runner_service/inventory.py", line 120, in __init__ Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: self.load() Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: File "/usr/lib/python3.7/site-packages/runner_service/inventory.py", line 148, in load Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: self.fd.close() Jan 09 17:22:27 localhost.localdomain gunicorn-3[19681]: AttributeError: 'NoneType' object has no attribute 'close' _________________________________________ *Regards,* *Prajith K Prasad.* On Mon, Dec 16, 2019 at 8:03 PM Steven Rosenberg <srosenbe@redhat.com> wrote:
Dear Amit,
There are a few scenarios, but on adding a new Host freshly installed, the errors are as follows:
2019-12-15 16:41:55,347+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] EVENT_ID: VDS_ANSIBLE_INSTALL_STARTED(560), Ansible host-deploy playbook execution has started on host Host1.
2019-12-15 16:41:55,622+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] Exception: Internal server error, please check Ansible runner error log.
2019-12-15 16:41:55,623+02 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] Host installation failed for host '0e2b5314-4313-4f40-9dca-d716403aa51a', 'Host1': Failed to execute Ansible host-deploy role: Internal server error, please check Ansible runner error log.. Please check logs for more details: /home/srosenbe/ovirt_engine_master/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20191215164155-10.35.0.35-67eb9528.log
2019-12-15 16:41:55,630+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] START, SetVdsStatusVDSCommand(HostName = Host1, SetVdsStatusVDSCommandParameters:{hostId='0e2b5314-4313-4f40-9dca-d716403aa51a', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 64eb8f0b
2019-12-15 16:41:55,640+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] FINISH, SetVdsStatusVDSCommand, return: , log id: 64eb8f0b
2019-12-15 16:41:55,669+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] EVENT_ID: VDS_INSTALL_FAILED(505), Host Host1 installation failed. Failed to execute Ansible host-deploy role: Internal server error, please check Ansible runner error log.. Please check logs for more details: /home/srosenbe/ovirt_engine_master/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20191215164155-10.35.0.35-67eb9528.log.
2019-12-15 16:41:55,683+02 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [67eb9528] Lock freed to object 'EngineLock:{exclusiveLocks='[0e2b5314-4313-4f40-9dca-d716403aa51a=VDS]', sharedLocks=''}'
The host-deploy directory is empty however and the gunicorn journal fails as follows:
Dec 15 16:41:55 srosenbe.tlv.csb gunicorn[3206]: Creating env/cmdline file: --private-key /home/srosenbe/ovirt_engine_master/etc/pki/ovirt-engine/keys/engine_id_rsa Dec 15 16:41:55 srosenbe.tlv.csb gunicorn[3206]: Exception on /api/v1/playbooks/ovirt-host-deploy.yml [POST] Dec 15 16:41:55 srosenbe.tlv.csb gunicorn[3206]: Traceback (most recent call last):
The traceback is quite long.
I can provide more information accordingly.
With Best Regards.
Steven.
On Mon, Dec 16, 2019 at 4:26 PM Amit Bawer <abawer@redhat.com> wrote:
what do you see for your host on engine.log ? any SSH timeout issue?
On Mon, Dec 16, 2019 at 4:18 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
On Mon, Dec 16, 2019 at 7:45 PM Amit Bawer <abawer@redhat.com> wrote:
from snippet is seems you are under ~/work/ovirt-engine-builds/11-12-global-options
This is my $PREFIX . ⌂120% [kmajumde:~/work/ovirt-engine-builds/11-12-global-options] $ ls var/log/ovirt-engine/ ansible boot.log cinderlib dump engine.log host-deploy notifier ova server.log setup ui.log
trying to ls var in relative path to your working directory is it possible you have omitted the leading backslash?
ls */*var/log/ovirt-engine/host-deploy/
On Mon, Dec 16, 2019 at 4:11 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
No logs whatsoever. ``` ls var/log/ovirt-engine/host-deploy/ ⌂142% [kmajumde:~/work/ovirt-engine-builds/11-12-global-options] $ ```
On Mon, Dec 16, 2019 at 7:39 PM Amit Bawer <abawer@redhat.com> wrote:
On Mon, Dec 16, 2019 at 2:46 PM Kaustav Majumder <kmajumde@redhat.com> wrote:
> Hi, > i have tried setting up my devel env on an updated fedora 30. Engine > is running well but when I am trying to add a new host to the engine > (Centos 7.7) it is taking 2+ hrs and has still not added the host. Is this > expected behaviour? Also I can't find any host deploy or ansible logs. >
Aren't any new logs under engine machine path at /var/log/ovirt-engine/host-deploy/ ?
> On Thu, Dec 12, 2019 at 2:11 AM Martin Perina <mperina@redhat.com> > wrote: > >> >> >> On Wed, Dec 11, 2019 at 4:29 PM Dominik Holler <dholler@redhat.com> >> wrote: >> >>> >>> >>> On Wed, Nov 27, 2019 at 8:37 AM Ondra Machacek < >>> omachace@redhat.com> wrote: >>> >>>> Hello, >>>> >>>> we are going to merge a series of patches to master branch, which >>>> integrates ansible-runner with oVirt engine. When the patches >>>> will be >>>> merged you will need to install new package called ansible-runner- >>>> service-dev, and follow instructions so your dev-env will keep >>>> working >>>> smoothly(all relevant info will be also in README.adoc): >>>> >>>> 1) sudo dnf update ovirt-release-master >>>> >>>> 2) sudo dnf install -y ansible-runner-service-dev >>>> >>>> >>> "dnf install -y ansible-runner-service-dev" did not work for me on >>> fedora 29. >>> >> >> You need to have at least FC30, because ansible-runner on FC29 is >> too old >> >>> I created manually the file /etc/yum.repos.d/centos.repo: >>> [centos-ovirt44-testing] >>> name=CentOS-7 - oVirt 4.4 >>> baseurl= >>> http://cbs.centos.org/repos/virt7-ovirt-44-testing/$basearch/os/ >>> gpgcheck=0 >>> enabled=1 >>> >>> which made the ansible-runner-service-dev available. >>> >>> >>>> 3) Edit `/etc/ansible-runner-service/config.yaml` file as follows: >>>> >>>> --- >>>> playbooks_root_dir: >>>> '$PREFIX/share/ovirt-engine/ansible-runner-service-project' >>>> ssh_private_key: >>>> '$PREFIX/etc/pki/ovirt-engine/keys/engine_id_rsa' >>>> port: 50001 >>>> target_user: root >>>> >>>> Where `$PREFIX` is the prefix of your development environment >>>> prefix, >>>> which you've specified during the compilation of the engine. >>>> >>>> 4) Restart and enable ansible-runner-service: >>>> >>>> # systemctl restart ansible-runner-service >>>> # systemctl enable ansible-runner-service >>>> >>>> That's it, your dev-env should start using the >>>> ansible-runner-service >>>> for host-deployment etc. >>>> >>>> Please note that only Fedora 30/31 and Centos7 was packaged, and >>>> are >>>> natively supported! >>>> >>>> Thanks, >>>> Ondra >>>> _______________________________________________ >>>> Devel mailing list -- devel@ovirt.org >>>> To unsubscribe send an email to devel-leave@ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AFKGTV4WDNONLN... >>>> >>> _______________________________________________ >>> Devel mailing list -- devel@ovirt.org >>> To unsubscribe send an email to devel-leave@ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/RMNEQG7KNFWSQX... >>> >> >> >> -- >> Martin Perina >> Manager, Software Engineering >> Red Hat Czech s.r.o. >> _______________________________________________ >> Devel mailing list -- devel@ovirt.org >> To unsubscribe send an email to devel-leave@ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5TMTVOWQLF7GQS... >> > > > -- > > Thanks, > > Kaustav Majumder >
--
Thanks,
Kaustav Majumder
--
Thanks,
Kaustav Majumder
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CGGJJGZOTOFWUD...