engine-setup failing on restarting ovirt-imageio
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ Here's the relevant log in engine-setup: 2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout: 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr: 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout: 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details. 2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio' -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export... 2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file = This means engine-setup did not create correct configuration file. Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update? This works in real setup, so I guess this is another OST environment issue. Nir --
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export... 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root seems like it's getting a recent enough version. What about above warning? -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user. To resolve it, I created a dir: mkdir /etc/ovirt-imageio And I did chown to the user running engine-setup.
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QS4QG3OMVNZYQR...
you don't need to do this with this patch applied: https://gerrit.ovirt.org/#/c/108582/ On Mon, Apr 27, 2020 at 5:02 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user. To resolve it, I created a dir: mkdir /etc/ovirt-imageio And I did chown to the user running engine-setup.
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QS4QG3OMVNZYQR...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CVQAPWJW54XFGK...
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev? I think this was fixed here: commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200 spec: allow imageio config in dev env Add $PREFIX into imageio config directory to allow imageio configuration in development environment when config files are installed into a specified directory. Currently it fails with: [Errno 13] Permission denied: '/etc/ovirt-imageio' Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com> Maybe this OST runs old engine without this fix?
To resolve it, I created a dir: mkdir /etc/ovirt-imageio And I did chown to the user running engine-setup.
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QS4QG3OMVNZYQR...
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com>
ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception>>>> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod>>>> method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovi rt_imageio/config.py", line 190, in _closeup_resatrt_service>>>> state=state,
File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state>>>> service=name,
RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'>>> Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ex ported-artifacts/test_logs/basic-suite-master/post-001_initialize_engine _pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/daemon. log
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed>>> Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main>>> server = Server(cfg)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__>>> self.remote_service = services.RemoteService(self.config, self.auth)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__>>> self._secure_server()
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server>>> raise errors.TlsConfigurationError(*required_config)
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/exp orted-artifacts/test_logs/basic-suite-master/post-001_initialize_engine_p ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev?
I think this was fixed here:
commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200
spec: allow imageio config in dev env
Add $PREFIX into imageio config directory to allow imageio configuration in development environment when config files are installed into a specified directory. Currently it fails with:
[Errno 13] Permission denied: '/etc/ovirt-imageio'
Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com>
Maybe this OST runs old engine without this fix?
OST runs in install-dev environment? IIRC not. Looking into the logs, imageio wasn't properly configured, it seems there's no certificate configuration: ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file = (from https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...) and as Nir wrote, /etc/ovirt-imageio is not archived looking into otopi log (https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...) it looks like the config ws installed into /usr/etc/ovirt-imageio instead of into /usr/etc (so far no idea why, also I guess we need to throw clear exception if the config file is not found): 2020-04-27 09:25:00,332-0400 DEBUG otopi.transaction transaction.commit:147 committing 'File transaction for '/usr/etc/ovirt-imageio/daemon.conf''
To resolve it, I created a dir: mkdir /etc/ovirt-imageio And I did chown to the user running engine-setup.
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QS4QG3OMVNZ YQRCCZNENKRZ7SB3RKIDZ/
On Mon, Apr 27, 2020 at 5:22 PM Vojtech Juranek <vjuranek@redhat.com> wrote:
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com>
ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception>>>> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod>>>> method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovi rt_imageio/config.py", line 190, in _closeup_resatrt_service>>>> state=state,
File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state>>>> service=name,
RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'>>> Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ex ported-artifacts/test_logs/basic-suite-master/post-001_initialize_engine _pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/daemon. log
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed>>> Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main>>> server = Server(cfg)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__>>> self.remote_service = services.RemoteService(self.config, self.auth)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__>>> self._secure_server()
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server>>> raise errors.TlsConfigurationError(*required_config)
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/exp orted-artifacts/test_logs/basic-suite-master/post-001_initialize_engine_p ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev?
I think this was fixed here:
commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200
spec: allow imageio config in dev env
Add $PREFIX into imageio config directory to allow imageio configuration in development environment when config files are installed into a specified directory. Currently it fails with:
[Errno 13] Permission denied: '/etc/ovirt-imageio'
Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com>
Maybe this OST runs old engine without this fix?
OST runs in install-dev environment? IIRC not.
Looking into the logs, imageio wasn't properly configured, it seems there's no certificate configuration:
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
(from https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...)
and as Nir wrote, /etc/ovirt-imageio is not archived
looking into otopi log (https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...) it looks like the config ws installed into /usr/etc/ovirt-imageio instead of into /usr/etc
Maybe PREFIX was /usr when generating the rpms? This is the trouble with code that does not have automated tests. This means endless time spent on manual debugging. (so far no idea why, also I guess we need to throw clear exception if the config file is not found):
2020-04-27 09:25:00,332-0400 DEBUG otopi.transaction transaction.commit:147 committing 'File transaction for '/usr/etc/ovirt-imageio/daemon.conf''
To resolve it, I created a dir: mkdir /etc/ovirt-imageio And I did chown to the user running engine-setup.
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QS4QG3OMVNZ YQRCCZNENKRZ7SB3RKIDZ/
Il giorno lun 27 apr 2020 alle ore 16:28 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 5:22 PM Vojtech Juranek <vjuranek@redhat.com> wrote:
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com>
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola < sbonazzo@redhat.com>
wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer < nsoffer@redhat.com>
ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola < sbonazzo@redhat.com>
wrote: > Failing job is > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ > > Here's the relevant log in engine-setup: > > > 2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd > plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', > 'ovirt-imageio.service'), executable='None', cwd='None', env=None > 2020-04-27 09:25:35,652-0400 DEBUG > otopi.plugins.otopi.services.systemd plugin.executeRaw:863 > execute-result: ('/usr/bin/systemctl', 'stop', > 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG > otopi.plugins.otopi.services.systemd plugin.execute:921 > execute-output: ('/usr/bin/systemctl', 'stop', > 'ovirt-imageio.service') stdout: > > > 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd > plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', > 'ovirt-imageio.service') stderr: > > > 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd > systemd.state:170 starting service ovirt-imageio 2020-04-27 > 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd > plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', > 'ovirt-imageio.service'), executable='None', cwd='None', env=None > 2020-04-27 09:25:36,631-0400 DEBUG > otopi.plugins.otopi.services.systemd plugin.executeRaw:863 > execute-result: ('/usr/bin/systemctl', 'start', > 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG > otopi.plugins.otopi.services.systemd plugin.execute:921 > execute-output: ('/usr/bin/systemctl', 'start', > 'ovirt-imageio.service') stdout: > > > 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd > plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', > 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed > because the control process exited with error code. See "systemctl > status ovirt-imageio.service" and "journalctl -xe" for details. > > 2020-04-27 09:25:36,633-0400 DEBUG otopi.context > context._executeMethod:145 method exception>>>> > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, > in _executeMethod>>>> > method['method']() > > File > "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovi > rt_imageio/config.py", line 190, in _closeup_resatrt_service>>>> > state=state, > > File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, > in state>>>> > service=name, > > RuntimeError: Failed to start service 'ovirt-imageio' > 2020-04-27 09:25:36,636-0400 ERROR otopi.context > context._executeMethod:154 Failed to execute stage 'Closing up': > Failed to start service 'ovirt-imageio'>>> Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ex
_
log
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed>>> Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main>>> server = Server(cfg)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__>>> self.remote_service = services.RemoteService(self.config, self.auth)
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__>>> self._secure_server()
File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server>>> raise errors.TlsConfigurationError(*required_config)
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections
need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/exp
orted-artifacts/test_logs/basic-suite-master/post-001_initialize_engine_p
ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev?
I think this was fixed here:
commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200
spec: allow imageio config in dev env
Add $PREFIX into imageio config directory to allow imageio configuration in development environment when config files are installed into a specified directory. Currently it fails with:
[Errno 13] Permission denied: '/etc/ovirt-imageio'
Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com>
Maybe this OST runs old engine without this fix?
OST runs in install-dev environment? IIRC not.
Looking into the logs, imageio wasn't properly configured, it seems
wrote: ported-artifacts/test_logs/basic-suite-master/post-001_initialize_engine pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/daemon. scripts there's no certificate configuration:
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls]
section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
(from
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export... )
and as Nir wrote, /etc/ovirt-imageio is not archived
looking into otopi log (
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...) it looks like the config ws installed into /usr/etc/ovirt-imageio instead of into /usr/etc
Maybe PREFIX was /usr when generating the rpms?
This is the trouble with code that does not have automated tests. This means endless time spent on manual debugging.
(so far no idea why, also I guess we need to throw clear exception if the config file is not found):
Looking at Makefile: IMAGEIO_SYSCONF_DIR=$(PREFIX)/etc/ovirt-imageio and this make sense to resolve to /usr/etc/ovirt-imageio. should have been: IMAGEIO_SYSCONF_DIR=$(SYSCONF_DIR)/ovirt-imageio In spec file: PREFIX=%{_prefix} and rpm resolves: $ rpm --eval "%{_prefix}" /usr so this couldn't have worked before at all.
On Mon, Apr 27, 2020 at 5:37 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 16:28 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 5:22 PM Vojtech Juranek <vjuranek@redhat.com> wrote:
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com>
ha scritto: > On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> > > wrote: >> Failing job is >> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ >> >> Here's the relevant log in engine-setup: >> >> >> 2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd >> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', >> 'ovirt-imageio.service'), executable='None', cwd='None', env=None >> 2020-04-27 09:25:35,652-0400 DEBUG >> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >> execute-result: ('/usr/bin/systemctl', 'stop', >> 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG >> otopi.plugins.otopi.services.systemd plugin.execute:921 >> execute-output: ('/usr/bin/systemctl', 'stop', >> 'ovirt-imageio.service') stdout: >> >> >> 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd >> plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', >> 'ovirt-imageio.service') stderr: >> >> >> 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd >> systemd.state:170 starting service ovirt-imageio 2020-04-27 >> 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd >> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', >> 'ovirt-imageio.service'), executable='None', cwd='None', env=None >> 2020-04-27 09:25:36,631-0400 DEBUG >> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >> execute-result: ('/usr/bin/systemctl', 'start', >> 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG >> otopi.plugins.otopi.services.systemd plugin.execute:921 >> execute-output: ('/usr/bin/systemctl', 'start', >> 'ovirt-imageio.service') stdout: >> >> >> 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd >> plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', >> 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed >> because the control process exited with error code. See "systemctl >> status ovirt-imageio.service" and "journalctl -xe" for details. >> >> 2020-04-27 09:25:36,633-0400 DEBUG otopi.context >> context._executeMethod:145 method exception>>>> >> Traceback (most recent call last): >> File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, >> in _executeMethod>>>> >> method['method']() >> >> File >> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovi >> rt_imageio/config.py", line 190, in _closeup_resatrt_service>>>> >> state=state, >> >> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, >> in state>>>> >> service=name, >> >> RuntimeError: Failed to start service 'ovirt-imageio' >> 2020-04-27 09:25:36,636-0400 ERROR otopi.context >> context._executeMethod:154 Failed to execute stage 'Closing up': >> Failed to start service 'ovirt-imageio'>>> > Looking at the logs at: > > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ex > ported-artifacts/test_logs/basic-suite-master/post-001_initialize_engine > _pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/daemon. > log > > 2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting > (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) > [server] Server failed>>> > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", > line 37, in main>>> > server = Server(cfg) > > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", > line 76, in __init__>>> > self.remote_service = services.RemoteService(self.config, self.auth) > > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", > line 80, in __init__>>> > self._secure_server() > > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", > line 95, in _secure_server>>> > raise errors.TlsConfigurationError(*required_config) > > ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] > section in daemon.conf), but not configured: ca_file = , cert_file = , > key_file = > > This means engine-setup did not create correct configuration file. > > Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ > I don't see /etc/ovirt-imageio directory, maybe the collections scripts > need update? > > This works in real setup, so I guess this is another OST environment > issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/exp orted-artifacts/test_logs/basic-suite-master/post-001_initialize_engine_p ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev?
I think this was fixed here:
commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200
spec: allow imageio config in dev env
Add $PREFIX into imageio config directory to allow imageio configuration in development environment when config files are installed into a specified directory. Currently it fails with:
[Errno 13] Permission denied: '/etc/ovirt-imageio'
Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com>
Maybe this OST runs old engine without this fix?
OST runs in install-dev environment? IIRC not.
Looking into the logs, imageio wasn't properly configured, it seems there's no certificate configuration:
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
(from https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...)
and as Nir wrote, /etc/ovirt-imageio is not archived
looking into otopi log (https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...) it looks like the config ws installed into /usr/etc/ovirt-imageio instead of into /usr/etc
Maybe PREFIX was /usr when generating the rpms?
This is the trouble with code that does not have automated tests. This means endless time spent on manual debugging.
(so far no idea why, also I guess we need to throw clear exception if the config file is not found):
Looking at Makefile: IMAGEIO_SYSCONF_DIR=$(PREFIX)/etc/ovirt-imageio
and this make sense to resolve to /usr/etc/ovirt-imageio.
should have been: IMAGEIO_SYSCONF_DIR=$(SYSCONF_DIR)/ovirt-imageio
In spec file: PREFIX=%{_prefix} and rpm resolves: $ rpm --eval "%{_prefix}" /usr
so this couldn't have worked before at all.
Looks like the patch adding $PREFIX was not tested with real setup, only with development setup. Vojta, do you want to test a fix?
On Mon, Apr 27, 2020 at 5:37 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 16:28 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 5:22 PM Vojtech Juranek <vjuranek@redhat.com> wrote:
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg <lrotenbe@redhat.com> wrote:
On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola <sbonazzo@redhat.com>
wrote: > Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer > <nsoffer@redhat.com> > > ha scritto: >> On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola >> <sbonazzo@redhat.com> >> >> wrote: >>> Failing job is >>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ >>> >>> Here's the relevant log in engine-setup: >>> >>> >>> 2020-04-27 09:25:35,457-0400 DEBUG >>> otopi.plugins.otopi.services.systemd >>> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', >>> 'ovirt-imageio.service'), executable='None', cwd='None', >>> env=None >>> 2020-04-27 09:25:35,652-0400 DEBUG >>> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >>> execute-result: ('/usr/bin/systemctl', 'stop', >>> 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 >>> DEBUG >>> otopi.plugins.otopi.services.systemd plugin.execute:921 >>> execute-output: ('/usr/bin/systemctl', 'stop', >>> 'ovirt-imageio.service') stdout: >>> >>> >>> 2020-04-27 09:25:35,653-0400 DEBUG >>> otopi.plugins.otopi.services.systemd >>> plugin.execute:926 execute-output: ('/usr/bin/systemctl', >>> 'stop', >>> 'ovirt-imageio.service') stderr: >>> >>> >>> 2020-04-27 09:25:35,653-0400 DEBUG >>> otopi.plugins.otopi.services.systemd >>> systemd.state:170 starting service ovirt-imageio 2020-04-27 >>> 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd >>> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', >>> 'ovirt-imageio.service'), executable='None', cwd='None', >>> env=None >>> 2020-04-27 09:25:36,631-0400 DEBUG >>> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >>> execute-result: ('/usr/bin/systemctl', 'start', >>> 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 >>> DEBUG >>> otopi.plugins.otopi.services.systemd plugin.execute:921 >>> execute-output: ('/usr/bin/systemctl', 'start', >>> 'ovirt-imageio.service') stdout: >>> >>> >>> 2020-04-27 09:25:36,632-0400 DEBUG >>> otopi.plugins.otopi.services.systemd >>> plugin.execute:926 execute-output: ('/usr/bin/systemctl', >>> 'start', >>> 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service >>> failed >>> because the control process exited with error code. See >>> "systemctl >>> status ovirt-imageio.service" and "journalctl -xe" for details. >>> >>> 2020-04-27 09:25:36,633-0400 DEBUG otopi.context >>> context._executeMethod:145 method exception>>>> >>> >>> Traceback (most recent call last): >>> File "/usr/lib/python3.6/site-packages/otopi/context.py", line >>> 132, >>> in _executeMethod>>>> >>> >>> method['method']() >>> >>> File >>> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-set >>> up/ovi >>> rt_imageio/config.py", line 190, in >>> _closeup_resatrt_service>>>> >>> >>> state=state, >>> >>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", >>> line 181, >>> in state>>>> >>> >>> service=name, >>> >>> RuntimeError: Failed to start service 'ovirt-imageio' >>> 2020-04-27 09:25:36,636-0400 ERROR otopi.context >>> context._executeMethod:154 Failed to execute stage 'Closing up': >>> Failed to start service 'ovirt-imageio'>>> >> >> Looking at the logs at: >> >> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/arti >> fact/ex >> ported-artifacts/test_logs/basic-suite-master/post-001_initialize >> _engine >> _pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/ >> daemon. >> log >> >> 2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting >> (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR >> (MainThread) >> [server] Server failed>>> >> >> Traceback (most recent call last): >> File >> "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", >> line 37, in main>>> >> >> server = Server(cfg) >> >> File >> "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", >> line 76, in __init__>>> >> >> self.remote_service = services.RemoteService(self.config, >> self.auth) >> >> File >> "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py" >> , >> line 80, in __init__>>> >> >> self._secure_server() >> >> File >> "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py" >> , >> line 95, in _secure_server>>> >> >> raise errors.TlsConfigurationError(*required_config) >> >> ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see >> [tls] >> section in daemon.conf), but not configured: ca_file = , >> cert_file = , >> key_file = >> >> This means engine-setup did not create correct configuration >> file. >> >> Looking in >> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ >> I don't see /etc/ovirt-imageio directory, maybe the collections >> scripts >> need update? >> >> This works in real setup, so I guess this is another OST >> environment >> issue. > > looking at dnf logs: > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artif > act/exp > orted-artifacts/test_logs/basic-suite-master/post-001_initialize_e > ngine_p > ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log > > 2020-04-27T13:16:13Z SUBDEBUG Installed: > ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > 2020-04-27T13:16:13Z SUBDEBUG Installed: > ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > 2020-04-27T13:16:13Z SUBDEBUG Installed: > ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > 2020-04-27T13:16:14Z SUBDEBUG Installed: > ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > 2020-04-27T13:16:14Z INFO warning: group Development/Debug does > not > exist - using root > > seems like it's getting a recent enough version. What about above > warning?
I had this problem in my local environment. I was running engine-setup with an unprivileged user.
For make install-dev?
I think this was fixed here:
commit 81a5e757fd8768ace825140ca7dc701cc2e011ce Author: Vojtech Juranek <vjuranek@redhat.com> Date: Thu Apr 23 11:44:41 2020 +0200
spec: allow imageio config in dev env
Add $PREFIX into imageio config directory to allow imageio configuration
in development environment when config files are installed into a specified directory.
Currently it fails with: [Errno 13] Permission denied: '/etc/ovirt-imageio'
Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc Signed-off-by: Vojtech Juranek <vjuranek@redhat.com>
Maybe this OST runs old engine without this fix?
OST runs in install-dev environment? IIRC not.
Looking into the logs, imageio wasn't properly configured, it seems
On pondělí 27. dubna 2020 16:47:56 CEST Nir Soffer wrote: there's no certificate configuration:
ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =>> > (from https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ exported-artifacts/test_logs/basic-suite-master/post-001_initialize_eng ine_pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/dae mon.log)
and as Nir wrote, /etc/ovirt-imageio is not archived
looking into otopi log (https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact /exported-artifacts/test_logs/basic-suite-master/post-001_initialize_en gine_pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-engine/set up/ovirt-engine-setup-20200427091928-9e7d0z.log) it looks like the config ws installed into /usr/etc/ovirt-imageio instead of into /usr/etc>>
Maybe PREFIX was /usr when generating the rpms?
This is the trouble with code that does not have automated tests. This means endless time spent on manual debugging.
(so far no idea why, also I guess we need to throw clear exception if
the config file is not found): Looking at Makefile: IMAGEIO_SYSCONF_DIR=$(PREFIX)/etc/ovirt-imageio
and this make sense to resolve to /usr/etc/ovirt-imageio.
should have been: IMAGEIO_SYSCONF_DIR=$(SYSCONF_DIR)/ovirt-imageio
In spec file: PREFIX=%{_prefix} and rpm resolves: $ rpm --eval "%{_prefix}" /usr
so this couldn't have worked before at all.
Looks like the patch adding $PREFIX was not tested with real setup, only with development setup.
yes, and actually I haven't finished testing it even in dev setup yet
Vojta, do you want to test a fix?
I'll, but need to look more into it, replacing $(PREFIX) with $(SYSCONF_DIR) doesn't seem to work as SYSCONF_DIR=$(PREFIX)/etc (see https://github.com/ oVirt/ovirt-engine/blob/master/Makefile#L41), so we will end up again with / usr/etc. We will probably have to do it as VMCONSOLE_* variable - hardcode config file and define new just install-dev environment (and do the proper testing before merge)
Il giorno lun 27 apr 2020 alle ore 16:58 Vojtech Juranek < vjuranek@redhat.com> ha scritto:
On pondělí 27. dubna 2020 16:47:56 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:37 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 16:28 Nir Soffer < nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 5:22 PM Vojtech Juranek <vjuranek@redhat.com>
On pondělí 27. dubna 2020 16:10:11 CEST Nir Soffer wrote:
On Mon, Apr 27, 2020 at 5:01 PM Liran Rotenberg < lrotenbe@redhat.com> wrote: > On Mon, Apr 27, 2020 at 4:58 PM Sandro Bonazzola > <sbonazzo@redhat.com> > > wrote: >> Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer >> <nsoffer@redhat.com> >> >> ha scritto: >>> On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola >>> <sbonazzo@redhat.com> >>> >>> wrote: >>>> Failing job is >>>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ >>>> >>>> Here's the relevant log in engine-setup: >>>> >>>> >>>> 2020-04-27 09:25:35,457-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd >>>> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', >>>> 'ovirt-imageio.service'), executable='None', cwd='None', >>>> env=None >>>> 2020-04-27 09:25:35,652-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >>>> execute-result: ('/usr/bin/systemctl', 'stop', >>>> 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 >>>> DEBUG >>>> otopi.plugins.otopi.services.systemd plugin.execute:921 >>>> execute-output: ('/usr/bin/systemctl', 'stop', >>>> 'ovirt-imageio.service') stdout: >>>> >>>> >>>> 2020-04-27 09:25:35,653-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd >>>> plugin.execute:926 execute-output: ('/usr/bin/systemctl', >>>> 'stop', >>>> 'ovirt-imageio.service') stderr: >>>> >>>> >>>> 2020-04-27 09:25:35,653-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd >>>> systemd.state:170 starting service ovirt-imageio 2020-04-27 >>>> 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd >>>> plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', >>>> 'ovirt-imageio.service'), executable='None', cwd='None', >>>> env=None >>>> 2020-04-27 09:25:36,631-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd plugin.executeRaw:863 >>>> execute-result: ('/usr/bin/systemctl', 'start', >>>> 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 >>>> DEBUG >>>> otopi.plugins.otopi.services.systemd plugin.execute:921 >>>> execute-output: ('/usr/bin/systemctl', 'start', >>>> 'ovirt-imageio.service') stdout: >>>> >>>> >>>> 2020-04-27 09:25:36,632-0400 DEBUG >>>> otopi.plugins.otopi.services.systemd >>>> plugin.execute:926 execute-output: ('/usr/bin/systemctl', >>>> 'start', >>>> 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service >>>> failed >>>> because the control process exited with error code. See >>>> "systemctl >>>> status ovirt-imageio.service" and "journalctl -xe" for
>>>> >>>> 2020-04-27 09:25:36,633-0400 DEBUG otopi.context >>>> context._executeMethod:145 method exception>>>> >>>> >>>> Traceback (most recent call last): >>>> File "/usr/lib/python3.6/site-packages/otopi/context.py",
>>>> 132, >>>> in _executeMethod>>>> >>>> >>>> method['method']() >>>> >>>> File >>>> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-set >>>> up/ovi >>>> rt_imageio/config.py", line 190, in >>>> _closeup_resatrt_service>>>> >>>> >>>> state=state, >>>> >>>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", >>>> line 181, >>>> in state>>>> >>>> >>>> service=name, >>>> >>>> RuntimeError: Failed to start service 'ovirt-imageio' >>>> 2020-04-27 09:25:36,636-0400 ERROR otopi.context >>>> context._executeMethod:154 Failed to execute stage 'Closing up': >>>> Failed to start service 'ovirt-imageio'>>> >>> >>> Looking at the logs at: >>> >>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/arti >>> fact/ex >>>
>>> _engine >>> _
wrote: details. line ported-artifacts/test_logs/basic-suite-master/post-001_initialize pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/
>>> daemon. >>> log >>> >>> 2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting >>> (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR >>> (MainThread) >>> [server] Server failed>>> >>> >>> Traceback (most recent call last): >>> File >>> "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", >>> line 37, in main>>> >>> >>> server = Server(cfg) >>> >>> File >>> "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", >>> line 76, in __init__>>> >>> >>> self.remote_service = services.RemoteService(self.config, >>> self.auth) >>> >>> File >>> "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py" >>> , >>> line 80, in __init__>>> >>> >>> self._secure_server() >>> >>> File >>> "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py" >>> , >>> line 95, in _secure_server>>> >>> >>> raise errors.TlsConfigurationError(*required_config) >>> >>> ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see >>> [tls] >>> section in daemon.conf), but not configured: ca_file = , >>> cert_file = , >>> key_file = >>> >>> This means engine-setup did not create correct configuration >>> file. >>> >>> Looking in >>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ >>> I don't see /etc/ovirt-imageio directory, maybe the
collections > >> > > >>> scripts > >> > > >>> need update? > >> > > >>> > >> > > >>> This works in real setup, so I guess this is another OST > >> > > >>> environment > >> > > >>> issue. > >> > > >> > >> > > >> looking at dnf logs: > >> > > >> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artif > >> > > >> act/exp > >> > > >> orted-artifacts/test_logs/basic-suite-master/post-001_initialize_e > >> > > >> ngine_p > >> > > >> ytest.py/lago-basic-suite-master-engine/_var_log/dnf.rpm.log > >> > > >> > >> > > >> 2020-04-27T13:16:13Z SUBDEBUG Installed: > >> > > >> ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > >> > > >> 2020-04-27T13:16:13Z SUBDEBUG Installed: > >> > > >> ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > >> > > >> 2020-04-27T13:16:13Z SUBDEBUG Installed: > >> > > >> ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > >> > > >> 2020-04-27T13:16:14Z SUBDEBUG Installed: > >> > > >> ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 > >> > > >> 2020-04-27T13:16:14Z INFO warning: group Development/Debug does > >> > > >> not > >> > > >> exist - using root > >> > > >> > >> > > >> seems like it's getting a recent enough version. What about above > >> > > >> warning? > >> > > > > >> > > > I had this problem in my local environment. I was running > >> > > > engine-setup > >> > > > with an unprivileged user. > >> > > > >> > > For make install-dev? > >> > > > >> > > I think this was fixed here: > >> > > > >> > > commit 81a5e757fd8768ace825140ca7dc701cc2e011ce > >> > > Author: Vojtech Juranek <vjuranek@redhat.com> > >> > > Date: Thu Apr 23 11:44:41 2020 +0200 > >> > > > >> > > spec: allow imageio config in dev env > >> > > > >> > > Add $PREFIX into imageio config directory to allow imageio > >> > > configuration > >> > > > >> > > in development environment when config files are installed into a > >> > > specified > >> > > directory. > >> > > > >> > > Currently it fails with: > >> > > [Errno 13] Permission denied: '/etc/ovirt-imageio' > >> > > > >> > > Change-Id: I9e13bba0e97f681b0b07d5a135be76113e391ddc > >> > > Signed-off-by: Vojtech Juranek <vjuranek@redhat.com> > >> > > > >> > > Maybe this OST runs old engine without this fix? > >> > > >> > OST runs in install-dev environment? IIRC not. > >> > > >> > Looking into the logs, imageio wasn't properly configured, it seems there's no certificate configuration: > >> > ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] > >> > section in daemon.conf), but not configured: ca_file = , cert_file > >> > = , key_file =>> > > >> > (from > >> > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/ > >> > exported-artifacts/test_logs/basic-suite-master/post-001_initialize_eng > >> > ine_pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-imageio/dae > >> > mon.log) > >> > > >> > and as Nir wrote, /etc/ovirt-imageio is not archived > >> > > >> > looking into otopi log > >> > ( https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact > >> > /exported-artifacts/test_logs/basic-suite-master/post-001_initialize_en > >> > gine_pytest.py/lago-basic-suite-master-engine/_var_log/ovirt-engine/set > >> > up/ovirt-engine-setup-20200427091928-9e7d0z.log) it looks like the > >> > config ws installed into /usr/etc/ovirt-imageio instead of into > >> > /usr/etc>> > >> Maybe PREFIX was /usr when generating the rpms? > >> > >> This is the trouble with code that does not have automated tests. This > >> means endless > >> time spent on manual debugging. > >> > >> (so far no idea why, also I guess we need to throw clear exception if > > > >> the config file is not found): > > Looking at Makefile: > > IMAGEIO_SYSCONF_DIR=$(PREFIX)/etc/ovirt-imageio > > > > and this make sense to resolve to /usr/etc/ovirt-imageio. > > > > should have been: > > IMAGEIO_SYSCONF_DIR=$(SYSCONF_DIR)/ovirt-imageio > > > > In spec file: > > PREFIX=%{_prefix} > > and rpm resolves: > > $ rpm --eval "%{_prefix}" > > /usr > > > > so this couldn't have worked before at all. > > Looks like the patch adding $PREFIX was not tested with real setup, > only with development setup.
yes, and actually I haven't finished testing it even in dev setup yet
Vojta, do you want to test a fix?
I'll, but need to look more into it, replacing $(PREFIX) with $(SYSCONF_DIR) doesn't seem to work as SYSCONF_DIR=$(PREFIX)/etc (see https://github.com/ oVirt/ovirt-engine/blob/master/Makefile#L41 <https://github.com/oVirt/ovirt-engine/blob/master/Makefile#L41>), so we will end up again with / usr/etc. We will probably have to do it as VMCONSOLE_* variable - hardcode config file and define new just install-dev environment (and do the proper testing before merge)
No. SYSCONF_DIR=$(PREFIX)/etc allow you to set PREFIX for developer work. But then in spec file it's overridden: SYSCONF_DIR=%{_sysconfdir} $ rpm --eval "%{_sysconfdir}" /etc -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
I'll, but need to look more into it, replacing $(PREFIX) with $(SYSCONF_DIR) doesn't seem to work as SYSCONF_DIR=$(PREFIX)/etc (see https://github.com/ oVirt/ovirt-engine/blob/master/Makefile#L41 <https://github.com/oVirt/ovirt-engine/blob/master/Makefile#L41>), so we will end up again with / usr/etc. We will probably have to do it as VMCONSOLE_* variable - hardcode config file and define new just install-dev environment (and do the proper testing before merge)
No. SYSCONF_DIR=$(PREFIX)/etc allow you to set PREFIX for developer work.
isn't it overwritten here: https://github.com/oVirt/ovirt-engine/blob/master/Makefile#L37 ? I posted https://gerrit.ovirt.org//c/108672/ which does it in the same way as ovirt-vmconsole, but if you prefer using SYSCONFIG_DIR and it work in all cases, no problem to change it this way
But then in spec file it's overridden: SYSCONF_DIR=%{_sysconfdir} $ rpm --eval "%{_sysconfdir}" /etc
Il giorno lun 27 apr 2020 alle ore 17:33 Vojtech Juranek < vjuranek@redhat.com> ha scritto:
I'll, but need to look more into it, replacing $(PREFIX) with $(SYSCONF_DIR) doesn't seem to work as SYSCONF_DIR=$(PREFIX)/etc (see https://github.com/ oVirt/ovirt-engine/blob/master/Makefile#L41 <https://github.com/oVirt/ovirt-engine/blob/master/Makefile#L41>), so we will end up again with / usr/etc. We will probably have to do it as VMCONSOLE_* variable - hardcode config file and define new just install-dev environment (and do the proper testing before merge)
No. SYSCONF_DIR=$(PREFIX)/etc allow you to set PREFIX for developer work.
isn't it overwritten here: https://github.com/oVirt/ovirt-engine/blob/master/Makefile#L37 ?
parameters passed to make command are overriding those included in Makefile so no, it's not overwritten.
I posted https://gerrit.ovirt.org//c/108672/ which does it in the same way as ovirt-vmconsole, but if you prefer using SYSCONFIG_DIR and it work in all cases, no problem to change it this way
No strong preference, if this works I'm fine keeping it.
But then in spec file it's overridden: SYSCONF_DIR=%{_sysconfdir} $ rpm --eval "%{_sysconfdir}" /etc
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On Mon, Apr 27, 2020 at 4:57 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I think this is: Group: Development/Libraries What would be a good value? is this needed? --
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
Il giorno lun 27 apr 2020 alle ore 16:07 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:57 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I think this is:
Group: Development/Libraries
What would be a good value? is this needed?
No, you should drop it
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On Mon, Apr 27, 2020 at 5:29 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 16:07 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:57 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 27 apr 2020 alle ore 15:52 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Mon, Apr 27, 2020 at 4:42 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Failing job is https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/
Here's the relevant log in engine-setup:
2020-04-27 09:25:35,457-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:35,652-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service'), rc=0 2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'stop', 'ovirt-imageio.service') stderr:
2020-04-27 09:25:35,653-0400 DEBUG otopi.plugins.otopi.services.systemd systemd.state:170 starting service ovirt-imageio 2020-04-27 09:25:35,654-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), executable='None', cwd='None', env=None 2020-04-27 09:25:36,631-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service'), rc=1 2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stdout:
2020-04-27 09:25:36,632-0400 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio.service') stderr: Job for ovirt-imageio.service failed because the control process exited with error code. See "systemctl status ovirt-imageio.service" and "journalctl -xe" for details.
2020-04-27 09:25:36,633-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio/config.py", line 190, in _closeup_resatrt_service state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 181, in state service=name, RuntimeError: Failed to start service 'ovirt-imageio' 2020-04-27 09:25:36,636-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio'
Looking at the logs at:
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27 09:25:39,887 INFO (MainThread) [server] Starting (pid=17137, version=2.0.5) 2020-04-27 09:25:39,944 ERROR (MainThread) [server] Server failed Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 37, in main server = Server(cfg) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/server.py", line 76, in __init__ self.remote_service = services.RemoteService(self.config, self.auth) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 80, in __init__ self._secure_server() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/services.py", line 95, in _secure_server raise errors.TlsConfigurationError(*required_config) ovirt_imageio.errors.TlsConfigurationError: TLS enabled (see [tls] section in daemon.conf), but not configured: ca_file = , cert_file = , key_file =
This means engine-setup did not create correct configuration file.
Looking in https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/ I don't see /etc/ovirt-imageio directory, maybe the collections scripts need update?
This works in real setup, so I guess this is another OST environment issue.
looking at dnf logs: https://jenkins.ovirt.org/job/ovirt-system-tests_manual/6794/artifact/export...
2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-common-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:13Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z SUBDEBUG Installed: ovirt-imageio-daemon-2.0.5-0.202004260154.gitaf86468.el8.x86_64 2020-04-27T13:16:14Z INFO warning: group Development/Debug does not exist - using root
seems like it's getting a recent enough version. What about above warning?
I think this is:
Group: Development/Libraries
What would be a good value? is this needed?
No, you should drop it
Remove here: https://gerrit.ovirt.org/c/108671/
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
participants (5)
-
Benny Zlotnik -
Liran Rotenberg -
Nir Soffer -
Sandro Bonazzola -
Vojtech Juranek