I doubt that patch has anything to do with ovirt-imageio not starting.

Daniel/Nir - any insight?

On Mon, Nov 27, 2017 at 5:11 PM, Dafna Ron <dron@redhat.com> wrote:

Hi,

We have a failure reported in ovirt 4.1 on ovirt-imageio not starting and causing initialize_engine to fail

I am not sure the reported patch is related but it was reported by cq as the cause of the failure.

Link and headline of suspected patches:

Renamed disable-multipath.sh script to blacklist_all_disks.sh - https://gerrit.ovirt.org/#/c/84520/


Link to Job:

http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1384/


Link to all logs:

http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1384/artifact


(Relevant) error snippet from the log:

<error>

From lago log: 

[ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio-proxy'



Stacktrace

  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
    testMethod()
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
    return func(get_test_prefix(), *args, **kwargs)
  File "/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-prevrelease-suite-4.1/test-scenarios-before-upgrade/001_initialize_engine.py", line 46, in test_initialize_engine
    result.code, 0, 'engine-setup failed. Exit code is %s' % result.code
  File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_
    raise AssertionError(msg or "%r != %r" % (a, b))
"engine-setup failed. Exit code is 1\n-------------------- >> begin captured logging << --------------------\nlago.plugins.vm: INFO:     * Copy /home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-prevrelease-suite-4.1/engine-answer-file.conf to lago-upgrade-from-prevrelease-suite-4-1-engine:/tmp/answer-file-pre: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:e48d997a-49c2-4a9e-a87a-b99e79b4d47b:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine:\nlago.ssh: DEBUG: Still got 100 tries for lago-upgrade-from-prevrelease-suite-4-1-engine\nlago.ssh: DEBUG: end task:e48d997a-49c2-4a9e-a87a-b99e79b4d47b:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine:\nlago.plugins.vm: INFO:     * Copy /home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-prevrelease-suite-4.1/engine-answer-file.conf to lago-upgrade-from-prevrelease-suite-4-1-engine:/tmp/answer-file-pre: \x1b[32mSuccess\x1b[0m (in 0:00:00)\nlago.ssh: DEBUG: start task:8d1fe1d8-ad8f-4523-a0ce-94e9849a4403:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine:\nlago.ssh: DEBUG: Still got 100 tries for lago-upgrade-from-prevrelease-suite-4-1-engine\nlago.ssh: DEBUG: end task:8d1fe1d8-ad8f-4523-a0ce-94e9849a4403:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine:\nlago.ssh: DEBUG: Running 50e4d668 on lago-upgrade-from-prevrelease-suite-4-1-engine: engine-setup --config-append=/tmp/answer-file-pre\nlago.ssh: DEBUG: Command 50e4d668 on lago-upgrade-from-prevrelease-suite-4-1-engine returned with 1\nlago.ssh: DEBUG: Command 50e4d668 on lago-upgrade-from-prevrelease-suite-4-1-engine output:\n [ INFO  ] Stage: Initializing\n[ INFO  ] Stage: Environment setup\n          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/tmp/answer-file-pre']\n          Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20171127081018-s7jb49.log\n          Version: otopi-1.5.2 (otopi-1.5.2-1.el7.centos)\n[ INFO  ] Stage: Environment packages setup\n[ INFO  ] Yum Status: Downloading Packages\n[ INFO  ] Yum Download/Verify: iproute-3.10.0-87.el7.x86_64\n[ INFO  ] Yum Status: Check Package Signatures\n[ INFO  ] Yum Status: Running Test Transaction\n[ INFO  ] Yum Status: Running Transaction\n[ INFO  ] Yum update: 1/2: iproute-3.10.0-87.el7.x86_64\n[ INFO  ] Yum updated: 2/2: iproute\n[ INFO  ] Yum Verify: 1/2: iproute.x86_64 0:3.10.0-87.el7 - u\n[ INFO  ] Yum Verify: 2/2: iproute.x86_64 0:3.10.0-54.el7 - ud\n[ INFO  ] Stage: Programs detection\n[ INFO  ] Stage: Environment setup\n[ INFO  ] Stage: Environment customization\n         \n          --== PRODUCT OPTIONS ==--\n         \n         \n          --== PACKAGES ==--\n         \n[ INFO  ] Checking for an update for Setup...\n          No update for Setup found\n         \n          --== NETWORK CONFIGURATION ==--\n         \n[WARNING] Host name engine has no domain suffix\n[WARNING] Failed to resolve engine using DNS, it can be resolved only locally\n[ INFO  ] firewalld will be configured as firewall manager.\n[WARNING] Host name engine has no domain suffix\n[WARNING] Host name engine has no domain suffix\n         \n          --== DATABASE CONFIGURATION ==--\n         \n         \n          --== OVIRT ENGINE CONFIGURATION ==--\n         \n         \n          --== STORAGE CONFIGURATION ==--\n         \n         \n          --== PKI CONFIGURATION ==--\n         \n         \n          --== APACHE CONFIGURATION ==--\n         \n         \n          --== SYSTEM CONFIGURATION ==--\n         \n         \n          --== MISC CONFIGURATION ==--\n         \n         \n          --== END OF CONFIGURATION ==--\n         \n[ INFO  ] Stage: Setup validation\n[WARNING] Less than 16384MB of memory is available\n         \n          --== CONFIGURATION PREVIEW ==--\n         \n          Application mode                        : both\n          Default SAN wipe after delete           : False\n          Firewall manager                        : firewalld\n          Update Firewall                         : True\n          Host FQDN                               : engine\n          Upgrade packages                        : True\n          Engine database secured connection      : False\n          Engine database host                    : localhost\n          Engine database user name               : engine\n          Engine database name                    : engine\n          Engine database port                    : 5432\n          Engine database host name validation    : False\n          DWH database secured connection         : False\n          DWH database host                       : localhost\n          DWH database user name                  : ovirt_engine_history\n          DWH database name                       : ovirt_engine_history\n          DWH database port                       : 5432\n          DWH database host name validation       : False\n          Engine installation                     : True\n          PKI organization                        : Test\n          Configure local Engine database         : True\n          Set application as default page         : True\n          Configure Apache SSL                    : True\n          DWH installation                        : True\n          Configure local DWH database            : True\n          Engine Host FQDN                        : engine\n          Configure Image I/O Proxy               : True\n          Configure VMConsole Proxy               : False\n          Configure WebSocket Proxy               : True\n[ INFO  ] Stage: Transaction setup\n[ INFO  ] Stopping engine service\n[ INFO  ] Stopping ovirt-fence-kdump-listener service\n[ INFO  ] Stopping dwh service\n[ INFO  ] Stopping Image I/O Proxy service\n[ INFO  ] Stopping websocket-proxy service\n[ INFO  ] Stage: Misc configuration\n[ INFO  ] Stage: Package installation\n[ INFO  ] Stage: Misc configuration\n[ INFO  ] Upgrading CA\n[ INFO  ] Initializing PostgreSQL\n[ INFO  ] Creating PostgreSQL 'engine' database\n[ INFO  ] Configuring PostgreSQL\n[ INFO  ] Creating PostgreSQL 'ovirt_engine_history' database\n[ INFO  ] Configuring PostgreSQL\n[ INFO  ] Creating CA\n[ INFO  ] Creating/refreshing Engine database schema\n[ INFO  ] Creating/refreshing DWH database schema\n[ INFO  ] Configuring Image I/O Proxy\n[ INFO  ] Configuring WebSocket Proxy\n[ INFO  ] Creating/refreshing Engine 'internal' domain database schema\n[ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'\n[ INFO  ] Stage: Transaction commit\n[ INFO  ] Stage: Closing up\n[ INFO  ] Starting engine service\n[ INFO  ] Starting dwh service\n[ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio-proxy'\n[ INFO  ] Stage: Clean up\n          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20171127081018-s7jb49.log\n[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20171127081126-setup.conf'\n[ INFO  ] Stage: Pre-termination\n[ INFO  ] Stage: Termination\n[ ERROR ] Execution of setup failed\n\n--------------------- >> end captured logging << ---------------------"


from setup log:

http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1384/artifact/exported-artifacts/upgrade-from-prevrelease-suit-4.1-el7/test_logs/upgrade-from-prevrelease-suite-4.1/post-001_initialize_engine.py/lago-upgrade-from-prevrelease-suite-4-1-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20171127081018-s7jb49.log


2017-11-27 08:11:26 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/bin/systemctl', 'start', 'ovirt-imageio-proxy.service') stderr:
Job for ovirt-imageio-proxy.service failed because the control process exited with error code. See "systemctl status ovirt-imageio-proxy.service" and "journalctl -xe" for details.

2017-11-27 08:11:26 DEBUG otopi.context context._executeMethod:142 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt_imageio_proxy/config.py", line 351, in _closeup
    state=state,
  File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state
    service=name,
RuntimeError: Failed to start service 'ovirt-imageio-proxy'
2017-11-27 08:11:26 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio-proxy'
2017-11-27 08:11:26 DEBUG otopi.context context.dumpEnvironment:760 ENVIRONMENT DUMP - BEGIN
2017-11-27 08:11:26 DEBUG otopi.context context.dumpEnvironment:770 ENV BASE/error=bool:'True'
2017-11-27 08:11:26 DEBUG otopi.context context.dumpEnvironment:770 ENV BASE/exceptionInfo=list:'[(<type 'exceptions.RuntimeError'>, RuntimeError("Failed to start service 'ovirt-imageio-proxy'",), <traceback object at 0x4332d40>)]'
2017-11-27 08:11:26 DEBUG otopi.context context.dumpEnvironment:774 ENVIRONMENT DUMP - END
2017-11-27 08:11:26 INFO otopi.context context.runSequence:687 Stage: Clean up
2017-11-27 08:11:26 DEBUG otopi.context context.runSequence:691 STAGE cleanup

</error>



_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel