[ovirt-devel] [ OST Failure Report ] [ oVirt master][09-05-2017][ 002_bootstrap.add_hosts ]

[Anton Marchukov]

Hello.

Today around 11 am the add host for master started to consistently fail.

We found the following in /var/log/messages (as I understand it might get
there through journald as the host deploy log claims that vdsmd unit
startup failed):

May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: Traceback
(most recent call last):
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/bin/vdsm-tool", line 219, in main
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: return
tool_command[cmd]["command"](*args)
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 48, in
upgrade_networks
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: netupgrade.upgrade()
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line
55, in upgrade
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool:
_create_unified_configuration(rconfig, NetInfo(netinfo(vdsmnets)))
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line
133, in _create_unified_configuration
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: RunningConfig.store()
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py",
line 204, in store
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: _store_net_config()
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py",
line 281, in _store_net_config
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool:
utils.rmTree(real_old_safeconf_dir)
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib/python2.7/site-packages/vdsm/utils.py", line 125, in rmTree
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool:
shutil.rmtree(directoryToRemove)
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib64/python2.7/shutil.py", line 232, in rmtree
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool:
onerror(os.path.islink, path, sys.exc_info())
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File
"/usr/lib64/python2.7/shutil.py", line 230, in rmtree
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: raise
OSError("Cannot call rmtree on a symbolic link")
May  9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: OSError:
Cannot call rmtree on a symbolic link
May  9 07:01:25 lago-basic-suite-master-host1 systemd:
vdsm-network.service: control process exited, code=exited status=1


The full logs of the run you can find in Jenkins artifacts:

http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6589/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/


Any vdsm patches merged that might cause this?

-- 

Anton Marchukov
Senior Software Engineer - RHEV CI - Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170509/937fcec2/attachment.html>