[ovirt-users] spm changes hourly due to unexpected error of getAllTasksStatuses

[like.ma at cs2c.com.cn]

Hello all,

In our ovirt environment(oVirt 3.4.4), we have 2 hosts, and the spm changes hourly between the 2 hosts. In the events of DataCenters there is following message each hour:
'Data Center is being initialized, please wait for initialization to complete'

In the vdsm.log i found following error message:
Thread-13::ERROR::2015-06-23 13:38:36,667::task::866::TaskManager.Task::(_setError) Task=`e450d7d6-771d-4c51-90b7-c6b10da37897`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 2125, in getAllTasksStatuses
    allTasksStatus = sp.getAllTasksStatuses()
  File "/usr/share/vdsm/storage/securable.py", line 73, in wrapper
    raise SecureError("Secured object is not in safe state")
SecureError: Secured object is not in safe state

The attachment is vdsm.log and engine.log. The error occured at 2015-06-23 13:38 and 2015-06-23 15:38 in vdsm.log.
(by the way, in the other host the error occured at 2015-06-23 12:38 and 2015-06-23 14:38)

And in the engine.log i found following error message:
2015-06-23 13:38:37,933 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-34) [781beaea] IrsBroker::Failed::UpdateVMVDS due to: XmlRpcException: <type 'exceptions.UnicodeDecodeError'>:'utf8' codec can't decode byte 0xeb in position 749: invalid continuation byte

2015-06-23 13:38:37,971 ERROR [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) [781beaea] Exception while trying to update or remove VMs/Templates ovf in Data Center Default.: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: java.lang.reflect.UndeclaredThrowableException (Failed with error VDS_NETWORK_ERROR and code 5022)
        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.OvfDataUpdater.executeUpdateVmInSpmCommand(OvfDataUpdater.java:383) [bll.jar:]
        at org.ovirt.engine.core.bll.OvfDataUpdater.performOvfUpdate(OvfDataUpdater.java:163) [bll.jar:]
        at org.ovirt.engine.core.bll.OvfDataUpdater.updateOvfForVmsOfStoragePool(OvfDataUpdater.java:135) [bll.jar:]
        at org.ovirt.engine.core.bll.OvfDataUpdater.ovfUpdate_timer(OvfDataUpdater.java:93) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source) [:1.7.0_65]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65]
        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]

Can anyone help? 
Thanks


like.ma at cs2c.com.cn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150623/dfe5840c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log.gz
Type: application/octet-stream
Size: 884461 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150623/dfe5840c/attachment-0001.obj>