
Hi all, We are running oVirt 4.1.1 on engine and nodes. The ISO storage domain is NFS. Everything has been working great for years. Now I need to move the ISO and Export domains to a different storage server. When I put the ISO domain in maintenance the Master domain and the Export domain start flipping active-non active. The Events tab in the Web UI shows this message scrolling every 30 sec. or so: "Datacenter is being initialized. Please wait for initialization to complete" ... but it never completes. It also starts cycling the SPM assignment from one node to another. As soon as I activate back the ISO domain everything calms down and no more errors are recorded in the logs. In engine.log I see this "Unrecognized message received" error: 2019-02-25 19:45:20,467-08 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages Unrecognized message received 2019-02-25 19:45:20,481-08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler6) [76363b98] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), Correlation ID: null, Call Stack: null, Custom E vent ID: -1, Message: VDSM command GetStoragePoolInfoVDS failed: Unrecognized message received 2019-02-25 19:45:20,481-08 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler6) [76363b98] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: VDSGenericExce ption: VDSNetworkException: Unrecognized message received , log id: 1635c439 2019-02-25 19:45:20,481-08 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler6) [76363b98] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetwor kException: Unrecognized message received In vdsm.log on the nodes I see this: 2019-02-25 19:45:29,817-0800 ERROR (upgrade/b5b7a10) [storage.StoragePool] Unhandled exception (utils:371) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 368, in wrapper return f(*a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/concurrent.py", line 180, in run return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 232, in _upgradePoolDomain self._finalizePoolUpgradeIfNeeded() File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper raise SecureError("Secured object is not in safe state") SecureError: Secured object is not in safe state And I believe that after that error it moves the SPM to another host, gets the same error and continues moving it to the next host and so on. I really need to move the ISO and Export domains to a different storage server. Any idea what is causing this or how to fix it? Appreciate any help. Thanks, -- Peter