[ovirt-users] 2 Master on Storage Pool [Event Error]

[michael pagdanganan]

My storage pool has 2 Master Domain(Stored2 on Node2,Node1Container on Node1). and my old master domain (DATANd01 on Node1 ) hung on preparing for maintenance. When I tried to activate my old master domain (DATANd01 on Node1 ) all  storage domain goes down and up master keep on rotating.

Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)


Event Error:Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain Stored2 is marked as master in ovirt engine Database but not on storage side Please consult with support

VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or it's version: u'SD=f3e372e3-1251-4195-a4b9-1027e40059df, pool=5a865884-0366-0330-02b8-0000
VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()

Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)

Here's logs from engine:
-------------------------------------------------------------------------------------------------------[root at dev2engine ~]# tail /var/log/messages
Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root.
Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root.
Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root.
Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root.
Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root.
Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root.
Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root.
Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root.
Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root.
Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.

-------------------------------------------------------------------------------------------------------[root at dev2engine ~]# tail /var/log/ovirt-engine/engine.log
2018-02-20 08:01:16,062+08 INFO  [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7-thread-32) [102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8-0000000002d4'. Clearing event queue
2018-02-20 08:01:27,825+08 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7-thread-23) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage
2018-02-20 08:01:27,862+08 WARN  [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-23) [213f42b9] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status PreparingForMaintenance
2018-02-20 08:01:27,882+08 INFO  [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7-thread-20) [929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8-0000000002d4'. Clearing event queue
2018-02-20 08:01:40,106+08 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7-thread-17) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage
2018-02-20 08:01:40,197+08 WARN  [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-17) [7af552c1] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status PreparingForMaintenance
2018-02-20 08:01:40,246+08 INFO  [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7-thread-22) [73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8-0000000002d4'. Clearing event queue
2018-02-20 08:01:51,809+08 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7-thread-26) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage
2018-02-20 08:01:51,846+08 WARN  [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-26) [20307cbe] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status PreparingForMaintenance
2018-02-20 08:01:51,866+08 INFO  [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7-thread-49) [2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8-0000000002d4'. Clearing event queue

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180220/ea30a4e7/attachment.html>