Hello, I'm having a problem with oVirt - we're running 4.3 on CentOS 7: [root@ovirt ~]# rpm -q ovirt-engine; uname -r ovirt-engine-4.3.4.3-1.el7.noarch 3.10.0-957.21.2.el7.x86_64 Currently the Data Center alternates between Non responsive and contending status, and SPM selection fails. The error in the events tab is: VDSM compute5.domain command HSMGetTaskStatusVDS failed: Volume does not exist: (u'2bffb8d0-dfb5-4b08-9c6b-716e11f280c2',) Full error: 2019-08-21 13:53:53,507-0700 ERROR (tasks/8) [storage.TaskManager.Task] (Task='3076bb8c-7462-4573-a832-337da478ae0e') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336, in run return self.cmd(*self.argslist, **self.argsdict) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 333, in startSpm self._upgradePool(expectedDomVersion, __securityOverride=True) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 484, in _upgradePool str(targetDomVersion)) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1108, in _convertDomain targetFormat) File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 447, in convert converter(repoPath, hostId, imageRepo, isMsd) File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 405, in v5DomainConverter domain.convert_volumes_metadata(target_version) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 813, in convert_volumes_metadata for vol in self.iter_volumes(): File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 764, in iter_volumes yield self.produceVolume(img_id, vol_id) File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 846, in produceVolume volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/glusterVolume.py", line 45, in __init__ volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 817, in __init__ self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line 71, in __init__ volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86, in __init__ self.validate() File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 112, in validate self.validateVolumePath() File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line 131, in validateVolumePath raise se.VolumeDoesNotExist(self.volUUID) VolumeDoesNotExist: Volume does not exist: (u'2bffb8d0-dfb5-4b08-9c6b-716e11f280c2',) Most of our VMs are still running, but we can't start or restart any VMs. All of the domains show as down since SPM selection fails. Any thoughts? Thanks, -Matthew