10:31 a.m.
Can you run the engine manually on another server ?
hosted-engine --vm-start
Best Regards,
Strahil NikolovOn Jul 5, 2019 18:05, Richard Chan <richard(a)treeboxsolutions.com>
wrote:
My engine is getting SpmStart failure.
What is the best way to troubleshoot this?
--
Richard Chan
10:37 a.m.
New subject: Upgraded to 4.3: getting SpmStart failure
Unfortunately, this is a separate engine running external to the oVirt
cluster.
The main error on the hosts is: any idea what this Volume is referring to?
I have 3 existings storage domains
ISO EXPORT and DATA all upgraded to 4.3
None have this UUID.
VolumeDoesNotExist: Volume does not exist:
(u'1d1b6dfd-197a-4583-8398-6b941dbca854',)
2019-07-05 23:02:15,954+0800 ERROR (tasks/0) [storage.StoragePool] failed:
Volume does not exist: (u'1d1b6dfd-197a-4583-8398-6b941dbca854',) (sp:384)
2019-07-05 23:02:15,954+0800 INFO (tasks/0) [storage.SANLock] Releasing
Lease(name='SDM',
path=u'/rhev/data-center/mnt/192.168.88.40:_genie_ovirt_data/f9aaa65a-0ac7-4c61-a27d-15af39016419/dom_md/leases',
offset=1048576) (clusterlock:488)
2019-07-05 23:02:15,955+0800 INFO (tasks/0) [storage.SANLock] Successfully
released Lease(name='SDM',
path=u'/rhev/data-center/mnt/192.168.88.40:_genie_ovirt_data/f9aaa65a-0ac7-4c61-a27d-15af39016419/dom_md/leases',
offset=1048576) (clusterlock:497)
2019-07-05 23:02:15,955+0800 ERROR (tasks/0) [storage.TaskManager.Task]
(Task='159b9026-7034-43c2-9587-8fec036e92ba') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
in _run
return fn(*args, **kargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336,
in run
return self.cmd(*self.argslist, **self.argsdict)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 333, in
startSpm
self._upgradePool(expectedDomVersion, __securityOverride=True)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line
79, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 484, in
_upgradePool
str(targetDomVersion))
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1108, in
_convertDomain
targetFormat)
File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py",
line 447, in convert
converter(repoPath, hostId, imageRepo, isMsd)
File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py",
line 405, in v5DomainConverter
domain.convert_volumes_metadata(target_version)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 813,
in convert_volumes_metadata
for vol in self.iter_volumes():
File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 764, in
iter_volumes
yield self.produceVolume(img_id, vol_id)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 846, in
produceVolume
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 817,
in __init__
self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
71, in __init__
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86,
in __init__
self.validate()
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 112,
in validate
self.validateVolumePath()
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
131, in validateVolumePath
raise se.VolumeDoesNotExist(self.volUUID)
On Fri, Jul 5, 2019 at 11:31 PM Strahil <hunter86_bg(a)yahoo.com> wrote:
Can you run the engine manually on another server ?
hosted-engine --vm-start
Best Regards,
Strahil Nikolov
On Jul 5, 2019 18:05, Richard Chan <richard(a)treeboxsolutions.com> wrote:
My engine is getting SpmStart failure.
What is the best way to troubleshoot this?
--
Richard Chan
--
Richard Chan
Chief Architect
TreeBox Solutions Pte Ltd
1 Commonwealth Lane #03-01
Singapore 149544
Tel: 6570 3725
http://www.treeboxsolutions.com
Co.Reg.No. 201100585R
8:37 a.m.
New subject: Upgraded to 4.3: getting SpmStart failure
On Fri, Jul 5, 2019 at 6:45 PM Richard Chan <richard(a)treeboxsolutions.com>
wrote:
Unfortunately, this is a separate engine running external to the
oVirt
cluster.
The main error on the hosts is: any idea what this Volume is referring to?
I have 3 existings storage domains
ISO EXPORT and DATA all upgraded to 4.3
None have this UUID.
VolumeDoesNotExist: Volume does not exist:
(u'1d1b6dfd-197a-4583-8398-6b941dbca854',)
2019-07-05 23:02:15,954+0800 ERROR (tasks/0) [storage.StoragePool] failed:
Volume does not exist: (u'1d1b6dfd-197a-4583-8398-6b941dbca854',) (sp:384)
2019-07-05 23:02:15,954+0800 INFO (tasks/0) [storage.SANLock] Releasing
Lease(name='SDM',
path=u'/rhev/data-center/mnt/192.168.88.40:_genie_ovirt_data/f9aaa65a-0ac7-4c61-a27d-15af39016419/dom_md/leases',
offset=1048576) (clusterlock:488)
2019-07-05 23:02:15,955+0800 INFO (tasks/0) [storage.SANLock]
Successfully released Lease(name='SDM',
path=u'/rhev/data-center/mnt/192.168.88.40:_genie_ovirt_data/f9aaa65a-0ac7-4c61-a27d-15af39016419/dom_md/leases',
offset=1048576) (clusterlock:497)
2019-07-05 23:02:15,955+0800 ERROR (tasks/0) [storage.TaskManager.Task]
(Task='159b9026-7034-43c2-9587-8fec036e92ba') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
in _run
return fn(*args, **kargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336,
in run
return self.cmd(*self.argslist, **self.argsdict)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 333, in
startSpm
self._upgradePool(expectedDomVersion, __securityOverride=True)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line
79, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 484, in
_upgradePool
str(targetDomVersion))
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1108,
in _convertDomain
targetFormat)
File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py",
line 447, in convert
converter(repoPath, hostId, imageRepo, isMsd)
File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py",
line 405, in v5DomainConverter
domain.convert_volumes_metadata(target_version)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line
813, in convert_volumes_metadata
for vol in self.iter_volumes():
File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 764, in
iter_volumes
yield self.produceVolume(img_id, vol_id)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 846, in
produceVolume
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
817, in __init__
self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
71, in __init__
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86,
in __init__
self.validate()
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
112, in validate
self.validateVolumePath()
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
131, in validateVolumePath
raise se.VolumeDoesNotExist(self.volUUID)
This usually means the volume does not have the right permissions.
130 if not self.oop.fileUtils.pathExists(volPath):
131 raise se.VolumeDoesNotExist(self.volUUID)
oop.fileUtils.pathExists() is:
130 if not self.oop.fileUtils.pathExists(volPath):
131 raise se.VolumeDoesNotExist(self.volUUID)
IOProcess.pathExists() is:
482 def pathExists(self, filename, writable=False):
483 check = os.R_OK
484
485 if writable:
486 check |= os.W_OK
487
488 if self.access(filename, check):
489 return True
490
491 return self.access(filename, check)
Bad permissions would fail converting a domain of course, so this real fix
is to
fix the bad permissions.
Nir
On Fri, Jul 5, 2019 at 11:31 PM Strahil <hunter86_bg(a)yahoo.com> wrote:
> Can you run the engine manually on another server ?
> hosted-engine --vm-start
>
> Best Regards,
> Strahil Nikolov
> On Jul 5, 2019 18:05, Richard Chan <richard(a)treeboxsolutions.com> wrote:
>
> My engine is getting SpmStart failure.
>
> What is the best way to troubleshoot this?
>
>
> --
> Richard Chan
>
>
--
Richard Chan
Chief Architect
TreeBox Solutions Pte Ltd
1 Commonwealth Lane #03-01
Singapore 149544
Tel: 6570 3725
http://www.treeboxsolutions.com
Co.Reg.No. 201100585R
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3A5Y2JMNHBT...
11:25 a.m.
New subject: Upgraded to 4.3: getting SpmStart failure
This is in the engine.log
[storage.StoragePool] Trying to upgrade master domain
`f9aaa65a-0ac7-4c61-a27d-15af3901
6419` (sp:479)
2019-07-06 00:24:14,056+0800 INFO (tasks/5) [storage.StorageDomain]
Converting domain f9aaa65a-0ac7-4c61-a27d-15af39016419 volumes
metadata from version 4 to version 5 (fileSD:811)
2019-07-06 00:24:14,121+0800 INFO (tasks/5) [storage.StorageDomain]
Removing remnants of deleted images [] (fileSD:740)
2019-07-06 00:24:14,121+0800 ERROR (tasks/5) [storage.StoragePool]
Unexpected error (sp:383)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 333, in
startSpm
On Fri, Jul 5, 2019 at 11:31 PM Strahil <hunter86_bg(a)yahoo.com> wrote:
Can you run the engine manually on another server ?
hosted-engine --vm-start
Best Regards,
Strahil Nikolov
On Jul 5, 2019 18:05, Richard Chan <richard(a)treeboxsolutions.com> wrote:
My engine is getting SpmStart failure.
What is the best way to troubleshoot this?
--
Richard Chan
--
Richard Chan
Chief Architect
TreeBox Solutions Pte Ltd
1 Commonwealth Lane #03-01
Singapore 149544
Tel: 6570 3725
http://www.treeboxsolutions.com
Co.Reg.No. 201100585R
1754
days inactive
1756
days old
3 comments
3 participants
participants (3)
-
Nir Soffer -
Richard Chan -
Strahil