11:46 p.m.
Due to some complications in my 4.4 to 4.5 upgrade that have taken down ovirt for a few
days I must go back to 4.4 to salvage what I can and rethink how I will upgrade.
Due to that im having to rebuild my engine although I took backups prior.
My issue currently is with 4.4 the engine deployment has an error regarding mirrors so I
was able to use the ansible script to deploy offline and I am at the activate storage
domain portion where it fails
I am setting it to glusterfs with the host:/path as gluster-1.d.com:/engine
The host machine i am building the engine on has a mount:
gluster-1.d.com:/engine 104804356 13827696 90976660 14%
/rhev/data-center/mnt/glusterSD/gluster-1.d.com:_engine
within the mapping :
drwxr-xr-x. 2 vdsm kvm 6 Apr 5 22:15 gluster-1.d.com:_data
drwxr-xr-x. 4 vdsm kvm 38 May 5 15:00 gluster-1.d.com:_engine
I am able to get into the temp engines ssh and ping the domain although I cannot seem to
find where my steps are incorrect.
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Activate storage domain]
[ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is
"[]". HTTP response code is 400.
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg":
"Fault reason is \"Operation Failed\". Fault detail is \"[]\".
HTTP response code is 400."}
Please specify the storage you would like to use (glusterfs, iscsi, fc,
nfs)[nfs]:
VDSM Logs are showing:
2022-05-05 15:30:32,719-0600 INFO (jsonrpc/6) [storage.StorageDomain]
sdUUID=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c (fileSD:535)
2022-05-05 15:30:32,734-0600 INFO (jsonrpc/6) [vdsm.api] FINISH getStorageDomainInfo
error=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) from=::ffff:192.168.222.116,53112,
flow_id=661d6831, task_id=480acb07-0a53-4bad-aa30-4cd18ce9e5f2 (api:52)
2022-05-05 15:30:32,734-0600 ERROR (jsonrpc/6) [storage.TaskManager.Task]
(Task='480acb07-0a53-4bad-aa30-4cd18ce9e5f2') Unexpected error (task:877)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 884, in
_run
return fn(*args, **kargs)
File "<decorator-gen-131>", line 2, in getStorageDomainInfo
File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2717, in
getStorageDomainInfo
dom = self.validateSdUUID(sdUUID)
File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 313, in
validateSdUUID
sdDom.validate()
File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in
validate
raise se.StorageDomainAccessError(self.sdUUID)
vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or
entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',)
2022-05-05 15:30:32,734-0600 INFO (jsonrpc/6) [storage.TaskManager.Task]
(Task='480acb07-0a53-4bad-aa30-4cd18ce9e5f2') aborting: Task is aborted:
"value=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) abortedcode=379" (task:1182)
2022-05-05 15:30:32,735-0600 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH
getStorageDomainInfo error=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) (dispatcher:83)
2022-05-05 15:30:32,735-0600 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call
StorageDomain.getInfo failed (error 379) in 0.02 seconds (__init__:312)
2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [vdsm.api] START
createStoragePool(spUUID='fd385045-8355-4ef8-9970-ade84698ad89',
poolName='LabNet', masterDom='7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',
domList=['7f7cf7a4-f8fa-41a4-8e6a-275230655a8c'], masterVersion=13,
lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3)
from=::ffff:192.168.222.116,53112, flow_id=661d6831,
task_id=b58a0503-98c3-41a9-b5d7-3330e2985133 (api:48)
2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StoragePool] updating pool
fd385045-8355-4ef8-9970-ade84698ad89 backend from type NoneType instance 0x7f767080b9d0 to
type StoragePoolDiskBackend instance 0x7f760c4dd308 (sp:168)
2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StoragePool]
spUUID=fd385045-8355-4ef8-9970-ade84698ad89 poolName=LabNet
master_sd=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c
domList=['7f7cf7a4-f8fa-41a4-8e6a-275230655a8c'] masterVersion=13
{'LEASERETRIES': 3, 'LEASETIMESEC': 60, 'LOCKRENEWALINTERVALSEC':
5, 'IOOPTIMEOUTSEC': 10} (sp:624)
2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StorageDomain]
sdUUID=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c (fileSD:535)
2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.StoragePool] Unexpected error
(sp:640)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 635, in
create
domain.validate()
File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in
validate
raise se.StorageDomainAccessError(self.sdUUID)
vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or
entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',)
2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [vdsm.api] FINISH createStoragePool
error=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) from=::ffff:192.168.222.116,53112,
flow_id=661d6831, task_id=b58a0503-98c3-41a9-b5d7-3330e2985133 (api:52)
2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.TaskManager.Task]
(Task='b58a0503-98c3-41a9-b5d7-3330e2985133') Unexpected error (task:877)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 635, in
create
domain.validate()
File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in
validate
raise se.StorageDomainAccessError(self.sdUUID)
vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or
entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 884, in
_run
return fn(*args, **kargs)
File "<decorator-gen-31>", line 2, in createStoragePool
File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 1006, in
createStoragePool
leaseParams)
File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 641, in
create
raise se.StorageDomainAccessError(sdUUID)
vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or
entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',)
2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [storage.TaskManager.Task]
(Task='b58a0503-98c3-41a9-b5d7-3330e2985133') aborting: Task is aborted:
"value=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) abortedcode=379" (task:1182)
2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH
createStoragePool error=Domain is either partially accessible or entirely inaccessible:
('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) (dispatcher:83)
2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
StoragePool.create failed (error 379) in 0.02 seconds (__init__:312)
2022-05-05 15:30:34,709-0600 INFO (periodic/0) [vdsm.api] START repoStats(domains=())
from=internal, task_id=1b759c4f-ab60-4f3a-8ede-2606f916bf4a (api:48)
2022-05-05 15:30:34,710-0600 INFO (periodic/0) [vdsm.api] FINISH repoStats return={}
from=internal, task_id=1b759c4f-ab60-4f3a-8ede-2606f916bf4a (api:54)
2022-05-05 15:30:36,324-0600 INFO (jsonrpc/7) [api.host] START getStats()
from=::ffff:192.168.222.116,53112 (api:48)
2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=())
from=::ffff:192.168.222.116,53112, task_id=3b40703b-83dc-4aff-ae92-41158e5ffefc (api:48)
2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={}
from=::ffff:192.168.222.116,53112, task_id=3b40703b-83dc-4aff-ae92-41158e5ffefc (api:54)
2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] START multipath_health()
from=::ffff:192.168.222.116,53112, task_id=00953608-b566-4d3d-be62-f20ea77ff7d9 (api:48)
2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] FINISH multipath_health
return={} from=::ffff:192.168.222.116,53112, task_id=00953608-b566-4d3d-be62-f20ea77ff7d9
(api:54)
2022-05-05 15:30:36,348-0600 INFO (jsonrpc/7) [api.host] FINISH getStats
return={'status': {'code': 0, 'message': 'Done'},
'info': (suppressed)} from=::ffff:192.168.222.116,53112 (api:54)
2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vdsm.api] START
getConnectedStoragePoolsList() from=internal, task_id=a5db87b7-5130-49da-a5c4-5622dbe9d74b
(api:48)
2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vdsm.api] FINISH
getConnectedStoragePoolsList return={'poollist': []} from=internal,
task_id=a5db87b7-5130-49da-a5c4-5622dbe9d74b (api:54)
2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vds] recovery: waiting for storage pool
to go up (clientIF:735)
2022-05-05 15:30:37,444-0600 INFO (jsonrpc/2) [api.host] START getAllVmStats()
from=::1,33062 (api:48)
2022-05-05 15:30:37,446-0600 INFO (jsonrpc/2) [api.host] FINISH getAllVmStats
return={'status': {'code': 0, 'message': 'Done'},
'statsList': (suppressed)} from=::1,33062 (api:54)
2022-05-05 15:30:37,451-0600 INFO (jsonrpc/3) [api.host] START getAllVmIoTunePolicies()
from=::1,33062 (api:48)
2022-05-05 15:30:37,452-0600 INFO (jsonrpc/3) [api.host] FINISH getAllVmIoTunePolicies
return={'status': {'code': 0, 'message': 'Done'},
'io_tune_policies_dict': {'da57699e-e76a-4125-81ce-50821bb09eef':
{'policy': [], 'current_values': []}}} from=::1,33062 (api:54)