1:08 p.m.
--=_fa659c011be33add9dea7033694abeb7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,=0A=0AI had an electrical failure on the server hosting the engine.=
=0AAfter the reboot it was able to gain access to it again, log into the=
GUI, but the currently online node is not leaving "not responsive" stat=
us.=0AOf course, the network storage paths are still mounted, the VMs ar=
e running, but I can't gain control again.=0A=0AIn vdsmd.log, I have a l=
ot of messages like this one :=0A2018-03-27 12:03:11,281+0200 INFO (vmre=
covery) [vds] recovery: waiting for storage pool to go up (clientIF:674)=
=0A2018-03-27 12:03:16,286+0200 INFO (vmrecovery) [vdsm.api] START getCo=
nnectedStoragePoolsList(options=3DNone) from=3Dinternal, task_id=3Db90f5=
50e-ee68-4a91-a7c6-3b60f11c3978 (api:46)=0A2018-03-27 12:03:16,286+0200=
INFO (vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return=
=3D{'poollist': []} from=3Dinternal, task_id=3Db90f550e-ee68-4a91-a7c6-3=
b60f11c3978 (api:52)=0A2018-03-27 12:03:16,287+0200 INFO (vmrecovery) [v=
ds] recovery: waiting for storage pool to go up (clientIF:674)=0A2018-03=
-27 12:03:18,413+0200 INFO (periodic/3) [vdsm.api] START repoStats(domai=
ns=3D()) from=3Dinternal, task_id=3D067714b4-8172-4eec-92bb-6ac16586a657=
(api:46)=0A2018-03-27 12:03:18,413+0200 INFO (periodic/3) [vdsm.api] FI=
NISH repoStats return=3D{} from=3Dinternal, task_id=3D067714b4-8172-4eec=
-92bb-6ac16586a657 (api:52)=0A2018-03-27 12:03:18,413+0200 INFO (periodi=
c/3) [vdsm.api] START multipath_health() from=3Dinternal, task_id=3De974=
21fb-5d5a-4291-9231-94bc1961cc49 (api:46)=0A2018-03-27 12:03:18,413+0200=
INFO (periodic/3) [vdsm.api] FINISH multipath_health return=3D{} from=
=3Dinternal, task_id=3De97421fb-5d5a-4291-9231-94bc1961cc49 (api:52)=0A2=
018-03-27 12:03:20,458+0200 INFO (jsonrpc/6) [api.host] START getAllVmSt=
ats() from=3D::1,57576 (api:46)=0A2018-03-27 12:03:20,462+0200 INFO (jso=
nrpc/6) [api.host] FINISH getAllVmStats return=3D{'status': {'message':=
'Done', 'code': 0}, 'statsList': (suppressed)} from=3D::1,57576
(api:52=
)=0A2018-03-27 12:03:20,464+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer=
] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:573)=
=0A2018-03-27 12:03:20,474+0200 INFO (jsonrpc/7) [api.host] START getAll=
VmIoTunePolicies() from=3D::1,57576 (api:46)=0A2018-03-27 12:03:20,475+0=
200 INFO (jsonrpc/7) [api.host] FINISH getAllVmIoTunePolicies return=3D{=
'status': {'message': 'Done', 'code': 0},
'io_tune_policies_dict': {'c33=
a30ba-7fe8-4ff4-aeac-80cb396b9670': {'policy': [], 'current_values':
[{'=
ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec':=
0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_bytes_sec': 0L},=
'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07ef=
a4fe-06bc-498e-8f42-035461aef900/images/593f6f61-cb7f-4c53-b6e7-617964c2=
22e9/329b2e8b-6cf9-4b39-9190-14a32697ce44', 'name': 'sda'}]},
'e8a90739-=
7737-413e-8edc-a373192f4476': {'policy': [], 'current_values':
[{'ioTune=
': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec': 0L, 'r=
ead_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L},
'path'=
: '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06=
bc-498e-8f42-035461aef900/images/97e078f7-69c6-46c2-b620-26474cd65929/bb=
b4a1fb-5594-4750-be71-c6b55dca3257', 'name': 'vda'}]},
'3aec5ce4-691f-48=
7c-a916-aa7f7a664d8c': {'policy': [], 'current_values':
[{'ioTune': {'wr=
ite_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L,
'read_byt=
es_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L},
'path': '/rhe=
v/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e=
-8f42-035461aef900/images/46a65a1b-d00a-452d-ab9b-70862bb5c053/a4d2ad44-=
5577-4412-9a8c-819d1f12647a', 'name': 'sda'}, {'ioTune':
{'write_bytes_s=
ec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L,
'read_bytes_sec': 0L=
, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-cent=
er/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-03546=
1aef900/images/0c3a13ce-8f7a-4034-a8cc-12f795b8aa17/c48e0e37-e54b-4ca3-b=
3ed-b66ead9fad44', 'name': 'sdb'}]},
'5de1de8f-ac01-459f-b4b8-6d1ed05c8c=
a3': {'policy': [], 'current_values': [{'ioTune':
{'write_bytes_sec': 0L=
, 'total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L,
'writ=
e_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-center/mnt/=
10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900=
/images/320ac81c-7db7-4ec0-a271-755e91442b6a/8bfc95c5-318c-43dd-817f-6c7=
a8a7a5b43', 'name': 'sda'}, {'ioTune':
{'write_bytes_sec': 0L, 'total_io=
ps_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L,
'write_iops_sec'=
: 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-center/mnt/10.100.2.13=
2:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/e7a=
d86bb-3c63-466b-82cf-687164c46f7b/613ea0ce-ed14-4185-b3fd-36490441f889',=
'name': 'sdb'}]}, '5d548a09-a397-4aac-8b1f-39002e014f5f':
{'policy': []=
, 'current_values': [{'ioTune': {'write_bytes_sec': 0L,
'total_iops_sec'=
: 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec':
0L, '=
total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volu=
me2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/c7421014-7=
c5f-45ad-a948-caa83b8ce3e7/ae0ba893-69af-4b67-a262-b739596d5c95', 'name'=
: 'sda'}]}, '168b01b1-5ec8-41dd-808e-fa9f66cea718': {'policy': [],
'curr=
ent_values': [{'ioTune': {'write_bytes_sec': 0L,
'total_iops_sec': 0L, '=
read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_b=
ytes_sec': 0L}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovi=
rt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/b9b7902a-7a62-482=
6-bfda-dff260b9fcd1/d05db17c-9908-4bfb-a74b-4aa944510a56', 'name':
'vda'=
}, {'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_s=
ec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_bytes_sec':=
0L}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1=
/07efa4fe-06bc-498e-8f42-035461aef900/images/564b3848-b6d5-4deb-910f-5b6=
f2fdbccc5/4f89ff25-2d3b-40b9-9bbc-9a6b6995346c', 'name': 'vdb'},
{'ioTun=
e': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec': 0L, '=
read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L},
'path=
': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-0=
6bc-498e-8f42-035461aef900/images/738e0704-8484-483b-ae67-091715496152/2=
f811423-6bab-4966-9c00-9d3b72429328', 'name': 'vdc'}]}}}
from=3D::1,5757=
6 (api:52)=0A2018-03-27 12:03:20,475+0200 INFO (jsonrpc/7) [jsonrpc.Json=
RpcServer] RPC call Host.getAllVmIoTunePolicies succeeded in 0.00 second=
s (__init__:573)=0A2018-03-27 12:03:21,292+0200 INFO (vmrecovery) [vdsm.=
api] START getConnectedStoragePoolsList(options=3DNone) from=3Dinternal,=
task_id=3Da35602b2-7d5c-4e87-86cd-ede17c62488f (api:46)=0A2018-03-27 12=
:03:21,292+0200 INFO (vmrecovery) [vdsm.api] FINISH getConnectedStorageP=
oolsList return=3D{'poollist': []} from=3Dinternal, task_id=3Da35602b2-7=
d5c-4e87-86cd-ede17c62488f (api:52)=0A2018-03-27 12:03:21,293+0200 INFO=
(vmrecovery) [vds] recovery: waiting for storage pool to go up (clientI=
F:674)=0A=0ASo i see no error.=0A=0ABut in messages :=0AMar 27 12:01:43=
pfm-srv-virt-2 libvirtd: 2018-03-27 10:01:43.569+0000: 71793: error : q=
emuDomainAgentAvailable:6030 : Guest agent is not responding: QEMU guest=
agent is not connected=0A=0AI have restarted libvirtd and vdsmd service=
s.=0A=0AIs there something else to do ?=0A=0ARegards =0A=0A-------------=
------------------------------------------------------------------------=
------------=0AFreeMail powered by mail.fr
--=_fa659c011be33add9dea7033694abeb7
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<div><span style=3D"font-family: arial, helvetica,sans-serif; font-size:=
10pt; color: #000000;">Hi,<br /><br />I had an electrical failure on
th=
e server hosting the engine.<br />After the reboot it was able to gain a=
ccess to it again, log into the GUI, but the currently online node is no=
t leaving "not responsive" status.<br />Of course, the network storage p=
aths are still mounted, the VMs are running, but I can't gain control ag=
ain.<br /><br />In vdsmd.log, I have a lot of messages like this one :<b=
r />2018-03-27 12:03:11,281+0200 INFO (vmrecovery) [vds] recovery:=
waiting for storage pool to go up (clientIF:674)<br />2018-03-27 12:03:=
16,286+0200 INFO (vmrecovery) [vdsm.api] START getConnectedStorage=
PoolsList(options=3DNone) from=3Dinternal, task_id=3Db90f550e-ee68-4a91-=
a7c6-3b60f11c3978 (api:46)<br />2018-03-27 12:03:16,286+0200 INFO =
(vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return=3D{'=
poollist': []} from=3Dinternal, task_id=3Db90f550e-ee68-4a91-a7c6-3b60f1=
1c3978 (api:52)<br />2018-03-27 12:03:16,287+0200 INFO (vmrecovery=
) [vds] recovery: waiting for storage pool to go up (clientIF:674)<br />=
2018-03-27 12:03:18,413+0200 INFO (periodic/3) [vdsm.api] START re=
poStats(domains=3D()) from=3Dinternal, task_id=3D067714b4-8172-4eec-92bb=
-6ac16586a657 (api:46)<br />2018-03-27 12:03:18,413+0200 INFO (per=
iodic/3) [vdsm.api] FINISH repoStats return=3D{} from=3Dinternal, task_i=
d=3D067714b4-8172-4eec-92bb-6ac16586a657 (api:52)<br />2018-03-27 12:03:=
18,413+0200 INFO (periodic/3) [vdsm.api] START multipath_health()=
from=3Dinternal, task_id=3De97421fb-5d5a-4291-9231-94bc1961cc49 (api:46=
)<br />2018-03-27 12:03:18,413+0200 INFO (periodic/3) [vdsm.api] F=
INISH multipath_health return=3D{} from=3Dinternal, task_id=3De97421fb-5=
d5a-4291-9231-94bc1961cc49 (api:52)<br />2018-03-27 12:03:20,458+0200 IN=
FO (jsonrpc/6) [api.host] START getAllVmStats() from=3D::1,57576 (=
api:46)<br />2018-03-27 12:03:20,462+0200 INFO (jsonrpc/6) [api.ho=
st] FINISH getAllVmStats return=3D{'status': {'message': 'Done',
'code':=
0}, 'statsList': (suppressed)} from=3D::1,57576 (api:52)<br />2018-03-2=
7 12:03:20,464+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC c=
all Host.getAllVmStats succeeded in 0.01 seconds (__init__:573)<br />201=
8-03-27 12:03:20,474+0200 INFO (jsonrpc/7) [api.host] START getAll=
VmIoTunePolicies() from=3D::1,57576 (api:46)<br />2018-03-27 12:03:20,47=
5+0200 INFO (jsonrpc/7) [api.host] FINISH getAllVmIoTunePolicies r=
eturn=3D{'status': {'message': 'Done', 'code': 0},
'io_tune_policies_dic=
t': {'c33a30ba-7fe8-4ff4-aeac-80cb396b9670': {'policy': [],
'current_val=
ues': [{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec':
0L, 'read_io=
ps_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_bytes_se=
c': 0L}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms=
__1/07efa4fe-06bc-498e-8f42-035461aef900/images/593f6f61-cb7f-4c53-b6e7-=
617964c222e9/329b2e8b-6cf9-4b39-9190-14a32697ce44', 'name': 'sda'}]},
'e=
8a90739-7737-413e-8edc-a373192f4476': {'policy': [], 'current_values':
[=
{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec'=
: 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_bytes_sec': 0L}=
, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07e=
fa4fe-06bc-498e-8f42-035461aef900/images/97e078f7-69c6-46c2-b620-26474cd=
65929/bbb4a1fb-5594-4750-be71-c6b55dca3257', 'name': 'vda'}]},
'3aec5ce4=
-691f-487c-a916-aa7f7a664d8c': {'policy': [], 'current_values':
[{'ioTun=
e': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec': 0L, '=
read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L},
'path=
': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-0=
6bc-498e-8f42-035461aef900/images/46a65a1b-d00a-452d-ab9b-70862bb5c053/a=
4d2ad44-5577-4412-9a8c-819d1f12647a', 'name': 'sda'},
{'ioTune': {'write=
_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L,
'read_bytes_=
sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L},
'path': '/rhev/d=
ata-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f=
42-035461aef900/images/0c3a13ce-8f7a-4034-a8cc-12f795b8aa17/c48e0e37-e54=
b-4ca3-b3ed-b66ead9fad44', 'name': 'sdb'}]},
'5de1de8f-ac01-459f-b4b8-6d=
1ed05c8ca3': {'policy': [], 'current_values': [{'ioTune':
{'write_bytes_=
sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L,
'read_bytes_sec': 0=
L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-cen=
ter/mnt/10.100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-0354=
61aef900/images/320ac81c-7db7-4ec0-a271-755e91442b6a/8bfc95c5-318c-43dd-=
817f-6c7a8a7a5b43', 'name': 'sda'}, {'ioTune':
{'write_bytes_sec': 0L, '=
total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L,
'write_i=
ops_sec': 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-center/mnt/10.=
100.2.132:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/im=
ages/e7ad86bb-3c63-466b-82cf-687164c46f7b/613ea0ce-ed14-4185-b3fd-364904=
41f889', 'name': 'sdb'}]},
'5d548a09-a397-4aac-8b1f-39002e014f5f': {'pol=
icy': [], 'current_values': [{'ioTune': {'write_bytes_sec':
0L, 'total_i=
ops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L,
'write_iops_sec=
': 0L, 'total_bytes_sec': 0L}, 'path':
'/rhev/data-center/mnt/10.100.2.1=
32:_volume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/c7=
421014-7c5f-45ad-a948-caa83b8ce3e7/ae0ba893-69af-4b67-a262-b739596d5c95'=
, 'name': 'sda'}]}, '168b01b1-5ec8-41dd-808e-fa9f66cea718':
{'policy': [=
], 'current_values': [{'ioTune': {'write_bytes_sec': 0L,
'total_iops_sec=
': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L,
'write_iops_sec': 0L,=
'total_bytes_sec': 0L}, 'path':
'/rhev/data-center/mnt/10.100.2.132:_vo=
lume2_ovirt__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/b9b7902a=
-7a62-4826-bfda-dff260b9fcd1/d05db17c-9908-4bfb-a74b-4aa944510a56', 'nam=
e': 'vda'}, {'ioTune': {'write_bytes_sec': 0L,
'total_iops_sec': 0L, 're=
ad_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_byt=
es_sec': 0L}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt=
__vms__1/07efa4fe-06bc-498e-8f42-035461aef900/images/564b3848-b6d5-4deb-=
910f-5b6f2fdbccc5/4f89ff25-2d3b-40b9-9bbc-9a6b6995346c', 'name':
'vdb'},=
{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L,
'read_iops_sec=
': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L,
'total_bytes_sec': 0L=
}, 'path': '/rhev/data-center/mnt/10.100.2.132:_volume2_ovirt__vms__1/07=
efa4fe-06bc-498e-8f42-035461aef900/images/738e0704-8484-483b-ae67-091715=
496152/2f811423-6bab-4966-9c00-9d3b72429328', 'name': 'vdc'}]}}}
from=3D=
::1,57576 (api:52)<br />2018-03-27 12:03:20,475+0200 INFO (jsonrpc=
/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmIoTunePolicies succeed=
ed in 0.00 seconds (__init__:573)<br />2018-03-27 12:03:21,292+0200 INFO=
(vmrecovery) [vdsm.api] START getConnectedStoragePoolsList(option=
s=3DNone) from=3Dinternal, task_id=3Da35602b2-7d5c-4e87-86cd-ede17c62488=
f (api:46)<br />2018-03-27 12:03:21,292+0200 INFO (vmrecovery) [vd=
sm.api] FINISH getConnectedStoragePoolsList return=3D{'poollist': []} fr=
om=3Dinternal, task_id=3Da35602b2-7d5c-4e87-86cd-ede17c62488f (api:52)<b=
r />2018-03-27 12:03:21,293+0200 INFO (vmrecovery) [vds] recovery:=
waiting for storage pool to go up (clientIF:674)<br /><br />So i see no=
error.<br /><br />But in messages :<br />Mar 27 12:01:43
pfm-srv-virt-2=
libvirtd: 2018-03-27 10:01:43.569+0000: 71793: error : qemuDomainAgentA=
vailable:6030 : Guest agent is not responding: QEMU guest agent is not c=
onnected<br /><br /><br />I have restarted libvirtd and vdsmd
services.<=
br /><br />Is there something else to do ?<br /><br
/>Regards</span></di=
v>=0A <br/><hr>FreeMail powered by <a
href=3D"https:/=
/mail.fr" target=3D"_blank">mail.fr</a>=0A
--=_fa659c011be33add9dea7033694abeb7--