Verify the ovirt-ha-agent service on the healthy node.

What type of storage are you using for the VM engine? Is it still reachable?

I don't think that the update broke your VM. The should at least boot.

On Sat, Jun 25, 2022, 09:44 <adam_xu@adagene.com.cn> wrote:

Vdsm log:

2022-06-25 01:24:48,206-0700 INFO  (jsonrpc/6) [vdsm.api] START multipath_health() from=::1,53638, task_id=33f083e2-1a06-449c-afce-66cf0c831be0 (api:48)

2022-06-25 01:24:48,206-0700 INFO  (jsonrpc/6) [vdsm.api] FINISH multipath_health return={} from=::1,53638, task_id=33f083e2-1a06-449c-afce-66cf0c831be0 (api:54)

2022-06-25 01:24:48,206-0700 WARN  (jsonrpc/6) [throttled] MOM not available. Error: [Errno 111] Connection refused (throttledlog:104)

2022-06-25 01:24:48,207-0700 WARN  (jsonrpc/6) [throttled] MOM not available, KSM stats will be missing. Error:  (throttledlog:104)

2022-06-25 01:24:48,207-0700 WARN  (jsonrpc/6) [root] Failed to retrieve Hosted Engine HA info, is Hosted Engine setup finished? (api:168)

2022-06-25 01:24:48,208-0700 INFO  (jsonrpc/6) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,53638 (api:54)

2022-06-25 01:24:49,323-0700 INFO  (jsonrpc/4) [vdsm.api] START repoStats(domains=['74ff7aac-f9e9-4e89-86db-2f40e48ddd85']) from=::1,53638, task_id=448f81dd-f0bd-4c92-92ca-9d6400be4a7e (api:48)

2022-06-25 01:24:49,323-0700 INFO  (jsonrpc/4) [vdsm.api] FINISH repoStats return={'74ff7aac-f9e9-4e89-86db-2f40e48ddd85': {'code': 0, 'lastCheck': '2.0', 'delay': '0.000278052', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,53638, task_id=448f81dd-f0bd-4c92-92ca-9d6400be4a7e (api:54)

2022-06-25 01:24:51,778-0700 INFO  (jsonrpc/1) [api.host] START getStats() from=::1,53638 (api:48)

2022-06-25 01:24:51,791-0700 INFO  (jsonrpc/1) [vdsm.api] START repoStats(domains=()) from=::1,53638, task_id=a9ab92c1-87e0-42df-a059-36e2588d5a78 (api:48)

2022-06-25 01:24:51,791-0700 INFO  (jsonrpc/1) [vdsm.api] FINISH repoStats return={'74ff7aac-f9e9-4e89-86db-2f40e48ddd85': {'code': 0, 'lastCheck': '1.1', 'delay': '0.000278052', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'fa5059e6-38ad-4f71-ad7d-0fc30aedf254': {'code': 0, 'lastCheck': '4.4', 'delay': '0.000361428', 'valid': True, 'version': 0, 'acquired': True, 'actual': True}} from=::1,53638, task_id=a9ab92c1-87e0-42df-a059-36e2588d5a78 (api:54)

2022-06-25 01:24:51,792-0700 INFO  (jsonrpc/1) [vdsm.api] START multipath_health() from=::1,53638, task_id=9845c9ca-e20e-425a-9fd2-f473b256380e (api:48)

2022-06-25 01:24:51,792-0700 INFO  (jsonrpc/1) [vdsm.api] FINISH multipath_health return={} from=::1,53638, task_id=9845c9ca-e20e-425a-9fd2-f473b256380e (api:54)

2022-06-25 01:24:51,793-0700 WARN  (jsonrpc/1) [root] Failed to retrieve Hosted Engine HA info, is Hosted Engine setup finished? (api:168)

2022-06-25 01:24:51,793-0700 INFO  (jsonrpc/1) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,53638 (api:54)

2022-06-25 01:24:52,482-0700 INFO  (jsonrpc/7) [api.virt] START getStats() from=::1,53638, vmId=39877e08-26da-4694-a6a0-53bda1f1f87d (api:48)

2022-06-25 01:24:52,482-0700 INFO  (jsonrpc/7) [api.virt] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'statsList': [{'statusTime': '11315340662', 'status': 'Down', 'vmId': '39877e08-26da-4694-a6a0-53bda1f1f87d', 'exitCode': 0, 'exitMessage': 'User shut down from within the guest', 'exitReason': 7}]} from=::1,53638, vmId=39877e08-26da-4694-a6a0-53bda1f1f87d (api:54)

发件人: adam_xu@adagene.com.cn <adam_xu@adagene.com.cn>
发送时间: 2022625 16:39
收件人: 'wodel youchi' <wodel.youchi@gmail.com>
抄送: 'users' <users@ovirt.org>
主题: [ovirt-users]回复: Re: can not access engine when hosted engine is up

 

I restart vdsmd process failed. And when I try to restart the only healthy node. Now no vm is running now.

Too bad.

# hosted-engine --vm-status

The hosted engine configuration has not been retrieved from shared storage yet,

please ensure that ovirt-ha-agent service is running.

 

I think the engine vm is down because of “dnf update” on it.

Is there any  recovery mode I can do now?

发件人: wodel youchi <wodel.youchi@gmail.com>
发送时间: 2022625 16:23
收件人: adam_xu@adagene.com.cn
抄送: users <users@ovirt.org>
主题: [ovirt-users] Re: can not access engine when hosted engine is up

 

Try restarting the vdmd process in that node and you can try to start the VM engine in another healthy hosted node.

 

On Sat, Jun 25, 2022, 09:07 <adam_xu@adagene.com.cn> wrote:

I can not ping the engine. And no kvm process of engine in any node.

I try to run “hosted-engine --vm-start”, it said:

# hosted-engine --vm-start

Traceback (most recent call last):

  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main

    "__main__", mod_spec)

  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code

    exec(code, run_globals)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 214, in <module>

    args.command(args)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 42, in func

    f(*args, **kwargs)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 91, in checkVmStatus

    cli = ohautil.connect_vdsm_json_rpc()

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/util.py", line 474, in connect_vdsm_json_rpc

    __vdsm_json_rpc_connect(logger, timeout)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/util.py", line 415, in __vdsm_json_rpc_connect

    timeout=VDSM_MAX_RETRY * VDSM_DELAY

RuntimeError: Couldn't  connect to VDSM within 60 seconds

Traceback (most recent call last):

  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main

    "__main__", mod_spec)

  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code

    exec(code, run_globals)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 214, in <module>

    args.command(args)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 42, in func

    f(*args, **kwargs)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/vdsm_helper.py", line 57, in create

    cli = ohautil.connect_vdsm_json_rpc()

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/util.py", line 474, in connect_vdsm_json_rpc

    __vdsm_json_rpc_connect(logger, timeout)

  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/util.py", line 415, in __vdsm_json_rpc_connect

    timeout=VDSM_MAX_RETRY * VDSM_DELAY

RuntimeError: Couldn't  connect to VDSM within 60 seconds

VM failed to launch

 

发件人: wodel youchi <wodel.youchi@gmail.com>
发送时间: 2022625 15:56
收件人: adam_xu@adagene.com.cn
抄送: users <users@ovirt.org>
主题: [ovirt-users] Re: can not access engine when hosted engine is up

 

Hi,

 

Can you ping the VM engine? If yes then it's up and running, you may ssh into it and verify the ovirt-engine service if it is running properly.

 

If the VM engine doesn't ping, search for its kvm process in your hosted nodes (all of them), for example: ps -ef | grep qemu-kvm | grep -i hosted

 

If the process exists then VM exists but may be it is paused, if it does not, then try to start the VM : hosted-engine --vm-start

 

When the global maintenance is active if the VM engine is rebooted or shutdown it is not restarted.

 

 

Regards.

 

On Sat, Jun 25, 2022, 08:42 <adam_xu@adagene.com.cn> wrote:

Hi ovirt list,

I need help.

I ran “dnf update” on my engine and reboot it. Then I lost connection of the engine.

When I access one of my host web management https://ovirthost1:9090, I saw engine status is “Hosted Engine is up!”, but another host of the cluster is in down status.

How can I bring up my engine since it is “UP” now.

 

_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DYVOECFXW57F2ZGOI7FKCMXYVUTVJQKP/