
Hello all, my broker.log files are suddenly filling up with lines that start with: Listener::DEBUG::2021-11-24 09:13:20,795::listener::103::ovirt_hosted_engine_ha.broker.listener.Action.get_stats::(wrapper) Executing RPC handler get_stats with params (<ovirt_hosted_engine_ha.broker.listener.ActionsHandler object at 0x7fd3f0411b70>,) Listener::DEBUG::2021-11-24 09:13:20,796::listener::145::ovirt_hosted_engine_ha.broker.listener._encode::(_encode) Encoded successfully: b'maintenance= Followed by a long list of nulls (0\x00) OS: CentOS 8 Ovirt version: 4.4 Any ideas ? -- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003 "In an open world, who needs windows or gates ?"

On Thu, Nov 25, 2021 at 4:32 PM Valerio Luccio <valerio.luccio@nyu.edu> wrote:
Hello all,
my broker.log files are suddenly filling up with lines that start with:
Listener::DEBUG::2021-11-24 09:13:20,795::listener::103::ovirt_hosted_engine_ha.broker.listener.Action.get_stats::(wrapper) Executing RPC handler get_stats with params (<ovirt_hosted_engine_ha.broker.listener.ActionsHandler object at 0x7fd3f0411b70>,) Listener::DEBUG::2021-11-24 09:13:20,796::listener::145::ovirt_hosted_engine_ha.broker.listener._encode::(_encode) Encoded successfully: b'maintenance=
Followed by a long list of nulls (0\x00)
OS: CentOS 8
Ovirt version: 4.4
Any ideas ?
Perhaps you or someone from your team changed the logging level to debug? Please check /etc/ovirt-hosted-engine-ha/broker-log.conf -> '[logger_root]' -> 'level'. The specific large lines 'Encoded successfully' - only in debug level, though - were added in 4.4, during the port to python3, and recently I decided they are useless and removed them [1] - only in master branch, so for 4.5. Best regards, [1] https://gerrit.ovirt.org/c/ovirt-hosted-engine-ha/+/117111
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003
"In an open world, who needs windows or gates ?"
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7BJPH6Q5RU5SCT...
-- Didi

Damn, you're right. I'm puzzled, I don't remember changing that and I could have sworn that the server had been rebooted since the last change date on that file, gremlins, gremlins. I guess there is no way to turn that off without restarting the engine and the VMs, right ? On 11/25/21 9:44 AM, Yedidyah Bar David wrote:
On Thu, Nov 25, 2021 at 4:32 PM Valerio Luccio <valerio.luccio@nyu.edu> wrote:
Hello all,
my broker.log files are suddenly filling up with lines that start with:
Listener::DEBUG::2021-11-24 09:13:20,795::listener::103::ovirt_hosted_engine_ha.broker.listener.Action.get_stats::(wrapper) Executing RPC handler get_stats with params (<ovirt_hosted_engine_ha.broker.listener.ActionsHandler object at 0x7fd3f0411b70>,) Listener::DEBUG::2021-11-24 09:13:20,796::listener::145::ovirt_hosted_engine_ha.broker.listener._encode::(_encode) Encoded successfully: b'maintenance=
Followed by a long list of nulls (0\x00)
OS: CentOS 8
Ovirt version: 4.4
Any ideas ?
Perhaps you or someone from your team changed the logging level to debug? Please check /etc/ovirt-hosted-engine-ha/broker-log.conf -> '[logger_root]' -> 'level'.
The specific large lines 'Encoded successfully' - only in debug level, though - were added in 4.4, during the port to python3, and recently I decided they are useless and removed them [1] - only in master branch, so for 4.5.
Best regards,
[1] https://gerrit.ovirt.org/c/ovirt-hosted-engine-ha/+/117111 <https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.ovirt.org_c_ovirt-2Dhosted-2Dengine-2Dha_-2B_117111&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=KqDyiHsM-kxO3kISbpMWGKeiMa3il3wvH_QlKkax6-__1KAaQiw_1jBtX67g-TY_&s=aP5tXbmxdyw8BX58hhPKe9TjR6Jo_K7e3dpluIuWHIU&e=>
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003
"In an open world, who needs windows or gates ?"
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=KqDyiHsM-kxO3kISbpMWGKeiMa3il3wvH_QlKkax6-__1KAaQiw_1jBtX67g-TY_&s=7g2dsXCaXxaA6OjxO0DBW7FjboDjjg8RscWx2gGNz8k&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=KqDyiHsM-kxO3kISbpMWGKeiMa3il3wvH_QlKkax6-__1KAaQiw_1jBtX67g-TY_&s=Jc6vTC_ykG9LJK855Kh-tV0TQNzbPE1x_kxrgO6PwAc&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7BJPH6Q5RU5SCT... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_7BJPH6Q5RU5SCTQHYHGVUTDHRKQDLQ2C_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=KqDyiHsM-kxO3kISbpMWGKeiMa3il3wvH_QlKkax6-__1KAaQiw_1jBtX67g-TY_&s=r8kmbJRFvJ3MSJkKe16LX-zULnvDE8ooLW4-dvPPmFo&e=>
-- Didi -- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003
"In an open world, who needs windows or gates ?"

The broker logs can be changed... No need to shutdown the VM. Just use the hosted-engine tool to put the vm in global maintenance, then change the log level and restart the broker/agent service. After several minutes (wait at least 5 min) 'hosted-engine --vm-status' should show that the situation is OK and you can remove the maintenance. Best Regards,Strahil Nikolov On Thu, Nov 25, 2021 at 18:05, Valerio Luccio<valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AIVNDIUJ7THFY5...

Thanks, I actually restarted all of the VMs, but I'm still getting all these nulls. Here's my broker-log.conf file: [loggers] keys=root [handlers] keys=syslog,logfile [formatters] keys=long,sysform [logger_root] level=ERROR handlers=syslog,logfile propagate=0 [handler_syslog] level=ERROR class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER) [handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/broker.log', 'd', 1, 7) level=ERROR formatter=long [formatter_long] format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s [formatter_sysform] format=ovirt-ha-broker %(name)s %(levelname)s %(message)s datefmt= I'm puzzled. Thanks Didi and Strahil On 11/25/21 6:32 PM, Strahil Nikolov wrote:
The broker logs can be changed... No need to shutdown the VM.
Just use the hosted-engine tool to put the vm in global maintenance, then change the log level and restart the broker/agent service.
After several minutes (wait at least 5 min) 'hosted-engine --vm-status' should show that the situation is OK and you can remove the maintenance.
Best Regards, Strahil Nikolov
On Thu, Nov 25, 2021 at 18:05, Valerio Luccio <valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=yYRS9SPg0aWhkSgOBg2oP6WhfwXuTkidSeZTXKNrRghfF09-lxyxvu9kYu642YsZ&s=eHQDywX3Rt5Wl_1qMvFxPsRB3_OSO0y4LJZIVohFEEA&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=yYRS9SPg0aWhkSgOBg2oP6WhfwXuTkidSeZTXKNrRghfF09-lxyxvu9kYu642YsZ&s=Pjl6h9W9GHJdiwZnQL18PUSOWOzAJqhniHuzb-THx18&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AIVNDIUJ7THFY5... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_AIVNDIUJ7THFY5GQ5TIFP7FYSXNQTWE2_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=yYRS9SPg0aWhkSgOBg2oP6WhfwXuTkidSeZTXKNrRghfF09-lxyxvu9kYu642YsZ&s=NGGiLiRv-6N6kxYqMvUsZ-FzzXHVX3MoQX4MGbIuH9g&e=>
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003 "In an open world, who needs windows or gates ?"

You need to restart ovirt-ha-broker.service and ovirt-ha-agent.service Best Regards,Strahil Nikolov On Mon, Nov 29, 2021 at 18:31, Valerio Luccio<valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABCACVKMVJ6YE2...

I stopped the VMs and then "hosted-engine --vm-shutdown" followed by "hosted-engine --vm-start", I thought that would restart everything, but apparently not. Now I just did "systemctl status ovirt-ha-broker" and got this ugly thing: ovirt-ha-broker ovirt_hosted_engine_ha.broker.notifications.Notifications ERRO> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/notific> timeout=float(cfg["smtp-timeout"])) File "/usr/lib64/python3.6/smtplib.py", line 251, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib64/python3.6/smtplib.py", line 336, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib64/python3.6/smtplib.py", line 307, in _get_socket self.source_address) File "/usr/lib64/python3.6/socket.py", line 724, in create_connection raise err File "/usr/lib64/python3.6/socket.py", line 713, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused And systemctl status --no-pager --full ovirt-ha-agent ● ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2021-11-19 15:17:19 EST; 2 weeks 1 days ago Main PID: 14290 (ovirt-ha-agent) Tasks: 2 (limit: 406449) Memory: 105.8M CGroup: /system.slice/ovirt-ha-agent.service └─14290 /usr/libexec/platform-python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent Dec 05 05:34:40 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 05:34:40 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 07:34:37 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 07:34:37 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 08:34:45 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 08:34:45 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 10:34:39 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 10:34:39 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 11:34:44 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 11:34:44 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Is the broker error causing the agent error or vice-versa ? All I did a couple of weeks ago was reboot the host (after having shut down the VMs and stopped the engine). The engine and VMs seem to be working fine. Thanks for all of your help. On 12/2/21 1:15 PM, Strahil Nikolov wrote:
You need to restart ovirt-ha-broker.service and ovirt-ha-agent.service
Best Regards, Strahil Nikolov
On Mon, Nov 29, 2021 at 18:31, Valerio Luccio <valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=rGhA8YehG-yXVxSDfgMelHF0JHbjFnZsW5jzxfNT5jg&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=JxOjRfL0kT413ILzviDUSds8hiIDdGsJkuMAlhDxXF8&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABCACVKMVJ6YE2... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_ABCACVKMVJ6YE2UE2TSOAQSFUOXRWNMN_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=r64TAl26D5BJ8aUrqSEt8g22Hd2tOGz7l6PLNvplKPY&e=>
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003 "In an open world, who needs windows or gates ?"

On Sun, Dec 5, 2021 at 8:13 PM Valerio Luccio <valerio.luccio@nyu.edu> wrote:
I stopped the VMs and then "hosted-engine --vm-shutdown" followed by "hosted-engine --vm-start", I thought that would restart everything, but apparently not.
Now I just did "systemctl status ovirt-ha-broker" and got this ugly thing:
ovirt-ha-broker ovirt_hosted_engine_ha.broker.notifications.Notifications ERRO> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/notific> timeout=float(cfg["smtp-timeout"])) File "/usr/lib64/python3.6/smtplib.py", line 251, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib64/python3.6/smtplib.py", line 336, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib64/python3.6/smtplib.py", line 307, in _get_socket self.source_address) File "/usr/lib64/python3.6/socket.py", line 724, in create_connection raise err File "/usr/lib64/python3.6/socket.py", line 713, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused
This is harmless, can be ignored. It most likely happens because you configured to send notifications via email to an unreachable mail server - the default is localhost. I suggest to check the log for more/other errors.
And
systemctl status --no-pager --full ovirt-ha-agent ● ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2021-11-19 15:17:19 EST; 2 weeks 1 days ago Main PID: 14290 (ovirt-ha-agent) Tasks: 2 (limit: 406449) Memory: 105.8M CGroup: /system.slice/ovirt-ha-agent.service └─14290 /usr/libexec/platform-python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent
Dec 05 05:34:40 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 05:34:40 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf
That's not good. You might find more details it the HA logs or vdsm logs.
Dec 05 07:34:37 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 07:34:37 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 08:34:45 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 08:34:45 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 10:34:39 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 10:34:39 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 11:34:44 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 11:34:44 argus.cbi.fas.nyu.edu ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf
Is the broker error causing the agent error or vice-versa ?
All I did a couple of weeks ago was reboot the host (after having shut down
the VMs and stopped the engine).
The engine and VMs seem to be working fine.
That's good - so not all hope is lost :-). But better find the root cause and handle it. Best regards,
Thanks for all of your help.
On 12/2/21 1:15 PM, Strahil Nikolov wrote:
You need to restart ovirt-ha-broker.service and ovirt-ha-agent.service
Best Regards, Strahil Nikolov
On Mon, Nov 29, 2021 at 18:31, Valerio Luccio <valerio.luccio@nyu.edu> <valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=rGhA8YehG-yXVxSDfgMelHF0JHbjFnZsW5jzxfNT5jg&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=JxOjRfL0kT413ILzviDUSds8hiIDdGsJkuMAlhDxXF8&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABCACVKMVJ6YE2... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_ABCACVKMVJ6YE2UE2TSOAQSFUOXRWNMN_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=r64TAl26D5BJ8aUrqSEt8g22Hd2tOGz7l6PLNvplKPY&e=>
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003
"In an open world, who needs windows or gates ?"
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AUYRVXQGSV5GKC...
-- Didi

Thanks for your input Didi. On 12/5/21 1:20 PM, Yedidyah Bar David wrote:
On Sun, Dec 5, 2021 at 8:13 PM Valerio Luccio <valerio.luccio@nyu.edu> wrote:
I stopped the VMs and then "hosted-engine --vm-shutdown" followed by "hosted-engine --vm-start", I thought that would restart everything, but apparently not.
Now I just did "systemctl status ovirt-ha-broker" and got this ugly thing:
ovirt-ha-broker ovirt_hosted_engine_ha.broker.notifications.Notifications ERRO> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/notific> timeout=float(cfg["smtp-timeout"])) File "/usr/lib64/python3.6/smtplib.py", line 251, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib64/python3.6/smtplib.py", line 336, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib64/python3.6/smtplib.py", line 307, in _get_socket self.source_address) File "/usr/lib64/python3.6/socket.py", line 724, in create_connection raise err File "/usr/lib64/python3.6/socket.py", line 713, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused
This is harmless, can be ignored. It most likely happens because you configured to send notifications via email to an unreachable mail server - the default is localhost. I suggest to check the log for more/other errors. Thanks, I'll dig around. Glad to hear it's harmless.
And
systemctl status --no-pager --full ovirt-ha-agent ● ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2021-11-19 15:17:19 EST; 2 weeks 1 days ago Main PID: 14290 (ovirt-ha-agent) Tasks: 2 (limit: 406449) Memory: 105.8M CGroup: /system.slice/ovirt-ha-agent.service └─14290 /usr/libexec/platform-python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent
Dec 05 05:34:40argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 05:34:40argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf
That's not good. You might find more details it the HA logs or vdsm logs. I'll see what I can find out.
Dec 05 07:34:37argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 07:34:37argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 08:34:45argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 08:34:45argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 10:34:39argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 10:34:39argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Dec 05 11:34:44argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable to extract HEVM OVF Dec 05 11:34:44argus.cbi.fas.nyu.edu <http://argus.cbi.fas.nyu.edu> ovirt-ha-agent[14290]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf
Is the broker error causing the agent error or vice-versa ?
All I did a couple of weeks ago was reboot the host (after having shut down the VMs and stopped the engine).
The engine and VMs seem to be working fine.
That's good - so not all hope is lost :-). But better find the root cause and handle it. I didn't check the status because everything was working. I didn't suspect a problem until I saw those huge log files.
Best regards,
Thanks for all of your help.
On 12/2/21 1:15 PM, Strahil Nikolov wrote:
You need to restart ovirt-ha-broker.service and ovirt-ha-agent.service
Best Regards, Strahil Nikolov
On Mon, Nov 29, 2021 at 18:31, Valerio Luccio <valerio.luccio@nyu.edu> <mailto:valerio.luccio@nyu.edu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=rGhA8YehG-yXVxSDfgMelHF0JHbjFnZsW5jzxfNT5jg&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=JxOjRfL0kT413ILzviDUSds8hiIDdGsJkuMAlhDxXF8&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABCACVKMVJ6YE2... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_ABCACVKMVJ6YE2UE2TSOAQSFUOXRWNMN_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=5xo7GuPW-Qg8DXHy8qKrpYQJtzOUAo8uFpShKUcGEsTg58MuzUGegNROFK8XTbNG&s=r64TAl26D5BJ8aUrqSEt8g22Hd2tOGz7l6PLNvplKPY&e=>
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003
"In an open world, who needs windows or gates ?"
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_privacy-2Dpolicy.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=BA0X8INpzd5_5mRfF9Ku9MvBgiJIZAX7ErJBKre2sNwSCizNaB8jC0uud5vA9lzw&s=orCEPbT1QL89Pw0TbLOnuIbwwlU6LhOLoHXoia_rOe0&e=> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_community_about_community-2Dguidelines_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=BA0X8INpzd5_5mRfF9Ku9MvBgiJIZAX7ErJBKre2sNwSCizNaB8jC0uud5vA9lzw&s=67UaVscMjVT1y-aRhpnUx3Y4_-1LQHPAnCQrwCNtPXE&e=> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AUYRVXQGSV5GKC... <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archives_list_users-40ovirt.org_message_AUYRVXQGSV5GKCIQB33UZZGAYMDJ27QJ_&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=zZK0dca4HNf-XwnAN9ais1C3ncS0n2x39pF7yr-muHY&m=BA0X8INpzd5_5mRfF9Ku9MvBgiJIZAX7ErJBKre2sNwSCizNaB8jC0uud5vA9lzw&s=2ywTbbNYnqs6DjV0gmx8UGvc76UpG5drJT6DMf_YNUk&e=>
-- Didi
-- Valerio Luccio High Performance Computing 10 Astor Place, Room 416D New York University New York, NY 10003 "In an open world, who needs windows or gates ?"
participants (3)
-
Strahil Nikolov
-
Valerio Luccio
-
Yedidyah Bar David