[ovirt-users] Re: After upgrade to vdsm-4.40.90.4-1.el8 - Internal JSON-RPC error - how to fix?

在 2022/1/6 15:53, Liran Rotenberg 写道: On Thu, Jan 6, 2022 at 9:20 AM Adam Xu <adam_xu(a)adagene.com.cn&gt; wrote: > I also got the error when I try to import an ova from vmware to my ovirt > cluster using a san storage domain. > > I resovled this by importing this ova to a standalone host which is using > its local storage. > 在 2021/11/18 17:41, John Mortensen 写道: > > Hi, > After we upgraded to vdsm-4.40.90.4-1.el8 on our two node cluster two things has happned: > 1. First node that was upgraded now continuously logs this error: > VDSM <node name> command Get Host Statistics failed: Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"} > > Hi, Looks like [1], however in that report the import process succeeded despite those errors. Can you please share the import logs? when I try to import ova to san storage. there are tons of log like: 2022-01-06 09:15:55,058+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-8) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt4.adagene.cn command Get Host Statistics failed: Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"} 2022-01-06 09:15:55,059+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-8) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 09:15:58,347+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-93) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:08,358+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:10,090+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:10,090+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Failed in 'Get Host Statistics' method 2022-01-06 09:16:10,090+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:10,092+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 09:16:18,367+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:25,099+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:25,099+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Failed in 'Get Host Statistics' method 2022-01-06 09:16:25,099+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] these logs seems endless, I have to stop importing tasks after several hours waitting.

--------------------------------------------------------------------------- when I try to import ova to a standalone host with local storage, logs like: 2022-01-06 12:31:21,737+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadP ool-Thread-9) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt6.adagene.cn command Get Host Statistics failed: Internal JSON-RPC error: {'reason': "'st r' object has no attribute 'decode'"} 2022-01-06 12:31:21,744+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 9) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "' str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:23,776+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-47) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:33,788+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-77) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:36,709+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:36,710+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Failed in 'Get Host Statistics' method 2022-01-06 12:31:36,710+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:36,712+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 85) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': " 'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:43,800+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-86) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:51,712+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:51,712+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Failed in 'Get Host Statistics' method 2022-01-06 12:31:51,713+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:51,714+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 36) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': " 'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:53,812+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-93) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:32:03,823+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-4) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4- 4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:32:06,816+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:32:06,816+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Failed in 'Get Host Statistics' method 2022-01-06 12:32:06,817+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] And it will succeed after serveral warning messages above. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2026809 > 2. During import of two virtual machines from VMware (has done multible before upgrade) the import seems to never finish - currently running on day 2-3... any clues how to fix this? > /John > > Import and engine logs would be helpful. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5AA3L6IEXNC... > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/MWD4ZKXB2SJ... > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/SP5K7MOQFUS... _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MP4OWTRLZQL...