[ovirt-users] Re: After upgrade to vdsm-4.40.90.4-1.el8 - Internal JSON-RPC error - how to fix?

在 2022/1/6 15:53, Liran Rotenberg 写道: when I try to import ova to san storage. there are tons of log like: 2022-01-06 09:15:55,058+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-8) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt4.adagene.cn command Get Host Statistics failed: Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"} 2022-01-06 09:15:55,059+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-8) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 09:15:58,347+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-93) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:08,358+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:10,090+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:10,090+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Failed in 'Get Host Statistics' method 2022-01-06 09:16:10,090+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:10,092+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-13) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 09:16:18,367+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [b8ca5e99-e3e5-4a09-8b66-e225f45fdf77] Command 'ImportVmFromOva' (id: 'a1ef50bc-5dba-40af-9ebd-48bb421db495') waiting on child command id: '7943bb90-5964-4458-a1b8-d957cff94f09' type:'ConvertOva' to complete 2022-01-06 09:16:25,099+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 09:16:25,099+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Failed in 'Get Host Statistics' method 2022-01-06 09:16:25,099+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-34) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] these logs seems endless, I have to stop importing tasks after several hours waitting. --------------------------------------------------------------------------- when I try to import ova to a standalone host with local storage, logs like: 2022-01-06 12:31:21,737+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadP ool-Thread-9) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt6.adagene.cn command Get Host Statistics failed: Internal JSON-RPC error: {'reason': "'st r' object has no attribute 'decode'"} 2022-01-06 12:31:21,744+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 9) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': "' str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:23,776+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-47) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:33,788+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-77) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:36,709+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:36,710+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Failed in 'Get Host Statistics' method 2022-01-06 12:31:36,710+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-85) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:36,712+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 85) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': " 'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:43,800+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-86) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:31:51,712+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:51,712+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Failed in 'Get Host Statistics' method 2022-01-06 12:31:51,713+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-36) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:31:51,714+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread- 36) [] Unable to GetStats: VDSErrorException: VDSGenericException: VDSErrorException: Failed to Get Host Statistics, error = Internal JSON-RPC error: {'reason': " 'str' object has no attribute 'decode'"}, code = -32603 2022-01-06 12:31:53,812+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-93) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4 -4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:32:03,823+08 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Th read-4) [9015b315-7fd1-408b-a231-41354d4b6384] Command 'ImportVmFromOva' (id: 'c8b98e81-bfc7-4e51-8082-4e4aec0d182e') waiting on child command id: '1378cdc5-a8d4- 4588-94d2-a72004cf268d' type:'ConvertOva' to complete 2022-01-06 12:32:06,816+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] 2022-01-06 12:32:06,816+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Failed in 'Get Host Statistics' method 2022-01-06 12:32:06,817+08 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool -Thread-76) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'str' object has no attribute 'decode'"}] And it will succeed after serveral warning messages above.