2018-09-10 05:27:19,084-0400 INFO (jsonrpc/4) [api.virt] FINISH migrate return={'status': {'message': 'Migration in progress', 'code': 0}, 'progress': 0} from=::ffff:192.168.200.2,error:I am seeing a failure in 4.2 which suggests the same issue is causing failure in 4.2 as well (on different test)full logs:
https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/ 3069/artifact/basic-suite.el7. x86_64/test_logs/basic-suite- 4.2/post-006_migrations.py/ 36202, flow_id=107cdbdd, vmId=9d5e4d63-41ad-47eb-bf04- 0fc
8a49d59a6 (api:52)
2018-09-10 05:27:19,085-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call VM.migrate succeeded in 0.01 seconds (__init__:573)
2018-09-10 05:27:21,583-0400 INFO (migsrc/9d5e4d63) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') Creation of destination VM took: 2 seconds (migration:473)
2018-09-10 05:27:21,583-0400 INFO (migsrc/9d5e4d63) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') starting migration to qemu+tls://lago-basic-suite-4- 2-host-1/system with miguri tcp://lago-basic-suite-4-2- host-1 (migration:502)
2018-09-10 05:27:25,044-0400 ERROR (periodic/0) [virt.periodic.Operation] <vdsm.virt.sampling.HostMonitor object at 0x7f0ddfc2b190> operation failed (periodic:222)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/virt/periodic. py", line 220, in __call__
self._func()
File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. py", line 576, in __call__
sample = HostSample(self._pid)
File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. py", line 240, in __init__
self.interfaces = _get_interfaces_and_samples()
File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. py", line 202, in _get_interfaces_and_samples
links_and_samples[link.name] = InterfaceSample(link)
File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. py", line 109, in __init__
self.speed = _getLinkSpeed(link)
File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. py", line 590, in _getLinkSpeed
speed = vlan.speed(dev.name)
File "/usr/lib/python2.7/site-packages/vdsm/network/link/ vlan.py", line 35, in speed
dev_speed = nic.read_speed_using_sysfs(dev_name)
File "/usr/lib/python2.7/site-packages/vdsm/network/link/ nic.py", line 47, in read_speed_using_sysfs
s = int(f.read())
IOError: [Errno 22] Invalid argument
2018-09-10 05:27:25,499-0400 INFO (libvirt/events) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') CPU stopped: onSuspend (vm:6203)
2018-09-10 05:27:27,167-0400 INFO (migsrc/9d5e4d63) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') migration took 7 seconds to complete (migration:514)
2018-09-10 05:27:27,168-0400 INFO (migsrc/9d5e4d63) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') Changed state to Down: Migration succeeded (code=4) (vm:1693)
2018-09-10 05:27:27,176-0400 INFO (migsrc/9d5e4d63) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') Stopping connection (guestagent:438)
2018-09-10 05:27:27,183-0400 INFO (jsonrpc/6) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:192.168.200.2,36202, vmId=9d5e4d63-41ad-47eb-bf04- 0fc8a49d59a6 (api:46)
2018-09-10 05:27:27,183-0400 INFO (jsonrpc/6) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') Release VM resources (vm:5283)
2018-09-10 05:27:27,183-0400 WARN (jsonrpc/6) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') trying to set state to Powering down when already Down (vm:612)
2018-09-10 05:27:27,183-0400 INFO (jsonrpc/6) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') Stopping connection (guestagent:438)
2018-09-10 05:27:27,183-0400 INFO (jsonrpc/6) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') _destroyVmGraceful attempt #0 (vm:5320)
2018-09-10 05:27:27,184-0400 WARN (jsonrpc/6) [virt.vm] (vmId='9d5e4d63-41ad-47eb-bf04-0fc8a49d59a6') VM '9d5e4d63-41ad-47eb-bf04- 0fc8a49d59a6' couldn't be destroyed in libvirt: Requested operation is not valid: domain is not running (vm
:5341)
2018-09-10 05:27:27,185-0400 INFO (jsonrpc/6) [vdsm.api] START teardownImage(sdUUID='8cbc7539-1ab3-4a3e-9504- 65b5002c6546', spUUID='3b4effaf-6bad-42b8- 810b-80faaa5d5a1e', imgUUID='90402f78-6c84-44cf- bcdc-7823f252d185', volUUID=None) from
=::ffff:192.168.200.2,36202, task_id=73c097e0-6cb6-449e-bebe-f0dfa55e2155 (api:46)
2018-09-10 05:27:27,186-0400 INFO (jsonrpc/6) [storage.StorageDomain] Removing image run directory '/var/run/vdsm/storage/8cbc7539-1ab3-4a3e-9504- 65b5002c6546/90402f78-6c84- 44cf-bcdc-7823f252d185' (blockSD:1360)
2018-09-10 05:27:27,186-0400 INFO (jsonrpc/6) [storage.fileUtils] Removing directory: /var/run/vdsm/storage/8cbc7539-1ab3-4a3e-9504- 65b5002c6546/90402f78-6c84- 44cf-bcdc-7823f252d185 (fileUtils:178)
2018-09-10 05:27:27,325-0400 INFO (jsonrpc/6) [storage.LVM] Deactivating lvs: vg=8cbc7539-1ab3-4a3e-9504-65b5002c6546 lvs=['715b9383-87ec-4381-aaff- f084df42b3ce', 'ba23b2ac-90db-41b7-ab0f- c49f0c4d1b1e', '9201288a-918b-41a8-8ff2- 6643833e1
d91'] (lvm:1310)
https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/ 3069/artifact/basic-suite.el7. x86_64/test_logs/basic-suite- 4.2/post-006_migrations.py/ On Mon, Sep 10, 2018 at 11:03 AM, Dafna Ron <dron@redhat.com> wrote:I don't think its related directly to the change but anything between
packaging: use system provided nimbus-jose-jwt also on el7 patch and current patch could have caused the failure.
https://gerrit.ovirt.org/#/q/project:ovirt-engine+status:Mer ged+branch:master Sandro, Ondra, can you please have a look?(Relevant) error snippet from the log:
<error>
018-09-10 04:05:33,850-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Installation of lago-upgrade-from-release-suit e-master-host-0. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDI R}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar -b1 --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt- host-deploy.tar
2018-09-10 04:05:33,850-04 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Tarball '/var/cache/ovirt-engine/ovirt -host-deploy.tar' refresh
2018-09-10 04:05:33,915-04 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] SSH execute 'root@lago-upgrade-from-releas e-suite-master-host-0' 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDI R}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar -b1 --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True '
2018-09-10 04:05:34,694-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLo gDirector] (VdsDeploy) [33bb53b] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suit e-master-host-0. Stage: Initializing.
2018-09-10 04:05:34,710-04 INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (VdsDeploy) [33bb53b] transaction rolled back
2018-09-10 04:05:34,719-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [33bb53b] Error during deploy dialog
2018-09-10 04:05:34,723-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Error during host lago-upgrade-from-release-suit e-master-host-0 install
2018-09-10 04:05:34,736-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLo gDirector] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR( 511), An error has occurred during installation of Host lago-upgrade-from-release-suit e-master-host-0: CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLExcept ion: ERROR: invalid byte sequence for encoding "UTF8": 0x00.
2018-09-10 04:05:34,737-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Error during host lago-upgrade-from-release-suit e-master-host-0 install, preferring first exception: CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLExcept ion: ERROR: invalid byte sequence for encoding "UTF8": 0x00
2018-09-10 04:05:34,737-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalComm and] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Host installation failed for host 'febee66d-fa4b-4321-9c99-cb053 ddd4817', 'lago-upgrade-from-release-sui te-master-host-0': CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLExcept ion: ERROR: invalid byte sequence for encoding "UTF8": 0x00
2018-09-10 04:05:34,760-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] START, SetVdsStatusVDSCommand(HostNam e = lago-upgrade-from-release-suit e-master-host-0, SetVdsStatusVDSCommandParamete rs:{hostId='febee66d-fa4b-4321 -9c99-cb053ddd4817', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 793ad1ed
2018-09-10 04:05:34,769-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] FINISH, SetVdsStatusVDSCommand, return: , log id: 793ad1ed
2018-09-10 04:05:34,781-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLo gDirector] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] EVENT_ID: VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-suit e-master-host-0 installation failed. CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLExcept ion: ERROR: invalid byte sequence for encoding "UTF8": 0x00.
2018-09-10 04:05:34,794-04 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalComm and] (EE-ManagedThreadFactory-engin e-Thread-1) [33bb53b] Lock freed to object 'EngineLock:{exclusiveLocks='[ febee66d-fa4b-4321-9c99-cb053d dd4817=VDS]', sharedLocks=''}'
2018-09-10 04:06:10,003-04 INFO [org.ovirt.engine.core.bll.quota.QuotaManager] (EE-ManagedThreadFactory-engin eScheduled-Thread-44) [] Quota Cache updated. (20 msec)
(END)</error>
_______________________________________________
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community- guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/ message/ TZGEPC56JYKKCOZ5XODOJMVLBR7MI6 KG/