Hello,
Thank you for helping with previous issue. Unfortunately we got another. We have RHV manager with several different hosts. After the backup, vm placed on RHEL shuts down. In engine.log I found moment when this happen: "VM '35183baa-1c70-4016-b7cd-528889876f19'(stor2rrd) moved from 'Up' --> 'Down'".  I attached whole logs to email. It doesn't matter if it's a full or incremental, results are always the same. RHVH hosts works properly.

Log fragment:
2020-12-08 10:18:33,845+01 INFO  [org.ovirt.engine.core.bll.storage.disk.image.ImageTransferUpdater] (default task-76) [60da31e3-92f6-4555-8c43-2f8afee272e0] Updating image transfer 87bdb42e-e64c-460d-97ac-218e923336a1 (image e57e4af0-5d0b-4f60-9e6c-e217c666e5e6) phase to Finalizing Success
2020-12-08 10:18:33,940+01 INFO  [org.ovirt.engine.core.bll.StopVmBackupCommand] (default task-76) [89ec1a77-4b46-42b0-9d0f-15e53d5f952a] Running command: StopVmBackupCommand internal: false. Entities affected :  ID: 35183baa-1c70-4016-b7cd-528889876f19 Type: VMAction group BACKUP_DISK with role type ADMIN,  ID: e57e4af0-5d0b-4f60-9e6c-e217c666e5e6 Type: DiskAction group BACKUP_DISK with role type ADMIN
2020-12-08 10:18:33,940+01 INFO  [org.ovirt.engine.core.bll.StopVmBackupCommand] (default task-76) [89ec1a77-4b46-42b0-9d0f-15e53d5f952a] Stopping VmBackup 'aae03819-cea6-45a1-9ee5-0f831af8464d'
2020-12-08 10:18:33,952+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.StopVmBackupVDSCommand] (default task-76) [89ec1a77-4b46-42b0-9d0f-15e53d5f952a] START, StopVmBackupVDSCommand(HostName = rhv-2, VmBackupVDSParameters:{hostId='afad6b8b-78a6-4e9a-a9bd-783ad42a2d47', backupId='aae03819-cea6-45a1-9ee5-0f831af8464d'}), log id: 78b2c27a
2020-12-08 10:18:33,958+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.StopVmBackupVDSCommand] (default task-76) [89ec1a77-4b46-42b0-9d0f-15e53d5f952a] FINISH, StopVmBackupVDSCommand, return: , log id: 78b2c27a
2020-12-08 10:18:33,975+01 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-76) [89ec1a77-4b46-42b0-9d0f-15e53d5f952a] EVENT_ID: VM_BACKUP_FINALIZED(10,794), Backup <UNKNOWN> for VM stor2rrd finalized (User: admin@internal-authz).
2020-12-08 10:18:35,221+01 INFO  [org.ovirt.engine.core.sso.servlets.OAuthRevokeServlet] (default task-73) [] User admin@internal successfully logged out
2020-12-08 10:18:35,236+01 INFO  [org.ovirt.engine.core.bll.aaa.TerminateSessionsForTokenCommand] (default task-81) [1b35276c] Running command: TerminateSessionsForTokenCommand internal: true.
2020-12-08 10:18:35,236+01 INFO  [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (default task-81) [1b35276c] Not removing session '90TxdK0PBueLijy+sCrFoHC/KNUGNzNpZuYMK/yKDAkbAefFr+8wOJsATsDKv18LxpyxCl+eX7hTHNxN23anAw==', session has running commands for user 'admin@internal-authz'.
2020-12-08 10:18:35,447+01 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-31) [] VM '35183baa-1c70-4016-b7cd-528889876f19' was reported as Down on VDS 'afad6b8b-78a6-4e9a-a9bd-783ad42a2d47'(rhv-2)
2020-12-08 10:18:35,448+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-31) [] START, DestroyVDSCommand(HostName = rhv-2, DestroyVmVDSCommandParameters:{hostId='afad6b8b-78a6-4e9a-a9bd-783ad42a2d47', vmId='35183baa-1c70-4016-b7cd-528889876f19', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 4f473135
2020-12-08 10:18:35,451+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-31) [] FINISH, DestroyVDSCommand, return: , log id: 4f473135
2020-12-08 10:18:35,451+01 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-31) [] VM '35183baa-1c70-4016-b7cd-528889876f19'(stor2rrd) moved from 'Up' --> 'Down'
2020-12-08 10:18:35,466+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-31) [] EVENT_ID: VM_DOWN_ERROR(119), VM stor2rrd is down with error. Exit message: Lost connection with qemu process.
2020-12-08 10:18:35,484+01 INFO  [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-34085) [3da40159] Running command: ProcessDownVmCommand internal: true.


The environment of faulty host is:
OS Version: RHEL - 8.3 - 1.0.el8
OS Description: Red Hat Enterprise Linux 8.3 (Ootpa)
Kernel Version: 4.18.0 - 240.1.1.el8_3.x86_64
KVM Version: 5.1.0 - 14.module+el8.3.0+8438+644aff69
LIBVIRT Version: libvirt-6.6.0-7.module+el8.3.0+8424+5ea525c5
VDSM Version: vdsm-4.40.26.3-1.el8ev
SPICE Version: 0.14.3 - 3.el8
GlusterFS Version: [N/A]
CEPH Version: librbd1-12.2.7-9.el8
Open vSwitch Version: openvswitch-2.11-7.el8ev
Nmstate Version: nmstate-0.3.4-13.el8_3

Regards

Łukasz Kołaciński

Junior Java Developer

e-mail: l.kolacinski@storware.eu


 

STORWARE

ul. Leszno 8/44
01-192 Warszawa
www.storware.eu

facebook

twitter

linkedin

Storware_Stopka_09

 

Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa 000510131 , NIP 5213672602. Wiadomość ta jest przeznaczona jedynie dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów. This message is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you have received this message in error, please contact the sender and remove the material from all of your computer systems.