[ovirt-users] Re: VM has been paused due to no Storage space error.

Tuesday, 5 November 2024

Hi,

The information is really unclear.
In previous emails you state you use preallocated disks, now you tell us 
it only happens on thin provisioned disks.
Also the VM disk was 300GB, and now its 80GB.

How did you check the LV size? Can you show me the command + output.
What does: vdsm-tool dump-volume-chains storage-id give for the disk 
that has the issue? The VM has 1 or more disks?
Can you also do a virsh --readonly dumpxml <vm> on the hypervisor where 
the vm runs and give the output?

Thanks
Jean-Louis

On 5/11/2024 11:55, suporte(a)logicworks.pt wrote:
...
 Hi,

 Can't find any error on vdsm log. What should I look for?
 The size of the Logical Volume for that disk is 80GB
 We have plenty free space.
 The VM cannot start again. it only happens with thin provision disks. 
 If I move the disk of the paused VM to another storage it works for a 
 day and than it pauses again.

 Thanks
 José

 ------------------------------------------------------------------------
 *De: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;
 *Para: *"suporte" <suporte(a)logicworks.pt&gt;, "Colin Coe" 
 <colin.coe(a)gmail.com&gt;
 *Cc: *"users" <users(a)ovirt.org&gt;
 *Itens enviados: *Segunda-feira, 4 de Novembro de 2024 12:14:30
 *Assunto: *Re: [ovirt-users] Re: VM has been paused due to no Storage 
 space error.

 Hi,

 I think you need to dig deeper into the issue.
 - What does vdsm logs show for example?
 - What is the size of the Logical Volume for that disk?
 - The Volume Group (checked on that hypervisor) still gives free space?
 - There is no way to start the VM again without getting it paused?

 Jean-Louis

 On 4/11/2024 13:09, suporte(a)logicworks.pt wrote:

     But we are in Version 4.5.4-1.el8, so still don't know.

     ------------------------------------------------------------------------
     *From: *"suporte" <suporte(a)logicworks.pt&gt;
     *To: *"Colin Coe" <colin.coe(a)gmail.com&gt;
     *Cc: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;, "users"
     <users(a)ovirt.org&gt;
     *Sent: *Monday, November 4, 2024 12:08:11 PM
     *Subject: *Re: [ovirt-users] Re: VM has been paused due to no
     Storage space error.

     if I understand correctly, upgrading will solve the issue!

     ------------------------------------------------------------------------
     *From: *"Colin Coe" <colin.coe(a)gmail.com&gt;
     *To: *"suporte" <suporte(a)logicworks.pt&gt;
     *Cc: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;, "users"
     <users(a)ovirt.org&gt;
     *Sent: *Monday, November 4, 2024 12:04:19 PM
     *Subject: *Re: [ovirt-users] Re: VM has been paused due to no
     Storage space error.

     I don't think this article will help you. It specifically talks
     about upgrading to RHV 4.4SP1 (which is oVirt 4.5). There is a
     config workaround but it says don't do this in RHV 4.4SP1.

     On Mon, 4 Nov 2024 at 19:30, José Ferradeira via Users
     <users(a)ovirt.org&gt; wrote:

         Hi,

         I found this: https://access.redhat.com/solutions/130843
         But have no access.

         Any help?

         Thanks
         José

         ------------------------------------------------------------------------
         *From: *"José Ferradeira via Users" <users(a)ovirt.org&gt;
         *To: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;
         *Cc: *"users" <users(a)ovirt.org&gt;
         *Sent: *Wednesday, August 21, 2024 11:27:47 AM
         *Subject: *[ovirt-users] Re: VM has been paused due to no
         Storage space error.

         Hi,
         Ok, discard is not enabled on the disk.
         Reading https://gitlab.com/qemu-project/qemu/-/issues/1621

         I'm sure the image did not grow over 110% of its virtual size

         Any idea?

         Thanks
         José

         ------------------------------------------------------------------------
         *From: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;
         *To: *suporte(a)logicworks.pt
         *Cc: *"users" <users(a)ovirt.org&gt;
         *Sent: *Wednesday, August 21, 2024 10:17:22 AM
         *Subject: *Re: [ovirt-users] VM has been paused due to no
         Storage space error.

         Hi,

         Not talking about discard on the storage domain, but on
         disk/vm level.
         Is it enabled there?

         Cause if that is the cause, it could be
         https://gitlab.com/qemu-project/qemu/-/issues/1621

         On 21/08/2024 11:14, suporte(a)logicworks.pt wrote:

             Hi Jean-Louis,

             The VM disk is 300GB with 216GB available, preallocated on
             a iSCSI storage with 3532GB total and 1932GB free.
             Have enable incremental backup. Discard After Delete is
             not enabled.

             Thanks
             José

             ------------------------------------------------------------------------
             *From: *"Jean-Louis Dupond" <jean-louis(a)dupond.be&gt;
             <mailto:jean-louis@dupond.be>
             *To: *suporte(a)logicworks.pt, "users" <users(a)ovirt.org&gt;
             <mailto:users@ovirt.org>
             *Sent: *Wednesday, August 21, 2024 9:12:53 AM
             *Subject: *Re: [ovirt-users] VM has been paused due to no
             Storage space error.

             Hi Jose,

             How big are the disks? And what size are they on the storage?
             I guess the images are qcow2?

             Is discard enabled?

             Jean-Louis

             On 20/08/2024 13:42, José Ferradeira via Users wrote:

                 Hello,

                 running ovirt Version 4.5.4-1.el8 on Centos 8,
                 randomly we have this error:
                 VM has been paused due to no Storage space error.
                 We have plenty of space on the iSCSI storage. This is
                 a preallocated disk, VirtIO-SCSi.
                 No user interaction. It happens, so far, with 3 VM,
                 Windows and Ubuntu.
                 This service was stopped: dnf-makecache.service

                 This is what I found on the engine log:

                 2024-08-19 01:04:35,522+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-25) [eb7e5f1] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Up' --> 'Paused'
                 2024-08-19 01:04:35,665+01 ERROR
                 [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
                 (ForkJoinPool-1-worker-25) [eb7e5f1] EVENT_ID:
                 VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to
                 no Storage space error.
                 2024-08-19 09:26:35,855+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-29) [72482216] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Paused' --> 'Down'
                 2024-08-19 09:26:48,114+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [72482216] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'WaitForLaunch' --> 'PoweringUp'
                 2024-08-19 09:27:50,062+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6)
                 [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo)
                 moved from 'PoweringUp' --> 'Up'
                 2024-08-19 09:29:25,145+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [72482216] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Up' --> 'Paused'
                 2024-08-19 09:29:25,273+01 ERROR
                 [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
                 (ForkJoinPool-1-worker-15) [72482216] EVENT_ID:
                 VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to
                 no Storage space error.
                 2024-08-19 09:37:26,128+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [6d88f065] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Paused' --> 'Down'
                 2024-08-19 09:41:43,300+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [6d88f065] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'WaitForLaunch' --> 'PoweringUp'
                 2024-08-19 09:42:14,882+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-23) [6d88f065] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'PoweringUp' --> 'Up'
                 2024-08-19 09:42:59,792+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [6d88f065] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Up' --> 'Paused'
                 2024-08-19 09:42:59,894+01 ERROR
                 [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
                 (ForkJoinPool-1-worker-15) [6d88f065] EVENT_ID:
                 VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to
                 no Storage space error.
                 2024-08-19 09:45:30,334+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [6b3d8ee] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Paused' --> 'Down'
                 2024-08-19 09:47:51,068+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [6b3d8ee] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'WaitForLaunch' --> 'PoweringUp'
                 2024-08-19 09:48:50,710+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-80)
                 [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo)
                 moved from 'PoweringUp' --> 'Up'
                 2024-08-19 10:06:38,810+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [1dd98021] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'PoweringDown' --> 'Down'
                 2024-08-19 10:08:11,606+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [1dd98021] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'WaitForLaunch' --> 'PoweringUp'
                 2024-08-19 10:09:12,507+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25)
                 [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo)
                 moved from 'PoweringUp' --> 'Up'
                 2024-08-19 10:21:13,835+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [63fa2421] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'Up' --> 'Down'
                 2024-08-19 10:25:19,302+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-15) [63fa2421] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'WaitForLaunch' --> 'PoweringUp'
                 2024-08-19 10:26:05,456+01 INFO
                 [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
                 (ForkJoinPool-1-worker-3) [63fa2421] VM
                 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved
                 from 'PoweringUp' --> 'Up'

                 And we cannot start the VM anymore.

                 Any idea?
                 Thanks

                 -- 
                 ------------------------------------------------------------------------
                 Jose Ferradeira
                 http://www.logicworks.pt

                 _______________________________________________
                 Users mailing list --users(a)ovirt.org
                 To unsubscribe send an email tousers-leave(a)ovirt.org
                 Privacy Statement:https://www.ovirt.org/privacy-policy.html
                 oVirt Code of
Conduct:https://www.ovirt.org/community/about/community-guidelines/
                 List
Archives:https://lists.ovirt.org/archives/list/users@ovirt.org/message/YK...

         _______________________________________________
         Users mailing list -- users(a)ovirt.org
         To unsubscribe send an email to users-leave(a)ovirt.org
         Privacy Statement: https://www.ovirt.org/privacy-policy.html
         oVirt Code of Conduct:
         https://www.ovirt.org/community/about/community-guidelines/
         List Archives:

https://lists.ovirt.org/archives/list/users@ovirt.org/message/OON4G6VUC6Z...
         _______________________________________________
         Users mailing list -- users(a)ovirt.org
         To unsubscribe send an email to users-leave(a)ovirt.org
         Privacy Statement: https://www.ovirt.org/privacy-policy.html
         oVirt Code of Conduct:
         https://www.ovirt.org/community/about/community-guidelines/
         List Archives:

https://lists.ovirt.org/archives/list/users@ovirt.org/message/AWZVVV4TEB6...

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

[ovirt-users] Re: VM has been paused due to no Storage space error.