[ovirt-users] VM has been paused due to no Storage space error.

Thu Apr 14 07:57:02 UTC 2016

Ok, that makes sense, thanks for the insight both Alex and Fred. I'm 
attaching the VDSM log of the SPM node at the time of the pause. I 
couldn't find anything that would clearly identify the problem, but 
maybe you'll be able to.

Thanks.

Regards.

El 2016-04-13 13:09, Fred Rolland escribió:
> Hi,
> 
> Yes, just as Alex explained, if the disk has been created as thin
> provisioning, the vdsm will extends once a watermark is reached.
> Usually it should not get to the state the Vm is paused.
> 
> From the log, you can see that the request for extension has been sent
> before the VM got to the No Space Error.
> Later, we can see the VM resuming.
> 
> INFO::2016-04-13
> 10:52:04,182::vm::1026::virt.vm::(extendDrivesIfNeeded)
> vmId=`f9cd282e-110a-4896-98d3-6d320662744d`::Requesting extension for
> volume
> ....
> INFO::2016-04-13 10:52:29,360::vm::3728::virt.vm::(onIOError)
> vmId=`f9cd282e-110a-4896-98d3-6d320662744d`::abnormal vm stop device
> virtio-disk0 error enospc
> ....
> INFO::2016-04-13 10:52:54,317::vm::5084::virt.vm::(_logGuestCpuStatus)
> vmId=`f9cd282e-110a-4896-98d3-6d320662744d`::CPU running: onResume
> 
> Note that the extension is done on the SPM host, so it would be
> interesting to see the vdsm log from the host that was in SPM role at
> this timeframe.
> 
> Regards,
> 
> Fred
> 
> On Wed, Apr 13, 2016 at 2:43 PM, Alex Crow <acrow at integrafin.co.uk>
> wrote:
> 
>> Hi,
>> 
>> If you have set up VM disks as Thin Provisioned, the VM has to
>> pause when the disk image needs to expand. You won't see this on VMs
>> with preallocated storage.
>> 
>> It's not the SAN that's running out of space, it's the VM image
>> needing to be expanded incrementally each time.
>> 
>> Cheers
>> 
>> Alex
>> 
>> On 13/04/16 12:04, nicolas at devels.es wrote:
>> Hi Fred,
>> 
>> This is an iSCSI storage. I'm attaching the VDSM logs from the host
>> where this machine has been running. Should you need any further
>> info, don't hesitate to ask.
>> 
>> Thanks.
>> 
>> Regards.
>> 
>> El 2016-04-13 11:54, Fred Rolland escribió:
>> Hi,
>> 
>> What kind of storage do you have ? (ISCSI,FC,NFS...)
>> Can you provide the vdsm logs from the host where this VM runs ?
>> 
>> Thanks,
>> 
>> Freddy
>> 
>> On Wed, Apr 13, 2016 at 1:02 PM, <nicolas at devels.es> wrote:
>> 
>> Hi,
>> 
>> We're running oVirt 3.6.4.1-1. Lately we're seeing a bunch of
>> events like these:
>> 
>> 2016-04-13 10:52:30,735 INFO 
>> [org.ovirt.engine.core.vdsbroker.VmAnalyzer]
>> (DefaultQuartzScheduler_Worker-86) [60dea18f] VM
>> 'f9cd282e-110a-4896-98d3-6d320662744d'(vm.domain.com [1] [1]) moved
>> from
>> 'Up' --> 'Paused'
>> 2016-04-13 10:52:30,815 INFO 
>> 
>> 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> 
>> (DefaultQuartzScheduler_Worker-86) [60dea18f] Correlation ID: null,
>> 
>> Call Stack: null, Custom Event ID: -1, Message: VM vm.domain.com
>> [1] [1]
>> has been paused.
>> 2016-04-13 10:52:30,898 ERROR
>> 
>> 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> 
>> (DefaultQuartzScheduler_Worker-86) [60dea18f] Correlation ID: null,
>> 
>> Call Stack: null, Custom Event ID: -1, Message: VM vm.domain.com
>> [1] [1]
>> has been paused due to no Storage space error.
>> 2016-04-13 10:52:52,320 WARN 
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData]
>> (org.ovirt.thread.pool-8-thread-38) [] domain
>> '5de4a000-a9c4-489c-8eee-10368647c413:iscsi01' in problem. vds:
>> 'host6.domain.com [2] [2]'
>> 2016-04-13 10:52:55,183 INFO 
>> [org.ovirt.engine.core.vdsbroker.VmAnalyzer]
>> (DefaultQuartzScheduler_Worker-70) [3da0f3d4] VM
>> 'f9cd282e-110a-4896-98d3-6d320662744d'(vm.domain.com [1] [1]) moved
>> from
>> 'Paused' --> 'Up'
>> 2016-04-13 10:52:55,318 INFO 
>> 
>> 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> 
>> (DefaultQuartzScheduler_Worker-70) [3da0f3d4] Correlation ID: null,
>> 
>> Call Stack: null, Custom Event ID: -1, Message: VM vm.domain.com
>> [1] [1]
>> has recovered from paused back to up.
>> 
>> The storage domain is far from being full, though (400+ G available
>> 
>> right now). Could this be related to this other issue [1]? If not,
>> how could I debug what's going on?
>> 
>> Thanks.
>> 
>>  [1]: https://www.mail-archive.com/users@ovirt.org/msg32079.html
>> [3]
>> [3]
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users [4] [4]
>> 
>> Links:
>> ------
>> [1] http://vm.domain.com [1]
>> [2] http://host6.domain.com [2]
>> [3] https://www.mail-archive.com/users@ovirt.org/msg32079.html [3]
>> [4] http://lists.ovirt.org/mailman/listinfo/users [4]
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users [4]
> 
> --
> This message is intended only for the addressee and may contain
> confidential information. Unless you are that person, you may not
> disclose its contents or use it in any way and are requested to delete
> the message along with any attachments and notify us immediately.
> This email is not intended to, nor should it be taken to, constitute
> advice.
> The information provided is correct to our knowledge & belief and must
> not
> be used as a substitute for obtaining tax, regulatory, investment,
> legal or
> any other appropriate advice.
> 
> "Transact" is operated by Integrated Financial Arrangements Ltd.
> 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020)
> 7608 5300.
> (Registered office: as above; Registered in England and Wales under
> number: 3727592). Authorised and regulated by the Financial Conduct
> Authority (entered on the Financial Services Register; no. 190856).
> 
> _______________________________________________
>  Users mailing list
>  Users at ovirt.org
>  http://lists.ovirt.org/mailman/listinfo/users [4]
> 
> 
> 
> Links:
> ------
> [1] http://vm.domain.com
> [2] http://host6.domain.com
> [3] https://www.mail-archive.com/users@ovirt.org/msg32079.html
> [4] http://lists.ovirt.org/mailman/listinfo/users
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vdsm.log.gz
Type: application/x-gzip
Size: 28146 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160414/db4fecdb/attachment-0001.gz>