I am running vdsm from packages as my interest is in developing for the engine and not vdsm.

I updated the vdsm package in an attempt to solve this, now I have:

# rpm -q vdsm

vdsm-4.10.3-10.fc18.x86_64

I noticed that when the storage domain crashes I can't even do "df -h" (hangs)

I'm also getting some errors in /var/log/messages:

Mar 24 19:57:44 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:45 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:46 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:47 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:48 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:49 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:50 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:51 bufferoverflow sanlock[1208]: 2013-03-24 19:57:51+0200 7412 [4759]: 1083422e close_task_aio 0 0x7ff3740008c0 busy

Mar 24 19:57:51 bufferoverflow sanlock[1208]: 2013-03-24 19:57:51+0200 7412 [4759]: 1083422e close_task_aio 1 0x7ff374000910 busy

Mar 24 19:57:51 bufferoverflow sanlock[1208]: 2013-03-24 19:57:51+0200 7412 [4759]: 1083422e close_task_aio 2 0x7ff374000960 busy

Mar 24 19:57:51 bufferoverflow sanlock[1208]: 2013-03-24 19:57:51+0200 7412 [4759]: 1083422e close_task_aio 3 0x7ff3740009b0 busy

Mar 24 19:57:51 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:52 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:53 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:54 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:55 bufferoverflow vdsm SuperVdsmProxy WARNING Connect to svdsm failed [Errno 2] No such file or directory

Mar 24 19:57:55 bufferoverflow vdsm Storage.Misc ERROR Panic: Couldn't connect to supervdsm

Mar 24 19:57:55 bufferoverflow respawn: slave '/usr/share/vdsm/vdsm' died, respawning slave

Mar 24 19:57:55 bufferoverflow vdsm fileUtils WARNING Dir /rhev/data-center/mnt already exists

Mar 24 19:57:58 bufferoverflow vdsm vds WARNING Unable to load the json rpc server module. Please make sure it is installed.

Mar 24 19:57:58 bufferoverflow vdsm vm.Vm WARNING vmId=`4d3d81b3-d083-4569-acc2-8e631ed51843`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found

Mar 24 19:57:58 bufferoverflow vdsm vm.Vm WARNING vmId=`4d3d81b3-d083-4569-acc2-8e631ed51843`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found

Mar 24 19:57:58 bufferoverflow vdsm vm.Vm WARNING vmId=`4d3d81b3-d083-4569-acc2-8e631ed51843`::_readPauseCode unsupported by libvirt vm

Mar 24 19:57:58 bufferoverflow kernel: [ 7402.688177] ata1: hard resetting link

Mar 24 19:57:59 bufferoverflow kernel: [ 7402.994510] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.005510] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20120711/psargs-359)

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.005517] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT0._GTF] (Node ffff880407c74d48), AE_NOT_FOUND (20120711/psparse-536)

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.015485] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20120711/psargs-359)

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.015493] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT0._GTF] (Node ffff880407c74d48), AE_NOT_FOUND (20120711/psparse-536)

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.016061] ata1.00: configured for UDMA/133

Mar 24 19:57:59 bufferoverflow kernel: [ 7403.016066] ata1: EH complete

Mar 24 19:58:01 bufferoverflow sanlock[1208]: 2013-03-24 19:58:01+0200 7422 [4759]: 1083422e close_task_aio 0 0x7ff3740008c0 busy

Mar 24 19:58:01 bufferoverflow sanlock[1208]: 2013-03-24 19:58:01+0200 7422 [4759]: 1083422e close_task_aio 1 0x7ff374000910 busy

Mar 24 19:58:01 bufferoverflow sanlock[1208]: 2013-03-24 19:58:01+0200 7422 [4759]: 1083422e close_task_aio 2 0x7ff374000960 busy

Mar 24 19:58:01 bufferoverflow sanlock[1208]: 2013-03-24 19:58:01+0200 7422 [4759]: 1083422e close_task_aio 3 0x7ff3740009b0 busy

Mar 24 19:58:01 bufferoverflow kernel: [ 7405.714145] device-mapper: table: 253:0: multipath: error getting device

Mar 24 19:58:01 bufferoverflow kernel: [ 7405.714148] device-mapper: ioctl: error adding target to table

Mar 24 19:58:01 bufferoverflow kernel: [ 7405.715051] device-mapper: table: 253:0: multipath: error getting device

Mar 24 19:58:01 bufferoverflow kernel: [ 7405.715053] device-mapper: ioctl: error adding target to table

ata1 is a 500GB SSD. (only SATA device on the system except a DVD drive)

Yuval

On Sun, Mar 24, 2013 at 2:52 PM, Dan Kenigsberg <danken@redhat.com> wrote:

On Fri, Mar 22, 2013 at 08:24:35PM +0200, Limor Gavish wrote:
> Hello,
>
> I am using Ovirt 3.2 on Fedora 18:
> [wil@bufferoverflow ~]$ rpm -q vdsm
> vdsm-4.10.3-7.fc18.x86_64
>
> (the engine is built from sources).
>
> I seem to have hit this bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=922515

This bug is only one part of the problem, but it's nasty enough that I
have just suggested it as a fix to the ovirt-3.2 branch of vdsm:
http://gerrit.ovirt.org/13303

Could you test if with it, vdsm relinquishes its spm role, and recovers
as operational?

>
> in the following configuration:
> Single host (no migrations)
> Created a VM, installed an OS inside (Fedora18)
> stopped the VM.
> created template from it.
> Created an additional VM from the template using thin provision.
> Started the second VM.
>
> in addition to the errors in the logs the storage domains (both data and
> ISO) crashed, i.e went to "unknown" and "inactive" states respectively.
> (see the attached engine.log)
>
> I attached the VDSM and engine logs.
>
> is there a way to work around this problem?
> It happens repeatedly.
>
> Yuval Meir