[Users] Problem with libvirt
Dafna Ron
dron at redhat.com
Thu Jan 31 08:43:49 EST 2013
I'm looking at your log.
you have a problem connecting to the storage.
are you sure that the host is non-responsive and not Non-Operational?
the problem started at 11:49:48
Thread-194::ERROR::2013-01-31
11:49:48,785::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error
while collecting domain `57d184a0-908b-49b5-926f-cd413b9e6526`
monitoring information
Traceback (most recent call last):
File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in
_monitorDomain
domain.selftest()
File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest
fileSD.FileStorageDomain.selftest(self)
File "/usr/share/vdsm/storage/fileSD.py", line 399, in selftest
self.oop.os.statvfs(self.domaindir)
File "/usr/share/vdsm/storage/processPool.py", line 63, in wrapper
return self.runExternally(func, *args, **kwds)
File "/usr/share/vdsm/storage/processPool.py", line 74, in runExternally
return self._procPool.runExternally(*args, **kwargs)
File "/usr/share/vdsm/storage/processPool.py", line 161, in runExternally
raise Timeout("Operation Stuck")
Timeout: Operation Stuck
Thread-194::DEBUG::2013-01-31
11:49:48,812::domainMonitor::130::Storage.DomainMonitor::(_monitorDomain) Domain
`57d184a0-908b-49b5-926f-cd413b9e6526` changed its status to Invalid
MainThread::INFO::2013-01-31 11:49:53,093::vmChannels::135::vds::(stop)
VM channels listener was stopped.
the network errors that you see in the engine log is the vdsm restarting
(this is part of the flow when we cannot see the domain).
MainThread::INFO::2013-01-31 11:51:49,885::vdsm::70::vds::(run) I am the
actual vdsm 4.10-10
please check the connectivity to the storage.
Dafna
On 01/31/2013 03:29 PM, Juan Jose wrote:
> Hello again,
>
> I attach vdsm.log and I have had to reboot the host node machine and
> the libvirtd.log doen't have information about this issue. I will try
> to reproduce the error again and I will attach the libvirtd.log. It is
> possible that you are right and it is possible a network problem, I
> will investigate the problem more.
>
> Many thanks in avanced,
>
> Juanjo.
>
> On Thu, Jan 31, 2013 at 1:26 PM, Dafna Ron <dron at redhat.com
> <mailto:dron at redhat.com>> wrote:
>
> non responsive state for host means that you either have a network
> issue
> (iptables perhaps) or vdsmd is down.
> please attach vdsm and libvirtd logs.
>
>
>
> On 01/31/2013 01:21 PM, Juan Jose wrote:
> > Hello everybody,
> >
> > I have installed and configured oVirt 3.1 engine in a Fedora 17
> with a
> > Fedora 17 node connected. Ihave defined a NFS domain for my VM and
> > another for ISOs. I try to start a Fedora 17 Server with Run
> once and
> > the machi start without problems, after that I preceed with the
> > installation in its wirtual disk but when I arrive to define
> > partitions in the virtual disk the machine is freeze and I start to
> > receive engine errors and the default data center go in non
> responsive
> > status.
> >
> > I can see this messages in /var/log/ovirt-engine/engine.log, which I
> > attach to this message:
> >
> > ....
> > 2013-01-31 11:43:23,957 INFO
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] Recieved a Device without an
> > address when processing VM da09284e-3189-428b-a879-6201f7a5ca87
> > devices, skipping device: {shared=false,
> > volumeID=1d0e9fdf-c4bc-4894-8ff1-7a5e185d57a4, index=0,
> > propagateErrors=off, format=raw, type=disk, truesize=8589938688,
> > reqsize=0, bootOrder=2, iface=virtio,
> > volumeChain=[Ljava.lang.Object;@1ea2bdf9,
> > imageID=49e21bfc-384b-4bea-8013-f02b1be137c7,
> > domainID=57d184a0-908b-49b5-926f-cd413b9e6526, specParams={},
> > optional=false, needExtend=false,
> >
> path=/rhev/data-center/d6e7e8b8-49c7-11e2-a261-000a5e429f63/57d184a0-908b-49b5-926f-cd413b9e6526/images/49e21bfc-384b-4bea-8013-f02b1be137c7/1d0e9fdf-c4bc-4894-8ff1-7a5e185d57a4,
> > device=disk, poolID=d6e7e8b8-49c7-11e2-a261-000a5e429f63,
> > readonly=false, deviceId=49e21bfc-384b-4bea-8013-f02b1be137c7,
> > apparentsize=8589934592}.
> > 2013-01-31 11:43:23,960 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=4dca1c64-dbf8-4e31-b359-82cf0e259f65,Device=qxl,Type=video,BootOrder=0,SpecParams={vram=65536},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=false,alias=
> > 2013-01-31 11:43:23,961 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=aba73f2f-e951-4eba-9da4-8fb58315df2c,Device=memballoon,Type=balloon,BootOrder=0,SpecParams={model=virtio},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=true,alias=
> > 2013-01-31 11:43:23,962 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=9bfb770c-13fa-4bf6-9f1f-414927bc31b0,Device=cdrom,Type=disk,BootOrder=0,SpecParams={path=},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=true,alias=
> > 2013-01-31 11:43:23,963 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=614bc0b4-64d8-4058-8bf8-83db62617e00,Device=bridge,Type=interface,BootOrder=0,SpecParams={},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=false,alias=
> > 2013-01-31 11:43:23,964 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-47) [75664f2b] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=49e21bfc-384b-4bea-8013-f02b1be137c7,Device=disk,Type=disk,BootOrder=0,SpecParams={},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=false,alias=
> > 2013-01-31 11:43:26,063 INFO
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-24) [7d021319] VM Fedora17
> > da09284e-3189-428b-a879-6201f7a5ca87 moved from WaitForLaunch -->
> > PoweringUp
> > 2013-01-31 11:43:26,064 INFO
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand]
> > (QuartzScheduler_Worker-24) [7d021319] START,
> FullListVdsCommand(vdsId
> > = 7d3491e8-49ce-11e2-8b2e-000a5e429f63, vds=null,
> > vmIds=[da09284e-3189-428b-a879-6201f7a5ca87]), log id: f68f564
> > 2013-01-31 11:43:26,086 INFO
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand]
> > (QuartzScheduler_Worker-24) [7d021319] FINISH, FullListVdsCommand,
> > return:
> > [Lorg.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcStruct;@33c68023, log
> > id: f68f564
> > 2013-01-31 11:43:26,091 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-24) [7d021319] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=aba73f2f-e951-4eba-9da4-8fb58315df2c,Device=memballoon,Type=balloon,BootOrder=0,SpecParams={model=virtio},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=true,alias=
> > 2013-01-31 11:43:26,092 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-24) [7d021319] VM
> > da09284e-3189-428b-a879-6201f7a5ca87 managed non plugable device was
> > removed unexpetedly from libvirt:
> >
> VmId=da09284e-3189-428b-a879-6201f7a5ca87,DeviceId=9bfb770c-13fa-4bf6-9f1f-414927bc31b0,Device=cdrom,Type=disk,BootOrder=0,SpecParams={path=},Address=,IsManaged=true,IsPlugged=true,IsReadOnly=true,alias=
> > 2013-01-31 11:43:31,721 INFO
> > [org.ovirt.engine.core.bll.SetVmTicketCommand]
> (ajp--0.0.0.0-8009-11)
> > [28d7a789] Running command: SetVmTicketCommand internal: false.
> > Entities affected : ID: da09284e-3189-428b-a879-6201f7a5ca87
> Type: VM
> > 2013-01-31 11:43:31,724 INFO
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand]
> > (ajp--0.0.0.0-8009-11) [28d7a789] START,
> SetVmTicketVDSCommand(vdsId =
> > 7d3491e8-49ce-11e2-8b2e-000a5e429f63,
> > vmId=da09284e-3189-428b-a879-6201f7a5ca87, ticket=qmcnuOICblb3,
> > validTime=120,m userName=admin at internal,
> > userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 6eaacb95
> > 2013-01-31 11:43:31,758 INFO
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand]
> > (ajp--0.0.0.0-8009-11) [28d7a789] FINISH, SetVmTicketVDSCommand, log
> > id: 6eaacb95
> >
> > ...
> >
> > 2013-01-31 11:49:13,392 WARN
> > [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> > (QuartzScheduler_Worker-81) [164eaa47] domain
> > 57d184a0-908b-49b5-926f-cd413b9e6526 in problem. vds: host1
> > 2013-01-31 11:49:54,121 ERROR
> > [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> > (QuartzScheduler_Worker-73) [73213e4f] vds::refreshVdsStats Failed
> > getVdsStats, vds = 7d3491e8-49ce-11e2-8b2e-000a5e429f63 : host1,
> > error = VDSNetworkException: VDSNetworkException:
> > 2013-01-31 11:49:54,172 WARN
> > [org.ovirt.engine.core.vdsbroker.VdsManager]
> > (QuartzScheduler_Worker-73) [73213e4f]
> > ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS ,
> vds =
> > 7d3491e8-49ce-11e2-8b2e-000a5e429f63 : host1, VDS Network Error,
> > continuing.
> > VDSNetworkException:
> >
> > ....
> >
> >
> > In the events windows after VM freezing, I have below events:
> >
> > 2013-Jan-31, 11:50:52 Failed to elect Host as Storage Pool Manager
> > for Data Center Default. Setting status to Non-Operational.
> > 2013-Jan-31, 11:50:52 VM Fedora17 was set to the Unknown status.
> > 2013-Jan-31, 11:50:52 Host host1 is non-responsive.
> > 2013-Jan-31, 11:49:55 Invalid status on Data Center Default.
> Setting
> > Data Center status to Non-Responsive (On host host1, Error: Network
> > error during communication with the Host.).
> > 2013-Jan-31, 11:44:25 VM Fedora17 started on Host host1
> >
> > Any suggest about the problem?. It seem a libvirt problem, I will
> > continue investigating.
> >
> > Many thanks in avanced,
> >
> > Juanjo.
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at ovirt.org <mailto:Users at ovirt.org>
> > http://lists.ovirt.org/mailman/listinfo/users
>
>
> --
> Dafna Ron
>
>
--
Dafna Ron
More information about the Users
mailing list