[ovirt-users] Error creating Disks
Maurice James
mjames at media-node.com
Wed Apr 16 14:40:27 UTC 2014
Im running CentOS 6.5 2.6.32-431.11.2.el6.x86_64
VDSM:
vdsm-python-4.14.6-15.git746e2e9.el6.x86_64
vdsm-hook-isolatedprivatevlan-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-vmfex-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-sriov-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-checkimages-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-smbios-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-hostusb-4.14.6-15.git746e2e9.el6.noarch
vdsm-4.14.6-15.git746e2e9.el6.x86_64
vdsm-hook-faqemu-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-promisc-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-scratchpad-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-qos-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-numa-4.14.6-15.git746e2e9.el6.noarch
vdsm-python-zombiereaper-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-qemucmdline-4.14.6-15.git746e2e9.el6.noarch
vdsm-cli-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-fileinject-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-directlun-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-vmdisk-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-macspoof-4.14.6-15.git746e2e9.el6.noarch
vdsm-gluster-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-floppy-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-hugepages-4.14.6-15.git746e2e9.el6.noarch
vdsm-xmlrpc-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-pincpu-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-openstacknet-4.14.6-15.git746e2e9.el6.noarch
Libvirt:
libvirt-lock-sanlock-0.10.2-29.el6_5.7.x86_64
libvirt-client-0.10.2-29.el6_5.7.x86_64
libvirt-0.10.2-29.el6_5.7.x86_64
libvirt-python-0.10.2-29.el6_5.7.x86_64
Ovirt-engine:
ovirt-engine-dwh-setup-3.4.1-0.0.master.20140406181125.git4081b13.el6.noarch
ovirt-engine-dwh-3.4.1-0.0.master.20140406181125.git4081b13.el6.noarch
ovirt-engine-websocket-proxy-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-setup-plugin-ovirt-engine-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-dbscripts-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-sdk-python-3.4.0.7-1.20140228.git19e14c5.el6.noarch
ovirt-engine-reports-3.4.1-0.0.master.20140410124141.gitbf81400.el6.noarch
ovirt-engine-setup-base-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-setup-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-backend-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-tools-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-reports-setup-3.4.1-0.0.master.20140410124141.gitbf81400.el6.noarch
ovirt-engine-setup-plugin-websocket-proxy-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-setup-plugin-allinone-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-webadmin-portal-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-cli-3.4.0.6-1.20140227.gite87e2bc.el6.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-userportal-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-lib-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
ovirt-engine-restapi-3.4.1-0.0.master.20140413010852.git43746c6.el6.noarch
Qemu:
qemu-kvm-tools-0.12.1.2-2.415.el6_5.7.x86_64
qemu-kvm-0.12.1.2-2.415.el6_5.7.x86_64
vdsm-hook-faqemu-4.14.6-15.git746e2e9.el6.noarch
vdsm-hook-qemucmdline-4.14.6-15.git746e2e9.el6.noarch
qemu-img-0.12.1.2-2.415.el6_5.7.x86_64
qemu-guest-agent-0.12.1.2-2.415.el6_5.7.x86_64
gpxe-roms-qemu-0.9.7-6.10.el6.noarch
----- Original Message -----
From: "Liron Aravot" <laravot at redhat.com>
To: dron at redhat.com
Cc: "Maurice James" <mjames at media-node.com>, users at ovirt.org
Sent: Wednesday, April 16, 2014 10:23:35 AM
Subject: Re: [ovirt-users] Error creating Disks
Hi Maurice, Dafna
The creation of the snapshot for each of the disks succeeds, but performing the "live" part of it fails - we can see the following error in the vdsm log.
Thread-4444::DEBUG::2014-04-16 09:40:39,071::vm::4007::vm.Vm::(snapshot) vmId=`50cf8bce-3982-491a-8b67-7d009c5c3243`::Snapshot faile
d using the quiesce flag, trying again without it (unsupported configuration: reuse is not supported with this QEMU binary)
Thread-4444::DEBUG::2014-04-16 09:40:39,081::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 67 edom: 10 level:
2 message: unsupported configuration: reuse is not supported with this QEMU binary
Thread-4444::ERROR::2014-04-16 09:40:39,082::vm::4011::vm.Vm::(snapshot) vmId=`50cf8bce-3982-491a-8b67-7d009c5c3243`::Unable to take
snapshot
Traceback (most recent call last):
File "/usr/share/vdsm/vm.py", line 4009, in snapshot
self._dom.snapshotCreateXML(snapxml, snapFlags)
File "/usr/share/vdsm/vm.py", line 859, in f
ret = attr(*args, **kwargs)
File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 92, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1636, in snapshotCreateXML
if ret is None:raise libvirtError('virDomainSnapshotCreateXML() failed', dom=self)
libvirtError: unsupported configuration: reuse is not supported with this QEMU binary
Thread-4444::DEBUG::2014-04-16 09:40:39,091::BindingXMLRPC::1074::vds::(wrapper) return vmSnapshot with {'status': {'message': 'Snap
shot failed', 'code': 48}}
there's open bug on possibly the same issue - but let's verify that it's the same
https://bugzilla.redhat.com/show_bug.cgi?id=1009100
what OS are you running? if it's not centos, please try to upgrade libvirt and try again.
if it's urgent - as i see that you already stopped your vm during those tries, as a temporary solution you can stop the vm, move the disk while it's stopped (which won't be live storage migration) and than start it.
----- Original Message -----
> From: "Dafna Ron" <dron at redhat.com>
> To: "Maurice James" <mjames at media-node.com>
> Cc: users at ovirt.org
> Sent: Wednesday, April 16, 2014 4:46:10 PM
> Subject: Re: [ovirt-users] Error creating Disks
>
> can you try to restart the engine?
> This should clean the cache and some of the tables holding temporary
> task info.
>
> Dafna
>
>
>
> On 04/16/2014 02:37 PM, Maurice James wrote:
> > I ran vdsClient -s 0 getAllTasksInfo and nothing was returned. What should
> > my next step be?
> >
> > ----- Original Message -----
> > From: "Dafna Ron" <dron at redhat.com>
> > To: "Maurice James" <mjames at media-node.com>
> > Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> > Sent: Wednesday, April 16, 2014 4:44:48 AM
> > Subject: Re: [ovirt-users] Error creating Disks
> >
> > ok... Now I am starting to understand what's going on and if you look at
> > the vdsm log, the live snapshot succeeds and no other ERROR are
> > reported. I think it's related to the task management on engine side.
> >
> > Can I ask you to run in the spm: vdsClient -s 0 getAllTasksInfo
> > If you have tasks we would have to stop and clear them (vdsClient -s 0
> > stopTask <task> ; vdsClient -s 0 clearTask <task>)
> > after you clear the tasks you will have to restart the engine
> >
> > Also, what version of ovirt are you using? is it 3.4? because this was
> > suppose to be fixed...
> >
> > here is the explanation from what I see in the logs:
> >
> > you are sending a command to Live Migrate
> >
> > 2014-04-15 12:50:54,381 INFO
> > [org.ovirt.engine.core.bll.MoveDisksCommand] (ajp--127.0.0.1-8702-4)
> > [4c089392] Running command: MoveDisksCommand internal: false. Entities
> > affected : ID: c24706d3-1872-4cd3-94a2-9c61ef032e29 Type: Disk
> > 2014-04-15 12:50:54,520 INFO
> > [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand]
> > (ajp--127.0.0.1-8702-4) [4c089392] Lock Acquired to object EngineLock
> > [exclusiveLocks= key: c24706d3-1872-4cd3-94a2-9c61ef032e29 value: DISK
> > , sharedLocks= key: ba49605b-fb7e-4a70-a380-6286d3903e50 value: VM
> > ]
> >
> > The fist step is creating the snapshot:
> >
> > 2014-04-15 12:50:54,734 INFO
> > [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
> > (org.ovirt.thread.pool-6-thread-47) Running command:
> > CreateAllSnapshotsFromVmCommand internal: true. Entities affected : ID:
> > ba49605b-fb7e-4a70-a3
> > 80-6286d3903e50 Type: VM
> > 2014-04-15 12:50:54,748 INFO
> > [org.ovirt.engine.core.bll.CreateSnapshotCommand]
> > (org.ovirt.thread.pool-6-thread-47) [23c23b4] Running command:
> > CreateSnapshotCommand internal: true. Entities affected : ID:
> > 00000000-0000-0000-0000-0000000
> > 00000 Type: Storage
> > 2014-04-15 12:50:54,760 INFO
> > [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand]
> > (org.ovirt.thread.pool-6-thread-47) [23c23b4] START,
> > CreateSnapshotVDSCommand( storagePoolId =
> > a106ab81-9d5f-49c1-aeaf-832a137b708c, ignor
> > eFailoverLimit = false, storageDomainId =
> > 3406665e-4adc-4fd4-aa1e-037547b29adb, imageGroupId =
> > c24706d3-1872-4cd3-94a2-9c61ef032e29, imageSizeInBytes = 107374182400,
> > volumeFormat = COW, newImageId = 015f7d9d-ff75-4a3c-a634-a00e82e04803,
> > newImageDescription = , imageId = e8442348-e28e-4e78-abe8-4f2848b47661,
> > sourceImageGroupId = c24706d3-1872-4cd3-94a2-9c61ef032e29), log id: f3b7d8b
> >
> >
> > which actually succeeds in vdsm and reported as successful to the engine:
> >
> > engine:
> >
> > 2014-04-15 12:51:07,291 INFO [org.ovirt.engine.core.bll.SPMAsyncTask]
> > (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task
> > db7f0f4d-f47d-472a-92bd-197b428fe417 (Parent Command LiveMigrateVmDisks,
> > Parameters Type
> > org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned
> > status finished, result 'success'.
> >
> > vdsm:
> >
> > Thread-4670::DEBUG::2014-04-15
> > 12:51:07,287::task::1185::TaskManager.Task::(prepare)
> > Task=`ea9d5846-656c-4a44-bca2-d5c091004078`::finished:
> > {'allTasksStatus': {'db7f0f4d-f47d-472a-92bd-197b428fe417': {'code': 0,
> > 'message': '1 jobs comple
> > ted successfully', 'taskState': 'finished', 'taskResult': 'success',
> > 'taskID': 'db7f0f4d-f47d-472a-92bd-197b428fe417'}}}
> >
> >
> > now engine has to end the task:
> >
> > 2014-04-15 12:51:07,301 INFO
> > [org.ovirt.engine.core.bll.CommandAsyncTask]
> > (DefaultQuartzScheduler_Worker-42) CommandAsyncTask::endAction: Ending
> > action for 1 tasks (command ID: 7c67ca08-9969-4a6e-87ef-0b7379d7edb6):
> > calling endAction .
> > 2014-04-15 12:51:07,303 INFO
> > [org.ovirt.engine.core.bll.CommandAsyncTask]
> > (org.ovirt.thread.pool-6-thread-42) CommandAsyncTask::EndCommandAction
> > [within thread] context: Attempting to endAction LiveMigrateVmDisks,
> > executionIndex: 0
> > 2014-04-15 12:51:07,315 INFO
> > [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand]
> > (org.ovirt.thread.pool-6-thread-42) Ending command successfully:
> > org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand
> > 2014-04-15 12:51:07,319 INFO
> > [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
> > (org.ovirt.thread.pool-6-thread-42) Ending command successfully:
> > org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand
> >
> > and than
> >
> > we see an exception:
> >
> > 2014-04-15 12:51:07,498 WARN
> > [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
> > (org.ovirt.thread.pool-6-thread-42) Wasnt able to live snapshot due to
> > error: VdcBLLException: VdcBLLException:
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> > VDSGenericException: VDSErrorException: Failed to SnapshotVDS, error =
> > Snapshot failed, code = 48 (Failed with error SNAPSHOT_FAILED and code
> > 48). VM will still be configured to the new created snapshot
> > 2014-04-15 12:51:07,543 INFO
> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> > (org.ovirt.thread.pool-6-thread-42) Correlation ID: null, Call Stack:
> > org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException:
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> > VDSGenericException: VDSErrorException: Failed to SnapshotVDS, error =
> > Snapshot failed, code = 48 (Failed with error SNAPSHOT_FAILED and code 48)
> > at
> > org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116)
> > at
> > org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33)
> > at
> > org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:1971)
> > at
> > org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand$2.runInTransaction(CreateAllSnapshotsFromVmCommand.java:354)
> > at
> > org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand$2.runInTransaction(CreateAllSnapshotsFromVmCommand.java:351)
> > at
> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174)
> > at
> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116)
> > at
> > org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.performLiveSnapshot(CreateAllSnapshotsFromVmCommand.java:351)
> > at
> > org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.endVmCommand(CreateAllSnapshotsFromVmCommand.java:273)
> > at
> > org.ovirt.engine.core.bll.VmCommand.endSuccessfully(VmCommand.java:304)
> > at
> > org.ovirt.engine.core.bll.CommandBase.internalEndSuccessfully(CommandBase.java:614)
> > at
> > org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:560)
> > at
> > org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1886)
> > at
> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:151)
> > at
> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:118)
> > at
> > org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:492)
> > at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:446)
> > at sun.reflect.GeneratedMethodAccessor513.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On 04/15/2014 05:54 PM, Maurice James wrote:
> >> Logs are attached.
> >>
> >> Live Migration failed
> >>
> >> ----- Original Message -----
> >> From: "Dafna Ron" <dron at redhat.com>
> >> To: "Maurice James" <mjames at media-node.com>
> >> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >> Sent: Tuesday, April 15, 2014 10:46:29 AM
> >> Subject: Re: [ovirt-users] Error creating Disks
> >>
> >> yeah, but is it the same error we see before in vdsm? :)
> >> lets try to follow the action in both logs so we can debug it better,
> >> can you do the following?
> >>
> >> 1. stop the vm and clean all snapshots.
> >> 2. add >>>> to the engine and vdsm logs using echo (#echo '>>>>>>>>' >>
> >> /var/log/ovirt-engine/engine.log and #echo '>>>>>>>>' >>
> >> /var/log/vdsm/vdsm.log)
> >> 3. start the vm
> >> 4. try to live migrate the disk
> >> 5. attach the full engine and vdsm logs.
> >>
> >> Thanks,
> >>
> >> Dafna
> >>
> >>
> >> On 04/15/2014 03:41 PM, Maurice James wrote:
> >>> It failed again. Its reported in the same location.
> >>>
> >>> Snip of engine.log
> >>>
> >>> 2014-04-15 10:41:17,174 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-34) Failed in SnapshotVDS method
> >>> 2014-04-15 10:41:17,178 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-34) Command SnapshotVDSCommand(HostName
> >>> = vhost3, HostId = bc9c25e6-714e-4eac-8af0-860ac76fd195,
> >>> vmId=ba49605b-fb7e-4a70-a380-6286d3903e50) execution failed. Exception:
> >>> VDSErrorException: VDSGenericException: VDSErrorException: Failed to
> >>> SnapshotVDS, error = Snapshot failed, code = 48
> >>> 2014-04-15 10:41:17,531 ERROR
> >>> [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand]
> >>> (org.ovirt.thread.pool-6-thread-34) Command
> >>> org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand throw Vdc Bll
> >>> exception. With error message VdcBLLException: Auto-generated live
> >>> snapshot for VM ba49605b-fb7e-4a70-a380-6286d3903e50 failed (Failed with
> >>> error imageErr and code 13)
> >>> 2014-04-15 10:41:17,534 ERROR
> >>> [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand]
> >>> (org.ovirt.thread.pool-6-thread-34) Reverting task unknown, handler:
> >>> org.ovirt.engine.core.bll.lsm.LiveSnapshotTaskHandler
> >>>
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>> From: "Dafna Ron" <dron at redhat.com>
> >>> To: "Maurice James" <mjames at media-node.com>
> >>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >>> Sent: Tuesday, April 15, 2014 10:37:41 AM
> >>> Subject: Re: [ovirt-users] Error creating Disks
> >>>
> >>> ok.
> >>> so the vm was down and you deleted all the snapshots.
> >>> in the webadmin, when you select the disk you want to live migrate, is
> >>> it the same storage reported in vdsm?
> >>>
> >>> Thanks,
> >>>
> >>> Dafna
> >>>
> >>>
> >>> On 04/15/2014 03:25 PM, Maurice James wrote:
> >>>> I restarted from the VM. The VM now has 0 snapshots. All of the
> >>>> snapshots in the screen shots never completed, so I deleted them while
> >>>> the VM was powerd off from the ui
> >>>>
> >>>> ----- Original Message -----
> >>>> From: "Dafna Ron" <dron at redhat.com>
> >>>> To: "Maurice James" <mjames at media-node.com>
> >>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >>>> Sent: Tuesday, April 15, 2014 10:21:06 AM
> >>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>
> >>>> did you restart the vm from the vm internally or stop -> start the vm?
> >>>>
> >>>> how many snapshots does the vm have?
> >>>>
> >>>> Thanks,
> >>>> Dafna
> >>>>
> >>>>
> >>>> On 04/15/2014 03:16 PM, Maurice James wrote:
> >>>>> I restarted the VM. Same issue. I was able to find the physical
> >>>>> location on disk. I attached 2 screen shots showing the location of
> >>>>> the disks on the file system.
> >>>>>
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>> From: "Dafna Ron" <dron at redhat.com>
> >>>>> To: "Maurice James" <mjames at media-node.com>
> >>>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >>>>> Sent: Tuesday, April 15, 2014 10:12:56 AM
> >>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>
> >>>>> it seems that the issue is the links between the engine and the vdsm
> >>>>>
> >>>>> why don't you check where the disk is physically located and than try
> >>>>> to
> >>>>> restart the vm?
> >>>>>
> >>>>> Thanks,
> >>>>> Dafna
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 04/15/2014 02:57 PM, Maurice James wrote:
> >>>>>> yes I did see a warning about restarting. It said:
> >>>>>> Failed to create live snapshot '20140415 for VM 'TIEATS_Racktables'.
> >>>>>> VM restart is recommended
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>> From: "Dafna Ron" <dron at redhat.com>
> >>>>>> To: "Maurice James" <mjames at media-node.com>
> >>>>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >>>>>> Sent: Tuesday, April 15, 2014 9:49:22 AM
> >>>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>>
> >>>>>> Hi Maurice,
> >>>>>>
> >>>>>> are you getting any errors/alert about restarting the vm in the event
> >>>>>> log (in the webadmin).
> >>>>>>
> >>>>>> I believe that you have a mismatch in the directory link between the
> >>>>>> db
> >>>>>> and vdsm which was created when you tried to live migrate and failed
> >>>>>> the
> >>>>>> fist time).
> >>>>>>
> >>>>>> we can see in the vdsm log that createVolume (which is part of the
> >>>>>> live
> >>>>>> migration) is failing with this error:
> >>>>>>
> >>>>>> OSError: [Errno 2] No such file or directory:
> >>>>>> '/rhev/data-center/a106ab81-9d5f-49c1-aeaf-832a137b708c/b7663d70-e658-41fa-b9f0-8da83c9eddce/images/348effd9-c5db-44fc-a9e5-67391647096b'
> >>>>>>
> >>>>>> so it seems that the engine is pointing to a link that does not exist
> >>>>>> and we need to find on what storage the disk actually exists on and
> >>>>>> where the engine is pointing to.
> >>>>>>
> >>>>>> you can look at the webadmin at the disk and see where it's suppose to
> >>>>>> be and than look in /rhev and see where the disk link is in vdsm (if
> >>>>>> you
> >>>>>> can see where the disk is actually is - physically in the storage,
> >>>>>> that
> >>>>>> would be good too).
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Dafna
> >>>>>>
> >>>>>>
> >>>>>> On 04/15/2014 01:16 PM, Maurice James wrote:
> >>>>>>> As far as the disk creation , I got that sorted out. It seem that my
> >>>>>>> vdsm versions were out of sync on my hosts. When I updated them the
> >>>>>>> disk creation began working, but im still having issues with live
> >>>>>>> disk migrations.
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>> From: "Dafna Ron" <dron at redhat.com>
> >>>>>>> To: "Maurice James" <mjames at media-node.com>
> >>>>>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, users at ovirt.org
> >>>>>>> Sent: Tuesday, April 15, 2014 5:31:48 AM
> >>>>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>>>
> >>>>>>> Also, before the task fails, there are ERRORs in the log which are
> >>>>>>> reporting problems in the DC
> >>>>>>>
> >>>>>>> 2014-04-14 09:56:05,161 ERROR
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-16) Command
> >>>>>>> GetCapabilitiesVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]) execution
> >>>>>>> failed. Exception: VDSNetworkException:
> >>>>>>> java.net.SocketTimeoutException:
> >>>>>>> connect timed out
> >>>>>>> 2014-04-14 09:56:10,226 ERROR
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-49) Command
> >>>>>>> GetCapabilitiesVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]) execution
> >>>>>>> failed. Exception: VDSNetworkException:
> >>>>>>> java.net.SocketTimeoutException:
> >>>>>>> connect timed out
> >>>>>>> 2014-04-14 09:56:15,283 ERROR
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-81) Command
> >>>>>>> GetCapabilitiesVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]) execution
> >>>>>>> failed. Exception: VDSNetworkException:
> >>>>>>> java.net.SocketTimeoutException:
> >>>>>>> connect timed out
> >>>>>>> 2014-04-14 09:56:20,342 ERROR
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-58) Command
> >>>>>>> GetCapabilitiesVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]) execution
> >>>>>>> failed. Exception: VDSNetworkException:
> >>>>>>> java.net.SocketTimeoutException:
> >>>>>>> connect timed out
> >>>>>>> 2014-04-14 09:56:25,409 ERROR
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-84) Command
> >>>>>>> GetCapabilitiesVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]) execution
> >>>>>>> failed. Exception: VDSNetworkException:
> >>>>>>> java.net.SocketTimeoutException:
> >>>>>>> connect timed out
> >>>>>>> 2014-04-14 09:56:30,542 INFO
> >>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> >>>>>>> (DefaultQuartzScheduler_Worker-91) START,
> >>>>>>> GetHardwareInfoVDSCommand(HostName = vhost3, HostId =
> >>>>>>> bc9c25e6-714e-4eac-8af0-860ac76fd195, vds=Host[vhost3]), log id:
> >>>>>>> 30f312c
> >>>>>>>
> >>>>>>>
> >>>>>>> than you stop the vm and fail the AddDisk.
> >>>>>>>
> >>>>>>>
> >>>>>>> is there a chance you were having problems with your storage?
> >>>>>>>
> >>>>>>> Dafna
> >>>>>>>
> >>>>>>>
> >>>>>>> On 04/14/2014 04:04 PM, Maurice James wrote:
> >>>>>>>> Snapshot creation seems to be failing as well
> >>>>>>>>
> >>>>>>>> ----- Original Message -----
> >>>>>>>> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
> >>>>>>>> To: "Maurice James" <mjames at media-node.com>, "Federico Simoncelli"
> >>>>>>>> <fsimonce at redhat.com>
> >>>>>>>> Cc: users at ovirt.org
> >>>>>>>> Sent: Monday, April 14, 2014 11:00:55 AM
> >>>>>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>>>>
> >>>>>>>> Hi Federico,
> >>>>>>>> Can you please take a look?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ----- Original Message -----
> >>>>>>>>> From: "Maurice James" <mjames at media-node.com>
> >>>>>>>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
> >>>>>>>>> Cc: users at ovirt.org
> >>>>>>>>> Sent: Monday, April 14, 2014 5:44:44 PM
> >>>>>>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>>>>>
> >>>>>>>>> Logs attached
> >>>>>>>>>
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
> >>>>>>>>> To: "Maurice James" <mjames at media-node.com>
> >>>>>>>>> Cc: users at ovirt.org
> >>>>>>>>> Sent: Monday, April 14, 2014 10:33:03 AM
> >>>>>>>>> Subject: Re: [ovirt-users] Error creating Disks
> >>>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>> IMHO not enough info is provided,
> >>>>>>>>> Can you please provide full engine.log and relevant vdsm.log?
> >>>>>>>>>
> >>>>>>>>> THanks,
> >>>>>>>>> Yair
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>>> From: "Maurice James" <mjames at media-node.com>
> >>>>>>>>>> To: users at ovirt.org
> >>>>>>>>>> Sent: Monday, April 14, 2014 5:00:37 PM
> >>>>>>>>>> Subject: [ovirt-users] Error creating Disks
> >>>>>>>>>>
> >>>>>>>>>> oVirt Engine Version:
> >>>>>>>>>> 3.4.1-0.0.master.20140412010845.git43746c6.el6
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> While attempting to create a disk on an NFS storage domain, it
> >>>>>>>>>> fails with
> >>>>>>>>>> the
> >>>>>>>>>> following error in the engine.log
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 2014-04-14 09:58:12,127 ERROR
> >>>>>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> >>>>>>>>>> (DefaultQuartzScheduler_Worker-72) Failed in
> >>>>>>>>>> HSMGetAllTasksStatusesVDS
> >>>>>>>>>> method
> >>>>>>>>>> 2014-04-14 09:58:12,139 ERROR
> >>>>>>>>>> [org.ovirt.engine.core.bll.SPMAsyncTask]
> >>>>>>>>>> (DefaultQuartzScheduler_Worker-72)
> >>>>>>>>>> BaseAsyncTask::LogEndTaskFailure: Task
> >>>>>>>>>> ee6ce682-bd76-467a-82d2-d227229cb9de (Parent Command AddDisk,
> >>>>>>>>>> Parameters
> >>>>>>>>>> Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters)
> >>>>>>>>>> ended
> >>>>>>>>>> with
> >>>>>>>>>> failure:
> >>>>>>>>>> 2014-04-14 09:58:12,159 ERROR
> >>>>>>>>>> [org.ovirt.engine.core.bll.AddDiskCommand]
> >>>>>>>>>> (org.ovirt.thread.pool-6-thread-9) [483e53d6] Ending command with
> >>>>>>>>>> failure:
> >>>>>>>>>> org.ovirt.engine.core.bll.AddDiskCommand
> >>>>>>>>>> 2014-04-14 09:58:12,212 ERROR
> >>>>>>>>>> [org.ovirt.engine.core.bll.AddImageFromScratchCommand]
> >>>>>>>>>> (org.ovirt.thread.pool-6-thread-9) [ab1e0be] Ending command with
> >>>>>>>>>> failure:
> >>>>>>>>>> org.ovirt.engine.core.bll.AddImageFromScratchCommand
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> Users mailing list
> >>>>>>>>>> Users at ovirt.org
> >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> Users mailing list
> >>>>>>>> Users at ovirt.org
> >>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >
>
>
> --
> Dafna Ron
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
More information about the Users
mailing list