[Users] oVirt 3.3.3 RC EL6 Live Snapshot
Dafna Ron
dron at redhat.com
Tue Feb 4 14:25:06 UTC 2014
No, the bug is about libvirt pausing the vm's for live snapshot
I am not sure about your question on clock skew... we have to take a lot
of snapshot to surpass the 5 minutes skew though :)
On 02/04/2014 01:45 PM, Steve Dainard wrote:
> Just for clarity on that bug report; are you suggesting that the guest
> must go into a paused state to take a snapshot and that the admin
> shouldn't be aware of this state?
>
> Although not horribly concerning for most tasks, wouldn't this lead to
> a 10 second clock skew every time there is a snapshot? Or does the
> guest sync from host hw clock on resume?
>
> Thanks,
>
> *Steve Dainard *
> IT Infrastructure Manager
> Miovision <http://miovision.com/> | /Rethink Traffic/
> 519-513-2407 ex.250
> 877-646-8476 (toll-free)
>
> *Blog <http://miovision.com/blog> | **LinkedIn
> <https://www.linkedin.com/company/miovision-technologies> | Twitter
> <https://twitter.com/miovision> | Facebook
> <https://www.facebook.com/miovision>*
> ------------------------------------------------------------------------
> Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener,
> ON, Canada | N2C 1L3
> This e-mail may contain information that is privileged or
> confidential. If you are not the intended recipient, please delete the
> e-mail and any attachments and notify us immediately.
>
>
> On Tue, Feb 4, 2014 at 3:44 AM, Dafna Ron <dron at redhat.com
> <mailto:dron at redhat.com>> wrote:
>
> On 02/04/2014 08:20 AM, Elad Ben Aharon wrote:
>
> From what I saw in the thread, libvirt pauses the VM, which
> effects the continuity of its operation.
> I checked it also in one of the latest builds of 3.3 and I
> observed the same behaviour:
> 2014-02-04 10:18:29,441 INFO
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (DefaultQuartzScheduler_Worker-22) [5cbc7ab5] VM nfs2-1
> 86728f5c-3583-420f-9536-bfabaf11b235 moved from Up --> Paused
>
> Dafna, I saw you've already opened a bug on that:
> https://bugzilla.redhat.com/show_bug.cgi?id=1057587
>
>
> just proves I'm getting senile :)
>
>
> ----- Original Message -----
> From: "Dafna Ron" <dron at redhat.com <mailto:dron at redhat.com>>
> To: "Maor Lipchuk" <mlipchuk at redhat.com
> <mailto:mlipchuk at redhat.com>>
> Cc: "Steve Dainard" <sdainard at miovision.com
> <mailto:sdainard at miovision.com>>, "Elad Ben Aharon"
> <ebenahar at redhat.com <mailto:ebenahar at redhat.com>>, "Karli
> Sjöberg" <Karli.Sjoberg at slu.se <mailto:Karli.Sjoberg at slu.se>>,
> users at ovirt.org <mailto:users at ovirt.org>
> Sent: Tuesday, February 4, 2014 12:15:42 AM
> Subject: Re: [Users] oVirt 3.3.3 RC EL6 Live Snapshot
>
> Maor I am not saying that we are not doing a live snapshot :) I am
> saying that we need a print in the log that states live
> snapshot command
> was called i.e: Print in the log: LiveSnapshotCommand -> this
> can call
> to the rest of snapshotVDSCreateCommand.
>
>
> On 02/03/2014 07:38 PM, Maor Lipchuk wrote:
>
> On 02/03/2014 07:46 PM, Dafna Ron wrote:
>
> On 02/03/2014 05:34 PM, Maor Lipchuk wrote:
>
> On 02/03/2014 07:18 PM, Dafna Ron wrote:
>
> Maor,
>
> If snapshotVDSCommand is for live snapshot,
> what is the offline create
> snapshot command?
>
> It is the CreateSnapshotVdsCommand which calls
> createVolume in VDSM
>
> but we need to be able to know that a live snapshot
> was sent and not an
> offline snapshot.
>
> Yes, at the logs we can see the all process :
>
> First a request to create a snapshot (new volume) sent to
> VDSM:
> 2014-02-02 09:41:09,557 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand]
> (pool-6-thread-49) [67ea047a] START, CreateSnapshotVDSCommand(
> storagePoolId = fcb89071-6cdb-4972-94d1-c9324cebf814,
> ignoreFailoverLimit = false, storageDomainId =
> a52938f7-2cf4-4771-acb2-0c78d14999e5, imageGroupId =
> c1cb6b66-655e-48c3-8568-4975295eb037, imageSizeInBytes =
> 21474836480,
> volumeFormat = COW, newImageId =
> 6d8c80a4-328f-4a53-86a2-a4080a2662ce,
> newImageDescription = , imageId =
> 5085422e-6592-415a-9da3-9e43dac9374b,
> sourceImageGroupId =
> c1cb6b66-655e-48c3-8568-4975295eb037), log id: 7875f3f5
>
> after the snapshot gets created :
> 2014-02-02 09:41:20,553 INFO
> [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
> (pool-6-thread-49) Ending command successfully:
> org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand
>
> then the engine calls the live snapshot (see also [1])
> 2014-02-02 09:41:30,234 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> (pool-6-thread-49) FINISH, SnapshotVDSCommand, log id:
> 7e0d7872
>
> Elad, somewhere in this flow we need to know that the
> snapshot was taken
> on a running vm :) this seems like a bug to me.
>
> we did not say that live snapshot did not
> succeed :) we said that the
> vm is paused and restarted - which is
> something that should not happen
> for live snapshot (or at least never did before).
>
> It's not sure that the restart is related to the
> live snapshot. but that
> should be observed in the libvirt/vdsm logs.
>
> yes, I am sure because the user is reporting it and
> the logs show it...
>
> as I wrote before, we know that vdsm is
> reporting the vm as paused, that
> is because libvirt is reporting the vm as
> paused and I think that its
> happening because libvirt is not doing a live
> snapshot and so pauses the
> vm while taking the snapshot.
>
> That sounds logic to me, it's need to be checked
> with libvirt, if that
> kind of behaviour could happen.
>
> Elad, can you please try to reproduce and open a bug
> to libvirt?
>
> Dafna
>
>
> On 02/03/2014 05:08 PM, Maor Lipchuk wrote:
>
> From the engine logs it seems that
> indeed live snapshot is called
> (The
> command is snapshotVDSCommand see [1]).
> This is done right after the snapshot has
> been created in the VM and it
> signals the qemu process to start using
> the new volume created.
>
> When live snapshot does not succeed we
> should see in the log something
> like "Wasn't able to live snapshot due to
> error:...", but it does not
> appear so it seems that this worked out fine.
>
> At some point I can see in the logs that
> VDSM reports to the engine
> that
> the VM is paused.
>
>
> [1]
> 2014-02-02 09:41:20,564 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> (pool-6-thread-49) START,
> SnapshotVDSCommand(HostName = ovirt002,
> HostId
> = 3080fb61-2d03-4008-b47f-9b66276a4257,
> vmId=e261e707-a21f-4ae8-9cff-f535f4430446), log
> id: 7e0d7872
> 2014-02-02 09:41:21,119 INFO
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (DefaultQuartzScheduler_Worker-93) VM
> snapshot-test
> e261e707-a21f-4ae8-9cff-f535f4430446 moved
> from Up --> Paused
> 2014-02-02 09:41:30,234 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> (pool-6-thread-49) FINISH,
> SnapshotVDSCommand, log id: 7e0d7872
> 2014-02-02 09:41:30,238 INFO
> [org.ovirt.engine.core.bll.CreateSnapshotCommand]
> (pool-6-thread-49)
> [67ea047a] Ending command successfully:
> org.ovirt.engine.core.bll.CreateSnapshotCommand
> ...
>
> Regards,
> Maor
>
> On 02/03/2014 06:24 PM, Dafna Ron wrote:
>
> Thanks Steve.
>
> from the logs I can see that the
> create snapshot succeeds and that the
> vm is resumed.
> the vm moves to pause as part of
> libvirt flows:
>
> 2014-02-02 14:41:20.872+0000: 5843:
> debug :
> qemuProcessHandleStop:728 :
> Transitioned guest snapshot-test to
> paused state
> 2014-02-02 14:41:30.031+0000: 5843:
> debug :
> qemuProcessHandleResume:776
> : Transitioned guest snapshot-test out
> of paused into resumed state
>
> There are bugs here but I am not sure
> yet if this is libvirt
> regression
> or engine.
>
> I'm adding Elad and Maor since in
> engine logs I can't see anything
> calling for live snapshot (only for
> snapshot) - Maor, shouldn't live
> snapshot command be logged somewhere
> in the logs?
> Is it possible that engine is calling
> to create snapshot and not
> create
> live snapshot which is why the vm pauses?
>
> Elad, if engine is not logging live
> snapshot anywhere I would open
> a bug
> for engine (to print that in the logs).
> Also, there is a bug in vdsm log for
> sdc where the below is logged as
> ERROR and not INFO:
>
> Thread-23::ERROR::2014-02-02
> 09:51:19,497::sdc::137::Storage.StorageDomainCache::(_findDomain)
> looking for unfetched domain
> a52938f7-2cf4-4771-acb2-0c78d14999e5
> Thread-23::ERROR::2014-02-02
> 09:51:19,497::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain)
>
>
> looking for domain
> a52938f7-2cf4-4771-acb2-0c78d14999e5
>
> If the engine was sending live
> snapshot or if there is no
> difference in
> the two commands in engine side than I
> would open a bug for libvirt
> for
> pausing the vm during live snapshot.
>
> Dafna
>
> On 02/03/2014 02:41 PM, Steve Dainard
> wrote:
>
> [root at ovirt002 ~]# vdsClient -s 0
> getStorageDomainInfo
> a52938f7-2cf4-4771-acb2-0c78d14999e5
> uuid =
> a52938f7-2cf4-4771-acb2-0c78d14999e5
> pool =
> ['fcb89071-6cdb-4972-94d1-c9324cebf814']
> lver = 5
> version = 3
> role = Master
> remotePath = gluster-store-vip:/rep1
> spm_id = 2
> type = NFS
> class = Data
> master_ver = 1
> name = gluster-store-rep1
>
>
> *Steve Dainard *
> IT Infrastructure Manager
> Miovision <http://miovision.com/>
> | /Rethink Traffic/
> 519-513-2407 <tel:519-513-2407> ex.250
> 877-646-8476
> <tel:877-646-8476> (toll-free)
>
> *Blog <http://miovision.com/blog>
> | **LinkedIn
> <https://www.linkedin.com/company/miovision-technologies>
> | Twitter
> <https://twitter.com/miovision> |
> Facebook
> <https://www.facebook.com/miovision>*
> ------------------------------------------------------------------------
>
>
> Miovision Technologies Inc. | 148
> Manitou Drive, Suite 101,
> Kitchener,
> ON, Canada | N2C 1L3
> This e-mail may contain
> information that is privileged or
> confidential. If you are not the
> intended recipient, please delete
> the
> e-mail and any attachments and
> notify us immediately.
>
>
> On Sun, Feb 2, 2014 at 2:55 PM,
> Dafna Ron <dron at redhat.com
> <mailto:dron at redhat.com>
> <mailto:dron at redhat.com
> <mailto:dron at redhat.com>>> wrote:
>
> please run vdsClient -s 0
> getStorageDomainInfo
>
> a52938f7-2cf4-4771-acb2-0c78d14999e5
>
> Thanks,
>
> Dafna
>
>
>
> On 02/02/2014 03:02 PM,
> Steve Dainard wrote:
>
> Logs attached with VM
> running on qemu-kvm-rhev packages
> installed.
>
> *Steve Dainard *
> IT Infrastructure Manager
> Miovision
> <http://miovision.com/> | /Rethink
> Traffic/
> 519-513-2407 <tel:519-513-2407>
> <tel:519-513-2407
> <tel:519-513-2407>> ex.250
>
> 877-646-8476 <tel:877-646-8476>
> <tel:877-646-8476
> <tel:877-646-8476>> (toll-free)
>
> *Blog
> <http://miovision.com/blog> |
> **LinkedIn
>
> <https://www.linkedin.com/company/miovision-technologies>
> |
> Twitter
> <https://twitter.com/miovision> |
> Facebook
>
> <https://www.facebook.com/miovision>*
>
> ------------------------------------------------------------------------
>
>
> Miovision Technologies
> Inc. | 148 Manitou Drive, Suite 101,
> Kitchener, ON, Canada
> | N2C 1L3
> This e-mail may
> contain information that is
> privileged or
> confidential. If you
> are not the intended recipient, please
> delete the e-mail and
> any attachments and notify us
> immediately.
>
>
> On Sun, Feb 2, 2014 at
> 5:05 AM, Dafna Ron
> <dron at redhat.com
> <mailto:dron at redhat.com>
>
> <mailto:dron at redhat.com
> <mailto:dron at redhat.com>>
> <mailto:dron at redhat.com
> <mailto:dron at redhat.com>
>
> <mailto:dron at redhat.com
> <mailto:dron at redhat.com>>>> wrote:
>
> can you please
> upload full engine, vdsm, libvirt and
> vm's
> qemu logs?
>
>
> On 02/02/2014
> 02:08 AM, Steve Dainard wrote:
>
> I have two
> CentOS 6.5 Ovirt hosts (ovirt001,
> ovirt002)
>
> I've installed
> the applicable qemu-kvm-rhev
> packages
> from this
> site:
> http://www.dreyou.org/ovirt/vdsm32/Packages/
> on
> ovirt002.
>
> On ovirt001 if
> I take a live snapshot:
>
> Snapshot 'test
> qemu-kvm' creation for VM
> 'snapshot-test' was
> initiated by
> admin at internal.
> The VM is paused
> Failed to
> create live snapshot 'test qemu-kvm'
> for VM
>
> 'snapshot-test'. VM restart is
> recommended.
> Failed to
> complete snapshot 'test qemu-kvm'
> creation
> for VM
> 'snapshot-test'.
> The VM is then
> started, and the status for the
> snapshot
> changes to OK.
>
> On ovirt002
> (with the packages from dreyou) I
> don't
> get any
> messages about
> a snapshot failing, but my VM is
> still
> paused
> to complete
> the snapshot. Is there something else
> other than
> the
> qemu-kvm-rhev packages that would
> enable this
> functionality?
>
> I've looked
> for some information on when the
> packages
> would be
> built as
> required in the CentOS repos, but I
> don't see
> anything
> definitive.
>
> http://lists.ovirt.org/pipermail/users/2013-December/019126.html
> Looks like one
> of the maintainers is waiting for
> someone to
> tell him what
> flags need to be set.
>
> Also, another
> thread here:
> http://comments.gmane.org/gmane.comp.emulators.ovirt.arch/1618
> same
> maintainer, mentioning that he
> hasn't seen
> anything in
> the bug tracker.
>
> There is a bug
> here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1009100
> that
> seems
> to have ended
> in finding a way for qemu to expose
> whether it
> supports live
> snapshots, rather than figuring
> out how
> to get
> the CentOS
> team the info they need to build the
> packages with
> the proper
> flags set.
>
> I have bcc'd
> both dreyou (packaged the
> qemu-kvm-rhev
> packages
> listed above)
> and Russ (CentOS maintainer
> mentioned in
> the
> other threads)
> if they wish to chime in and perhaps
> collaborate on
> which flags, if any, should be set
> for the
> qemu-kvm
> builds so we can get a CentOS bug
> report
> going and
> hammer this out.
>
> Thanks everyone.
>
> **crosses
> fingers and hopes for live snapshots
> soon**
>
>
>
> *Steve Dainard *
> IT
> Infrastructure Manager
> Miovision
> <http://miovision.com/> | /Rethink
> Traffic/
> 519-513-2407 <tel:519-513-2407>
> <tel:519-513-2407
> <tel:519-513-2407>>
> <tel:519-513-2407 <tel:519-513-2407>
> <tel:519-513-2407
> <tel:519-513-2407>>>
> <tel:519-513-2407
> <tel:519-513-2407>
> <tel:519-513-2407 <tel:519-513-2407>>
>
> <tel:519-513-2407
> <tel:519-513-2407>
> <tel:519-513-2407
> <tel:519-513-2407>>>> ex.250
> 877-646-8476 <tel:877-646-8476>
> <tel:877-646-8476
> <tel:877-646-8476>>
> <tel:877-646-8476 <tel:877-646-8476>
> <tel:877-646-8476
> <tel:877-646-8476>>>
> <tel:877-646-8476
> <tel:877-646-8476>
> <tel:877-646-8476 <tel:877-646-8476>>
>
>
> <tel:877-646-8476
> <tel:877-646-8476>
> <tel:877-646-8476
> <tel:877-646-8476>>>> (toll-free)
>
> *Blog
> <http://miovision.com/blog> |
> **LinkedIn
>
> <https://www.linkedin.com/company/miovision-technologies>
> |
> Twitter
> <https://twitter.com/miovision> |
> Facebook
>
> <https://www.facebook.com/miovision>*
> ------------------------------------------------------------------------
>
>
>
>
> Miovision
> Technologies Inc. | 148 Manitou Drive,
> Suite
> 101,
> Kitchener, ON,
> Canada | N2C 1L3
> This e-mail
> may contain information that is
> privileged or
> confidential.
> If you are not the intended
> recipient,
> please
> delete the
> e-mail and any attachments and
> notify us
> immediately.
>
>
> On Fri, Jan
> 31, 2014 at 1:26 PM, Steve Dainard
>
> <sdainard at miovision.com
> <mailto:sdainard at miovision.com>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>>>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>>
>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>
>
> <mailto:sdainard at miovision.com
> <mailto:sdainard at miovision.com>>>>> wrote:
>
>
> How
> would you developers, speaking for the
> oVirt-community,
> propose to
> solve
> this for CentOS _now_ ?
>
> I
> would imagine that the easiest way is
> that
> you build and
> host
> this one
> package(qemu-kvm-rhev), since you´ve
> basically
> already
> have
> the source
> and
> recipe (since you´re already
> providing it
> for RHEV
>
> anyway). Then,
> once
> that´s in place, it´s more a
> question of
> where to
> host the
>
> packages, in what repository. Be
> it your
> own,
> or some
> other
> repo
> set up
> for
> the SIG.
>
> This
> is my view, how I as a user view this
> issue.
>
>
>
> I think
> this is a pretty valid view.
>
> What would
> it take to get the correct qemu
> package
> hosted
> in the
> ovirt repo?
>
> --
>
> Med
> Vänliga Hälsningar
>
> -------------------------------------------------------------------------------
>
>
>
> Karli
> Sjöberg
>
> Swedish University of Agricultural
> Sciences
> Box 7079
> (Visiting
> Address
>
> Kronåsvägen 8)
> S-750
> 07 Uppsala, Sweden
> Phone:
> +46-(0)18-67 15 66
> <tel:%2B46-%280%2918-67%2015%2066>
> <tel:%2B46-%280%2918-67%2015%2066>
> <tel:%2B46-%280%2918-67%2015%2066>
> <tel:%2B46-%280%2918-67%2015%2066>
> karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>>
>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>>>
>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>
>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>
>
> <mailto:karli.sjoberg at slu.se
> <mailto:karli.sjoberg at slu.se>>>>
>
> _______________________________________________
> Users
> mailing list
> Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>
>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>>
>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>
>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>>>
>
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>
>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>>
>
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> -- Dafna Ron
>
>
>
>
> -- Dafna Ron
>
>
>
>
>
> --
> Dafna Ron
>
>
--
Dafna Ron
More information about the Users
mailing list