[Users] oVirt 3.3.3 RC EL6 Live Snapshot

Dafna Ron dron at redhat.com
Mon Feb 3 22:15:42 UTC 2014


Maor I am not saying that we are not doing a live snapshot :) I am 
saying that we need a print in the log that states live snapshot command 
was called i.e: Print in the log: LiveSnapshotCommand -> this can call 
to the rest of snapshotVDSCreateCommand.


On 02/03/2014 07:38 PM, Maor Lipchuk wrote:
> On 02/03/2014 07:46 PM, Dafna Ron wrote:
>> On 02/03/2014 05:34 PM, Maor Lipchuk wrote:
>>> On 02/03/2014 07:18 PM, Dafna Ron wrote:
>>>> Maor,
>>>>
>>>> If snapshotVDSCommand is for live snapshot, what is the offline create
>>>> snapshot command?
>>> It is the CreateSnapshotVdsCommand which calls createVolume in VDSM
>> but we need to be able to know that a live snapshot was sent and not an
>> offline snapshot.
> Yes, at the logs we can see the all process :
>
> First a request to create a snapshot (new volume) sent to VDSM:
> 2014-02-02 09:41:09,557 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand]
> (pool-6-thread-49) [67ea047a] START, CreateSnapshotVDSCommand(
> storagePoolId = fcb89071-6cdb-4972-94d1-c9324cebf814,
> ignoreFailoverLimit = false, storageDomainId =
> a52938f7-2cf4-4771-acb2-0c78d14999e5, imageGroupId =
> c1cb6b66-655e-48c3-8568-4975295eb037, imageSizeInBytes = 21474836480,
> volumeFormat = COW, newImageId = 6d8c80a4-328f-4a53-86a2-a4080a2662ce,
> newImageDescription = , imageId = 5085422e-6592-415a-9da3-9e43dac9374b,
> sourceImageGroupId = c1cb6b66-655e-48c3-8568-4975295eb037), log id: 7875f3f5
>
> after the snapshot gets created :
> 2014-02-02 09:41:20,553 INFO
> [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
> (pool-6-thread-49) Ending command successfully:
> org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand
>
> then the engine calls the live snapshot (see also [1])
> 2014-02-02 09:41:30,234 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
> (pool-6-thread-49) FINISH, SnapshotVDSCommand, log id: 7e0d7872
>
>> Elad, somewhere in this flow we need to know that the snapshot was taken
>> on a running vm :) this seems like a bug to me.
>>>> we did not say that live snapshot did not succeed :)  we said that the
>>>> vm is paused and restarted - which is something that should not happen
>>>> for live snapshot (or at least never did before).
>>> It's not sure that the restart is related to the live snapshot. but that
>>> should be observed in the libvirt/vdsm logs.
>> yes, I am sure because the user is reporting it and the logs show it...
>>>> as I wrote before, we know that vdsm is reporting the vm as paused, that
>>>> is because libvirt is reporting the vm as paused and I think that its
>>>> happening because libvirt is not doing a live snapshot and so pauses the
>>>> vm while taking the snapshot.
>>> That sounds logic to me, it's need to be checked with libvirt, if that
>>> kind of behaviour could happen.
>> Elad, can you please try to reproduce and open a bug to libvirt?
>>
>>>> Dafna
>>>>
>>>>
>>>> On 02/03/2014 05:08 PM, Maor Lipchuk wrote:
>>>>>    From the engine logs it seems that indeed live snapshot is called
>>>>> (The
>>>>> command is snapshotVDSCommand see [1]).
>>>>> This is done right after the snapshot has been created in the VM and it
>>>>> signals the qemu process to start using the new volume created.
>>>>>
>>>>> When live snapshot does not succeed we should see in the log something
>>>>> like "Wasn't able to live snapshot due to error:...", but it does not
>>>>> appear so it seems that this worked out fine.
>>>>>
>>>>> At some point I can see in the logs that VDSM reports to the engine
>>>>> that
>>>>> the VM is paused.
>>>>>
>>>>>
>>>>> [1]
>>>>> 2014-02-02 09:41:20,564 INFO
>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
>>>>> (pool-6-thread-49) START, SnapshotVDSCommand(HostName = ovirt002,
>>>>> HostId
>>>>> = 3080fb61-2d03-4008-b47f-9b66276a4257,
>>>>> vmId=e261e707-a21f-4ae8-9cff-f535f4430446), log id: 7e0d7872
>>>>> 2014-02-02 09:41:21,119 INFO
>>>>> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
>>>>> (DefaultQuartzScheduler_Worker-93) VM snapshot-test
>>>>> e261e707-a21f-4ae8-9cff-f535f4430446 moved from Up --> Paused
>>>>> 2014-02-02 09:41:30,234 INFO
>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
>>>>> (pool-6-thread-49) FINISH, SnapshotVDSCommand, log id: 7e0d7872
>>>>> 2014-02-02 09:41:30,238 INFO
>>>>> [org.ovirt.engine.core.bll.CreateSnapshotCommand] (pool-6-thread-49)
>>>>> [67ea047a] Ending command successfully:
>>>>> org.ovirt.engine.core.bll.CreateSnapshotCommand
>>>>> ...
>>>>>
>>>>> Regards,
>>>>> Maor
>>>>>
>>>>> On 02/03/2014 06:24 PM, Dafna Ron wrote:
>>>>>> Thanks Steve.
>>>>>>
>>>>>> from the logs I can see that the create snapshot succeeds and that the
>>>>>> vm is resumed.
>>>>>> the vm moves to pause as part of libvirt flows:
>>>>>>
>>>>>> 2014-02-02 14:41:20.872+0000: 5843: debug :
>>>>>> qemuProcessHandleStop:728 :
>>>>>> Transitioned guest snapshot-test to paused state
>>>>>> 2014-02-02 14:41:30.031+0000: 5843: debug :
>>>>>> qemuProcessHandleResume:776
>>>>>> : Transitioned guest snapshot-test out of paused into resumed state
>>>>>>
>>>>>> There are bugs here but I am not sure yet if this is libvirt
>>>>>> regression
>>>>>> or engine.
>>>>>>
>>>>>> I'm adding Elad and Maor since in engine logs I can't see anything
>>>>>> calling for live snapshot (only for snapshot) - Maor, shouldn't live
>>>>>> snapshot command be logged somewhere in the logs?
>>>>>> Is it possible that engine is calling to create snapshot and not
>>>>>> create
>>>>>> live snapshot which is why the vm pauses?
>>>>>>
>>>>>> Elad, if engine is not logging live snapshot anywhere I would open
>>>>>> a bug
>>>>>> for engine (to print that in the logs).
>>>>>> Also, there is a bug in vdsm log for sdc where the below is logged as
>>>>>> ERROR and not INFO:
>>>>>>
>>>>>> Thread-23::ERROR::2014-02-02
>>>>>> 09:51:19,497::sdc::137::Storage.StorageDomainCache::(_findDomain)
>>>>>> looking for unfetched domain a52938f7-2cf4-4771-acb2-0c78d14999e5
>>>>>> Thread-23::ERROR::2014-02-02
>>>>>> 09:51:19,497::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain)
>>>>>>
>>>>>>
>>>>>> looking for domain a52938f7-2cf4-4771-acb2-0c78d14999e5
>>>>>>
>>>>>> If the engine was sending live snapshot or if there is no
>>>>>> difference in
>>>>>> the two commands in engine side than I would open a bug for libvirt
>>>>>> for
>>>>>> pausing the vm during live snapshot.
>>>>>>
>>>>>> Dafna
>>>>>>
>>>>>> On 02/03/2014 02:41 PM, Steve Dainard wrote:
>>>>>>> [root at ovirt002 ~]# vdsClient -s 0 getStorageDomainInfo
>>>>>>> a52938f7-2cf4-4771-acb2-0c78d14999e5
>>>>>>> uuid = a52938f7-2cf4-4771-acb2-0c78d14999e5
>>>>>>> pool = ['fcb89071-6cdb-4972-94d1-c9324cebf814']
>>>>>>> lver = 5
>>>>>>> version = 3
>>>>>>> role = Master
>>>>>>> remotePath = gluster-store-vip:/rep1
>>>>>>> spm_id = 2
>>>>>>> type = NFS
>>>>>>> class = Data
>>>>>>> master_ver = 1
>>>>>>> name = gluster-store-rep1
>>>>>>>
>>>>>>>
>>>>>>> *Steve Dainard *
>>>>>>> IT Infrastructure Manager
>>>>>>> Miovision <http://miovision.com/> | /Rethink Traffic/
>>>>>>> 519-513-2407 ex.250
>>>>>>> 877-646-8476 (toll-free)
>>>>>>>
>>>>>>> *Blog <http://miovision.com/blog> | **LinkedIn
>>>>>>> <https://www.linkedin.com/company/miovision-technologies>  | Twitter
>>>>>>> <https://twitter.com/miovision>  | Facebook
>>>>>>> <https://www.facebook.com/miovision>*
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> Miovision Technologies Inc. | 148 Manitou Drive, Suite 101,
>>>>>>> Kitchener,
>>>>>>> ON, Canada | N2C 1L3
>>>>>>> This e-mail may contain information that is privileged or
>>>>>>> confidential. If you are not the intended recipient, please delete
>>>>>>> the
>>>>>>> e-mail and any attachments and notify us immediately.
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Feb 2, 2014 at 2:55 PM, Dafna Ron <dron at redhat.com
>>>>>>> <mailto:dron at redhat.com>> wrote:
>>>>>>>
>>>>>>>        please run vdsClient -s 0 getStorageDomainInfo
>>>>>>>        a52938f7-2cf4-4771-acb2-0c78d14999e5
>>>>>>>
>>>>>>>        Thanks,
>>>>>>>
>>>>>>>        Dafna
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>        On 02/02/2014 03:02 PM, Steve Dainard wrote:
>>>>>>>
>>>>>>>            Logs attached with VM running on qemu-kvm-rhev packages
>>>>>>> installed.
>>>>>>>
>>>>>>>            *Steve Dainard *
>>>>>>>            IT Infrastructure Manager
>>>>>>>            Miovision <http://miovision.com/> | /Rethink Traffic/
>>>>>>>            519-513-2407 <tel:519-513-2407> ex.250
>>>>>>>
>>>>>>>            877-646-8476 <tel:877-646-8476> (toll-free)
>>>>>>>
>>>>>>>            *Blog <http://miovision.com/blog> | **LinkedIn
>>>>>>>           
>>>>>>> <https://www.linkedin.com/company/miovision-technologies>  |
>>>>>>>            Twitter <https://twitter.com/miovision>  | Facebook
>>>>>>>            <https://www.facebook.com/miovision>*
>>>>>>>         
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>            Miovision Technologies Inc. | 148 Manitou Drive, Suite 101,
>>>>>>>            Kitchener, ON, Canada | N2C 1L3
>>>>>>>            This e-mail may contain information that is privileged or
>>>>>>>            confidential. If you are not the intended recipient, please
>>>>>>>            delete the e-mail and any attachments and notify us
>>>>>>> immediately.
>>>>>>>
>>>>>>>
>>>>>>>            On Sun, Feb 2, 2014 at 5:05 AM, Dafna Ron <dron at redhat.com
>>>>>>>            <mailto:dron at redhat.com> <mailto:dron at redhat.com
>>>>>>>            <mailto:dron at redhat.com>>> wrote:
>>>>>>>
>>>>>>>                can you please upload full engine, vdsm, libvirt and
>>>>>>> vm's
>>>>>>>            qemu logs?
>>>>>>>
>>>>>>>
>>>>>>>                On 02/02/2014 02:08 AM, Steve Dainard wrote:
>>>>>>>
>>>>>>>                    I have two CentOS 6.5 Ovirt hosts (ovirt001,
>>>>>>> ovirt002)
>>>>>>>
>>>>>>>                    I've installed the applicable qemu-kvm-rhev
>>>>>>> packages
>>>>>>>            from this
>>>>>>>                    site:
>>>>>>> http://www.dreyou.org/ovirt/vdsm32/Packages/ on
>>>>>>>            ovirt002.
>>>>>>>
>>>>>>>                    On ovirt001 if I take a live snapshot:
>>>>>>>
>>>>>>>                    Snapshot 'test qemu-kvm' creation for VM
>>>>>>>            'snapshot-test' was
>>>>>>>                    initiated by admin at internal.
>>>>>>>                    The VM is paused
>>>>>>>                    Failed to create live snapshot 'test qemu-kvm'
>>>>>>> for VM
>>>>>>>                    'snapshot-test'. VM restart is recommended.
>>>>>>>                    Failed to complete snapshot 'test qemu-kvm'
>>>>>>> creation
>>>>>>>            for VM
>>>>>>>                    'snapshot-test'.
>>>>>>>                    The VM is then started, and the status for the
>>>>>>> snapshot
>>>>>>>                    changes to OK.
>>>>>>>
>>>>>>>                    On ovirt002 (with the packages from dreyou) I don't
>>>>>>>            get any
>>>>>>>                    messages about a snapshot failing, but my VM is
>>>>>>> still
>>>>>>>            paused
>>>>>>>                    to complete the snapshot. Is there something else
>>>>>>>            other than
>>>>>>>                    the qemu-kvm-rhev packages that would enable this
>>>>>>>            functionality?
>>>>>>>
>>>>>>>                    I've looked for some information on when the
>>>>>>> packages
>>>>>>>            would be
>>>>>>>                    built as required in the CentOS repos, but I
>>>>>>> don't see
>>>>>>>                    anything definitive.
>>>>>>>
>>>>>>>          
>>>>>>> http://lists.ovirt.org/pipermail/users/2013-December/019126.html
>>>>>>>                    Looks like one of the maintainers is waiting for
>>>>>>>            someone to
>>>>>>>                    tell him what flags need to be set.
>>>>>>>
>>>>>>>                    Also, another thread here:
>>>>>>>           
>>>>>>> http://comments.gmane.org/gmane.comp.emulators.ovirt.arch/1618
>>>>>>>                    same maintainer, mentioning that he hasn't seen
>>>>>>>            anything in
>>>>>>>                    the bug tracker.
>>>>>>>
>>>>>>>                    There is a bug here:
>>>>>>>            https://bugzilla.redhat.com/show_bug.cgi?id=1009100 that
>>>>>>> seems
>>>>>>>                    to have ended in finding a way for qemu to expose
>>>>>>>            whether it
>>>>>>>                    supports live snapshots, rather than figuring
>>>>>>> out how
>>>>>>>            to get
>>>>>>>                    the CentOS team the info they need to build the
>>>>>>>            packages with
>>>>>>>                    the proper flags set.
>>>>>>>
>>>>>>>                    I have bcc'd both dreyou (packaged the
>>>>>>> qemu-kvm-rhev
>>>>>>>            packages
>>>>>>>                    listed above) and Russ (CentOS maintainer
>>>>>>> mentioned in
>>>>>>> the
>>>>>>>                    other threads) if they wish to chime in and perhaps
>>>>>>>                    collaborate on which flags, if any, should be set
>>>>>>> for the
>>>>>>>                    qemu-kvm builds so we can get a CentOS bug report
>>>>>>>            going and
>>>>>>>                    hammer this out.
>>>>>>>
>>>>>>>                    Thanks everyone.
>>>>>>>
>>>>>>>                    **crosses fingers and hopes for live snapshots
>>>>>>> soon**
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                    *Steve Dainard *
>>>>>>>                    IT Infrastructure Manager
>>>>>>>                    Miovision <http://miovision.com/> | /Rethink
>>>>>>> Traffic/
>>>>>>>            519-513-2407 <tel:519-513-2407> <tel:519-513-2407
>>>>>>>            <tel:519-513-2407>> <tel:519-513-2407 <tel:519-513-2407>
>>>>>>>                    <tel:519-513-2407 <tel:519-513-2407>>> ex.250
>>>>>>>            877-646-8476 <tel:877-646-8476> <tel:877-646-8476
>>>>>>>            <tel:877-646-8476>> <tel:877-646-8476 <tel:877-646-8476>
>>>>>>>
>>>>>>>                    <tel:877-646-8476 <tel:877-646-8476>>> (toll-free)
>>>>>>>
>>>>>>>                    *Blog <http://miovision.com/blog> | **LinkedIn
>>>>>>>                        
>>>>>>> <https://www.linkedin.com/company/miovision-technologies>  |
>>>>>>>                    Twitter <https://twitter.com/miovision>  | Facebook
>>>>>>>                    <https://www.facebook.com/miovision>*
>>>>>>>                        
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                    Miovision Technologies Inc. | 148 Manitou Drive,
>>>>>>> Suite
>>>>>>>            101,
>>>>>>>                    Kitchener, ON, Canada | N2C 1L3
>>>>>>>                    This e-mail may contain information that is
>>>>>>> privileged or
>>>>>>>                    confidential. If you are not the intended
>>>>>>> recipient,
>>>>>>>            please
>>>>>>>                    delete the e-mail and any attachments and notify us
>>>>>>>            immediately.
>>>>>>>
>>>>>>>
>>>>>>>                    On Fri, Jan 31, 2014 at 1:26 PM, Steve Dainard
>>>>>>>                    <sdainard at miovision.com
>>>>>>>            <mailto:sdainard at miovision.com>
>>>>>>> <mailto:sdainard at miovision.com
>>>>>>>            <mailto:sdainard at miovision.com>>
>>>>>>>                    <mailto:sdainard at miovision.com
>>>>>>>            <mailto:sdainard at miovision.com>
>>>>>>>
>>>>>>>                    <mailto:sdainard at miovision.com
>>>>>>>            <mailto:sdainard at miovision.com>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>                            How would you developers, speaking for the
>>>>>>>                    oVirt-community,
>>>>>>>                            propose to
>>>>>>>                            solve this for CentOS _now_ ?
>>>>>>>
>>>>>>>                            I would imagine that the easiest way is
>>>>>>> that
>>>>>>>            you build and
>>>>>>>                            host this one
>>>>>>>                            package(qemu-kvm-rhev), since you´ve
>>>>>>> basically
>>>>>>>            already
>>>>>>>                    have
>>>>>>>                            the source
>>>>>>>                            and recipe (since you´re already
>>>>>>> providing it
>>>>>>>            for RHEV
>>>>>>>                            anyway). Then,
>>>>>>>                            once that´s in place, it´s more a
>>>>>>> question of
>>>>>>>            where to
>>>>>>>                    host the
>>>>>>>                            packages, in what repository. Be it your
>>>>>>> own,
>>>>>>>            or some
>>>>>>>                    other
>>>>>>>                            repo set up
>>>>>>>                            for the SIG.
>>>>>>>
>>>>>>>                            This is my view, how I as a user view this
>>>>>>> issue.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                        I think this is a pretty valid view.
>>>>>>>
>>>>>>>                        What would it take to get the correct qemu
>>>>>>> package
>>>>>>>            hosted
>>>>>>>                    in the
>>>>>>>                        ovirt repo?
>>>>>>>
>>>>>>>                            --
>>>>>>>
>>>>>>>                            Med Vänliga Hälsningar
>>>>>>>
>>>>>>>                                       
>>>>>>> -------------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                            Karli Sjöberg
>>>>>>>                            Swedish University of Agricultural Sciences
>>>>>>>            Box 7079
>>>>>>>                    (Visiting
>>>>>>>                            Address
>>>>>>>                            Kronåsvägen 8)
>>>>>>>                            S-750 07 Uppsala, Sweden
>>>>>>>                            Phone: +46-(0)18-67 15 66
>>>>>>>            <tel:%2B46-%280%2918-67%2015%2066>
>>>>>>>                    <tel:%2B46-%280%2918-67%2015%2066>
>>>>>>>                    <tel:%2B46-%280%2918-67%2015%2066>
>>>>>>>            karli.sjoberg at slu.se <mailto:karli.sjoberg at slu.se>
>>>>>>>            <mailto:karli.sjoberg at slu.se <mailto:karli.sjoberg at slu.se>>
>>>>>>>                    <mailto:karli.sjoberg at slu.se
>>>>>>>            <mailto:karli.sjoberg at slu.se> <mailto:karli.sjoberg at slu.se
>>>>>>>            <mailto:karli.sjoberg at slu.se>>>
>>>>>>>
>>>>>>>                          
>>>>>>> _______________________________________________
>>>>>>>                            Users mailing list
>>>>>>>            Users at ovirt.org <mailto:Users at ovirt.org>
>>>>>>>            <mailto:Users at ovirt.org <mailto:Users at ovirt.org>>
>>>>>>>                    <mailto:Users at ovirt.org <mailto:Users at ovirt.org>
>>>>>>>            <mailto:Users at ovirt.org <mailto:Users at ovirt.org>>>
>>>>>>>
>>>>>>>            http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                    _______________________________________________
>>>>>>>                    Users mailing list
>>>>>>>            Users at ovirt.org <mailto:Users at ovirt.org>
>>>>>>>            <mailto:Users at ovirt.org <mailto:Users at ovirt.org>>
>>>>>>>
>>>>>>>            http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                --     Dafna Ron
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>        --     Dafna Ron
>>>>>>>
>>>>>>>
>>


-- 
Dafna Ron



More information about the Users mailing list