Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state. Best Regards, Strahil Nikolov On May 10, 2019 15:37, Derek Atkins <derek@ihtfp.com> wrote:
Hi,
Michael Blanchard <michael@wanderingmad.com> writes:
If you haven't seen my other posts, I'm not a very experienced Linux admin, so I'm trying to make it as easy as possible to run and maintain. It's hard enough for me to not break ovirt in crazy ways
This has nothing to do with ovirt.
You could use rdiff-backup on any running machine, be it virtual or bare metal. It's just a way to use a combination of diff and rsync to backup machines. Indeed, I was using it with my vmware-based systems and, when I migrated them to ovirt, the backups just continued working.
Get Outlook for Android
-derek -- Derek Atkins 617-623-3745 derek@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CBMHSCOBQ3MQHK...
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM. What I do (on systems that have a running database) is to run a "flush" operation to sync the database to disk, and then from within the flush operation I create an LVM snapshot, and then I backup off the snapshot. If I'm not running a database, then I just create the snapshot directly.
Best Regards, Strahil Nikolov
-derek -- Derek Atkins 617-623-3745 derek@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant
On Mon, May 13, 2019 at 3:47 PM Derek Atkins <derek@ihtfp.com> wrote:
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM.
Actually doable, depending on DB architecture and if some "logging/archiviing" supported/enabled on database. See here related threads where I was involved: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJFQOLIWU224R2... and here: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/J65N7E2PY3KWUCZ... When you install qemu-guest-agent (in my case version 0.12.1.2-2.506.el6_10.1) you also get an example for MySQL in /usr/share/qemu-kvm/qemu-ga/fsfreeze-hook.d/mysql-flush.sh.sample and eventually updated code (actually I see it is dated 2013...) here: https://github.com/guillon/qemu-plugins/blob/master/scripts/qemu-guest-agent... In case of PostgreSQL you would need to have WAL enabled and do an Exclusive Low Level Backup, because the pre-snapshot and post-snapshot could not be inside the same RDBMS session connection: https://www.postgresql.org/docs/10/continuous-archiving.html#BACKUP-BASE-BAC... so the freeze option of your freeze-hook script would contain a call to a connection that runs SELECT pg_start_backup('label', true); to minimize time to do checkpoint and the thaw option would contain SELECT pg_stop_backup(); This needs also that archives during the elapsed of backup (and the last generaed when stopping backup) are copied in some way externally in respect of the VM OS for complete recoverability, if you don't already have it for your standard archiving policy. And thoroughly test the whole workflow, both backups and restores, especially when upgrading your software layers, oVirt, VM OS, PostgreSQL.... HIH, Gianluca
I know that a robust API is provided for performing VM backup functions, but it seems crazy to me that oVirt has not implemented a simple method to backup VM to a storage domain such as an NFS export domain. Is this functionality being left out for a reason (perhaps not wanting to step on toes of 3rd party software vendors providing backup solutions for oVirt/RHEV)? On Mon, May 13, 2019 at 11:21 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 13, 2019 at 3:47 PM Derek Atkins <derek@ihtfp.com> wrote:
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM.
Actually doable, depending on DB architecture and if some "logging/archiviing" supported/enabled on database. See here related threads where I was involved:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJFQOLIWU224R2... and here:
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/J65N7E2PY3KWUCZ...
When you install qemu-guest-agent (in my case version 0.12.1.2-2.506.el6_10.1) you also get an example for MySQL in /usr/share/qemu-kvm/qemu-ga/fsfreeze-hook.d/mysql-flush.sh.sample and eventually updated code (actually I see it is dated 2013...) here:
https://github.com/guillon/qemu-plugins/blob/master/scripts/qemu-guest-agent...
In case of PostgreSQL you would need to have WAL enabled and do an Exclusive Low Level Backup, because the pre-snapshot and post-snapshot could not be inside the same RDBMS session connection:
https://www.postgresql.org/docs/10/continuous-archiving.html#BACKUP-BASE-BAC...
so the freeze option of your freeze-hook script would contain a call to a connection that runs SELECT pg_start_backup('label', true); to minimize time to do checkpoint
and the thaw option would contain SELECT pg_stop_backup();
This needs also that archives during the elapsed of backup (and the last generaed when stopping backup) are copied in some way externally in respect of the VM OS for complete recoverability, if you don't already have it for your standard archiving policy.
And thoroughly test the whole workflow, both backups and restores, especially when upgrading your software layers, oVirt, VM OS, PostgreSQL....
HIH, Gianluca _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2RAVXSCEPPCWLS...
I've been using oVirtBackup in the past few years and also I've done more restores from this backup than I'd have liked. All restores were successful. I'm pretty happy with this script: https://github.com/wefixit-AT/oVirtBackup Well documented, easy to install and configure. In summary the script uses the API to take a snapshot, convert it to a VM, export the VM, remove the snapshot. It also removes old backups, checks for free storage space, and so on. It's basically set and forget. I just wrote my own little bash wrapper that will grab the output of the script and email it to me. Hope this helps, -- Peter On Mon, May 13, 2019 at 8:12 AM Jayme <jaymef@gmail.com> wrote:
I know that a robust API is provided for performing VM backup functions, but it seems crazy to me that oVirt has not implemented a simple method to backup VM to a storage domain such as an NFS export domain. Is this functionality being left out for a reason (perhaps not wanting to step on toes of 3rd party software vendors providing backup solutions for oVirt/RHEV)?
On Mon, May 13, 2019 at 11:21 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Mon, May 13, 2019 at 3:47 PM Derek Atkins <derek@ihtfp.com> wrote:
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM.
Actually doable, depending on DB architecture and if some "logging/archiviing" supported/enabled on database. See here related threads where I was involved:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJFQOLIWU224R2... and here:
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/J65N7E2PY3KWUCZ...
When you install qemu-guest-agent (in my case version 0.12.1.2-2.506.el6_10.1) you also get an example for MySQL in /usr/share/qemu-kvm/qemu-ga/fsfreeze-hook.d/mysql-flush.sh.sample and eventually updated code (actually I see it is dated 2013...) here:
https://github.com/guillon/qemu-plugins/blob/master/scripts/qemu-guest-agent...
In case of PostgreSQL you would need to have WAL enabled and do an Exclusive Low Level Backup, because the pre-snapshot and post-snapshot could not be inside the same RDBMS session connection:
https://www.postgresql.org/docs/10/continuous-archiving.html#BACKUP-BASE-BAC...
so the freeze option of your freeze-hook script would contain a call to a connection that runs SELECT pg_start_backup('label', true); to minimize time to do checkpoint
and the thaw option would contain SELECT pg_stop_backup();
This needs also that archives during the elapsed of backup (and the last generaed when stopping backup) are copied in some way externally in respect of the VM OS for complete recoverability, if you don't already have it for your standard archiving policy.
And thoroughly test the whole workflow, both backups and restores, especially when upgrading your software layers, oVirt, VM OS, PostgreSQL....
HIH, Gianluca _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2RAVXSCEPPCWLS...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7DVSYQLZ6DGVPG...
Works for smaller vms but that process is pretty inefficient imo. It had to clone the vm then export it, takes a lot of space and time. I like the way vprotect does it, snapshot then streams the image via the API. There’s no cloning to another vm step and also handles incremental backups of vms as well so incremental backups take a fraction of time. On Mon, May 13, 2019 at 3:04 PM Wood Peter <peterwood.sd@gmail.com> wrote:
I've been using oVirtBackup in the past few years and also I've done more restores from this backup than I'd have liked. All restores were successful. I'm pretty happy with this script:
https://github.com/wefixit-AT/oVirtBackup
Well documented, easy to install and configure.
In summary the script uses the API to take a snapshot, convert it to a VM, export the VM, remove the snapshot. It also removes old backups, checks for free storage space, and so on.
It's basically set and forget.
I just wrote my own little bash wrapper that will grab the output of the script and email it to me.
Hope this helps,
-- Peter
On Mon, May 13, 2019 at 8:12 AM Jayme <jaymef@gmail.com> wrote:
I know that a robust API is provided for performing VM backup functions, but it seems crazy to me that oVirt has not implemented a simple method to backup VM to a storage domain such as an NFS export domain. Is this functionality being left out for a reason (perhaps not wanting to step on toes of 3rd party software vendors providing backup solutions for oVirt/RHEV)?
On Mon, May 13, 2019 at 11:21 AM Gianluca Cecchi < gianluca.cecchi@gmail.com> wrote:
On Mon, May 13, 2019 at 3:47 PM Derek Atkins <derek@ihtfp.com> wrote:
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM.
Actually doable, depending on DB architecture and if some "logging/archiviing" supported/enabled on database. See here related threads where I was involved:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJFQOLIWU224R2... and here:
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/J65N7E2PY3KWUCZ...
When you install qemu-guest-agent (in my case version 0.12.1.2-2.506.el6_10.1) you also get an example for MySQL in /usr/share/qemu-kvm/qemu-ga/fsfreeze-hook.d/mysql-flush.sh.sample and eventually updated code (actually I see it is dated 2013...) here:
https://github.com/guillon/qemu-plugins/blob/master/scripts/qemu-guest-agent...
In case of PostgreSQL you would need to have WAL enabled and do an Exclusive Low Level Backup, because the pre-snapshot and post-snapshot could not be inside the same RDBMS session connection:
https://www.postgresql.org/docs/10/continuous-archiving.html#BACKUP-BASE-BAC...
so the freeze option of your freeze-hook script would contain a call to a connection that runs SELECT pg_start_backup('label', true); to minimize time to do checkpoint
and the thaw option would contain SELECT pg_stop_backup();
This needs also that archives during the elapsed of backup (and the last generaed when stopping backup) are copied in some way externally in respect of the VM OS for complete recoverability, if you don't already have it for your standard archiving policy.
And thoroughly test the whole workflow, both backups and restores, especially when upgrading your software layers, oVirt, VM OS, PostgreSQL....
HIH, Gianluca _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2RAVXSCEPPCWLS...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/7DVSYQLZ6DGVPG...
On Tue, May 14, 2019 at 1:19 AM Jayme <jaymef@gmail.com> wrote:
Works for smaller vms but that process is pretty inefficient imo. It had to clone the vm then export it, takes a lot of space and time. I like the way vprotect does it, snapshot then streams the image via the API. There’s no cloning to another vm step and also handles incremental backups of vms as well so incremental backups take a fraction of time.
I am not following this closely, but it seems like CBT is intended to be included in a later 4.3 release, see e.g.: https://ovirt.org/develop/release-management/features/storage/incremental-ba... https://bugzilla.redhat.com/show_bug.cgi?id=1217820
On Mon, May 13, 2019 at 3:04 PM Wood Peter <peterwood.sd@gmail.com> wrote:
I've been using oVirtBackup in the past few years and also I've done more restores from this backup than I'd have liked. All restores were successful. I'm pretty happy with this script:
https://github.com/wefixit-AT/oVirtBackup
Well documented, easy to install and configure.
In summary the script uses the API to take a snapshot, convert it to a VM, export the VM, remove the snapshot. It also removes old backups, checks for free storage space, and so on.
It's basically set and forget.
I just wrote my own little bash wrapper that will grab the output of the script and email it to me.
Hope this helps,
-- Peter
On Mon, May 13, 2019 at 8:12 AM Jayme <jaymef@gmail.com> wrote:
I know that a robust API is provided for performing VM backup functions, but it seems crazy to me that oVirt has not implemented a simple method to backup VM to a storage domain such as an NFS export domain.
oVirt does provide an _example_ python sdk script to do that, in case you want to hack something of your own around it: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/vm_backup...
Is this functionality being left out for a reason (perhaps not wanting to step on toes of 3rd party software vendors providing backup solutions for oVirt/RHEV)?
I do not think oVirt should do _everything_, do you? It should allow easy integration with other tools. And it does, IMO.
On Mon, May 13, 2019 at 11:21 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 13, 2019 at 3:47 PM Derek Atkins <derek@ihtfp.com> wrote:
Strahil <hunter86_bg@yahoo.com> writes:
Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state.
Yet another reason to do it from inside the VM.
Actually doable, depending on DB architecture and if some "logging/archiviing" supported/enabled on database. See here related threads where I was involved: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJFQOLIWU224R2... and here: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/J65N7E2PY3KWUCZ...
When you install qemu-guest-agent (in my case version 0.12.1.2-2.506.el6_10.1) you also get an example for MySQL in /usr/share/qemu-kvm/qemu-ga/fsfreeze-hook.d/mysql-flush.sh.sample and eventually updated code (actually I see it is dated 2013...) here: https://github.com/guillon/qemu-plugins/blob/master/scripts/qemu-guest-agent...
In case of PostgreSQL you would need to have WAL enabled and do an Exclusive Low Level Backup, because the pre-snapshot and post-snapshot could not be inside the same RDBMS session connection: https://www.postgresql.org/docs/10/continuous-archiving.html#BACKUP-BASE-BAC...
so the freeze option of your freeze-hook script would contain a call to a connection that runs SELECT pg_start_backup('label', true); to minimize time to do checkpoint
and the thaw option would contain SELECT pg_stop_backup();
This needs also that archives during the elapsed of backup (and the last generaed when stopping backup) are copied in some way externally in respect of the VM OS for complete recoverability, if you don't already have it for your standard archiving policy.
And thoroughly test the whole workflow, both backups and restores, especially when upgrading your software layers, oVirt, VM OS, PostgreSQL....
HIH, Gianluca _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2RAVXSCEPPCWLS...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7DVSYQLZ6DGVPG...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UQYP3CEIUFJAOW...
-- Didi
participants (6)
-
Derek Atkins -
Gianluca Cecchi -
Jayme -
Strahil -
Wood Peter -
Yedidyah Bar David