Thanks Jayme, much appreciated! On 10.02.2020 16:59, Jayme wrote: > I've been part of this mailing list for a while now and have received a > lot of great advice and help on various subjects. I read the list daily > and one thing I've noticed is that many users are curious about backup > options for oVirt (myself included). I wanted to share with the > community a solution I've come up with to easily backup multiple running > oVirt VMs to OVA format using some basic Ansible playbooks. I've put > together a blog post detailing the process which also includes links to > a Github repo containing the playbooks here: > https://blog.silverorange.com/backing-up-ovirt-vms-with-ansible-4c2fca8b3b43 > > Any feedback, suggestions or questions are welcome. I hope this > information is helpful. > > Thanks! > > - Jayme > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U65CV5A6WC6... > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/SLMGCTPHMQX...

On Mon, Feb 10, 2020 at 5:01 PM Jayme <jaymef(a)gmail.com&gt; wrote: > I've been part of this mailing list for a while now and have received a > lot of great advice and help on various subjects. I read the list daily and > one thing I've noticed is that many users are curious about backup options > for oVirt (myself included). I wanted to share with the community a > solution I've come up with to easily backup multiple running oVirt VMs to > OVA format using some basic Ansible playbooks. I've put together a blog > post detailing the process which also includes links to a Github repo > containing the playbooks here: > https://blog.silverorange.com/backing-up-ovirt-vms-with-ansible-4c2fca8b3b43 > > Any feedback, suggestions or questions are welcome. I hope this > information is helpful. > > Thanks! > > - Jayme > > Hi Jayme, sorry in advance for the long mail, where I try to give details; I don't know your Ansible experience. A very nice and clean article indeed, with useful details (apart from text not justified: I prefer it but YMMV) and pretty fair with vProtect work and also pros and cons of their solution. I met Pawel Maczka from vProtect during oVirt Summit last year and I was able to appreciate his kindness and skill and efforts in integrating with oVirt/RHV. That said, I have some suggestions for you. In the next days I could work on a similar need for a customer, so it will be nice to share efforts and hopefully results... ;-) This week I have not much time but if you can elaborate and test what below, we can share. 1) engine parameters you could use ansible vault to encrypt credential files, to have better security and so you can disclose the playbook files without having to care abut sensitive information In my case I put username, password, ovirt mgr fqdn, ovirt ca file all in a file and then encrypt it (and also engine database ones, see below). Then I create a securely protected vault file named "vault_file" where I store the vault password and then I recall the playbook with: ansible-playbook --vault-password-file=vault_file backup_ovirt_vms.yml Alternatively you are prompted for the vault password each time you run the playbook 2) The best practice of using the oVirt SSO token in Ansible is to use a block of kind: tasks: - name: Ansible block to export as OVA block: - name: Obtain SSO token using username/password credentials ovirt_auth: url: https://{{ url_name }}/ovirt-engine/api username: "{{ ovirt_username }}" password: "{{ ovirt_password }}" ca_file: "{{ ovirt_ca }}" - name: "Backup VMs" include_tasks: export_vm.yml loop: "{{ vms }}" always: - name: Revoke SSO token ovirt_auth: state: absent ovirt_auth: "{{ ovirt_auth }}" So that anyway, thanks to the "always" section, you are sure to revoke the token 3) To manage timing of export to ova that fires up and suddenly completes from an ansible job point of view. Possibly overkill, I don't know if ovirt_job module in any way could do the same, but I try to solve using the engine db Please note that engine db credentials are on engine inside file: /etc/ovirt-engine/engine.conf.d/10-setup-database.conf and by default you can connect remotely to the database with engine user and with password being encrypted over the network thanks to pg_hba.conf in directory /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf # TYPE DATABASE USER ADDRESS METHOD host engine engine 0.0.0.0/0 md5 There are two tables involved "job_subject_entity" table that you query and where entity_id should be the id of the VM you are exportng and from this table you get the related job_id "job" table where you query for the job_id matching from what you got from previous query (eventually to refine if you have more concurrent jobs running against your VM and you have to filter what interests to you, eg action_type= ExportVmToOva ... TBD) Eg: export running engine=> \x Expanded display is on. engine=> select * from job_subject_entity where entity_id='442a1321-e366-4ea2-81bc-cad6e860a517'; -[ RECORD 1 ]------------------------------------- job_id | 1d4797f3-b1f9-4c19-8c8d-fb8c019399b1 entity_id | 442a1321-e366-4ea2-81bc-cad6e860a517 entity_type | VM engine=> select * from job where job_id='1d4797f3-b1f9-4c19-8c8d-fb8c019399b1'; -[ RECORD 1 ]---------+------------------------------------------------------------------------------------------------------- job_id | 1d4797f3-b1f9-4c19-8c8d-fb8c019399b1 action_type | ExportVmToOva description | Exporting VM c8 as an OVA to /rhev/data-center/mnt/10.4.192.69:_export_ovirt/dump/c8.ova on Host ov301 status | STARTED owner_id | 58823863-00d4-0257-0094-0000000002f3 visible | t start_time | 2020-02-18 12:00:16.629+01 end_time | last_update_time | 2020-02-18 12:00:33.905+01 correlation_id | 2bc48fc7-4af7-4287-a090-7245bac6f84b is_external | f is_auto_cleared | t engine_session_seq_id | 1091 The status field is STARTED and the end_time has no value When completed: engine=> select * from job where job_id='1d4797f3-b1f9-4c19-8c8d-fb8c019399b1'; -[ RECORD 1 ]---------+------------------------------------------------------------------------------------------------------- job_id | 1d4797f3-b1f9-4c19-8c8d-fb8c019399b1 action_type | ExportVmToOva description | Exporting VM c8 as an OVA to /rhev/data-center/mnt/10.4.192.69:_export_ovirt/dump/c8.ova on Host ov301 status | FINISHED owner_id | 58823863-00d4-0257-0094-0000000002f3 visible | t start_time | 2020-02-18 12:00:16.629+01 end_time | 2020-02-18 12:11:04.795+01 last_update_time | 2020-02-18 12:11:04.795+01 correlation_id | 2bc48fc7-4af7-4287-a090-7245bac6f84b is_external | f is_auto_cleared | t engine_session_seq_id | 1091 engine=> The status field is FINISHED (or FAILED if something went wrong, to be tested, verified) and end_time has the completion (or failure perhaps, TBV) timestamp Please note that the records remains into the two tables for some minutes (it seems 15) after FINISHED status has been reached. From ansible point of view you use postgresql_query module. The export_vm.yml becomes sort of this where you register the ovirt_vm task result and then you monitor via postgresql queries - name: "Export VM to OVA" ovirt_vm: auth: "{{ ovirt_auth }}" name: "{{ item }}" state: exported cluster: "{{ cluster }}" export_ova: host: "{{ host }}" filename: "{{ item }}.{{ file_ext }}" directory: "{{ in_progress_path }}" register: export_result - name: Debug export vm as ova result debug: var: export_result - name: Monitor export job status include_tasks: query_task.yml vars: entity_id: "{{ export_result.id }}" Example of export_result registered variable is TASK [Debug export vm as ova result] ******************************************************************** ok: [localhost] => { "export_result": { "changed": true, "diff": { "after": {}, "before": {} }, "failed": false, "id": "442a1321-e366-4ea2-81bc-cad6e860a517", "vm": { . . . } } } You give the export_result.id as the value of entity_id variable to the included task The query_task.yml task file contains: --- - name: query job_subject_entity table postgresql_query: login_host: "{{ dbserver }}" db: "{{ db }}" login_user: "{{ username }}" login_password: "{{ password }}" query: "select * from job_subject_entity where entity_id='{{ entity_id }}'" register: job_id - name: debug query result debug: var: job_id - name: query job table postgresql_query: login_host: "{{ dbserver }}" db: "{{ db }}" login_user: "{{ username }}" login_password: "{{ password }}" query: "select * from job where job_id='{{ job_id.query_result[0].job_id }}'" register: job_status until: job_status.query_result[0].status == "FINISHED" retries: 15 delay: 60 - name: debug query result debug: var: job_status ... In my case I set 15 retries with delay of 60 seconds between one and another Here below a complete run for one vm (for which the export has an elapsed of about 10 minutes). After the line TASK [query job table] ***************************************************************************** you will get the prompt until the export has finished and filled with the lines of type FAILED - RETRYING: query job table (15 retries left). FAILED - RETRYING: query job table (14 retries left). ecc depending on delay and retries set. $ ansible-playbook --vault-password-file=vault_pw backup_ovirt_vms.yml PLAY [localhost] *********************************************************************************** TASK [Gathering Facts] ***************************************************************************** ok: [localhost] TASK [Obtain SSO token using username/password credentials] **************************************** ok: [localhost] TASK [Backup VMs] ********************************************************************************** included: /home/g.cecchi/ovirt/export_vm.yml for localhost TASK [Export VM to OVA] **************************************************************************** changed: [localhost] TASK [Debug export vm as ova result] *************************************************************** ok: [localhost] => { "export_result.id": "442a1321-e366-4ea2-81bc-cad6e860a517" } TASK [Monitor export job status] ******************************************************************* included: /home/g.cecchi/ovirt/query_task.yml for localhost TASK [query job_subject_entity table] ************************************************************** ok: [localhost] TASK [debug query result] ************************************************************************** ok: [localhost] => { "job_id": { "changed": false, "failed": false, "query": "select * from job_subject_entity where entity_id='442a1321-e366-4ea2-81bc-cad6e860a517'", "query_result": [ { "entity_id": "442a1321-e366-4ea2-81bc-cad6e860a517", "entity_type": "VM", "job_id": "8064059e-4b74-48a2-83be-a7dfd5067172" } ], "rowcount": 1, "statusmessage": "SELECT 1" } } TASK [query job table] ***************************************************************************** FAILED - RETRYING: query job table (15 retries left). FAILED - RETRYING: query job table (14 retries left). FAILED - RETRYING: query job table (13 retries left). FAILED - RETRYING: query job table (12 retries left). FAILED - RETRYING: query job table (11 retries left). FAILED - RETRYING: query job table (10 retries left). FAILED - RETRYING: query job table (9 retries left). FAILED - RETRYING: query job table (8 retries left). FAILED - RETRYING: query job table (7 retries left). FAILED - RETRYING: query job table (6 retries left). FAILED - RETRYING: query job table (5 retries left). ok: [localhost] TASK [debug query result] ************************************************************************** ok: [localhost] => { "job_status": { "attempts": 12, "changed": false, "failed": false, "query": "select * from job where job_id='8064059e-4b74-48a2-83be-a7dfd5067172'", "query_result": [ { "action_type": "ExportVmToOva", "correlation_id": "ed3d7454-c52c-46ce-bfbe-97ee8e1e885e", "description": "Exporting VM c8 as an OVA to /rhev/data-center/mnt/10.4.192.69:_export_ovirt/dump/c8.ova on Host ov301", "end_time": "2020-02-18T13:00:19.314000+01:00", "engine_session_seq_id": 1093, "is_auto_cleared": true, "is_external": false, "job_id": "8064059e-4b74-48a2-83be-a7dfd5067172", "last_update_time": "2020-02-18T13:00:19.314000+01:00", "owner_id": "58823863-00d4-0257-0094-0000000002f3", "start_time": "2020-02-18T12:49:36.862000+01:00", "status": "FINISHED", "visible": true } ], "rowcount": 1, "statusmessage": "SELECT 1" } } TASK [Revoke SSO token] **************************************************************************** ok: [localhost] PLAY RECAP ***************************************************************************************** localhost : ok=11 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 On host for these preliminary tests I'm using export domain, even if deprecated, so that I can use whatever host I want in the cluster and I have [root@ov301 dump]# ll total 35650236 -rw-------. 1 root root 21478390272 Feb 18 13:00 c8.ova I think there is a bug open about root permissions of this file here. Possibly they depend on way how export domain is mounted? https://bugzilla.redhat.com/show_bug.cgi?id=1645229 Improvements to intercept failures and in case the automatically created snapshot has not been deleted to manage it, consolidating... Let me know what do you think about it. HIH, Gianluca