- Users - oVirt List Archives

What happens when shared storage is down?
by Vinícius Ferrão 10 Jun '20

10 Jun '20

Hello, This is a pretty vague and difficult question to answer. But what happens if the shared storage holding the VMs is down or unavailable for a period of time? I’m aware that a longer timeout may put the VMs on pause state, but how this is handled? Is it a time limit? Requests limit? Who manages this? In an event of self recovery of the storage backend what happens next? Manual intervention is required? The VMs may be down or they just continue to run? It depends on the guest OS running like in XenServer where different scenarios may happen? I’ve looked here: https://www.ovirt.org/documentation/admin-guide/chap-Storage.html but there’s nothing that goes about this question. Thanks, Sent from my iPhone

3 4

Ovirt Manager HA
by Vijay Sachdeva 10 Jun '20

10 Jun '20

Hi All, Is there a documentation on creating Ovirt-Manager in HA or is it possible to make the manager to run in HA environment. Since it seems to be a single point of failure..!! Thanks Vijay Sachdeva

4 3

Cannot start ppc64le VM's
by Vinícius Ferrão 10 Jun '20

10 Jun '20

Hello, I’m trying to run ppc64le VM’s on POWER9 but qemu-kvm fails complaining about NUMA issues: VM ppc64le.local.versatushpc.com.br<http://ppc64le.local.versatushpc.com.br> is down with error. Exit message: internal error: qemu unexpectedly closed the monitor: 2020-06-05T06:16:10.716052Z qemu-kvm: warning: CPU(s) not present in any NUMA nodes: CPU 4 [core-id: 4], CPU 5 [core-id: 5], CPU 6 [core-id: 6], CPU 7 [core-id: 7], CPU 8 [core-id: 8], CPU 9 [core-id: 9], CPU 10 [core-id: 10], CPU 11 [core-id: 11], CPU 12 [core-id: 12], CPU 13 [core-id: 13], CPU 14 [core-id: 14], CPU 15 [core-id: 15] 2020-06-05T06:16:10.716067Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future 2020-06-05T06:16:11.155924Z qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs. Any idea of what’s happening? I found some links, but I’m not sure if they are related or not: https://bugzilla.redhat.com/show_bug.cgi?id=1732726 https://bugzilla.redhat.com/show_bug.cgi?id=1592648 Thanks,

2 5

Re: Global Maintenance
by Yedidyah Bar David 10 Jun '20

10 Jun '20

On Tue, Jun 9, 2020 at 8:03 PM Anton Louw <Anton.Louw(a)voxtelecom.co.za> wrote: > > > > Hi Didi, > > I followed the below steps in order to move the HE to a standalone engine: > > Backup and remove Hosted Engine: > > 1. Backup Hosted Engine (engine-backup --scope=all --mode=backup --file=Full --log=Log_Full) > 2. Download the backup files from HE using WinSCP > 3. Enable global Maintenance (hosted-engine --set-maintenance --mode=global) > 4. Power down hosted engine (hosted-engine --vm-shutdown) > > --------------------------------------------------------------------------------------------- > Redeploy Hosted Engine: > > Build new CentOS VM in another environment > *Same IP > *Same name > *Same resources > > 1. sudo yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm > 2. sudo yum install -y ovirt-engine > 3. Copy backup files to newly deployed CentOS VM > 4. engine-backup --mode=restore --file=Full --log=Log_Full --provision-db --provision-dwh-db --restore-permissions > 5. after restore has completed, run engine-setup > > Yes, I get “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode” error message when running anything to do with engine-setup. In this case, I would like to reconfigure the Websocket proxy. > > How will I go about updating the below: > > update vds_statistics set ha_global_maintenance=f; Please remember that this isn't tested. Please thoroughly test first on a test setup. One way to do that is: /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c 'update vds_statistics set ha_global_maintenance=f;' As I said, it's probably best to do this while the engine is down, to prevent caching issues etc. Best regards, > > Thank you > > Anton Louw > Cloud Engineer: Storage and Virtualization at Vox > ________________________________ > T: 087 805 0000 | D: 087 805 1572 > M: N/A > E: anton.louw(a)voxtelecom.co.za > A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > > > > > > Disclaimer > > The contents of this email are confidential to the sender and the intended recipient. Unless the contents are clearly and entirely of a personal nature, they are subject to copyright in favour of the holding company of the Vox group of companies. Any recipient who receives this email in error should immediately report the error to the sender and permanently delete this email from all storage devices. > > This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here. > > > > > -----Original Message----- > From: Yedidyah Bar David <didi(a)redhat.com> > Sent: 09 June 2020 15:19 > To: Anton Louw <Anton.Louw(a)voxtelecom.co.za> > Cc: users(a)ovirt.org > Subject: Re: [ovirt-users] Global Maintenance > > On Tue, Jun 9, 2020 at 9:39 AM Anton Louw via Users <users(a)ovirt.org> wrote: > > > > > > > > Hi Everybody, > > > > > > > > Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues. > > How exactly did you do that? > > > Just one thing, when I want to reconfigure settings on my self-hosted engine, it says: > > > > “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode” > > This message is in engine-setup, right? > > engine-setup's code is checking this by: > > /usr/share/ovirt-engine/setup/plugins/ovirt-engine-common/ovirt-engine/system/he.py > > SELECT vm_guid, run_on_vds > FROM vms > WHERE vm_name = %(HostedEngineVmName)s; > > where HostedEngineVmName is either the option of same name from vdc_options, or 'HostedEngine' if missing. Then, where VdsId is 'run_on_vds' column of the result of above: > > SELECT vds_id, ha_global_maintenance > FROM vds_statistics > WHERE vds_id = %(VdsId)s; > > So if this isn't a hosted-engine setup anymore, it should probably be safe to do: > > update vds_statistics set ha_global_maintenance=f; > > and perhaps restart the engine (in case it also does similar checks itself but caches this data). > > Please note that I didn't try this myself. > > > > > > > > > When trying to enable global maintenance from my node, I also get the below: > > > > > > > > “Cannot connect to the HA daemon, please check the logs” > > This is expected, no? You say it's not a hosted-engine setup anymore. > > > > > > > > > Is there something I missed when I moved my self-hosted engine to a standalone manager? > > Either Yes, or the procedure you followed is incomplete. If latter, please file a bug about it. Thanks! > > Best regards, > -- > Didi -- Didi

1 0

Re: Global Maintenance
by Yedidyah Bar David 09 Jun '20

09 Jun '20

On Tue, Jun 9, 2020 at 9:39 AM Anton Louw via Users <users(a)ovirt.org> wrote: > > > > Hi Everybody, > > > > Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues. How exactly did you do that? > Just one thing, when I want to reconfigure settings on my self-hosted engine, it says: > > “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode” This message is in engine-setup, right? engine-setup's code is checking this by: /usr/share/ovirt-engine/setup/plugins/ovirt-engine-common/ovirt-engine/system/he.py SELECT vm_guid, run_on_vds FROM vms WHERE vm_name = %(HostedEngineVmName)s; where HostedEngineVmName is either the option of same name from vdc_options, or 'HostedEngine' if missing. Then, where VdsId is 'run_on_vds' column of the result of above: SELECT vds_id, ha_global_maintenance FROM vds_statistics WHERE vds_id = %(VdsId)s; So if this isn't a hosted-engine setup anymore, it should probably be safe to do: update vds_statistics set ha_global_maintenance=f; and perhaps restart the engine (in case it also does similar checks itself but caches this data). Please note that I didn't try this myself. > > > > When trying to enable global maintenance from my node, I also get the below: > > > > “Cannot connect to the HA daemon, please check the logs” This is expected, no? You say it's not a hosted-engine setup anymore. > > > > Is there something I missed when I moved my self-hosted engine to a standalone manager? Either Yes, or the procedure you followed is incomplete. If latter, please file a bug about it. Thanks! Best regards, -- Didi

2 1

ovirt vm backup tool
by Ayansh Rocks 09 Jun '20

09 Jun '20

Hi Guys, Can you please suggest a good backup tool for ovirt virtual machines..? Thanks Shashank

4 3

Power Management on IBM AC922 Power9 (ppc64le)
by Vinícius Ferrão 09 Jun '20

09 Jun '20

Hello, I would like to know how to enable Power Management on AC922 hardware from IBM. It’s ppc64le architecture and runs OpenBMC as manager. I only get Test failed: Internal JSON-RPC error when adding the infos with ipmilan on the engine. From the command line I can use ipmitool but without specifying any user. On the Engine I must specify an user. There’s no way to leave it blank. Thanks,

4 4

First ovirt 4.4 installation failing
by wart＠caltech.edu 09 Jun '20

09 Jun '20

I'm having some trouble setting up my first oVirt system. I have the CentOS 8 installation on the bare metal (ovirt1.ldas.ligo-la.caltech.edu) the ovirt4.4 packages installed, and then try running 'hosted-engine --deploy' to set up my engine (ovirt-engine1.ldas.ligo-la.caltech.edu) For this initial deployment, I accept almost all of the defaults (other than local network-specific settings). However, the hosted-engine deployment fails with: [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/pass word credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait for the host to be up] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": false, "ov irt_hosts": []} [...cleanup...] [ INFO ] TASK [ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} However, when I run 'virsh list', I can still see a HostedEngine1 vm running. In virt-hosted-engine-setup-20200522153439-e7iw3k.log I see the error: 2020-05-25 11:57:03,897-0500 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:103 {'changed': False, 'ovirt_hosts': [], 'invocation': {'module_args': {'pattern': 'name=ovirt1.ldas.ligo-la.caltech.edu', 'fetch_nested': False, 'nested_attributes': [], 'all_content': False, 'cluster_version': None}}, '_ansible_no_log': False, 'attempts': 120} 2020-05-25 11:57:03,998-0500 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:107 fatal: [localhost]: FAILED! => {"attempts": 120, "changed": false, "ovirt_hosts": []} In ovirt-hosted-engine-setup-ansible-bootstrap_local_vm-20200525112504-y2mmzu.log I see the following ansible errors: 2020-05-25 11:36:22,300-0500 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Always revoke the SSO token kwargs 2020-05-25 11:36:23,766-0500 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "invocation": { "module_args": { "ca_file": null, "compress": true, "headers": null, "hostname": null, "insecure": null, "kerberos": false, "ovirt_auth": { "ansible_facts": { "ovirt_auth": { "ca_file": null, "compress": true, "headers": null, "insecure": true, "kerberos": false, "timeout": 0, "token": "tF4ZMU0Q23zS13W2vzyhkswGMB4XAXZCFiPg9IVvbJXkPq9MFmne40wvCKaQOJO_TkYOpfxe78r9HHJcSrUWCQ", "url": "https://ovirt-engine1.ldas.ligo-la.caltech.edu/ovirt-engine/api" } }, "attempts": 1, "changed": false, "failed": false }, "password": null, "state": "absent", "timeout": 0, "token": null, "url": null, "username": null } }, "msg": "You must specify either 'url' or 'hostname'." }, "ansible_task": "Always revoke the SSO token", "ansible_type": "task", "status": "FAILED", "task_duration": 2 } 2020-05-25 11:36:23,767-0500 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f15adaffa58> kwargs ignore_errors:True Then further down: 2020-05-25 11:57:05,063-0500 DEBUG var changed: host "localhost" var "ansible_failed_result" type "<class 'dict'>" value: "{ "_ansible_no_log": false, "_ansible_parsed": true, "attempts": 120, "changed": false, "failed": true, "invocation": { "module_args": { "all_content": false, "cluster_version": null, "fetch_nested": false, "nested_attributes": [], "pattern": "name=ovirt1.ldas.ligo-la.caltech.edu" } }, "ovirt_hosts": [] }" 2020-05-25 11:57:05,063-0500 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "attempts": 120, "changed": false, "invocation": { "module_args": { "all_content": false, "cluster_version": null, "fetch_nested": false, "nested_attributes": [], "pattern": "name=ovirt1.ldas.ligo-la.caltech.edu" } }, "ovirt_hosts": [] }, "ansible_task": "Wait for the host to be up", "ansible_type": "task", "status": "FAILED", "task_duration": 1235 } 2020-05-25 11:57:05,063-0500 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f15ad92dcc0> kwargs ignore_errors:None Not being very familiar with ansible, I'm not sure where to look next for the root cause of the problem. --Michael Thomas

5 11

Global Maintenance
by Anton Louw 09 Jun '20

09 Jun '20

Hi Everybody, Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues. Just one thing, when I want to reconfigure settings on my self-hosted engine, it says: "It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode" When trying to enable global maintenance from my node, I also get the below: "Cannot connect to the HA daemon, please check the logs" Is there something I missed when I moved my self-hosted engine to a standalone manager? Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

1 0

Upgrade from 4.3.7 to 4.4 report (through scratch reinstall)
by Gianluca Cecchi 09 Jun '20

09 Jun '20

Just to give a report of final success updating from 4.3.7 to 4.4 one of my home labs, composed by a single host. hw: Intel NUC6i5SY with 32G of ram and 2 SSD disks (250Gb and 500Gb) source sw: oVirt 4.3.7 single host with CentOS 7 OS and storage provided through the host itself via NFS (not officially supported, but working, apart when shutdown needed) Two main VMs to migrate to the new environment: Fedora 30 and Slackware Current (just to not forget the first love... ;-) Exported the VMs to an export storage domain offered through an external USB disk dest sw: oVirt node ng 4.4 configured with HCI single host wizard. Installed on the 250Gb disk. I pre-clean the disks (dd of the first 100Mb of the disks) before install, because during the beta/rc phase I noticed the installer was not so smart to cleanup pre-existing configurations. I had problems in the first run, but with engine cleanup and redeploying it went ok. See here for more details: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/6QODLB6J5Z74YC… I configured the gluster domains in the wizard on the whole second disk. I then imported the 2 VMs without problems. Before starting them up I changed their inherited "BIOS Type" from Legacy to "Default Cluster" and they both started without any problem. While in 4.4 I was able to update Fedora VM from 30 to 31 and then 32 and also refresh the slackware-current one that was about a month behind from latest current. With "Default Cluster" BIOS type the VMs start with the following options: " -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu Broadwell-noTSX" I was then able to download a CentOS 8 cloud image from the predefined ovirt-image-repository storage domain and convert it to template. Created a VM from this template and cloud-init was able to inject the ssh public key and set timezone. Changed cluster type of ths VM to Default as above with a warning from oVirt but without any problem inside the VM and also changed the disk type from virtio to virtio-scsi without problems. Now I enabled incremental backup at engine level and also at disk level of this CentOS 8 VM so that I can make some tests in this regard. Also, I was able to successfully test the ovirt-ansible-shutdown-env ansible role to make a clean overall shutdown of the environment, one of the things that in my previous unsupported setup was a little cumbersome. Right after install I noticed that in CentOS 8 ovirt-node-ng setup, intel_pstate was the default scaling driver setup, but my cpu was almost always crying with core at 2.6Ghz (and temp around 90 degrees) even if attempted to setup conservative profiles and without nothing running, apart engine VM. The NUC is under my working day table and I don't need always performance from it... So I modified (notice also the rhgb and quite omissions...) these files and reboot: - /etc/default/grub GRUB_CMDLINE_LINUX='crashkernel=auto resume=/dev/mapper/onn-swap rd.lvm.lv=onn/ovirt-node-ng-4.4.0-0.20200521.0+1 rd.lvm.lv=onn/swap intel_pstate=disable' - /boot/grub2/grub.cfg set default_kernelopts="root=UUID=85212719-8feb-43aa-9819-2820d4672795 ro crashkernel=auto ipv6.disable=1 intel_pstate=disable " - /boot/loader/entries/ovirt-node-ng-4.4.0-0.20200521.0+1-4.18.0-147.8.1.el8_1.x86_64.conf options intel_pstate=disable boot=UUID=b717ab4f-ca71-469a-8836-ff92cebc7650 crashkernel=auto rd.lvm.lv=onn/swap root=/dev/onn/ovirt-node-ng-4.4.0-0.20200521.0+1 resume=/dev/mapper/onn-swap rootflags=discard rd.lvm.lv=onn/ovirt-node-ng-4.4.0-0.20200521.0+1 img.bootid=ovirt-node-ng-4.4.0-0.20200521.0+1 null - /boot/efi/EFI/centos/grub.cfg set default_kernelopts="root=/dev/mapper/onn-root ro crashkernel=auto resume=/dev/mapper/onn-swap rd.lvm.lv=onn/root rd.lvm.lv=onn/swap intel_pstate=disable " And now with the default/old scaling driver all is quite silent and still working ok for my needs, with web admin quite usable. I updated my two VMs as described above after cpompleting these changes. Right now for example I have hosted engine running + slackware one + CentOS 8 one with [root@ovirt01 ~]# cat /proc/cpuinfo | grep Hz model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz cpu MHz : 648.431 model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz cpu MHz : 628.307 model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz cpu MHz : 648.859 model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz cpu MHz : 663.792 [root@ovirt01 ~]# and [root@ovirt01 ~# cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor powersave powersave powersave powersave [root@ovirt01 g.cecchi]# Thanks to all that helped throughout ... Cheers, Gianluca

2 2