Cannot start ppc64le VM's
by Vinícius Ferrão
Hello, I’m trying to run ppc64le VM’s on POWER9 but qemu-kvm fails complaining about NUMA issues:
VM ppc64le.local.versatushpc.com.br<http://ppc64le.local.versatushpc.com.br> is down with error. Exit message: internal error: qemu unexpectedly closed the monitor: 2020-06-05T06:16:10.716052Z qemu-kvm: warning: CPU(s) not present in any NUMA nodes: CPU 4 [core-id: 4], CPU 5 [core-id: 5], CPU 6 [core-id: 6], CPU 7 [core-id: 7], CPU 8 [core-id: 8], CPU 9 [core-id: 9], CPU 10 [core-id: 10], CPU 11 [core-id: 11], CPU 12 [core-id: 12], CPU 13 [core-id: 13], CPU 14 [core-id: 14], CPU 15 [core-id: 15] 2020-06-05T06:16:10.716067Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future 2020-06-05T06:16:11.155924Z qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs.
Any idea of what’s happening?
I found some links, but I’m not sure if they are related or not:
https://bugzilla.redhat.com/show_bug.cgi?id=1732726
https://bugzilla.redhat.com/show_bug.cgi?id=1592648
Thanks,
4 years, 5 months
Re: Global Maintenance
by Yedidyah Bar David
On Tue, Jun 9, 2020 at 8:03 PM Anton Louw <Anton.Louw(a)voxtelecom.co.za> wrote:
>
>
>
> Hi Didi,
>
> I followed the below steps in order to move the HE to a standalone engine:
>
> Backup and remove Hosted Engine:
>
> 1. Backup Hosted Engine (engine-backup --scope=all --mode=backup --file=Full --log=Log_Full)
> 2. Download the backup files from HE using WinSCP
> 3. Enable global Maintenance (hosted-engine --set-maintenance --mode=global)
> 4. Power down hosted engine (hosted-engine --vm-shutdown)
>
> ---------------------------------------------------------------------------------------------
> Redeploy Hosted Engine:
>
> Build new CentOS VM in another environment
> *Same IP
> *Same name
> *Same resources
>
> 1. sudo yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm
> 2. sudo yum install -y ovirt-engine
> 3. Copy backup files to newly deployed CentOS VM
> 4. engine-backup --mode=restore --file=Full --log=Log_Full --provision-db --provision-dwh-db --restore-permissions
> 5. after restore has completed, run engine-setup
>
> Yes, I get “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode” error message when running anything to do with engine-setup. In this case, I would like to reconfigure the Websocket proxy.
>
> How will I go about updating the below:
>
> update vds_statistics set ha_global_maintenance=f;
Please remember that this isn't tested. Please thoroughly test first
on a test setup.
One way to do that is:
/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c 'update
vds_statistics set ha_global_maintenance=f;'
As I said, it's probably best to do this while the engine is down, to
prevent caching issues etc.
Best regards,
>
> Thank you
>
> Anton Louw
> Cloud Engineer: Storage and Virtualization at Vox
> ________________________________
> T: 087 805 0000 | D: 087 805 1572
> M: N/A
> E: anton.louw(a)voxtelecom.co.za
> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg
> www.vox.co.za
>
>
>
>
>
>
>
>
> Disclaimer
>
> The contents of this email are confidential to the sender and the intended recipient. Unless the contents are clearly and entirely of a personal nature, they are subject to copyright in favour of the holding company of the Vox group of companies. Any recipient who receives this email in error should immediately report the error to the sender and permanently delete this email from all storage devices.
>
> This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.
>
>
>
>
> -----Original Message-----
> From: Yedidyah Bar David <didi(a)redhat.com>
> Sent: 09 June 2020 15:19
> To: Anton Louw <Anton.Louw(a)voxtelecom.co.za>
> Cc: users(a)ovirt.org
> Subject: Re: [ovirt-users] Global Maintenance
>
> On Tue, Jun 9, 2020 at 9:39 AM Anton Louw via Users <users(a)ovirt.org> wrote:
> >
> >
> >
> > Hi Everybody,
> >
> >
> >
> > Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues.
>
> How exactly did you do that?
>
> > Just one thing, when I want to reconfigure settings on my self-hosted engine, it says:
> >
> > “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode”
>
> This message is in engine-setup, right?
>
> engine-setup's code is checking this by:
>
> /usr/share/ovirt-engine/setup/plugins/ovirt-engine-common/ovirt-engine/system/he.py
>
> SELECT vm_guid, run_on_vds
> FROM vms
> WHERE vm_name = %(HostedEngineVmName)s;
>
> where HostedEngineVmName is either the option of same name from vdc_options, or 'HostedEngine' if missing. Then, where VdsId is 'run_on_vds' column of the result of above:
>
> SELECT vds_id, ha_global_maintenance
> FROM vds_statistics
> WHERE vds_id = %(VdsId)s;
>
> So if this isn't a hosted-engine setup anymore, it should probably be safe to do:
>
> update vds_statistics set ha_global_maintenance=f;
>
> and perhaps restart the engine (in case it also does similar checks itself but caches this data).
>
> Please note that I didn't try this myself.
>
> >
> >
> >
> > When trying to enable global maintenance from my node, I also get the below:
> >
> >
> >
> > “Cannot connect to the HA daemon, please check the logs”
>
> This is expected, no? You say it's not a hosted-engine setup anymore.
>
> >
> >
> >
> > Is there something I missed when I moved my self-hosted engine to a standalone manager?
>
> Either Yes, or the procedure you followed is incomplete. If latter, please file a bug about it. Thanks!
>
> Best regards,
> --
> Didi
--
Didi
4 years, 5 months
Re: Global Maintenance
by Yedidyah Bar David
On Tue, Jun 9, 2020 at 9:39 AM Anton Louw via Users <users(a)ovirt.org> wrote:
>
>
>
> Hi Everybody,
>
>
>
> Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues.
How exactly did you do that?
> Just one thing, when I want to reconfigure settings on my self-hosted engine, it says:
>
> “It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode”
This message is in engine-setup, right?
engine-setup's code is checking this by:
/usr/share/ovirt-engine/setup/plugins/ovirt-engine-common/ovirt-engine/system/he.py
SELECT vm_guid, run_on_vds
FROM vms
WHERE vm_name = %(HostedEngineVmName)s;
where HostedEngineVmName is either the option of same name from
vdc_options, or 'HostedEngine' if missing. Then, where VdsId is
'run_on_vds' column of the result of above:
SELECT vds_id, ha_global_maintenance
FROM vds_statistics
WHERE vds_id = %(VdsId)s;
So if this isn't a hosted-engine setup anymore, it should probably be
safe to do:
update vds_statistics set ha_global_maintenance=f;
and perhaps restart the engine (in case it also does similar checks
itself but caches this data).
Please note that I didn't try this myself.
>
>
>
> When trying to enable global maintenance from my node, I also get the below:
>
>
>
> “Cannot connect to the HA daemon, please check the logs”
This is expected, no? You say it's not a hosted-engine setup anymore.
>
>
>
> Is there something I missed when I moved my self-hosted engine to a standalone manager?
Either Yes, or the procedure you followed is incomplete. If latter,
please file a bug about it. Thanks!
Best regards,
--
Didi
4 years, 5 months
ovirt vm backup tool
by Ayansh Rocks
Hi Guys,
Can you please suggest a good backup tool for ovirt virtual machines..?
Thanks
Shashank
4 years, 5 months
Power Management on IBM AC922 Power9 (ppc64le)
by Vinícius Ferrão
Hello,
I would like to know how to enable Power Management on AC922 hardware from IBM. It’s ppc64le architecture and runs OpenBMC as manager.
I only get Test failed: Internal JSON-RPC error when adding the infos with ipmilan on the engine. From the command line I can use ipmitool but without specifying any user. On the Engine I must specify an user. There’s no way to leave it blank.
Thanks,
4 years, 5 months
First ovirt 4.4 installation failing
by wart＠caltech.edu
I'm having some trouble setting up my first oVirt system. I have the CentOS 8 installation on the bare metal (ovirt1.ldas.ligo-la.caltech.edu), the ovirt4.4 packages installed, and then try running 'hosted-engine --deploy' to set up my engine (ovirt-engine1.ldas.ligo-la.caltech.edu). For this initial deployment, I accept almost all of the defaults (other than local network-specific settings). However, the hosted-engine deployment fails with:
[ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/pass
word credentials]
[ INFO ] ok: [localhost]
[ INFO ] TASK [ovirt.hosted_engine_setup : Wait for the host to be up]
[ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": false, "ov
irt_hosts": []}
[...cleanup...]
[ INFO ] TASK [ovirt.hosted_engine_setup : Notify the user about a failure]
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"}
However, when I run 'virsh list', I can still see a HostedEngine1 vm running.
In virt-hosted-engine-setup-20200522153439-e7iw3k.log I see the error:
2020-05-25 11:57:03,897-0500 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:103 {'changed': False, 'ovirt_hosts': [], 'invocation': {'module_args': {'pattern': 'name=ovirt1.ldas.ligo-la.caltech.edu', 'fetch_nested': False, 'nested_attributes': [], 'all_content': False, 'cluster_version': None}}, '_ansible_no_log': False, 'attempts': 120}
2020-05-25 11:57:03,998-0500 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:107 fatal: [localhost]: FAILED! => {"attempts": 120, "changed": false, "ovirt_hosts": []}
In ovirt-hosted-engine-setup-ansible-bootstrap_local_vm-20200525112504-y2mmzu.log I see the following ansible errors:
2020-05-25 11:36:22,300-0500 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Always revoke the SSO token kwargs
2020-05-25 11:36:23,766-0500 ERROR ansible failed {
"ansible_host": "localhost",
"ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
"ansible_result": {
"_ansible_no_log": false,
"changed": false,
"invocation": {
"module_args": {
"ca_file": null,
"compress": true,
"headers": null,
"hostname": null,
"insecure": null,
"kerberos": false,
"ovirt_auth": {
"ansible_facts": {
"ovirt_auth": {
"ca_file": null,
"compress": true,
"headers": null,
"insecure": true,
"kerberos": false,
"timeout": 0,
"token": "tF4ZMU0Q23zS13W2vzyhkswGMB4XAXZCFiPg9IVvbJXkPq9MFmne40wvCKaQOJO_TkYOpfxe78r9HHJcSrUWCQ",
"url": "https://ovirt-engine1.ldas.ligo-la.caltech.edu/ovirt-engine/api"
}
},
"attempts": 1,
"changed": false,
"failed": false
},
"password": null,
"state": "absent",
"timeout": 0,
"token": null,
"url": null,
"username": null
}
},
"msg": "You must specify either 'url' or 'hostname'."
},
"ansible_task": "Always revoke the SSO token",
"ansible_type": "task",
"status": "FAILED",
"task_duration": 2
}
2020-05-25 11:36:23,767-0500 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f15adaffa58> kwargs ignore_errors:True
Then further down:
2020-05-25 11:57:05,063-0500 DEBUG var changed: host "localhost" var "ansible_failed_result" type "<class 'dict'>" value: "{
"_ansible_no_log": false,
"_ansible_parsed": true,
"attempts": 120,
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"all_content": false,
"cluster_version": null,
"fetch_nested": false,
"nested_attributes": [],
"pattern": "name=ovirt1.ldas.ligo-la.caltech.edu"
}
},
"ovirt_hosts": []
}"
2020-05-25 11:57:05,063-0500 ERROR ansible failed {
"ansible_host": "localhost",
"ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
"ansible_result": {
"_ansible_no_log": false,
"attempts": 120,
"changed": false,
"invocation": {
"module_args": {
"all_content": false,
"cluster_version": null,
"fetch_nested": false,
"nested_attributes": [],
"pattern": "name=ovirt1.ldas.ligo-la.caltech.edu"
}
},
"ovirt_hosts": []
},
"ansible_task": "Wait for the host to be up",
"ansible_type": "task",
"status": "FAILED",
"task_duration": 1235
}
2020-05-25 11:57:05,063-0500 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f15ad92dcc0> kwargs ignore_errors:None
Not being very familiar with ansible, I'm not sure where to look next for the root cause of the problem.
--Michael Thomas
4 years, 5 months
Global Maintenance
by Anton Louw
Hi Everybody,
Just a bit of background. A couple of weeks back I moved my self-hosted engine to a standalone manager. Everything went smooth, my environment is up and running without any issues. Just one thing, when I want to reconfigure settings on my self-hosted engine, it says:
"It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode"
When trying to enable global maintenance from my node, I also get the below:
"Cannot connect to the HA daemon, please check the logs"
Is there something I missed when I moved my self-hosted engine to a standalone manager?
Anton Louw
Cloud Engineer: Storage and Virtualization
______________________________________
D: 087 805 1572 | M: N/A
A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg
anton.louw(a)voxtelecom.co.za
www.vox.co.za
4 years, 5 months
Upgrade from 4.3.7 to 4.4 report (through scratch reinstall)
by Gianluca Cecchi
Just to give a report of final success updating from 4.3.7 to 4.4 one of my
home labs, composed by a single host.
hw: Intel NUC6i5SY with 32G of ram and 2 SSD disks (250Gb and 500Gb)
source sw: oVirt 4.3.7 single host with CentOS 7 OS and storage provided
through the host itself via NFS (not officially supported, but working,
apart when shutdown needed)
Two main VMs to migrate to the new environment: Fedora 30 and Slackware
Current (just to not forget the first love... ;-)
Exported the VMs to an export storage domain offered through an external
USB disk
dest sw: oVirt node ng 4.4 configured with HCI single host wizard.
Installed on the 250Gb disk. I pre-clean the disks (dd of the first 100Mb
of the disks) before install, because during the beta/rc phase I noticed
the installer was not so smart to cleanup pre-existing configurations.
I had problems in the first run, but with engine cleanup and redeploying it
went ok.
See here for more details:
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/6QODLB6J5Z74...
I configured the gluster domains in the wizard on the whole second disk.
I then imported the 2 VMs without problems. Before starting them up I
changed their inherited "BIOS Type" from Legacy to "Default Cluster" and
they both started without any problem.
While in 4.4 I was able to update Fedora VM from 30 to 31 and then 32 and
also refresh the slackware-current one that was about a month behind from
latest current.
With "Default Cluster" BIOS type the VMs start with the following options:
" -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off -cpu
Broadwell-noTSX"
I was then able to download a CentOS 8 cloud image from the predefined
ovirt-image-repository storage domain and convert it to template.
Created a VM from this template and cloud-init was able to inject the ssh
public key and set timezone.
Changed cluster type of ths VM to Default as above with a warning from
oVirt but without any problem inside the VM and also changed the disk type
from virtio to virtio-scsi without problems.
Now I enabled incremental backup at engine level and also at disk level of
this CentOS 8 VM so that I can make some tests in this regard.
Also, I was able to successfully test the ovirt-ansible-shutdown-env
ansible role to make a clean overall shutdown of the environment, one of
the things that in my previous unsupported setup was a little cumbersome.
Right after install I noticed that in CentOS 8 ovirt-node-ng setup,
intel_pstate was the default scaling driver setup, but my cpu was almost
always crying with core at 2.6Ghz (and temp around 90 degrees) even if
attempted to setup conservative profiles and without nothing running, apart
engine VM.
The NUC is under my working day table and I don't need always performance
from it...
So I modified (notice also the rhgb and quite omissions...) these files and
reboot:
- /etc/default/grub
GRUB_CMDLINE_LINUX='crashkernel=auto resume=/dev/mapper/onn-swap
rd.lvm.lv=onn/ovirt-node-ng-4.4.0-0.20200521.0+1
rd.lvm.lv=onn/swap intel_pstate=disable'
- /boot/grub2/grub.cfg
set default_kernelopts="root=UUID=85212719-8feb-43aa-9819-2820d4672795 ro
crashkernel=auto ipv6.disable=1 intel_pstate=disable "
-
/boot/loader/entries/ovirt-node-ng-4.4.0-0.20200521.0+1-4.18.0-147.8.1.el8_1.x86_64.conf
options intel_pstate=disable boot=UUID=b717ab4f-ca71-469a-8836-ff92cebc7650
crashkernel=auto rd.lvm.lv=onn/swap
root=/dev/onn/ovirt-node-ng-4.4.0-0.20200521.0+1
resume=/dev/mapper/onn-swap rootflags=discard
rd.lvm.lv=onn/ovirt-node-ng-4.4.0-0.20200521.0+1
img.bootid=ovirt-node-ng-4.4.0-0.20200521.0+1 null
- /boot/efi/EFI/centos/grub.cfg
set default_kernelopts="root=/dev/mapper/onn-root ro crashkernel=auto
resume=/dev/mapper/onn-swap rd.lvm.lv=onn/root rd.lvm.lv=onn/swap
intel_pstate=disable "
And now with the default/old scaling driver all is quite silent and still
working ok for my needs, with web admin quite usable. I updated my two VMs
as described above after cpompleting these changes.
Right now for example I have hosted engine running + slackware one + CentOS
8 one with
[root@ovirt01 ~]# cat /proc/cpuinfo | grep Hz
model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
cpu MHz : 648.431
model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
cpu MHz : 628.307
model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
cpu MHz : 648.859
model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
cpu MHz : 663.792
[root@ovirt01 ~]#
and
[root@ovirt01 ~# cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
powersave
powersave
powersave
powersave
[root@ovirt01 g.cecchi]#
Thanks to all that helped throughout ...
Cheers,
Gianluca
4 years, 5 months
oVirt 4.3 and cinderlib integration (for ceph) on CentOS 7 - centos-release-openstack-pike
by Mathias Schwenke
At https://www.ovirt.org/develop/release-management/features/storage/cinderl... ist described the cinderlib integration into oVirt:
Installation:
- install centos-release-openstack-pike on engine and all hosts
- install openstack-cinder and python-pip on engine
- pip install cinderlib on engine
- install python2-os-brick on all hosts
- install ceph-common on engine and on all hosts
Which software versions do you use on CentOS 7 whith oVirt 4.3.10?
The package centos-release-openstack-pike, as described at the above-mentioned Managed Block Storage feature page, doesn't exist anymore in the CentOS repositories, so I have to switch to centos-release-openstack-queens or newer (rocky, stein, train). So I get (for using with ceph luminous 12):
- openstack-cinder 12.0.10
- cinderlib 1.0.1
- ceph-common 12.2.11
- python2-os-brick 2.3.9
4 years, 5 months
migration
by eevans＠digitaldatatechs.com
I am running Ovirt 4.310. When I update the engine server it updates the user extensions to
ovirt-engine-ui-extensions.noarch 0:1.0.13-1.20200303git3b594b8.el7 and migration fails. I have to downgrade to ovirt-engine-ui-extensions-1.0.10-1.el7.noarch.rpm inorder to get migration to work again.
This needs to be passed on to the developers to correct the updated version.
Also, I had to download ovirt-engine-ui-extensions-1.0.10-1.el7.noarch.rpm and run yum downgrade ovirt-engine-ui-extensions-1.0.10-1.el7.noarch.rpm for the downgrade to complete successfully.
After the downgrade, migration works.
4 years, 5 months
