Changing Cluster Compatibility Version from 4.6 to 4.7 issue
by Alexandr Mikhailov
Hi!
Just uprgaded from 4.4. to 4.5. Had all the problems with this update, such as postgresql-jdbc version and with stripeCount in cli.y . But I managed it, everything works more or less.
Now I cannot raise the Cluster compatibility level. The problem is that increasing the level tries to change something in the HE configuration but cannot.
This is error massage:
Error while executing action: Cannot update cluster because the update triggered update of the VMs/Templates and it failed for the following: HostedEngine. "There was an attempt to change Hosted Engine VM values that are locked." is one of the error(s).
To fix the issue, please go to each VM/Template, edit, change the Custom Compatibility Version (or other fields changed previously in the cluster dialog) and press OK. If the save does not pass, fix the dialog validation. After successful cluster update, you can revert your Custom Compatibility Version change (or other changes). If the problem still persists, you may refer to the engine.log file for further details.
If i trying to edit HE machine without changing anything i se next error: There was an attempt to change Hosted Engine VM values that are locked/ I think this is linked issues.
Log from engine log when i trying to update Cluster version:
2022-05-27 14:20:54,410+06 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-212) [1b8b6b78] EVENT_ID: CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Ca
nnot update compatibility version of Vm/Template: [HostedEngine], Message: There was an attempt to change Hosted Engine VM values that are locked.
Log from engine log when i trying to save HE configuration without any changing:
2022-05-27 14:34:10,965+06 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-220) [9cdfe99b-b7a1-46a4-ab3f-fc110b939f08] Lock Acquired to object 'EngineLock:{exclusiveLocks='[HostedEngine=
VM_NAME]', sharedLocks='[4d6a0ffb-a221-4ef8-9846-6ada7690e74a=VM]'}'
2022-05-27 14:34:10,968+06 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-220) [9cdfe99b-b7a1-46a4-ab3f-fc110b939f08] Validation of action 'UpdateVm' failed for user admin@internal-auth
z. Reasons: VAR__ACTION__UPDATE,VAR__TYPE__VM,VM_CANNOT_UPDATE_HOSTED_ENGINE_FIELD
2022-05-27 14:34:10,969+06 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-220) [9cdfe99b-b7a1-46a4-ab3f-fc110b939f08] Lock freed to object 'EngineLock:{exclusiveLocks='[HostedEngine=VM_
NAME]', sharedLocks='[4d6a0ffb-a221-4ef8-9846-6ada7690e74a=VM]'}'
It is not clear what is happening and what changes to the configuration are trying to be saved and what to do about it. Help please.
1 year, 8 months
ovirt-engine manager, certificate issue
by david
hello
I have a problem to log in to ovirt-engine manager in my browser
the warning message in the browser display me this text:
PKIX path validation failed: java.security.cert.CertPathValidatorException:
validity check failed
to solve this problem I am offered to run engine-setup
and here is a question: the engine-setup will have no impact to the
hosts(hypervisors) working?
ovirt version 4.4.4.7-1.el8
thanks
1 year, 8 months
Issue with oVirt 4.5 and Data Warehouse installed on a Separate Machine
by Igor Davidoff
Hello,
i have an issue with 'engine-seup' step on DWH (separate server) aufter upgrade from 4.4.10 to 4.5.
It looks like the ovirt-engine-setup are looking for rpm-package 'ovirt-engine' instead of 'ovirt-engine-dwh'.
the reporting error is:
"
--== END OF CONFIGURATION ==--
[ INFO ] Stage: Setup validation
[ ERROR ] Failed to execute stage 'Setup validation': Command '/usr/bin/rpm' failed to execute
[ INFO ] Stage: Clean up
Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20220502100751-fqwb07.log
[WARNING] Remote engine was not configured to be able to access DWH, please check the logs.
[ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20220502101130-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
"
in Setup log i found:
"
2022-05-02 10:11:30,000+0000 DEBUG otopi.context context._executeMethod:127 Stage validation METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages.Plugin._validation
2022-05-02 10:11:30,001+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.executeRaw:813 execute: ('/usr/bin/rpm', '-q', '--queryformat=%{version}-%{release}', 'ovirt-engine'), executable='None', cwd='None', env=None
2022-05-02 10:11:30,013+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.executeRaw:863 execute-result: ('/usr/bin/rpm', '-q', '--queryformat=%{version}-%{release}', 'ovirt-engine'), rc=1
2022-05-02 10:11:30,013+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.execute:921 execute-output: ('/usr/bin/rpm', '-q', '--queryformat=%{version}-%{release}', 'ovirt-engine') stdout:
package ovirt-engine is not installed
2022-05-02 10:11:30,013+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.execute:926 execute-output: ('/usr/bin/rpm', '-q', '--queryformat=%{version}-%{release}', 'ovirt-engine') stderr:
2022-05-02 10:11:30,013+0000 DEBUG otopi.context context._executeMethod:145 method exception
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod
method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-common/distro-rpm/packages.py", line 463, in _validation
oenginecons.Const.ENGINE_PACKAGE_NAME,
File "/usr/lib/python3.6/site-packages/otopi/plugin.py", line 931, in execute
command=args[0],
RuntimeError: Command '/usr/bin/rpm' failed to execute
2022-05-02 10:11:30,015+0000 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Setup validation': Command '/usr/bin/rpm' failed to execute
"
usually the upgrade of minor versions in 4.4 was just:
# yum update ovirt\*setup\*
# engine-setup
# yum update
as it did not work, i tried the fresh installation of centos8 stream and recovery of DWH Database and configuration:
# engine-backup --mode=restore --file=backup.bck --provision-all-databases
-> no luck
the last idea was fresh installation centos8 stream + fresh installation of ovirt-engine-dwh 4.5 (without recovery)
-> the same error.
the engine side works fine.
i compared the current setup logs with the installation and all the minor upgrades of ovirt-engine-dwh before 4.5
and only found the rpm-validation for the package 'ovirt-engine-dwh':
"
2022-02-08 16:11:29,846+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.executeRaw:813 execute: ('/usr/bin/rpm', '-q', 'ovirt-engine-dwh'), executable='None', cwd='None', env=None
2022-02-08 16:11:29,877+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.executeRaw:863 execute-result: ('/usr/bin/rpm', '-q', 'ovirt-engine-dwh'), rc=0
2022-02-08 16:11:29,878+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.execute:921 execute-output: ('/usr/bin/rpm', '-q', 'ovirt-engine-dwh') stdout:
ovirt-engine-dwh-4.4.10-1.el8.noarch
2022-02-08 16:11:29,878+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_common.distro-rpm.packages plugin.execute:926 execute-output: ('/usr/bin/rpm', '-q', 'ovirt-engine-dwh') stderr:
2022-02-08 16:11:29,878+0000 DEBUG otopi.transaction transaction.commit:152 committing 'DWH Engine database Transaction'
"
It looks like engine-setup knows it is the DWH-Server, but is trying to validate the wrong rpm package.
Any Ideas, how to work around this.
Thank you!
1 year, 8 months
VM access to infiniband network
by Roberto Bertucci
Hi all,
i am facing a problem while trying to associate a Mellanox infiniband interface to a network and using it for VM traffic.
vdsm log shows the following message:
The bridge <bridge name> cannot use IP over InfiniBand interface <interface name> as port. Please use RoCE interface instead.
Did anybody face the same problem and solve it?
Actually ib interface is configured with an ip address and we are mounting NFS filesystems on cluster nodes through infiniband network.
1 year, 9 months
Problems with selinux after updating an ovirt node
by Giorgio Biacchi
Hi folks,
today I got a problem with vdsm and selinux after updating a host:
[root@host04 ~]# nodectl check
Status: WARN
Bootloader ... OK
Layer boot entries ... OK
Valid boot entries ... OK
Mount points ... OK
Separate /var ... OK
Discard is used ... OK
Basic storage ... OK
Initialized VG ... OK
Initialized Thin Pool ... OK
Initialized LVs ... OK
Thin storage ... OK
Checking available space in thinpool ... OK
Checking thinpool auto-extend ... OK
vdsmd ... BAD
So I run:
[root@host04 ~]# /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
vdsm: Running mkdirs
vdsm: Running configure_vdsm_logs
vdsm: Running run_init_hooks
vdsm: Running check_is_configured
lvm is configured for vdsm
Current revision of multipath.conf detected, preserving
Managed volume database is already configured
abrt is already configured for vdsm
libvirt is already configured for vdsm
sanlock is configured for vdsm
Modules sebool are not configured
Error:
One of the modules is not configured to work with VDSM.
To configure the module use the following:
'vdsm-tool configure [--module module-name]'.
If all modules are not configured try to use:
'vdsm-tool configure --force'
(The force flag will stop the module's service and start it
afterwards automatically to load the new configuration.)
vdsm: stopped during execute check_is_configured task (task returned
with error code 1).
But also runnining this gave me an error:
[root@host04 ~]# vdsm-tool configure --module sebool
Checking configuration status...
Running configure...
libsepol.context_from_record: type cloud_what_var_cache_t is not defined
libsepol.context_from_record: could not create context structure
libsepol.context_from_string: could not create context structure
libsepol.sepol_context_to_sid: could not convert
system_u:object_r:cloud_what_var_cache_t:s0 to sid
invalid context system_u:object_r:cloud_what_var_cache_t:s0
libsemanage.semanage_validate_and_compile_fcontexts: setfiles returned
error code 255.
Traceback (most recent call last):
File "/usr/bin/vdsm-tool", line 209, in main
return tool_command[cmd]["command"](*args)
File "/usr/lib/python3.6/site-packages/vdsm/tool/__init__.py", line
40, in wrapper
func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/tool/configurator.py",
line 145, in configure
_configure(c)
File "/usr/lib/python3.6/site-packages/vdsm/tool/configurator.py",
line 92, in _configure
getattr(module, 'configure', lambda: None)()
File
"/usr/lib/python3.6/site-packages/vdsm/tool/configurators/sebool.py",
line 88, in configure
_setup_booleans(True)
File
"/usr/lib/python3.6/site-packages/vdsm/tool/configurators/sebool.py",
line 60, in _setup_booleans
sebool_obj.finish()
File "/usr/lib/python3.6/site-packages/seobject.py", line 340, in finish
self.commit()
File "/usr/lib/python3.6/site-packages/seobject.py", line 330, in commit
rc = semanage_commit(self.sh)
OSError: [Errno 0] Error
I managed to solve this by running:
[root@host04 ~]# semodule -i
/usr/share/selinux/packages/ovirt-vmconsole/ovirt_vmconsole.pp
[root@host04 ~]# vdsm-tool configure --module sebool
Checking configuration status...
Running configure...
Done configuring modules to VDSM.
Regards
--
gb
PGP Key: http://pgp.mit.edu/
Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
1 year, 9 months
Unable to install on a bonded NIC
by weeglos＠yahoo.com
So I'm running a fresh install of oVirt on a new Centos Stream node. Fresh install.
I installed the OS with bonded interfaces. I bonded them during the install via anaconda.
I followed the doc here: https://ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engine_...
When I got to the hosted-engine --deploy step, it errored out saying, "Only Team devices are present. Teaming is unsupported."
However, I'm not teaming my network adapters at all. I'm bonding them:
[root@mustafar ~]# cat /etc/sysconfig/network-scripts/ifcfg-Bond_connection_1
BONDING_OPTS="mode=balance-rr downdelay=0 miimon=1 updelay=0"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME="Bond connection 1"
UUID=[redacted]
DEVICE=bond0
ONBOOT=yes
IPADDR=192.168.5.83
PREFIX=24
GATEWAY=192.168.5.1
DNS1=192.168.5.2
DNS2=192.168.5.3
DNS3=192.168.5.4
DOMAIN=[redacted]
[root@mustafar ~]#
What gives with this?
1 year, 9 months
VM hangs after migration
by Giorgio Biacchi
Hi,
I have a fresh Ovirt installation (4.4.10.7-1.el8 engine and oVirt Node
4.4.10) on a Dell VRTX chassis. There are 3 blades, two of them are
identical hardware (PowerEdge M630) and the third is a little newer
(PowerEdge M640). The third has different CPUs, more RAM, and slower
NICs. I also have a bunch of data domains some on the shared PERC
internal storage and others on an external iSCSI storage, all seems
configured correctly and all the hosts are operational.
I can migrate a VM back and forth from the first two blades without any
problem, I can migrate a VM to the third blade but when I migrate a VM
from the third blade to any of the other two the task terminate
successfully, the VM is marked as up on the target host but the VM
hangs, the console is frozen and the VM stops to respond to ping.
I have no clues about why this is happening and I'm looking for
suggestions about how to debug and hopefully fix this issue.
Thanks in advance
--
gb
PGP Key: http://pgp.mit.edu/
Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
1 year, 10 months
Unable to ugprade cluster level to 4.7 for the hosted engine (only)
by lists＠pequod.io
Hello,
i upgraded my engine and nodes to 4.5 a few days ago and now planning to upgrade the cluster level compatibility from 4.6 to 4.7. First i tried doing this from the cluster settings, but it fails because hosted-engine settings are locked. So i tried it by hand but again got the locked error, i found i cant change any values on the hosted engine. Changing compatiblity level on all other VMs worked fine and there are on 4.7 now.
I read about the timezone issue in 4.4.8, so i checked the timezone of my hosted engine it is filled with "Standard: (GMTZ) Greenwhich Standard Time". To be sure, i just did a "/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "update vm_static SET time_zone='Etc/GMT' where vm_name='HostedEngine';"" and it changed the timezone, but settings are still locked and i am unable to change the compatibility level.
Any idea how to solve this?
1 year, 10 months
failed to mount hosted engine gluster storage - how to debug?
by diego.ercolani＠ssis.sm
Hello, I have an issue probably related to my particular implementation but I think some controls are missing
Here is the story.
I have a cluster of two nodes 4.4.10.3 with an upgraded kernel as the cpu (Ryzen 5) suffer from an incompatibility issue with the kernel provided by 4.4.10.x series.
On each node there are three glusterfs "partitions" in replica mode, one for the hosted_engine, the other two are for user usage.
The third node was an old i3 workstation only used to provide the arbiter partition to the glusterfs cluster.
I installed a new server (ryzen processor) with 4.5.0 and successfully installed glusterfs 10.1 and inserted the arbiter bricks implemented on glusterfs 10.1 while the replica bricks are 8.6 after removing the old i3 provided bricks.
I successfully imported the new node in the ovirt engine (after updating the engine to 4.5)
The problem is that the ovirt-ha-broker doesn't start complaining that is not possible to connect the storage. (I suppose the hosted_engine storage) so I did some digs that I'm going to show here:
####
1. The node seem to be correctly configured:
[root@ovirt-node3 devices]# vdsm-tool validate-config
SUCCESS: ssl configured to true. No conflicts
[root@ovirt-node3 devices]# vdsm-tool configure
Checking configuration status...
libvirt is already configured for vdsm
SUCCESS: ssl configured to true. No conflicts
sanlock is configured for vdsm
Managed volume database is already configured
lvm is configured for vdsm
Current revision of multipath.conf detected, preserving
Running configure...
Done configuring modules to VDSM.
[root@ovirt-node3 devices]# vdsm-tool validate-config
SUCCESS: ssl configured to true. No conflicts
####
2. the node refuses to mount via hosted-engine (same error in broker.log)
[root@ovirt-node3 devices]# hosted-engine --connect-storage
Traceback (most recent call last):
File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/connect_storage_server.py", line 30, in <module>
timeout=ohostedcons.Const.STORAGE_SERVER_TIMEOUT,
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 312, in connect_storage_server
sserver.connect_storage_server(timeout=timeout)
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/storage_server.py", line 451, in connect_storage_server
'Connection to storage server failed'
RuntimeError: Connection to storage server failed
#####
3. manually mount of glusterfs work correctly
[root@ovirt-node3 devices]# grep storage /etc/ovirt-hosted-engine/hosted-engine.conf
storage=ovirt-node2.ovirt:/gveng
# The following are used only for iSCSI storage
[root@ovirt-node3 devices]#
[root@ovirt-node3 devices]# mount -t glusterfs ovirt-node2.ovirt:/gveng /mnt/tmp/
[root@ovirt-node3 devices]# ls -l /mnt/tmp
total 0
drwxr-xr-x. 6 vdsm kvm 64 Dec 15 19:04 7b8f1cc9-e3de-401f-b97f-8c281ca30482
What else should I control? Thank you and sorry for the long message
Diego
1 year, 10 months
