Problems with selinux after updating an ovirt node
by Giorgio Biacchi
Hi folks,
today I got a problem with vdsm and selinux after updating a host:
[root@host04 ~]# nodectl check
Status: WARN
Bootloader ... OK
Layer boot entries ... OK
Valid boot entries ... OK
Mount points ... OK
Separate /var ... OK
Discard is used ... OK
Basic storage ... OK
Initialized VG ... OK
Initialized Thin Pool ... OK
Initialized LVs ... OK
Thin storage ... OK
Checking available space in thinpool ... OK
Checking thinpool auto-extend ... OK
vdsmd ... BAD
So I run:
[root@host04 ~]# /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
vdsm: Running mkdirs
vdsm: Running configure_vdsm_logs
vdsm: Running run_init_hooks
vdsm: Running check_is_configured
lvm is configured for vdsm
Current revision of multipath.conf detected, preserving
Managed volume database is already configured
abrt is already configured for vdsm
libvirt is already configured for vdsm
sanlock is configured for vdsm
Modules sebool are not configured
Error:
One of the modules is not configured to work with VDSM.
To configure the module use the following:
'vdsm-tool configure [--module module-name]'.
If all modules are not configured try to use:
'vdsm-tool configure --force'
(The force flag will stop the module's service and start it
afterwards automatically to load the new configuration.)
vdsm: stopped during execute check_is_configured task (task returned
with error code 1).
But also runnining this gave me an error:
[root@host04 ~]# vdsm-tool configure --module sebool
Checking configuration status...
Running configure...
libsepol.context_from_record: type cloud_what_var_cache_t is not defined
libsepol.context_from_record: could not create context structure
libsepol.context_from_string: could not create context structure
libsepol.sepol_context_to_sid: could not convert
system_u:object_r:cloud_what_var_cache_t:s0 to sid
invalid context system_u:object_r:cloud_what_var_cache_t:s0
libsemanage.semanage_validate_and_compile_fcontexts: setfiles returned
error code 255.
Traceback (most recent call last):
File "/usr/bin/vdsm-tool", line 209, in main
return tool_command[cmd]["command"](*args)
File "/usr/lib/python3.6/site-packages/vdsm/tool/__init__.py", line
40, in wrapper
func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/tool/configurator.py",
line 145, in configure
_configure(c)
File "/usr/lib/python3.6/site-packages/vdsm/tool/configurator.py",
line 92, in _configure
getattr(module, 'configure', lambda: None)()
File
"/usr/lib/python3.6/site-packages/vdsm/tool/configurators/sebool.py",
line 88, in configure
_setup_booleans(True)
File
"/usr/lib/python3.6/site-packages/vdsm/tool/configurators/sebool.py",
line 60, in _setup_booleans
sebool_obj.finish()
File "/usr/lib/python3.6/site-packages/seobject.py", line 340, in finish
self.commit()
File "/usr/lib/python3.6/site-packages/seobject.py", line 330, in commit
rc = semanage_commit(self.sh)
OSError: [Errno 0] Error
I managed to solve this by running:
[root@host04 ~]# semodule -i
/usr/share/selinux/packages/ovirt-vmconsole/ovirt_vmconsole.pp
[root@host04 ~]# vdsm-tool configure --module sebool
Checking configuration status...
Running configure...
Done configuring modules to VDSM.
Regards
--
gb
PGP Key: http://pgp.mit.edu/
Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
1 year, 9 months
Unable to install on a bonded NIC
by weeglos＠yahoo.com
So I'm running a fresh install of oVirt on a new Centos Stream node. Fresh install.
I installed the OS with bonded interfaces. I bonded them during the install via anaconda.
I followed the doc here: https://ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engine_...
When I got to the hosted-engine --deploy step, it errored out saying, "Only Team devices are present. Teaming is unsupported."
However, I'm not teaming my network adapters at all. I'm bonding them:
[root@mustafar ~]# cat /etc/sysconfig/network-scripts/ifcfg-Bond_connection_1
BONDING_OPTS="mode=balance-rr downdelay=0 miimon=1 updelay=0"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME="Bond connection 1"
UUID=[redacted]
DEVICE=bond0
ONBOOT=yes
IPADDR=192.168.5.83
PREFIX=24
GATEWAY=192.168.5.1
DNS1=192.168.5.2
DNS2=192.168.5.3
DNS3=192.168.5.4
DOMAIN=[redacted]
[root@mustafar ~]#
What gives with this?
1 year, 9 months
VM hangs after migration
by Giorgio Biacchi
Hi,
I have a fresh Ovirt installation (4.4.10.7-1.el8 engine and oVirt Node
4.4.10) on a Dell VRTX chassis. There are 3 blades, two of them are
identical hardware (PowerEdge M630) and the third is a little newer
(PowerEdge M640). The third has different CPUs, more RAM, and slower
NICs. I also have a bunch of data domains some on the shared PERC
internal storage and others on an external iSCSI storage, all seems
configured correctly and all the hosts are operational.
I can migrate a VM back and forth from the first two blades without any
problem, I can migrate a VM to the third blade but when I migrate a VM
from the third blade to any of the other two the task terminate
successfully, the VM is marked as up on the target host but the VM
hangs, the console is frozen and the VM stops to respond to ping.
I have no clues about why this is happening and I'm looking for
suggestions about how to debug and hopefully fix this issue.
Thanks in advance
--
gb
PGP Key: http://pgp.mit.edu/
Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
1 year, 10 months
failed to mount hosted engine gluster storage - how to debug?
by diego.ercolani＠ssis.sm
Hello, I have an issue probably related to my particular implementation but I think some controls are missing
Here is the story.
I have a cluster of two nodes 4.4.10.3 with an upgraded kernel as the cpu (Ryzen 5) suffer from an incompatibility issue with the kernel provided by 4.4.10.x series.
On each node there are three glusterfs "partitions" in replica mode, one for the hosted_engine, the other two are for user usage.
The third node was an old i3 workstation only used to provide the arbiter partition to the glusterfs cluster.
I installed a new server (ryzen processor) with 4.5.0 and successfully installed glusterfs 10.1 and inserted the arbiter bricks implemented on glusterfs 10.1 while the replica bricks are 8.6 after removing the old i3 provided bricks.
I successfully imported the new node in the ovirt engine (after updating the engine to 4.5)
The problem is that the ovirt-ha-broker doesn't start complaining that is not possible to connect the storage. (I suppose the hosted_engine storage) so I did some digs that I'm going to show here:
####
1. The node seem to be correctly configured:
[root@ovirt-node3 devices]# vdsm-tool validate-config
SUCCESS: ssl configured to true. No conflicts
[root@ovirt-node3 devices]# vdsm-tool configure
Checking configuration status...
libvirt is already configured for vdsm
SUCCESS: ssl configured to true. No conflicts
sanlock is configured for vdsm
Managed volume database is already configured
lvm is configured for vdsm
Current revision of multipath.conf detected, preserving
Running configure...
Done configuring modules to VDSM.
[root@ovirt-node3 devices]# vdsm-tool validate-config
SUCCESS: ssl configured to true. No conflicts
####
2. the node refuses to mount via hosted-engine (same error in broker.log)
[root@ovirt-node3 devices]# hosted-engine --connect-storage
Traceback (most recent call last):
File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/connect_storage_server.py", line 30, in <module>
timeout=ohostedcons.Const.STORAGE_SERVER_TIMEOUT,
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 312, in connect_storage_server
sserver.connect_storage_server(timeout=timeout)
File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/storage_server.py", line 451, in connect_storage_server
'Connection to storage server failed'
RuntimeError: Connection to storage server failed
#####
3. manually mount of glusterfs work correctly
[root@ovirt-node3 devices]# grep storage /etc/ovirt-hosted-engine/hosted-engine.conf
storage=ovirt-node2.ovirt:/gveng
# The following are used only for iSCSI storage
[root@ovirt-node3 devices]#
[root@ovirt-node3 devices]# mount -t glusterfs ovirt-node2.ovirt:/gveng /mnt/tmp/
[root@ovirt-node3 devices]# ls -l /mnt/tmp
total 0
drwxr-xr-x. 6 vdsm kvm 64 Dec 15 19:04 7b8f1cc9-e3de-401f-b97f-8c281ca30482
What else should I control? Thank you and sorry for the long message
Diego
1 year, 10 months
After attaching the Storage domain, the VMs are disappeared from the VM import
by aminur.rahman＠iongroup.com
Hi,
We're noticing some weird issue while re-attaching the storage domain. After re-attach the storage domain, some VMs are completely missing from the VM Import. Before detaching the storage domain, all the VMs were shutdown gracefully.
I also noticed some disks are exists with no Alias under the disk import on the storage domain and I can't import those disks. Its failed to register the disk with <UNKONOWN> error.
We're using Ovirt 4.2 with multiple Dell hosts in the cluster and Compellent SAN with iSCSI volumes.
Please kindly advise if I am missing anything before detach the storage domain.
Thanks
1 year, 10 months
Cannot log into oVirt Manager - certificate issue
by Diggy Mc
I cannot log into oVirt Manager. My browser gave me a warning that the site's certificate has expired. Then when I try to log in, I receive the following error message:
"PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed"
How can I fix this problem? In advance, thank you for your help.
hosted-engine: v4.4.8.6
hosts: oVirt Node v4.4.8.3
1 year, 10 months
can't use vmconsole anymore
by Nathanaël Blanchet
Hi,
I was used to use the vmconsole proxy, but since a while, I'm getting
this issue (currently 4.4.5):
# ssh -t -p 2222 ovirt-vmconsole(a)air.v100.abes.fr connect
ovirt-vmconsole(a)air.v100.abes.fr: Permission denied (publickey).
I found following in the engine.log
2021-04-15 17:55:43,094+02 ERROR
[org.ovirt.engine.core.services.VMConsoleProxyServlet] (default task-4)
[] Error validating ticket: :
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
at
java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at
java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at
java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at
org.ovirt.engine.core.uutils//org.ovirt.engine.core.uutils.crypto.CertificateChain.buildCertPath(CertificateChain.java:128)
at
org.ovirt.engine.core.uutils//org.ovirt.engine.core.uutils.crypto.ticket.TicketDecoder.decode(TicketDecoder.java:89)
at
deployment.engine.ear.services.war//org.ovirt.engine.core.services.VMConsoleProxyServlet.validateTicket(VMConsoleProxyServlet.java:175)
at
deployment.engine.ear.services.war//org.ovirt.engine.core.services.VMConsoleProxyServlet.doPost(VMConsoleProxyServlet.java:225)
The user key is the good one, I use the same with my other engines and I
can successfully connect to vm consoles.
Thank you for helping
--
Nathanaël Blanchet
Supervision réseau
SIRE
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
1 year, 10 months
VM HostedEngine is down with error
by souvaliotimaria＠mail.com
Hello everyone,
I have a replica 2 + arbiter installation and this morning the Hosted Engine gave the following error on the UI and resumed on a different node (node3) than the one it was originally running(node1). (The original node has more memory than the one it ended up, but it had a better memory usage percentage at the time). Also, the only way I discovered the migration had happened and there was an Error in Events, was because I logged in the web interface of ovirt for a routine inspection. Βesides that, everything was working properly and still is.
The error that popped is the following:
VM HostedEngine is down with error. Exit message: internal error: qemu unexpectedly closed the monitor:
2020-09-01T06:49:20.749126Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future
2020-09-01T06:49:20.927274Z qemu-kvm: -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0x7,drive=drive-ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2,id=ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2,bootindex=1,write-cache=on: Failed to get "write" lock
Is another process using the image?.
Which from what I could gather concerns the following snippet from the HostedEngine.xml and it's the virtio disk of the Hosted Engine:
<disk type='file' device='disk' snapshot='no'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads' iothread='1'/>
<source file='/var/run/vdsm/storage/80f6e393-9718-4738-a14a-64cf43c3d8c2/d5de54b6-9f8e-4fba-819b-ebf6780757d2/a48555f4-be23-4467-8a54-400ae7baf9d7'>
<seclabel model='dac' relabel='no'/>
</source>
<target dev='vda' bus='virtio'/>
<serial>d5de54b6-9f8e-4fba-819b-ebf6780757d2</serial>
<alias name='ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</disk>
I've tried looking into the logs and the sar command but I couldn't find anything to relate with the above errors and determining the reason for it to happen. Is this a Gluster or a QEMU problem?
The Hosted Engine was manually migrated five days before on node1.
Is there a standard practice I could follow to determine what happened and secure my system?
Thank you very much for your time,
Maria Souvalioti
1 year, 11 months
