On 7/11/22 16:22, Jiří Sléžka wrote:

On 7/11/22 15:57, Strahil Nikolov wrote:

...

Can you check for AVC denials and the error message like the described in https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183 ?

thanks for reply, there are two unrelated (qemu-kvm) avc denials logged (related probably to sanlock recovery)

also I cannot find glustereventsd in any related log... is it really used by vdsm-gluster?

this service runs on no hosts

systemctl status glustereventsd ● glustereventsd.service - Gluster Events Notifier    Loaded: loaded (/usr/lib/systemd/system/glustereventsd.service; disabled; vendor preset: disabled)    Active: inactive (dead)

it looks like root of the problem is that Gluster service is disabled in cluster settings and cannot be enabled. But it was enabled before... also I have to manually install vdsm-gluster when I (re)install new host, but bricks from this host are in unknown state in admin. Maybe vdsm-gluster is not correctly configured? Maybe glustereventsd is not running? I am just guessing... I have no access to other HCI installation so I cannot compare differences. I would be really happy if someone could tell me what circumstances could disable Gluster service checkbox in admin and how to enable it again... Cheers, Jiri

Cheers,

Jiri

...

Best Regards, Strahil Nikolov

    On Mon, Jul 11, 2022 at 16:44, Jiří Sléžka     <jiri.slezka@slu.cz> wrote:     Hello,

    On 7/11/22 14:34, Strahil Nikolov wrote:      > Can you check something on the host:      > cat /etc/glusterfs/eventsconfig.json

    cat /etc/glusterfs/eventsconfig.json     {          "log-level": "INFO",          "port": 24009,          "disable-events-log": false     }

     > semanage port -l | grep $(awk -F ':' '/port/ {gsub(",","",$2); print      > $2}' /etc/glusterfs/eventsconfig.json)

    semanage port -l | grep 24009

    returns empty set, it looks like this port is not labeled

    Cheers,

    Jiri

     >      > Best Regards,      > Strahil Nikolov      > В понеделник, 11 юли 2022 г., 02:18:57 ч. Гринуич+3, Jiří Sléžka      > <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> написа:      >      >      > Hi,      >      > I would like to change CPU Type in my oVirt 4.4.10 HCI cluster     (based on      > 3 glusterfs/virt hosts). When I try to I got this error      >      > Error while executing action: Cannot disable gluster service on the      > cluster as it contains volumes.      >      > As I remember I had Gluster Service enabled on this cluster but     now both      > (Enable Virt Services and Enable Gluster Service) checkboxes are     grayed      > out and Gluster Service is unchecked.      >      > Also Storage / Volumes displays my volumes... well, displays one     brick      > on particular host in unknown state (? mark) which is new     situation. As      > I can see from command line all bricks are online, no healing in      > progress, all looks good...      >      > I am not sure if the second issue is relevant to first one so main      > question is how can I (re)enable gluster service in my cluster?      >      > Thanks in advance,      >      > Jiri      > _______________________________________________      > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>      > To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>

     > <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      > Privacy Statement: https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html >      > <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      > oVirt Code of Conduct:      > https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/ >      > <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      > List Archives:      >

https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR7P7K7OICSA5F253BVA/>>

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P...

On 7/12/22 22:28, Strahil Nikolov wrote:

glustereventad will notify the engine when something changes - like a new volume is created from the cli (or bad things happened ;) ), so it should be running. > You can use the workaround from the github issue and reatart the glustereventsd service.

ok, workaround applied, glustereventsd service enabled and started on all hosts. I can see this log entry in volume Events Detected change in status of brick 10.0.4.11:/gluster_bricks/engine/engine of volume engine of cluster McHosting from UNKNOWN to UP via gluster event. but Bricks tab shows still two (.12 and .13) of three bricks in Unknown state. From command line point of view all bricks are up and healthy. it looks like engine thinks that gluster service is disabled in cluster but I cannot enable it because checkbox is disabled. In my other (FC based) oVirt instance Gluster Service checkbox is not selected but not disabled. So I am interested what could make that checkbox inactive...

For the vdsm, you can always run '/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start' which is executed by the vdsmd.service before every start (ExecStartPre stanza) and see if it complains about something.

[root@ovirt-hci03 ~]# /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start vdsm: Running mkdirs vdsm: Running configure_vdsm_logs vdsm: Running run_init_hooks vdsm: Running check_is_configured sanlock is configured for vdsm lvm is configured for vdsm abrt is already configured for vdsm Managed volume database is already configured Current revision of multipath.conf detected, preserving libvirt is already configured for vdsm vdsm: Running validate_configuration SUCCESS: ssl configured to true. No conflicts vdsm: Running prepare_transient_repository vdsm: Running syslog_available vdsm: Running nwfilter vdsm: Running dummybr vdsm: Running tune_system vdsm: Running test_space vdsm: Running test_lo retcode 0, all looks ok... Cheers, Jiri

Best Regards, Strahil Nikolov

On Tue, Jul 12, 2022 at 11:12, Jiří Sléžka <jiri.slezka@slu.cz> wrote: On 7/11/22 16:22, Jiří Sléžka wrote: > On 7/11/22 15:57, Strahil Nikolov wrote: >> Can you check for AVC denials and the error message like the described >> in >> https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183 <https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183 >? > > thanks for reply, there are two unrelated (qemu-kvm) avc denials logged > (related probably to sanlock recovery) > > also I cannot find glustereventsd in any related log... is it really > used by vdsm-gluster? > > this service runs on no hosts > > systemctl status glustereventsd > ● glustereventsd.service - Gluster Events Notifier >     Loaded: loaded (/usr/lib/systemd/system/glustereventsd.service; > disabled; vendor preset: disabled) >     Active: inactive (dead)

it looks like root of the problem is that Gluster service is disabled in cluster settings and cannot be enabled. But it was enabled before... also I have to manually install vdsm-gluster when I (re)install new host, but bricks from this host are in unknown state in admin. Maybe vdsm-gluster is not correctly configured? Maybe glustereventsd is not running? I am just guessing...

I have no access to other HCI installation so I cannot compare differences.

I would be really happy if someone could tell me what circumstances could disable Gluster service checkbox in admin and how to enable it again...

Cheers,

Jiri

> > Cheers, > > Jiri > > >> >> >> Best Regards, >> Strahil Nikolov >> >>     On Mon, Jul 11, 2022 at 16:44, Jiří Sléžka >>     <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote: >>     Hello, >> >>     On 7/11/22 14:34, Strahil Nikolov wrote: >>      > Can you check something on the host: >>      > cat /etc/glusterfs/eventsconfig.json >> >>     cat /etc/glusterfs/eventsconfig.json >>     { >>          "log-level": "INFO", >>          "port": 24009, >>          "disable-events-log": false >>     } >> >> >>      > semanage port -l | grep $(awk -F ':' '/port/ {gsub(",","",$2); >> print >>      > $2}' /etc/glusterfs/eventsconfig.json) >> >>     semanage port -l | grep 24009 >> >>     returns empty set, it looks like this port is not labeled >> >>     Cheers, >> >>     Jiri >> >>      > >>      > Best Regards, >>      > Strahil Nikolov >>      > В понеделник, 11 юли 2022 г., 02:18:57 ч. Гринуич+3, Jiří Sléžka >>      > <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz> <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> написа: >>      > >>      > >>      > Hi, >>      > >>      > I would like to change CPU Type in my oVirt 4.4.10 HCI cluster >>     (based on >>      > 3 glusterfs/virt hosts). When I try to I got this error >>      > >>      > Error while executing action: Cannot disable gluster service on >> the >>      > cluster as it contains volumes. >>      > >>      > As I remember I had Gluster Service enabled on this cluster but >>     now both >>      > (Enable Virt Services and Enable Gluster Service) checkboxes are >>     grayed >>      > out and Gluster Service is unchecked. >>      > >>      > Also Storage / Volumes displays my volumes... well, displays one >>     brick >>      > on particular host in unknown state (? mark) which is new >>     situation. As >>      > I can see from command line all bricks are online, no healing in >>      > progress, all looks good... >>      > >>      > I am not sure if the second issue is relevant to first one so main >>      > question is how can I (re)enable gluster service in my cluster? >>      > >>      > Thanks in advance, >>      > >>      > Jiri >>      > _______________________________________________ >>      > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>> >>     <mailto:users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>>> >>      > To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> >>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>> >> >>      > <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>> >>      > Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >>     <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html >> >>      > <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >>     <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>>> >>      > oVirt Code of Conduct: >>      > https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >>     <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/ >> >>      > <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >>     <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>>> >>      > List Archives: >>      > >> >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... > >> >> >> <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... > >> >>      > >>      > >> >> <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... > >> >> >> <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR7P7K7OICSA5F253BVA/>>>

>> >> > > > _______________________________________________ > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> > To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2PEBYEZWLPX5YUDHD76A/>

Well... not yet.Check if the engine detects the volumes and verify again that all glustereventsd work. I would even consider restarting the engine, just to be on the safe side. What is your oVirt version ? Maybe an update could solve your problem. Best Regards,Strahil Nikolov  On Wed, Jul 13, 2022 at 17:05, Jiří Sléžka<jiri.slezka@slu.cz> wrote: On 7/13/22 14:53, Jiří Sléžka wrote:

On 7/12/22 22:28, Strahil Nikolov wrote:

...

glustereventad will notify the engine when something changes - like a new volume is created from the cli (or bad things happened ;) ), so it should be running. > You can use the workaround from the github issue and reatart the glustereventsd service.

ok, workaround applied, glustereventsd service enabled and started on all hosts.

I can see this log entry in volume Events

Detected change in status of brick 10.0.4.11:/gluster_bricks/engine/engine of volume engine of cluster McHosting from UNKNOWN to UP via gluster event.

but Bricks tab shows still two (.12 and .13) of three bricks in Unknown state. From command line point of view all bricks are up and healthy.

it looks like engine thinks that gluster service is disabled in cluster but I cannot enable it because checkbox is disabled. In my other (FC based) oVirt instance Gluster Service checkbox is not selected but not disabled. So I am interested what could make that checkbox inactive...

well, on db side it looks like cluster has gluster_service disabled... engine=# select virt_service, gluster_service from cluster;   virt_service | gluster_service --------------+-----------------   t            | f (1 row) still don't know why the checkbox is disabled. Would it be safe to enabled gluster_service directly in db? I suppose no... :-) Cheers, Jiri

...

For the vdsm, you can always run '/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start' which is executed by the vdsmd.service before every start (ExecStartPre stanza) and see if it complains about something.

[root@ovirt-hci03 ~]# /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start vdsm: Running mkdirs vdsm: Running configure_vdsm_logs vdsm: Running run_init_hooks vdsm: Running check_is_configured sanlock is configured for vdsm lvm is configured for vdsm abrt is already configured for vdsm Managed volume database is already configured Current revision of multipath.conf detected, preserving libvirt is already configured for vdsm vdsm: Running validate_configuration SUCCESS: ssl configured to true. No conflicts vdsm: Running prepare_transient_repository vdsm: Running syslog_available vdsm: Running nwfilter vdsm: Running dummybr vdsm: Running tune_system vdsm: Running test_space vdsm: Running test_lo

retcode 0, all looks ok...

Cheers,

Jiri

...

Best Regards, Strahil Nikolov

    On Tue, Jul 12, 2022 at 11:12, Jiří Sléžka     <jiri.slezka@slu.cz> wrote:     On 7/11/22 16:22, Jiří Sléžka wrote:      > On 7/11/22 15:57, Strahil Nikolov wrote:      >> Can you check for AVC denials and the error message like the     described      >> in      >>     https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183     <https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183      >?      >      > thanks for reply, there are two unrelated (qemu-kvm) avc denials     logged      > (related probably to sanlock recovery)      >      > also I cannot find glustereventsd in any related log... is it really      > used by vdsm-gluster?      >      > this service runs on no hosts      >      > systemctl status glustereventsd      > ● glustereventsd.service - Gluster Events Notifier      >     Loaded: loaded (/usr/lib/systemd/system/glustereventsd.service;      > disabled; vendor preset: disabled)      >     Active: inactive (dead)

    it looks like root of the problem is that Gluster service is     disabled in     cluster settings and cannot be enabled. But it was enabled before...     also I have to manually install vdsm-gluster when I (re)install new     host, but bricks from this host are in unknown state in admin. Maybe     vdsm-gluster is not correctly configured? Maybe glustereventsd is not     running? I am just guessing...

    I have no access to other HCI installation so I cannot compare     differences.

    I would be really happy if someone could tell me what circumstances     could disable Gluster service checkbox in admin and how to enable it     again...

    Cheers,

    Jiri

     >      > Cheers,      >      > Jiri      >      >      >>      >>      >> Best Regards,      >> Strahil Nikolov      >>      >>     On Mon, Jul 11, 2022 at 16:44, Jiří Sléžka      >>     <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote:      >>     Hello,      >>      >>     On 7/11/22 14:34, Strahil Nikolov wrote:      >>      > Can you check something on the host:      >>      > cat /etc/glusterfs/eventsconfig.json      >>      >>     cat /etc/glusterfs/eventsconfig.json      >>     {      >>          "log-level": "INFO",      >>          "port": 24009,      >>          "disable-events-log": false      >>     }      >>      >>      >>      > semanage port -l | grep $(awk -F ':' '/port/     {gsub(",","",$2);      >> print      >>      > $2}' /etc/glusterfs/eventsconfig.json)      >>      >>     semanage port -l | grep 24009      >>      >>     returns empty set, it looks like this port is not labeled      >>      >>     Cheers,      >>      >>     Jiri      >>      >>      >      >>      > Best Regards,      >>      > Strahil Nikolov      >>      > В понеделник, 11 юли 2022 г., 02:18:57 ч. Гринуич+3, Jiří     Sléžka      >>      > <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> написа:      >>      >      >>      >      >>      > Hi,      >>      >      >>      > I would like to change CPU Type in my oVirt 4.4.10 HCI     cluster      >>     (based on      >>      > 3 glusterfs/virt hosts). When I try to I got this error      >>      >      >>      > Error while executing action: Cannot disable gluster     service on      >> the      >>      > cluster as it contains volumes.      >>      >      >>      > As I remember I had Gluster Service enabled on this     cluster but      >>     now both      >>      > (Enable Virt Services and Enable Gluster Service)     checkboxes are      >>     grayed      >>      > out and Gluster Service is unchecked.      >>      >      >>      > Also Storage / Volumes displays my volumes... well,     displays one      >>     brick      >>      > on particular host in unknown state (? mark) which is new      >>     situation. As      >>      > I can see from command line all bricks are online, no     healing in      >>      > progress, all looks good...      >>      >      >>      > I am not sure if the second issue is relevant to first     one so main      >>      > question is how can I (re)enable gluster service in my     cluster?      >>      >      >>      > Thanks in advance,      >>      >      >>      > Jiri      >>      > _______________________________________________      >>      > Users mailing list -- users@ovirt.org     <mailto:users@ovirt.org> <mailto:users@ovirt.org     <mailto:users@ovirt.org>>      >>     <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>>      >>      > To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      >>      >>      > <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>>>      >>      > Privacy Statement:     https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html >>      >>      > <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>>      >>      > oVirt Code of Conduct:      >>      >     https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/ >>      >>      >     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>>      >>      > List Archives:      >>      >      >>      >>     https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

    <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

    <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >      >>      >      >>      >>     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

    <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

    <https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR7P7K7OICSA5F253BVA/>>>

     >>      >>      >      >      > _______________________________________________      > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>      > To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      > Privacy Statement: https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      > oVirt Code of Conduct:     https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      > List Archives:     https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P...

    <https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2PEBYEZWLPX5YUDHD76A/>

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/YVDPYIPFHHFEVZ...

On 7/14/22 14:30, Jiří Sléžka wrote:

On 7/14/22 00:34, Strahil Nikolov wrote:

...

Well... not yet. Check if the engine detects the volumes and verify again that all glustereventsd work.

I would even consider restarting the engine, just to be on the safe side.

engine restarted (I also yum updated it before), glustereventsd is running on all hosts, selinux's port label is set. Still only one brick is up and two are in unknown state in manager. See the screenshot.

...

What is your oVirt version ? Maybe an update could solve your problem.

latest in 4.4 version -> 4.4.10.7-1.el8. Engine and hosts are Rocky Linux 8.6 based.

ale here is output of select * from gluster_volume_bricks; I dont know if it is relevant but it shows old (2022-06-12 14:32:46.476558+02) dates in _update_date filed in part of UNKNOWN state bricks. Also UP state bricks has timestamp i past (around 1 day), is it normal?

https://pastebin.com/hQnj1en3

Could you suggest some relevant strings in engine.log | vdsm.log I could looks for? My blind grepping just reveleated

grep "GLUSTER" /var/log/ovirt-engine/engine.log

2022-07-13 13:10:19,263+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-3) [] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick 10.0.4.11:/gluster_bricks/engine/engine of volume engine of cluster McHosting from UNKNOWN to UP via gluster event. 2022-07-13 13:10:19,492+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-3) [] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick 10.0.4.11:/gluster_bricks/vms/vms of volume vms of cluster McHosting from UNKNOWN to UP via gluster event. 2022-07-13 13:10:21,185+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-3) [] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick 10.0.4.11:/gluster_bricks/vms2/vms2 of volume vms of cluster McHosting from UNKNOWN to UP via gluster event.

it match the engine log and also time I run glustereventsd service on host 10.0.4.11

but I see also repeatedly

2022-07-13 14:14:14,440+02 WARN [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [5fb948bc] Validation of action 'UpdateCluster' failed for user SYSTEM. Reasons: VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES

so it looks to me like there is some action running which wants disable gluster services on cluster and it cannot (and it is right!). But it probably blocks that gluster services checkbox in cluster settings in manager. What do you think?

sorry, date implies it is my yesterday's testing invoking of cluster edit window... there are no running jobs or tasks in db (as I can see)... so still dont know what is wrong :-( Cheers, Jiri

Cheers, Jiri

...

Best Regards, Strahil Nikolov

    On Wed, Jul 13, 2022 at 17:05, Jiří Sléžka     <jiri.slezka@slu.cz> wrote:     On 7/13/22 14:53, Jiří Sléžka wrote:      > On 7/12/22 22:28, Strahil Nikolov wrote:      >> glustereventad will notify the engine when something changes -     like a      >> new volume is created from the cli (or bad things happened ;) ),     so it      >> should be running. >      >> You can use the workaround from the github issue and reatart the      >> glustereventsd service.      >      > ok, workaround applied, glustereventsd service enabled and     started on      > all hosts.      >      > I can see this log entry in volume Events      >      > Detected change in status of brick      > 10.0.4.11:/gluster_bricks/engine/engine of volume engine of cluster      > McHosting from UNKNOWN to UP via gluster event.      >      > but Bricks tab shows still two (.12 and .13) of three bricks in     Unknown      > state. From command line point of view all bricks are up and healthy.      >      > it looks like engine thinks that gluster service is disabled in     cluster      > but I cannot enable it because checkbox is disabled. In my other (FC      > based) oVirt instance Gluster Service checkbox is not selected     but not      > disabled. So I am interested what could make that checkbox     inactive...

    well, on db side it looks like cluster has gluster_service disabled...

    engine=# select virt_service, gluster_service from cluster;        virt_service | gluster_service     --------------+-----------------        t            | f     (1 row)

    still don't know why the checkbox is disabled. Would it be safe to     enabled gluster_service directly in db? I suppose no... :-)

    Cheers,

    Jiri

     >      >> For the vdsm, you can always run      >> '/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start' which is     executed      >> by the vdsmd.service before every start (ExecStartPre stanza)     and see      >> if it complains about something.      >      > [root@ovirt-hci03 <mailto:root@ovirt-hci03> ~]#     /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start      > vdsm: Running mkdirs      > vdsm: Running configure_vdsm_logs      > vdsm: Running run_init_hooks      > vdsm: Running check_is_configured      > sanlock is configured for vdsm      > lvm is configured for vdsm      > abrt is already configured for vdsm      > Managed volume database is already configured      > Current revision of multipath.conf detected, preserving      > libvirt is already configured for vdsm      > vdsm: Running validate_configuration      > SUCCESS: ssl configured to true. No conflicts      > vdsm: Running prepare_transient_repository      > vdsm: Running syslog_available      > vdsm: Running nwfilter      > vdsm: Running dummybr      > vdsm: Running tune_system      > vdsm: Running test_space      > vdsm: Running test_lo      >      > retcode 0, all looks ok...      >      > Cheers,      >      > Jiri      >      >>      >>      >> Best Regards,      >> Strahil Nikolov      >>      >>     On Tue, Jul 12, 2022 at 11:12, Jiří Sléžka      >>     <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote:      >>     On 7/11/22 16:22, Jiří Sléžka wrote:      >>      > On 7/11/22 15:57, Strahil Nikolov wrote:      >>      >> Can you check for AVC denials and the error message like the      >>     described      >>      >> in      >>      >>      >>      >>

https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183

<https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183>      >>      >>

<https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183

<https://github.com/gluster/glusterfs-selinux/issues/27#issue-1097225183>      >>      >?      >>      >      >>      > thanks for reply, there are two unrelated (qemu-kvm) avc     denials      >>     logged      >>      > (related probably to sanlock recovery)      >>      >      >>      > also I cannot find glustereventsd in any related log...     is it      >> really      >>      > used by vdsm-gluster?      >>      >      >>      > this service runs on no hosts      >>      >      >>      > systemctl status glustereventsd      >>      > ● glustereventsd.service - Gluster Events Notifier      >>      >     Loaded: loaded      >> (/usr/lib/systemd/system/glustereventsd.service;      >>      > disabled; vendor preset: disabled)      >>      >     Active: inactive (dead)      >>      >>     it looks like root of the problem is that Gluster service is      >>     disabled in      >>     cluster settings and cannot be enabled. But it was enabled     before...      >>     also I have to manually install vdsm-gluster when I     (re)install new      >>     host, but bricks from this host are in unknown state in     admin. Maybe      >>     vdsm-gluster is not correctly configured? Maybe     glustereventsd is not      >>     running? I am just guessing...      >>      >>     I have no access to other HCI installation so I cannot compare      >>     differences.      >>      >>     I would be really happy if someone could tell me what     circumstances      >>     could disable Gluster service checkbox in admin and how to     enable it      >>     again...      >>      >>     Cheers,      >>      >>     Jiri      >>      >>      >>      >      >>      > Cheers,      >>      >      >>      > Jiri      >>      >      >>      >      >>      >>      >>      >>      >>      >> Best Regards,      >>      >> Strahil Nikolov      >>      >>      >>      >>     On Mon, Jul 11, 2022 at 16:44, Jiří Sléžka      >>      >>     <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote:      >>      >>     Hello,      >>      >>      >>      >>     On 7/11/22 14:34, Strahil Nikolov wrote:      >>      >>      > Can you check something on the host:      >>      >>      > cat /etc/glusterfs/eventsconfig.json      >>      >>      >>      >>     cat /etc/glusterfs/eventsconfig.json      >>      >>     {      >>      >>          "log-level": "INFO",      >>      >>          "port": 24009,      >>      >>          "disable-events-log": false      >>      >>     }      >>      >>      >>      >>      >>      >>      > semanage port -l | grep $(awk -F ':' '/port/      >>     {gsub(",","",$2);      >>      >> print      >>      >>      > $2}' /etc/glusterfs/eventsconfig.json)      >>      >>      >>      >>     semanage port -l | grep 24009      >>      >>      >>      >>     returns empty set, it looks like this port is not     labeled      >>      >>      >>      >>     Cheers,      >>      >>      >>      >>     Jiri      >>      >>      >>      >>      >      >>      >>      > Best Regards,      >>      >>      > Strahil Nikolov      >>      >>      > В понеделник, 11 юли 2022 г., 02:18:57 ч.     Гринуич+3, Jiří      >>     Sléžka      >>      >>      > <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>      >>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>>> написа:      >>      >>      >      >>      >>      >      >>      >>      > Hi,      >>      >>      >      >>      >>      > I would like to change CPU Type in my oVirt     4.4.10 HCI      >>     cluster      >>      >>     (based on      >>      >>      > 3 glusterfs/virt hosts). When I try to I got this     error      >>      >>      >      >>      >>      > Error while executing action: Cannot disable gluster      >>     service on      >>      >> the      >>      >>      > cluster as it contains volumes.      >>      >>      >      >>      >>      > As I remember I had Gluster Service enabled on this      >>     cluster but      >>      >>     now both      >>      >>      > (Enable Virt Services and Enable Gluster Service)      >>     checkboxes are      >>      >>     grayed      >>      >>      > out and Gluster Service is unchecked.      >>      >>      >      >>      >>      > Also Storage / Volumes displays my volumes... well,      >>     displays one      >>      >>     brick      >>      >>      > on particular host in unknown state (? mark)     which is new      >>      >>     situation. As      >>      >>      > I can see from command line all bricks are online, no      >>     healing in      >>      >>      > progress, all looks good...      >>      >>      >      >>      >>      > I am not sure if the second issue is relevant to     first      >>     one so main      >>      >>      > question is how can I (re)enable gluster service     in my      >>     cluster?      >>      >>      >      >>      >>      > Thanks in advance,      >>      >>      >      >>      >>      > Jiri      >>      >>      > _______________________________________________      >>      >>      > Users mailing list -- users@ovirt.org     <mailto:users@ovirt.org>      >>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>     <mailto:users@ovirt.org <mailto:users@ovirt.org>      >>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>>      >>      >>     <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>      >>     <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>>>      >>      >>      > To unsubscribe send an email to     users-leave@ovirt.org <mailto:users-leave@ovirt.org>      >>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      >>      >>     <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>>>      >>      >>      >>      >>      > <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >>     <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>> <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>>>      >>      >>      > Privacy Statement:      >> https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html >>>      >>      >>      > <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>>>      >>      >>      > oVirt Code of Conduct:      >>      >>      >      >> https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >>      >> <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/ >>>      >>      >>      >      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >>      >> <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>>>      >>      >>      > List Archives:      >>      >>      >      >>      >>      >>      >>      >>      >>

https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >      >>      >>      >>      >>      >>      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >      >>      >>      >>      >>      >      >>      >>      >      >>      >>      >>      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >      >>      >>      >>      >>      >>      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

     >      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S4NVCQ33ZSJSHR7P7K7OICSA5F253BVA/>>>>

     >>      >>      >>      >>      >>      >>      >>      >      >>      >      >>      > _______________________________________________      >>      > Users mailing list -- users@ovirt.org     <mailto:users@ovirt.org> <mailto:users@ovirt.org     <mailto:users@ovirt.org>>      >>      > To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      >>      > Privacy Statement:     https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >>     <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >>      > oVirt Code of Conduct:      >> https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >>     <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >>      > List Archives:      >>      >>

https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P...

     >      >>      >>      >>

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2P...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/KJZMKJCPGC2E2PEBYEZWLPX5YUDHD76A/>>

     >>      >>      >      >      > _______________________________________________      > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>      > To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      > Privacy Statement: https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      > oVirt Code of Conduct:     https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      > List Archives:

https://lists.ovirt.org/archives/list/users@ovirt.org/message/YVDPYIPFHHFEVZ...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/YVDPYIPFHHFEVZZMMEGDVJCVL5DXDPIW/>

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LA5YEMMY4UWHV4...

Dne 7/14/22 v 21:21 Strahil Nikolov napsal(a):

Go to the UI, select the volume , pres 'Start' and mark the checkbox for 'Force'-fully start .

well, it worked :-) Now all bricks are in UP state. In fact from commandline point of view all volumes were active and all bricks up all the time.

At least it should update the engine that everything is running . Have you checked if the checkmark for the Gluster service is available if you set the Host into maintenance?

which host do you mean? If all hosts in the cluster I have to plan an outage... will try... Thanks, Jiri

Best Regards, Strahil Nikolov

On Thu, Jul 14, 2022 at 16:08, Jiří Sléžka <jiri.slezka@slu.cz> wrote: _______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>

On 7/16/22 07:53, Strahil Nikolov wrote:

Try first with a single host. Set it into maintenance and check if the checkmark is available.

setting single host to maintenance didn't change state of the gluster services checkbox in cluster settings.

If not, try to 'reinstall' (UI, Hosts, Installation, Reinstall) the host. During the setup, it should give you to update if the host can run the HE and it should allow you to select the checkmark for Gluster.

well, in my oVirt install there is no way to setup glusterfs services during host reinstall. There are only choices to configure firewall, activate host after install, reboot host after install and deploy/undeploy hosted engine... I think that gluster related stuff is installed automatically as it is configured on cluster level (where in my case are gluster services disabled).

Let's work with a single node before being so drastic and outage-ing a cluster.

Cheers, Jiri

Best Regards, Strahil Nikolov

On Thu, Jul 14, 2022 at 23:03, Jiří Sléžka <jiri.slezka@slu.cz> wrote: Dne 7/14/22 v 21:21 Strahil Nikolov napsal(a): > Go to the UI, select the volume , pres 'Start' and mark the checkbox for > 'Force'-fully start .

well, it worked :-) Now all bricks are in UP state. In fact from commandline point of view all volumes were active and all bricks up all the time.

> At least it should update the engine that everything is running . > Have you checked if the checkmark for the Gluster service is available > if you set the Host into maintenance?

which host do you mean? If all hosts in the cluster I have to plan an outage... will try...

Thanks,

Jiri

> > Best Regards, > Strahil Nikolov > >    On Thu, Jul 14, 2022 at 16:08, Jiří Sléžka >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote: >    _______________________________________________ >    Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>> >    To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org>

>    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>> >    Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >    <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>> >    oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >    <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>> >    List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/> > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>> >

On 7/19/22 22:40, Strahil Nikolov wrote:

Then, just ensure that the glusterd.service is enabled on all hosts and leave it as it is.

If it worries you, you will have to move one of the hosts in another cluster (probably a new one) and slowly migrate the VMs from the old to the new one. Yet, if you use only 3 hosts that can put your VMs in risk (new cluster having a single host could lead to downtimes).

well, it blocks me from any changes on cluster so it is serious problem... but personally I don't like this "new cluster and migration" approach :-(

To be honest, I wouldn't change DB if it's a productive cluster. If you decide to go that one -> make an engine backup before that.

Would anyone from oVirt/gluster developers have a look? Thanks in advance, Jiri

Best Regards, Strahil Nikolov

On Tue, Jul 19, 2022 at 12:25, Jiří Sléžka <jiri.slezka@slu.cz> wrote: On 7/16/22 07:53, Strahil Nikolov wrote: > Try first with a single host. Set it into maintenance and check if the > checkmark is available.

setting single host to maintenance didn't change state of the gluster services checkbox in cluster settings.

> If not, try to 'reinstall' (UI, Hosts, Installation, Reinstall) the > host. During the setup, it should give you to update if the host can run > the HE and it should allow you to select the checkmark for Gluster.

well, in my oVirt install there is no way to setup glusterfs services during host reinstall. There are only choices to configure firewall, activate host after install, reboot host after install and deploy/undeploy hosted engine...

I think that gluster related stuff is installed automatically as it is configured on cluster level (where in my case are gluster services disabled).

> Let's work with a single node before being so drastic and outage-ing a > cluster.

Cheers,

Jiri

> > Best Regards, > Strahil Nikolov > >    On Thu, Jul 14, 2022 at 23:03, Jiří Sléžka >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote: >    Dne 7/14/22 v 21:21 Strahil Nikolov napsal(a): >      > Go to the UI, select the volume , pres 'Start' and mark the >    checkbox for >      > 'Force'-fully start . > >    well, it worked :-) Now all bricks are in UP state. In fact from >    commandline point of view all volumes were active and all bricks up all >    the time. > >      > At least it should update the engine that everything is running . >      > Have you checked if the checkmark for the Gluster service is >    available >      > if you set the Host into maintenance? > >    which host do you mean? If all hosts in the cluster I have to plan an >    outage... will try... > >    Thanks, > >    Jiri > >      > >      > Best Regards, >      > Strahil Nikolov >      > >      >    On Thu, Jul 14, 2022 at 16:08, Jiří Sléžka >      >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz> <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote: >      >    _______________________________________________ >      >    Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>> >    <mailto:users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>>> >      >    To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>> > >      >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>> >      >    Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >    <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>> >      >    <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >    <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>>> >      >    oVirt Code of Conduct: >      > https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >    <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>> >      > <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >    <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>>> >      >    List Archives: >      > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/> > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>> >      > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/> > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>> >      > >

_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVK... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVKK4LTFJEYR27B76ZEWB/>

By the way, have you tried to set each host into maintenanceand then 'Reinstall' from the Admin Portal ? Best Regards,Strahil Nikolov  On Sun, Jul 24, 2022 at 0:53, Strahil Nikolov<hunter86_bg@yahoo.com> wrote: Did you identify any errors in the Engine log that could provide any clue ? Best Regards,Strahil Nikolov  On Wed, Jul 20, 2022 at 16:15, Jiří Sléžka<jiri.slezka@slu.cz> wrote: On 7/19/22 22:40, Strahil Nikolov wrote:

Then, just ensure that the glusterd.service is enabled on all hosts and leave it as it is.

If it worries you, you will have to move one of the hosts in another cluster (probably a new one) and slowly migrate the VMs from the old to the new one. Yet, if you use only 3 hosts that can put your VMs in risk (new cluster having a single host could lead to downtimes).

well, it blocks me from any changes on cluster so it is serious problem... but personally I don't like this "new cluster and migration" approach :-(

To be honest, I wouldn't change DB if it's a productive cluster. If you decide to go that one -> make an engine backup before that.

Would anyone from oVirt/gluster developers have a look? Thanks in advance, Jiri

Best Regards, Strahil Nikolov

    On Tue, Jul 19, 2022 at 12:25, Jiří Sléžka     <jiri.slezka@slu.cz> wrote:     On 7/16/22 07:53, Strahil Nikolov wrote:       > Try first with a single host. Set it into maintenance and check     if the       > checkmark is available.

    setting single host to maintenance didn't change state of the gluster     services checkbox in cluster settings.

      > If not, try to 'reinstall' (UI, Hosts, Installation, Reinstall) the       > host. During the setup, it should give you to update if the host     can run       > the HE and it should allow you to select the checkmark for Gluster.

    well, in my oVirt install there is no way to setup glusterfs services     during host reinstall. There are only choices to configure firewall,     activate host after install, reboot host after install and     deploy/undeploy hosted engine...

    I think that gluster related stuff is installed automatically as it is     configured on cluster level (where in my case are gluster services     disabled).

      > Let's work with a single node before being so drastic and     outage-ing a       > cluster.

    Cheers,

    Jiri

      >       > Best Regards,       > Strahil Nikolov       >       >    On Thu, Jul 14, 2022 at 23:03, Jiří Sléžka       >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote:       >    Dne 7/14/22 v 21:21 Strahil Nikolov napsal(a):       >      > Go to the UI, select the volume , pres 'Start' and mark the       >    checkbox for       >      > 'Force'-fully start .       >       >    well, it worked :-) Now all bricks are in UP state. In fact from       >    commandline point of view all volumes were active and all     bricks up all       >    the time.       >       >      > At least it should update the engine that everything is     running .       >      > Have you checked if the checkmark for the Gluster service is       >    available       >      > if you set the Host into maintenance?       >       >    which host do you mean? If all hosts in the cluster I have to     plan an       >    outage... will try...       >       >    Thanks,       >       >    Jiri       >       >      >       >      > Best Regards,       >      > Strahil Nikolov       >      >       >      >    On Thu, Jul 14, 2022 at 16:08, Jiří Sléžka       >      >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote:       >      >    _______________________________________________       >      >    Users mailing list -- users@ovirt.org     <mailto:users@ovirt.org> <mailto:users@ovirt.org     <mailto:users@ovirt.org>>       >    <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>>       >      >    To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>       >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>       >       >      >    <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>>>       >      >    Privacy Statement:     https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>       >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>       >      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>       >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>>       >      >    oVirt Code of Conduct:       >      >     https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>       >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>       >      >      <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>       >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>>       >      >    List Archives:       >      >       >     https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>       >      <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>       >      >       >      <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>       >      <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>>       >      >       >

    _______________________________________________     Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>     To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>     Privacy Statement: https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>     oVirt Code of Conduct:     https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>     List Archives:     https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVK...     <https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVKK4LTFJEYR27B76ZEWB/>

but some webhook is registered on host... ovirt-hci.mch.local is resolvable (through /etc/hosts) [root@ovirt-hci01 ~]# gluster-eventsapi status Webhooks: http://ovirt-hci.mch.local:80/ovirt-engine/services/glusterevents +-----------+-------------+-----------------------+ | NODE | NODE STATUS | GLUSTEREVENTSD STATUS | +-----------+-------------+-----------------------+ | 10.0.4.12 | UP | OK | | 10.0.4.13 | UP | OK | | localhost | UP | OK | +-----------+-------------+-----------------------+ Jiri Dne 8/3/22 v 00:38 Jiří Sléžka napsal(a):

Dne 7/23/22 v 23:53 Strahil Nikolov napsal(a):

...

Did you identify any errors in the Engine log that could provide any clue ?

unfortunately no.

but funny thing... today I looked into html source of cluster settings page (via Firefox's web developer console). Gluster checkbox has this html code

<input type="checkbox" value="on" id="ClusterPopupView_enableGlusterService" tabindex="17" style="vertical-align: top;" disabled="">

when I edited and removed disabled="" part, I was able check that checkbox. After pressing Ok everything seems to be set but there are finally three relevant errors in the engine.log

2022-08-03 00:22:36,795+02 ERROR [org.ovirt.engine.core.bll.InitGlusterCommandHelper] (default task-2701) [6f4a736] Could not sync webhooks to gluster server 'ovirt-hci03.mch.local': null 2022-08-03 00:22:37,842+02 ERROR [org.ovirt.engine.core.bll.InitGlusterCommandHelper] (default task-2701) [6471654f] Could not sync webhooks to gluster server 'ovirt-hci01.mch.local': null 2022-08-03 00:22:39,051+02 ERROR [org.ovirt.engine.core.bll.InitGlusterCommandHelper] (default task-2701) [6970bf5a] Could not sync webhooks to gluster server 'ovirt-hci02.mch.local': null

Any idea why?

few lines before first error

2022-08-03 00:22:36,501+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-2701) [7078c5b6] START, GlusterServersListVDSCommand(HostName = ovirt-hci01.mch.local, VdsIdVDSCommandParametersBase:{hostId='41722608-413e-4444-a8bb-08ad783ec186'}), log id: 6083708e 2022-08-03 00:22:36,616+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-2701) [7078c5b6] FINISH, GlusterServersListVDSCommand, return: [10.0.3.51/24:CONNECTED, 10.0.4.12:CONNECTED, 10.0.4.13:CONNECTED], log id: 6083708e 2022-08-03 00:22:36,619+02 INFO [org.ovirt.engine.core.bll.gluster.AddGlusterWebhookInternalCommand] (default task-2701) [6f4a736] Running command: AddGlusterWebhookInternalCommand internal: true. Entities affected : ID: d03909e7-aca1-496c-9ff6-4a513c961ae3 Type: Cluster 2022-08-03 00:22:36,624+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterWebhookVDSCommand] (default task-2701) [6f4a736] START, AddGlusterWebhookVDSCommand(HostName = ovirt-hci01.mch.local, GlusterWebhookVDSParameters:{hostId='41722608-413e-4444-a8bb-08ad783ec186'}), log id: 1dfd0a44 2022-08-03 00:22:36,793+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterWebhookVDSCommand] (default task-2701) [6f4a736] FINISH, AddGlusterWebhookVDSCommand, return: , log id: 1dfd0a44 2022-08-03 00:22:36,795+02 ERROR [org.ovirt.engine.core.bll.InitGlusterCommandHelper] (default task-2701) [6f4a736] Could not sync webhooks to gluster server 'ovirt-hci03.mch.local': null

Cheers,

Jiri

...

Best Regards, Strahil Nikolov

    On Wed, Jul 20, 2022 at 16:15, Jiří Sléžka     <jiri.slezka@slu.cz> wrote:     On 7/19/22 22:40, Strahil Nikolov wrote:      > Then, just ensure that the glusterd.service is enabled on all     hosts and      > leave it as it is.      >      > If it worries you, you will have to move one of the hosts in another      > cluster (probably a new one) and slowly migrate the VMs from the     old to      > the new one.      > Yet, if you use only 3 hosts that can put your VMs in risk (new     cluster      > having a single host could lead to downtimes).

    well, it blocks me from any changes on cluster so it is serious     problem... but personally I don't like this "new cluster and migration"     approach :-(

     > To be honest, I wouldn't change DB if it's a productive cluster.     If you      > decide to go that one -> make an engine backup before that.

    Would anyone from oVirt/gluster developers have a look?

    Thanks in advance,

    Jiri

     >      > Best Regards,      > Strahil Nikolov      >      >      >      >      >      >    On Tue, Jul 19, 2022 at 12:25, Jiří Sléžka      >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote:      >    On 7/16/22 07:53, Strahil Nikolov wrote:      >      > Try first with a single host. Set it into maintenance and     check      >    if the      >      > checkmark is available.      >      >    setting single host to maintenance didn't change state of the     gluster      >    services checkbox in cluster settings.      >      >      > If not, try to 'reinstall' (UI, Hosts, Installation,     Reinstall) the      >      > host. During the setup, it should give you to update if     the host      >    can run      >      > the HE and it should allow you to select the checkmark for     Gluster.      >      >    well, in my oVirt install there is no way to setup glusterfs     services      >    during host reinstall. There are only choices to configure     firewall,      >    activate host after install, reboot host after install and      >    deploy/undeploy hosted engine...      >      >    I think that gluster related stuff is installed automatically     as it is      >    configured on cluster level (where in my case are gluster services      >    disabled).      >      >      > Let's work with a single node before being so drastic and      >    outage-ing a      >      > cluster.      >      >      >    Cheers,      >      >    Jiri      >      >      >      >      > Best Regards,      >      > Strahil Nikolov      >      >      >      >    On Thu, Jul 14, 2022 at 23:03, Jiří Sléžka      >      >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote:      >      >    Dne 7/14/22 v 21:21 Strahil Nikolov napsal(a):      >      >      > Go to the UI, select the volume , pres 'Start' and     mark the      >      >    checkbox for      >      >      > 'Force'-fully start .      >      >      >      >    well, it worked :-) Now all bricks are in UP state. In     fact from      >      >    commandline point of view all volumes were active and all      >    bricks up all      >      >    the time.      >      >      >      >      > At least it should update the engine that everything is      >    running .      >      >      > Have you checked if the checkmark for the Gluster     service is      >      >    available      >      >      > if you set the Host into maintenance?      >      >      >      >    which host do you mean? If all hosts in the cluster I     have to      >    plan an      >      >    outage... will try...      >      >      >      >    Thanks,      >      >      >      >    Jiri      >      >      >      >      >      >      >      > Best Regards,      >      >      > Strahil Nikolov      >      >      >      >      >      >    On Thu, Jul 14, 2022 at 16:08, Jiří Sléžka      >      >      >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>      >    <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>     <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>>> wrote:      >      >      >    _______________________________________________      >      >      >    Users mailing list -- users@ovirt.org     <mailto:users@ovirt.org>      >    <mailto:users@ovirt.org <mailto:users@ovirt.org>>     <mailto:users@ovirt.org <mailto:users@ovirt.org>      >    <mailto:users@ovirt.org <mailto:users@ovirt.org>>>      >      >    <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>      >    <mailto:users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>>>      >      >      >    To unsubscribe send an email to     users-leave@ovirt.org <mailto:users-leave@ovirt.org>      >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      >      >    <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org> <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>>>      >      >      >      >      >    <mailto:users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>      >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>>>      >      >      >    Privacy Statement:      > https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>>      >      >      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>>>      >      >      >    oVirt Code of Conduct:      >      >      >      > https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >      > <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>>      >      >      >      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >      > <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>>>      >      >      >    List Archives:      >      >      >      >      >      >

https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>

     > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>

     >      >      > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>

     > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>>

     >      >      >      >      >      > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>

     > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>

     >      >      > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>

     > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV5...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/624NH3C5REFDV55K4NPKF6IU4IHG6FPK/>>>>

     >      >      >      >      >      >      >    _______________________________________________      >    Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>     <mailto:users@ovirt.org <mailto:users@ovirt.org>>      >    To unsubscribe send an email to users-leave@ovirt.org     <mailto:users-leave@ovirt.org>      >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>      >    Privacy Statement: https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>      >    <https://www.ovirt.org/privacy-policy.html     <https://www.ovirt.org/privacy-policy.html>>      >    oVirt Code of Conduct:      > https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>      >    <https://www.ovirt.org/community/about/community-guidelines/     <https://www.ovirt.org/community/about/community-guidelines/>>      >    List Archives:      >

https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVK...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVKK4LTFJEYR27B76ZEWB/>

     > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVK...

<https://lists.ovirt.org/archives/list/users@ovirt.org/message/BPORMHV6GU3VVKK4LTFJEYR27B76ZEWB/>>

     >

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/57DUD5CA2XFONI...

Dne 8/3/22 v 03:06 Strahil Nikolov napsal(a):

I think it's related to Compute -> Clusters -> Cluster Name -> Gluster Hooks

I think https://access.redhat.com/solutions/6644151 should solve the problem (you can use a developer subscription to access it).

thanks, I really had 5 hook conflicts /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select id,name,hook_status,content_type,conflict_status from gluster_hooks where conflict_status != 0"; id | name | hook_status | content_type | conflict_status --------------------------------------+--------------------+-------------+--------------+----------------- 517462b4-104d-40d1-ac94-3f8baea8e80b | 30samba-start.sh | ENABLED | TEXT | 4 d428056d-f6fd-4e56-a48a-ccbdd273b774 | 30samba-set.sh | ENABLED | TEXT | 4 a1d8857a-9378-42af-81a8-89a4c75eb52e | 30samba-stop.sh | ENABLED | TEXT | 4 af362bbf-d1ea-4d5e-ae07-492c7ce0966f | 29CTDBsetup.sh | ENABLED | TEXT | 4 d3bdf3df-13f1-48d8-92d9-03d09989516f | 29CTDB-teardown.sh | ENABLED | TEXT | 4 (5 rows) I removed them and then synced gluster hooks in cluster Also diagnostic step rpm -qV glusterfs-server revealed that on one of hosts are some hooks missing [root@ovirt-hci01 ~]# rpm -qV glusterfs-server .M....... c /var/lib/glusterd/glusterd.info missing /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh missing /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh missing /var/lib/glusterd/hooks/1/start/post/S30samba-start.sh missing /var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh missing /var/lib/glusterd/hooks/1/stop/pre/S30samba-stop.sh I reinstalled glusterfs-server package there Well, I do this only to change CPU Type in cluster but now when Gluster services are checked and I try to change CPU Type I got "Error while executing action: Cannot update cluster because the update triggered update of the VMs/Templates and it failed for the following: HostedEngine. To fix the issue, please go to each of them, edit, change the Custom Compatibility Version (or other fields changed previously in the cluster dialog) and press OK. If the save does not pass, fix the dialog validation. After successful cluster update, you can revert your Custom Compatibility Version change (or other changes). If the problem still persists, you may refer to the engine.log file for further details." Strange thing and probably bug - this action disables Gluster services checkbox in cluster!!! Will try to report it... Also I have no idea what is wrong with HostedEngine as there is (as I can see) no custom settings on it... but I cannot change for example memory on it because "There was an attempt to change Hosted Engine VM values that are locked." 2022-08-03 22:32:01,436+02 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-3193) [b93958d9-b27d-4f1b-97f0-d78312c2d346] Validation of action 'UpdateVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__UPDATE,VAR__TYPE__VM,VM_CANNOT_UPDATE_HOSTED_ENGINE_FIELD Cheers, Jiri

Best Regards, Strahil Nikolov

On Wed, Aug 3, 2022 at 1:51, Jiří Sléžka <jiri.slezka@slu.cz> wrote: _______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWB... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWBYGHSIGNVIUGL4EFWT5/>

Hi, finally this post helped https://lists.ovirt.org/archives/list/users@ovirt.org/message/CL4MI3IJH6MPDX... invisible locked entry is missing time_zone in HostedEngine configuration... /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "update vm_static set time_zone='Etc/GMT' where vm_name='HostedEngine'" after this I can change CPU type in cluster and gluster services checkbox stay checked. Thanks for support, Jiri On 8/4/22 20:38, Strahil Nikolov wrote:

Go to the host running the HostedEngine VM and dump the xml via virsh. Then power cycle the engine and check if it fixed the issue with the CPU.

Best Regards, Strahil Nikolov

On Wed, Aug 3, 2022 at 23:58, Jiří Sléžka <jiri.slezka@slu.cz> wrote: Dne 8/3/22 v 03:06 Strahil Nikolov napsal(a): > I think it's related to Compute -> Clusters -> Cluster Name -> Gluster Hooks > > I think https://access.redhat.com/solutions/6644151 <https://access.redhat.com/solutions/6644151 >should solve the > problem (you can use a developer subscription to access it).

thanks, I really had 5 hook conflicts

/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select id,name,hook_status,content_type,conflict_status from gluster_hooks where conflict_status != 0";

                  id                  |        name        | hook_status | content_type | conflict_status --------------------------------------+--------------------+-------------+--------------+-----------------   517462b4-104d-40d1-ac94-3f8baea8e80b | 30samba-start.sh  | ENABLED   | TEXT        |              4   d428056d-f6fd-4e56-a48a-ccbdd273b774 | 30samba-set.sh    | ENABLED   | TEXT        |              4   a1d8857a-9378-42af-81a8-89a4c75eb52e | 30samba-stop.sh    | ENABLED   | TEXT        |              4   af362bbf-d1ea-4d5e-ae07-492c7ce0966f | 29CTDBsetup.sh    | ENABLED   | TEXT        |              4   d3bdf3df-13f1-48d8-92d9-03d09989516f | 29CTDB-teardown.sh | ENABLED   | TEXT        |              4 (5 rows)

I removed them and then synced gluster hooks in cluster

Also diagnostic step

rpm -qV glusterfs-server

revealed that on one of hosts are some hooks missing

[root@ovirt-hci01 <mailto:root@ovirt-hci01> ~]# rpm -qV glusterfs-server .M.......  c /var/lib/glusterd/glusterd.info missing    /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh missing    /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh missing    /var/lib/glusterd/hooks/1/start/post/S30samba-start.sh missing    /var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh missing    /var/lib/glusterd/hooks/1/stop/pre/S30samba-stop.sh

I reinstalled glusterfs-server package there

Well, I do this only to change CPU Type in cluster but now when Gluster services are checked and I try to change CPU Type I got

"Error while executing action: Cannot update cluster because the update triggered update of the VMs/Templates and it failed for the following: HostedEngine. To fix the issue, please go to each of them, edit, change the Custom Compatibility Version (or other fields changed previously in the cluster dialog) and press OK. If the save does not pass, fix the dialog validation. After successful cluster update, you can revert your Custom Compatibility Version change (or other changes). If the problem still persists, you may refer to the engine.log file for further details."

Strange thing and probably bug - this action disables Gluster services checkbox in cluster!!! Will try to report it...

Also I have no idea what is wrong with HostedEngine as there is (as I can see) no custom settings on it... but I cannot change for example memory on it because "There was an attempt to change Hosted Engine VM values that are locked."

2022-08-03 22:32:01,436+02 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-3193) [b93958d9-b27d-4f1b-97f0-d78312c2d346] Validation of action 'UpdateVm' failed for user admin@internal-authz. <mailto:admin@internal-authz.> Reasons: VAR__ACTION__UPDATE,VAR__TYPE__VM,VM_CANNOT_UPDATE_HOSTED_ENGINE_FIELD

Cheers,

Jiri

> > Best Regards, > Strahil Nikolov > >    On Wed, Aug 3, 2022 at 1:51, Jiří Sléžka >    <jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>> wrote: >    _______________________________________________ >    Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org>> >    To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> >    <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>> >    Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >    <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>> >    oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >    <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>> >    List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWB... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWBYGHSIGNVIUGL4EFWT5/> > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWB... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/HNGTNEBDBB2GWBYGHSIGNVIUGL4EFWT5/>> >

_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MUD4TF6MT33PNG... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/MUD4TF6MT33PNGVCQPLHSABJ3VKX64YG/>