- Users - oVirt List Archives

libvirtd.service is down
by jenia.ivlev＠gmail.com 22 Mar '21

22 Mar '21

Hello. I have an error in my oVirt node 4.4: internal error: "Network is already in use by interface ens3" This is the output from journalctl: [root@localhost ~]# sudo journalctl --unit libvirtd.service -- Logs begin at Sun 2021-03-21 16:13:11 UTC, end at Sun 2021-03-21 16:20:37 UTC. -- Mar 21 16:13:24 localhost.localdomain systemd[1]: Starting Virtualization daemon... Mar 21 16:13:25 localhost.localdomain systemd[1]: Started Virtualization daemon. Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: libvirt version: 6.6.0, package: 13.el8 (CBS <cbs(a)centos.org>, 2021-03-01-13:29:02, ) Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: hostname: localhost.localdomain Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: internal error: Network is already in use by interface ens3 Mar 21 16:15:25 localhost.localdomain systemd[1]: libvirtd.service: Succeeded. Mar 21 16:19:30 localhost.localdomain systemd[1]: Starting Virtualization daemon... Mar 21 16:19:30 localhost.localdomain systemd[1]: Started Virtualization daemon. And this is the output from systemctl: [root@localhost ~]# sudo systemctl status libvirtd.service â libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/libvirtd.service.d ââunlimited-core.conf Active: inactive (dead) since Sun 2021-03-21 16:15:25 UTC; 2min 36s ago Docs: man:libvirtd(8) https://libvirt.org Process: 1626 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=0/SUCCESS) Main PID: 1626 (code=exited, status=0/SUCCESS) Mar 21 16:13:24 localhost.localdomain systemd[1]: Starting Virtualization daemon... Mar 21 16:13:25 localhost.localdomain systemd[1]: Started Virtualization daemon. Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: libvirt version: 6.6.0, package: 13.el8 (CBS <cbs(a)centos.org>, 2021-03-01-13:29:02, ) Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: hostname: localhost.localdomain Mar 21 16:13:25 localhost.localdomain libvirtd[1626]: internal error: Network is already in use by interface ens3 Mar 21 16:15:25 localhost.localdomain systemd[1]: libvirtd.service: Succeeded. I managed to successfully start libvirtd.service by doing systemctl start libvirtd.service but I want to correct this bug somehow. Anyone encountered this before? Thanks

2 1

Re: oVirt process that writes to Gluster mount
by Ben 22 Mar '21

22 Mar '21

Well, I'll be -- you're absolutely right, and I'm a bit embarrassed I didn't consider that before. The node that's not healing shows connections from 2 FUSE clients, which I expect, and 3 glustershd, which I also expect. [root@gluster1 ~]# gluster volume status ssd-san client-list Client connections for volume ssd-san Name count ----- ------ glustershd 3 fuse 2 total clients for volume ssd-san : 5 ----------------------------------------------------------------- But the secondary node, which is constantly healing, *shows that it's missing a FUSE connection*: [root@gluster2 ~]# gluster volume status ssd-san client-list Client connections for volume ssd-san Name count ----- ------ glustershd 3 fuse 1 total clients for volume ssd-san : 4 ----------------------------------------------------------------- I had to restart the glusterd service on node 2 twice before the FUSE client reconnected and stayed connected. Thanks a ton, I really appreciate your help! On Mon, Mar 22, 2021 at 12:37 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: > Healing must happen only after a maintenance (for example patch + reboot) > on any of the nodes . > Once the node is up , the FUSE client (any host) should reconnect to all > gluster bricks and write to all bricks simultaneously. > > If you got constant healing, this indicates that a client is not writing > to all bricks. > > Check with the following command if there is such client: > 'gluster volume status *VOLNAME* clients' > > > Best Regards, > Strahil Nikolov > > On Mon, Mar 22, 2021 at 3:24, Ben > <gravyfish(a)gmail.com> wrote: > Sorry, just saw this -- I'm not sure I understand what you mean, but in > any case, the healing process does complete when I stop all of my VMs, > which I believe indicates that something about the oVirt writes to Gluster > is causing the problem in the first place. > > On Sun, Mar 14, 2021 at 8:06 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> > wrote: > > Are you sure that gluster volume client's count is the same on all nodes ? > > Best Regards, > Strahil Nikolov > > On Sat, Mar 13, 2021 at 23:58, Ben > <gravyfish(a)gmail.com> wrote: > Hi, I could use some help with a problem I'm having with the Gluster > storage servers I use in my oVirt data center. I first noticed the problem > when files would constantly heal after rebooting one of the Gluster nodes > -- in the replica 2/arbiter, the node that remained online and the arbiter > would begin healing files and never finish. > > I raised the issue with the helpful folks over at Gluster: > https://github.com/gluster/glusterfs/issues/2226 > > The short version is this: after running a tcpdump and noticing malformed > RPC calls to Gluster from one of my oVirt nodes, they're looking for a > stack trace of whatever process is running I/O on the Gluster cluster from > oVirt in order to figure out what it's doing and if the write problems > could cause the indefinite healing I'm seeing. After checking the qemu > PIDs, it doesn't look like they are actually performing the writes -- is > there a particular part of the oVirt stack I can look at to find the write > operations to Gluster? I don't see anything else doing read/write on the VM > image files on the Gluster mount, but I could be missing something. > > NB: I'm using a traditional Gluster setup with the FUSE client, not > hyperconverged. > > Thanks in advance for any assistance. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/76H5BY6IH55GC… > >

1 0

oVirt process that writes to Gluster mount
by Ben 22 Mar '21

22 Mar '21

Hi, I could use some help with a problem I'm having with the Gluster storage servers I use in my oVirt data center. I first noticed the problem when files would constantly heal after rebooting one of the Gluster nodes -- in the replica 2/arbiter, the node that remained online and the arbiter would begin healing files and never finish. I raised the issue with the helpful folks over at Gluster: https://github.com/gluster/glusterfs/issues/2226 The short version is this: after running a tcpdump and noticing malformed RPC calls to Gluster from one of my oVirt nodes, they're looking for a stack trace of whatever process is running I/O on the Gluster cluster from oVirt in order to figure out what it's doing and if the write problems could cause the indefinite healing I'm seeing. After checking the qemu PIDs, it doesn't look like they are actually performing the writes -- is there a particular part of the oVirt stack I can look at to find the write operations to Gluster? I don't see anything else doing read/write on the VM image files on the Gluster mount, but I could be missing something. NB: I'm using a traditional Gluster setup with the FUSE client, not hyperconverged. Thanks in advance for any assistance.

1 1

virbr0 down in oVirt node 4.4, cannot connect to console
by jenia.ivlev＠gmail.com 21 Mar '21

21 Mar '21

Hello. I'm trying to deploy a oVirt instance on my computer in a virtual machine. I started off by installing into a VM the oVirt Node 4.4 and I expected to be able to login into the web console from my browser at 192.168.122.1:9090. But instead what I realized is that inside the oVirt Node, the virbr0 interface 192.168.122.1 is down: https://photos.app.goo.gl/gARdRKdfvwDALtEC8 When I tried to bridge the virbr0 interface back up, it says "failed to access /etc/resolve.conf: No such file or directory": https://photos.app.goo.gl/epVqSjPsrN41ZVCZ8 What did I do wrong? Thanks

1 1

oVirt4.4.5 and gluster op-version
by Jiří Sléžka 20 Mar '21

20 Mar '21

Hello, I have just upgraded my 2hosts+1arbiter hci cluster to 4.4.5. Gluster is not managed by oVirt and currently is in op.version 70200 gluster volume get all cluster.op-version cluster.op-version 70200 after gluster upgrade max-op-version is gluster volume get all cluster.max-op-version cluster.max-op-version 80000 can I (or should I) switch to 80000 op-version? Thanks, Jiri

2 2

What would be the process to restore master host.
by miguel.garcia＠toshibagcs.com 20 Mar '21

20 Mar '21

We have a cluster with one master and 3 ovirt-nodes and the master server got the hard disk fried however the vms are still running and working in shape. We have concerns if restore master by reinstall OS and restore from latest ovirt backup would be enough or if we had to resintall ovirt cluster from scratch along with each node. Thanks in advance for your input.

2 2

Networking
by Jason Alexander Hazen Valliant-Saunders 19 Mar '21

19 Mar '21

I'm trying to create vlans to mirror my vlans on my external switch. is there a simple way to simply define a port trunk and have ovirt engine pass the traffic on? In VMWare; this can be accomplished by nic teaming/trunking (802.3 ad) and then pass the traffic to the switch. From what I have read in oVirt I need to define ovs switches first; I'd like to simply pass L2 to the switch directly. Regards, Hazen

3 5

Host Add - Host not found check DNS or /etc/hosts
by penguin pages 19 Mar '21

19 Mar '21

Fresh install CentOS8 Streams. Gluster wizard runs and deploys. Engine installs on node. I copy over /etc/hosts for primary site resources to ovirt engine so it can find primary cluster nodes without DNS being booted / up at that time. I check ovirt engine can ssh to the three nodes... so as root I generate SSH key and do ssh-copy-id to all three nodes. So as much as I can validate communication is working.. i have. When I "add host" -> use FDN -> input root password But it shows up in GUI host list with "install fails" Only message is : [root@ovirt01 ~]# cat /var/log/ovirt-engine/ansible-runner-service.log 2021-03-18 09:44:15,998 - runner_service.controllers.hosts - DEBUG - Adding host odin.penguinpages.local to group ovirt 2021-03-18 09:44:16,344 - runner_service.services.hosts - ERROR - SSH - NOCONN:SSH error - 'odin.penguinpages.local' not found; check DNS or /etc/hosts 2021-03-18 10:07:15,970 - root - INFO - Analysing local configuration options from /etc/ansible-runner-service/config.yaml 2021-03-18 10:07:15,975 - root - INFO - - setting playbooks_root_dir to /usr/share/ovirt-engine/ansible-runner-service-project 2021-03-18 10:07:15,975 - root - INFO - - setting ssh_private_key to /etc/pki/ovirt-engine/keys/engine_id_rsa 2021-03-18 10:07:15,975 - root - INFO - - setting port to 50001 2021-03-18 10:07:15,975 - root - INFO - - setting target_user to root 2021-03-18 10:07:15,975 - root - INFO - - setting log_path to /var/log/ovirt-engine 2021-03-18 10:07:15,975 - root - INFO - Analysing runtime overrides from environment variables 2021-03-18 10:07:15,975 - root - INFO - No configuration settings overridden 2021-03-18 10:07:15,995 - root - INFO - Loaded logging configuration from /etc/ansible-runner-service/logging.yaml 2021-03-18 10:07:16,004 - runner_service.controllers.hosts - DEBUG - Request received, content-type :None 2021-03-18 10:07:16,005 - runner_service.controllers.hosts - INFO - 127.0.0.1 - GET /api/v1/hosts/thor.penguinpages.local 2021-03-18 10:07:16,013 - runner_service.controllers.hosts - DEBUG - Request received, content-type :application/json; charset=UTF-8 2021-03-18 10:07:16,013 - runner_service.controllers.hosts - INFO - 127.0.0.1 - POST /api/v1/hosts/thor.penguinpages.local/groups/ovirt 2021-03-18 10:07:16,013 - runner_service.controllers.hosts - DEBUG - additional args received 2021-03-18 10:07:16,014 - runner_service.controllers.hosts - DEBUG - Adding host thor.penguinpages.local to group ovirt 2021-03-18 10:07:16,339 - runner_service.services.hosts - ERROR - SSH - NOCONN:SSH error - 'thor.penguinpages.local' not found; check DNS or /etc/hosts 2021-03-18 10:07:38,333 - runner_service.controllers.hosts - DEBUG - Request received, content-type :None 2021-03-18 10:07:38,334 - runner_service.controllers.hosts - INFO - 127.0.0.1 - GET /api/v1/hosts/thor.penguinpages.local 2021-03-18 10:07:38,339 - runner_service.controllers.hosts - DEBUG - Request received, content-type :application/json; charset=UTF-8 2021-03-18 10:07:38,339 - runner_service.controllers.hosts - INFO - 127.0.0.1 - POST /api/v1/hosts/thor.penguinpages.local/groups/ovirt 2021-03-18 10:07:38,339 - runner_service.controllers.hosts - DEBUG - additional args received 2021-03-18 10:07:38,340 - runner_service.controllers.hosts - DEBUG - Adding host thor.penguinpages.local to group ovirt 2021-03-18 10:07:38,713 - runner_service.services.hosts - ERROR - SSH - NOCONN:SSH error - 'thor.penguinpages.local' not found; check DNS or /etc/hosts But ssh is fine to all hosts works fine. [root@ovirt01 ~]# ssh thor.penguinpages.local Web console: https://thor.penguinpages.local:9090/ or https://172.16.100.101:9090/ Last login: Wed Mar 17 11:27:34 2021 from 172.16.101.103 [root@thor ~]# exit logout Connection to thor.penguinpages.local closed. [root@ovirt01 ~]# ssh medusa.penguinpages.local Activate the web console with: systemctl enable --now cockpit.socket Last login: Thu Mar 18 10:06:40 2021 from 172.16.100.186 [root@medusa ~]# exit logout Connection to medusa.penguinpages.local closed. [root@ovirt01 ~]# ssh odin.penguinpages.local Activate the web console with: systemctl enable --now cockpit.socket Last login: Wed Mar 17 16:02:46 2021 from 172.16.100.186 [root@odin ~]# exit logout Connection to odin.penguinpages.local closed. [root@ovirt01 ~]#

2 1

Quick Question: Degraded Gluster
by Christian Reiss 18 Mar '21

18 Mar '21

Hey folks, quick question: I have a 3 node cluster (HC) in a production environment. I recently swapped all the storage SSDs with bigger enterprise grades SSDs. So I took a host down (migrating all VMs away, maintenance, shutdown, fix raid, gluster repair, back online). This worked well and all three nodes are now hardware-wise replaced. The last one is still gluster-healing (roughly 20% done). The other two nodes are happily serving the vms while the third one is still in maintenance. my question is: Can I online the maintenance node while gluster is still healing? Or will will open up a can of worms I would not like? It would be cool having all three nodes back online soon while the gluster is still repairing. We have no urge tho. If there is a risk I would wait. If this is somewhat risk-free and do-able why not :) -- with kind regards, mit freundlichen Gruessen, Christian Reiss

2 1

Upgrading DWH from 4.4.4 to 4.4.5
by Nur Imam Febrianto 18 Mar '21

18 Mar '21

Hi, I’m currently trying to upgrade my oVirt Cluster into 4.4.5. Hosted engine upgrade work successfully. But whenever I tried upgrading DWH on separate machine it keep giving me this error : Failed to execute stage 'Misc configuration': 'OVESETUP_SYSTEM/selinuxBooleans' In engine-setup log : 2021-03-18 23:34:10,976+0700 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-common/apache/selinux.py", line 40, in _misc osetupcons.SystemEnv.SELINUX_BOOLEANS KeyError: 'OVESETUP_SYSTEM/selinuxBooleans' 2021-03-18 23:34:10,977+0700 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': 'OVESETUP_SYSTEM/selinuxBooleans' Using CentOS 8.3 as separate DWH machine. Anybody maybe can help me ? Thanks before. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

1 0