- Users - oVirt List Archives

Re: Help with 4.3.3 re-setup
by Strahil 21 Apr '19

21 Apr '19

Hi Michael, I know how you feel abiut VDO. I wass in the same boat until I starrted using systemd's mount units.Since then I do not have any issues. For now try the following: Boot 1 host via rescue dvd , chroot into the system and install the vdo and kvdo packages (maybe reinstall is more correct) . Once then, try to run the vdo service (still in chroot) and try to make it work. Then, just rebuild the initramfs (dracut -f -k kernel-version) and reboot. Check if your VDO, bricks and Gluster are operational again. If yes, do the other nodes. Once done - run 'gluster volume heal my-volume full' (replace my-volume with actual name). This will force gluster to sync all bricks. P.S.: Now, I'm using gluster snapshot for recovery between engine upgrades (HostedEngine is stopped while makking the snapahot). Best Regards, Strahil NikolovOn Apr 21, 2019 20:44, michael(a)wanderingmad.com wrote: > > I updated my ovirt from 4.3.2 to 4.3.3 and everything went bad, my gluster volumes were constantly dropping out and getting unsynced and everything was just running slower than molasses. So I thought I would reload the hosts from ovirt node to Centos and re-do the storage, thinking that may help. I tried using centos, but I was never able to add centos hosts back to the ovirt engine due to a gluster error in the logs, so I went and downloaded ovirt node 4.3.3, and now after spending an hour reloading all the servers with ovirt node, One of the nodes I can't access the cockpit on, and the other two have broken VDO modules, the modules are installed but don't exist in the kernel, and when I removed the module, I lost all network configuration, and when I re-installed it, I get a scriptlet error on compilation and there is still a missing VDO module. This is now the 3rd time i've hit this VDO issue and i'm really frustrated right now, should I just go back to 4.3.2? I just redownloa > ded the latest 4.3.3 ISO today. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QA2SMYRZCXQYR…

1 0

Help with 4.3.3 re-setup
by michael＠wanderingmad.com 21 Apr '19

21 Apr '19

I updated my ovirt from 4.3.2 to 4.3.3 and everything went bad, my gluster volumes were constantly dropping out and getting unsynced and everything was just running slower than molasses. So I thought I would reload the hosts from ovirt node to Centos and re-do the storage, thinking that may help. I tried using centos, but I was never able to add centos hosts back to the ovirt engine due to a gluster error in the logs, so I went and downloaded ovirt node 4.3.3, and now after spending an hour reloading all the servers with ovirt node, One of the nodes I can't access the cockpit on, and the other two have broken VDO modules, the modules are installed but don't exist in the kernel, and when I removed the module, I lost all network configuration, and when I re-installed it, I get a scriptlet error on compilation and there is still a missing VDO module. This is now the 3rd time i've hit this VDO issue and i'm really frustrated right now, should I just go back to 4.3.2? I just redownloa ded the latest 4.3.3 ISO today.

1 0

Issues post-upgrade from 4.2.8 to 4.3.2
by Vrgotic, Marko 21 Apr '19

21 Apr '19

Dear oVirt team, We have 3 Hosts HA Cluster with SHE engine. Storage is NetApp. Recently we have executed an upgrade from 4.2.8 to 4.3.2. Since then we are seeing strange behavior when trying to put Hosts to Maintenance or manually Migrate VMs between Hosts. The most repeated WARN/ERROR I am observing is following: The UI is constantly throwing an exception when trying to put HA host to maintenance or increase the SPM level. The repeated log line from engine.log I see, with regards or failing to migrate all hosts is: 2019-04-18 13:05:30,760Z WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [63d33dd1] Validation of action 'MigrateVm' failed for user SYSTEM. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL But there is more than enough. Also from UI.log: 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Permutation name: 0D2DB7A91B469CC36C64386E5632FAC5 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Uncaught exception: com.google.gwt.event.shared.Umbrell aException: Exception caught: (TypeError) : oab(...) is null at java.lang.Throwable.Throwable(Throwable.java:70) [rt.jar:1.8.0_201] at java.lang.RuntimeException.RuntimeException(RuntimeException.java:32) [rt.jar:1.8.0_201] at com.google.web.bindery.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:64) [gwt-servlet.jar:] at com.google.gwt.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:25) [gwt-servlet.jar:] at com.google.gwt.event.shared.HandlerManager.$fireEvent(HandlerManager.java:117) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.event.dom.client.DomEvent.fireNativeEvent(DomEvent.java:110) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.DOM.dispatchEvent(DOM.java:1415) [gwt-servlet.jar:] at com.google.gwt.user.client.impl.DOMImplStandard.dispatchEvent(DOMImplStandard.java:312) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.apply(Impl.java:236) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:275) [gwt-servlet.jar:] at Unknown.Su/<(https://ovirt-staging-engine.avinity.tv/ovirt-engine/webadmin/?locale=en_US… line 9 > scriptElement) at Unknown.anonymous(Unknown) Caused by: com.google.gwt.core.client.JavaScriptException: (TypeError) : oab(...) is null at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.$onAddHost(ClusterGuideModel.java:533) at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.executeCommand(ClusterGuideModel.java:617) at org.ovirt.engine.ui.uicommonweb.UICommand.$execute(UICommand.java:163) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget.$lambda$4(AbstractModelBoundPopupPresenterWidget.java:306) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget$lambda$4$Type.onClick(AbstractModelBoundPopupPresenterWidget.java:306) at com.google.gwt.event.dom.client.ClickEvent.dispatch(ClickEvent.java:55) [gwt-servlet.jar:] at com.google.gwt.event.shared.GwtEvent.dispatch(GwtEvent.java:76) [gwt-servlet.jar:] at com.google.web.bindery.event.shared.SimpleEventBus.$doFire(SimpleEventBus.java:173) [gwt-servlet.jar:] ... 12 more Can you give me some advice on how to proceed? This started happening since upgrade. Kindly awaiting your reply, Marko Vrgotic

1 0

VM Snapshots not erasable and not bootable
by Jonathan Baecker 21 Apr '19

21 Apr '19

Hello, I make automatically backups of my VMs and last night there was making some new one. But somehow ovirt could not delete the snapshots anymore, in the log it show that it tried the hole day to delete them but they had to wait until the merge command was done. In the evening the host was totally crashed and started again. Now I can not delete the snapshots manually and I can also not start the VMs anymore. In the web interface I get the message: VM timetrack is down with error. Exit message: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'serial': 'fd3b80fd-49ad-44ac-9efd-1328300582cd', 'index': 0, 'iface': 'scsi', 'apparentsize': '1572864', 'specParams': {}, 'cache': 'none', 'imageID': 'fd3b80fd-49ad-44ac-9efd-1328300582cd', 'truesize': '229888', 'type': 'disk', 'domainID': '9c3f06cf-7475-448e-819b-f4f52fa7d782', 'reqsize': '0', 'format': 'cow', 'poolID': '59ef3a18-002f-02d1-0220-000000000124', 'device': 'disk', 'path': '/rhev/data-center/59ef3a18-002f-02d1-0220-000000000124/9c3f06cf-7475-448e-819b-f4f52fa7d782/images/fd3b80fd-49ad-44ac-9efd-1328300582cd/47c0f42e-8bda-4e3f-8337-870899238788', 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': '47c0f42e-8bda-4e3f-8337-870899238788', 'diskType': 'file', 'alias': 'ua-fd3b80fd-49ad-44ac-9efd-1328300582cd', 'discard': False}. When I check the path permission is correct and there are also files in it. Is there any ways to fix that? Or to prevent this issue in the future? In the attachment I send also the engine.log Regards Jonathan

2 10

Re: Ovirt gluster arbiter within hosted VM
by Strahil 20 Apr '19

20 Apr '19

With GlusterD2 , you can use thin arbiter - which can be deployed in the cloud. Do not try to setup regular arbiter far away from the data bicks or yoir performance will be awful. Best Regards, Strahil NikolovOn Apr 19, 2019 13:28, Alex K <rightkicktech(a)gmail.com> wrote: > > > > On Fri, Apr 19, 2019 at 1:14 PM Scott Worthington <scott.c.worthington(a)gmail.com> wrote: >> >> And where, magically, is that node's storage going to live? > > In the disk of the VM. Let me know if I need to clarify further. >> >> >> You can't fake a proper setup of gluster. > > Not trying to fake it. Trying to find a solution with the available options. >> >> >> On Fri, Apr 19, 2019, 5:00 AM Alex K <rightkicktech(a)gmail.com> wrote: >>> >>> Hi all, >>> >>> I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. >>> >>> What do you think about this setup in regards to stability and performance? >>> I am running ovirt 4.2. >>> >>> Thanx, >>> Alex >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LTDOAMBKVE5BP…

2 1

Re: Ovirt gluster arbiter within hosted VM
by Strahil 20 Apr '19

20 Apr '19

As long as the VM is not hosted by those 2 hyperconverged nodes - there should be no problem. Another option is a small machine with a single SSD. I'm using Lenovo Tiny M-series as an arbiter. Best Regards, Strahil NikolovOn Apr 19, 2019 12:59, Alex K <rightkicktech(a)gmail.com> wrote: > > Hi all, > > I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. > > What do you think about this setup in regards to stability and performance? > I am running ovirt 4.2. > > Thanx, > Alex

1 0

Replica 3 distribute-replicated - data placement and fault tolerance
by Leo David 19 Apr '19

19 Apr '19

Hello Everyone, I did some fio performance tests on a particular vm and I have noticed things that I do not understand regarding how data is placed along the bricks. I am sure this is a lack of knowledge, but I would really appreciate any help in understanding this, I did a bit of research on the internet, but just could't find something relevant. I have one replica 3 distributed-replicated arbitrated volume, across 18 bricks ( 9 nodes, 2 jbods per node ). The volume was created as: node1 - brick1, node-2, brick1,.....node9-brick1, node1 - brick2, node-2, brick2,.....node9-brick2 As far as i've understood, under the hood there are sets of 3 times replicated data ( subvolumes ) which are assigned to the first 3 bricks, next set of replicated data to the next set of 3 bricks, and so on.. Now, I have this testing vm runign one node one. When I've started the fio test, i've noticed incresed gluster traffic from node 1 to node2,4 and 5 So I assumed that the vm disk is data resides on a subvolume allocated to bricks from these hosts. Then I have migrated the vm on node 2, and did the same test. Now the increased treffic is generated from node2 to node1, 4, and 5.. What I do not understand is: - why gluster client ( ovirt host ) is sending data to 3 bricks if the volume is arbitrated - shouldn't it send only to 2 of them ? - why are there 4 brick implicated in this subvolume - what would it be the fault tolerance level in this setup ,ie: how many hosts can I take down and still having the volume serving IO requests; can they be random ? I am sorry for my lack of knowledge, I am just trying to understand what is happening so I can deploy a decent proper setup of an hci environment. Thank you, Leo -- Best regards, Leo David

1 0

Re: Ovirt gluster arbiter within hosted VM
by Alex K 19 Apr '19

19 Apr '19

On Fri, Apr 19, 2019 at 1:14 PM Scott Worthington < scott.c.worthington(a)gmail.com> wrote: > And where, magically, is that node's storage going to live? > In the disk of the VM. Let me know if I need to clarify further. > > You can't fake a proper setup of gluster. > Not trying to fake it. Trying to find a solution with the available options. > > On Fri, Apr 19, 2019, 5:00 AM Alex K <rightkicktech(a)gmail.com> wrote: > >> Hi all, >> >> I have a two node hyper-converged setup which are causing me split-brains >> when network issues are encountered. Since I cannot add a third hardware >> node, I was thinking to add a dedicated guest VM hosted in same >> hyper-converged cluster which would do the arbiter for the volumes. >> >> What do you think about this setup in regards to stability and >> performance? >> I am running ovirt 4.2. >> >> Thanx, >> Alex >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/LTDOAMBKVE5BP… >> >

1 0

Ovirt gluster arbiter within hosted VM
by Alex K 19 Apr '19

19 Apr '19

Hi all, I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. What do you think about this setup in regards to stability and performance? I am running ovirt 4.2. Thanx, Alex

1 0

Ovirt-engine self hostd
by Dennis Perfors 19 Apr '19

19 Apr '19

Hi, We have build Ovirt-engine self hosted. We have created 2 vlans 1 services and 1 student. This works good. Services vlan 192.168.60.0/24 gateway ip 192.168.60.1 on this card Student vlan 192.168.10.0/24 gateway ip 192.168.10.1 on this network card Everything seems to be oke, but the probem is on the vlan. On the node 192.168.60.3, there is no internet connection when we do yum update -y But there is no internet connection on the host. I cant do yum update -y or do anything else. Here more information: ^Chttp://mirror.cj2.nl/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#56 - "Callback aborted" Trying other mirror. ^Chttp://mirror.sitbv.nl/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#56 - "Callback aborted" Trying other mirror. http://mirror.serverius.net/centos/7.6.1810/extras/x86_64/repodata/repomd.x…: [Errno 14] curl#7 - "Failed to connect to 2a03:3f40:1::15: Network is unreachable" [root@bastion ~]# ping nu.nl PING nu.nl (52.85.140.48) 56(84) bytes of data. 64 bytes from server-52-85-140-48.man50.r.cloudfront.net (52.85.140.48): icmp_seq=1 ttl=241 time=12.8 ms ^C [root@bastion ~]# ping alibabia.com PING alibabia.com (137.175.26.243) 56(84) bytes of data. 64 bytes from 137.175.26.243 (137.175.26.243): icmp_seq=1 ttl=115 time=144 ms 64 bytes from 137.175.26.243 (137.175.26.243): icmp_seq=2 ttl=115 time=144 ms ^C ipv4.method: manual ipv4.dns: 1.1.1.1 ipv4.dns-search: -- ipv4.dns-options: "" ipv4.dns-priority: 0 ipv4.addresses: 192.168.60.3/24 ipv4.gateway: 192.168.60.1 ipv4.routes: -- ipv4.route-metric: -1 ipv4.route-table: 0 (unspec) ipv4.ignore-auto-routes: no ipv4.ignore-auto-dns: no public (active) target: default icmp-block-inversion: no interfaces: ovirtmgmt sources: services: ssh dhcpv6-client ovirt-postgres ovirt-https ovn-central-firewall-service ovirt-fence-kdump-listener ovirt-imageio-proxy ovirt-websocket-proxy ovirt-http ovirt-vmconsole-proxy ovirt-provider-ovn https http cockpit libvirt-tls snmp vdsm ovirt-imageio ovirt-vmconsole ports: 9986/tcp 22/tcp 6081/udp 6100/tcp protocols: masquerade: yes forward-ports: port=1337:proto=tcp:toport=22:toaddr=192.168.60.3 port=80:proto=tcp:toport=80:toaddr=192.168.60.5 source-ports: icmp-blocks: rich rules: rule family="ipv4" source address="0.0.0.0/24" accept [root@athena ~]# Portfowarding also works to the host 192.168.60.3. so that is no poiint. But only traffic from the insidehost to outside is not working. Thanks

1 0