August 2017 - Users - oVirt List Archives

High Availability Setup with oVirt
by Anantha Raghava 20 Aug '17

20 Aug '17

This is a multi-part message in MIME format. --------------2016BA6EE2CEFB11BE2C28C2 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I am trying to setup ovirt clusters in two different locations and replicate the data between two sites so that in case of failure in primary site, I can bring the VM up in secondary site in quick time with minimum data loss. Can we do this oVirt? I was thinking of using DRBD, however read somewhere that oVirt plugins are still under development -- Thanks & Regards, Anantha Raghava Do not print this e-mail unless required. Save Paper & trees. --------------2016BA6EE2CEFB11BE2C28C2 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hi,</p> <p>I am trying to setup ovirt clusters in two different locations and replicate the data between two sites so that in case of failure in primary site, I can bring the VM up in secondary site in quick time with minimum data loss. Can we do this oVirt?</p> <p>I was thinking of using DRBD, however read somewhere that oVirt plugins are still under development<br> </p> <div class="moz-signature"> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <title></title> <meta name="generator" content="LibreOffice 5.0.3.2 (Linux)"> <meta name="created" content="00:00:00"> <meta name="changedby" content="Anantha Raghava"> <meta name="changed" content="2016-01-05T17:20:50.677541300"> <meta name="created" content="00:00:00"> <meta name="changedby" content="Anantha Raghava"> <meta name="changed" content="2015-12-20T09:03:26.251763811"> <meta name="created" content="2015-02-21T00:00:00"> <meta name="changedby" content="Anantha Raghava"> <meta name="changed" content="2015-12-20T09:02:11.666821134"> <style type="text/css"> @page { margin: 2cm } p { margin-bottom: 0.25cm; color: #000000; line-height: 120% } address { color: #000000 } a:link { so-language: zxx </style> <p>-- </p> <p style="margin-bottom: 0cm; line-height: 100%"><font face="Times New Roman, serif">Thanks & Regards,</font></p> <p style="margin-bottom: 0cm; line-height: 100%"><br> </p> <address style="line-height: 100%"><font face="Times New Roman, serif">Anantha Raghava</font></address> <br> <p style="margin-bottom: 0cm; line-height: 100%"><font color="#66cc00"><font face="Times New Roman, serif">Do not print this e-mail unless required. Save Paper & trees.</font></font></p> </div> </body> </html> --------------2016BA6EE2CEFB11BE2C28C2--

2 1

vdsm vds.dispatcher errors
by Gary Pedretty 19 Aug '17

19 Aug '17

--Apple-Mail=_3C58C5A6-FF18-4157-8715-87EEC5B8B53D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Why on even a new clean install of a glusterized self-hosted engine = setup do all the hosts fill the messages log with=20 journal: vdsm vds.dispatcher ERROR SSL error during reading data: = unexpected eof journal: vdsm vds.dispatcher ERROR SSL error during reading data: = unexpected eof journal: vdsm vds.dispatcher ERROR SSL error during reading data: = unexpected eof journal: vdsm vds.dispatcher ERROR SSL error during reading data: = unexpected eof journal: vdsm vds.dispatcher ERROR SSL error during reading data: = unexpected eof An entry every few seconds even with no VMs setup and just the hosted = engine running. I know it is listed as a bug on redhat bugzilla page, = but is this going to be normal until version 4.1 comes out? Gary ------------------------------------------------------------------------ Gary Pedretty gary(a)ravnalaska.net = <mailto:gary@eraalaska.net> Systems Manager www.flyravn.com = <http://www.flyravn.com/> Ravn Alaska /\ 907-450-7251 5245 Airport Industrial Road / \/\ 907-450-7238 fax Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your = neighbor as Green, green as far as the eyes can see yourself=E2=80=9D = Matt 22:39 ------------------------------------------------------------------------ --Apple-Mail=_3C58C5A6-FF18-4157-8715-87EEC5B8B53D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><span style=3D"font-family: LucidaGrande;" class=3D"">Why on = even a new clean install of a glusterized self-hosted engine setup do = all the hosts fill the messages log with </span><div class=3D"" = style=3D"font-family: LucidaGrande;"><br class=3D""></div><div class=3D"" = style=3D"font-family: LucidaGrande;">journal: vdsm vds.dispatcher ERROR = SSL error during reading data: unexpected eof</div><div class=3D"" = style=3D"font-family: LucidaGrande;"><div class=3D"">journal: vdsm = vds.dispatcher ERROR SSL error during reading data: unexpected = eof</div></div><div class=3D"" style=3D"font-family: LucidaGrande;"><div = class=3D"">journal: vdsm vds.dispatcher ERROR SSL error during reading = data: unexpected eof</div></div><div class=3D"" style=3D"font-family: = LucidaGrande;"><div class=3D"">journal: vdsm vds.dispatcher ERROR SSL = error during reading data: unexpected eof</div></div><div class=3D"" = style=3D"font-family: LucidaGrande;"><div class=3D"">journal: vdsm = vds.dispatcher ERROR SSL error during reading data: unexpected = eof</div></div><div class=3D"" style=3D"font-family: LucidaGrande;"><br = class=3D""></div><div class=3D"" style=3D"font-family: = LucidaGrande;"><br class=3D""></div><div class=3D"" style=3D"font-family: = LucidaGrande;">An entry every few seconds even with no VMs setup and = just the hosted engine running.  I know it is listed as a bug on = redhat bugzilla page, but is this going to be normal until version 4.1 = comes out?</div><div class=3D"" style=3D"font-family: LucidaGrande;"><br = class=3D""></div><div class=3D"" style=3D"font-family: = LucidaGrande;">Gary</div><div class=3D""><br class=3D""></div><div = class=3D""> <div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><div style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><div style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><div style=3D"text-align: start; = text-indent: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; = -webkit-line-break: after-white-space;" class=3D""><div = style=3D"text-align: start; text-indent: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"text-align: start; text-indent: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><div style=3D"text-align: start; = text-indent: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; = -webkit-line-break: after-white-space;" class=3D""><div = style=3D"text-align: start; text-indent: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"text-align: start; text-indent: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><div style=3D"text-align: start; = text-indent: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; = -webkit-line-break: after-white-space;" class=3D""><font face=3D"Menlo" = style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px;" class=3D""><div = class=3D"">---------------------------------------------------------------= ---------</div><div class=3D"">Gary Pedretty         =                     =            <a = href=3D"mailto:gary@eraalaska.net" = class=3D"">gary(a)ravnalaska.net</a></div><div class=3D"">Systems Manager =                     =                     =  <a href=3D"http://www.flyravn.com" = class=3D"">www.flyravn.com</a></div><div class=3D"">Ravn Alaska   =                     =     /\                 =    907-450-7251</div><div class=3D"">5245 Airport Industrial = Road         /  \/\         =     907-450-7238 fax</div><div class=3D"">Fairbanks, Alaska =  99709        /\  /    \ \ = Second greatest commandment</div></font><font face=3D"Monaco" = class=3D""><span style=3D"font-size: 12px;" class=3D"">Serving All of = Alaska          /  \/  /\  \ \/\ =   =E2=80=9CLove your neighbor as</span></font><br = style=3D"font-family: Monaco;" class=3D""><font face=3D"Menlo" = style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: = none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: = 0px;" class=3D""><span style=3D"font-size: 12px;" class=3D"">Green, = green as far as the eyes can see           =   yourself=E2=80=9D Matt 22:39</span></font><div style=3D"color: = rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: = Menlo;" class=3D""></div><font face=3D"Menlo" style=3D"font-size: 12px;" = class=3D""></font><span style=3D"color: rgb(0, 0, 0); letter-spacing: = normal; text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font = face=3D"Menlo" class=3D""><div = class=3D"">---------------------------------------------------------------= ---------</div></font></span><div style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font = face=3D"Menlo" style=3D"font-size: 12px;" class=3D""><br = class=3D""></font></div></div><span style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" = class=3D""><br class=3D"Apple-interchange-newline"></span></div><span = style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: = none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: = 0px; font-size: 12px;" class=3D""><br = class=3D"Apple-interchange-newline"></span></div><span style=3D"color: = rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: = 12px;" class=3D""><br class=3D"Apple-interchange-newline"></span></div><br= class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"></div><br = class=3D"Apple-interchange-newline"><br = class=3D"Apple-interchange-newline"> </div> <br class=3D""></body></html>= --Apple-Mail=_3C58C5A6-FF18-4157-8715-87EEC5B8B53D--

1 0

testing REST API
by wodel youchi 19 Aug '17

19 Aug '17

Hi; I am reading the REST API Guide, it's the first time that I am testing this. I have hard time to use OAuth authentication to make queries. I can get the token with curl and with a firefox REST module *curl --cacert ca.crt -X POST -H 'Content-Type: application/x-www-form-urlencoded' -H 'Accept: application/json' -d 'grant_type=password&scope=ovirt-appapi&username=admin%40internal&password=mypass' https://engine101.example.com/ovirt-engine/sso/oauth/token <https://engine101.example.com/ovirt-engine/sso/oauth/token> HTTP/1.1* engine log : [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-1) [] User admin@internal successfully logged in with scopes: ovirt-appapi But when I try to use it I get this error : *curl --cacert ca.crt -X GET -H 'application/xml' -H 'Authorization: Bearer fT0knxah-wEOyi-VdhmozKv-hz-wohVm268BBJts-MYxNZ548K0UZCSmv5nY18Z6gPiFdl-VAySjqr_N4gPGAA' https://engine101.example.com/ovirt-engine/api/vms <https://engine101.example.com/ovirt-engine/api/vms> HTTP/1.1* engine log: ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-30) [] Cannot authenticate using authentication Headers: The required scope ovirt-app-api is not granted. With basic authentication, it's working, so I don't know where is the problem. Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Garanti sans virus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#m_8952078234548713351_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 1

Recovering from a multi-node failure
by Jim Kusznir 19 Aug '17

19 Aug '17

Hi all: Today has been rough. two of my three nodes went down today, and self heal has not been healing well. 4 hours later, VMs are running. but the engine is not happy. It claims the storage domain is down (even though it is up on all hosts and VMs are running). I'm getting a ton of these messages logging: VDSM engine3 command HSMGetAllTasksStatusesVDS failed: Not SPM Aug 4, 2017 7:23:00 PM VDSM engine3 command SpmStatusVDS failed: Error validating master storage domain: ('MD read error',) Aug 4, 2017 7:22:49 PM VDSM engine3 command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=5868392a-0148-02cf-014d-000000000121, msdUUID=cdaf180c-fde6-4cb3-b6e5-b6bd869c8770' Aug 4, 2017 7:22:47 PM VDSM engine1 command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=5868392a-0148-02cf-014d-000000000121, msdUUID=cdaf180c-fde6-4cb3-b6e5-b6bd869c8770' Aug 4, 2017 7:22:46 PM VDSM engine2 command SpmStatusVDS failed: Error validating master storage domain: ('MD read error',) Aug 4, 2017 7:22:44 PM VDSM engine2 command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=5868392a-0148-02cf-014d-000000000121, msdUUID=cdaf180c-fde6-4cb3-b6e5-b6bd869c8770' Aug 4, 2017 7:22:42 PM VDSM engine1 command HSMGetAllTasksStatusesVDS failed: Not SPM: () ------------ I cannot set an SPM as it claims the storage domain is down; I cannot set the storage domain up. Also in the storage realm, one of my exports shows substantially less data than is actually there. Here's what happened, as best as I understood them: I went to do maintence on ovirt2 (needed to replace a faulty ram stick and rework the disk). I put it in maintence mode, then shut it down and did my work. In the process, much of the disk contents was lost (all the gluster data). I figure, no big deal, the gluster data is redundant on the network, it will heal when it comes back up. While I was doing maintence, all but one of the VMs were running on engine1. When I turned on engine2, all of the sudden, all vms including the main engine stop and go non-responsive. As far as I can tell, this should not have happened, as I turned ON one host, but none the less, I waited for recovery to occur (while customers started calling asking why everything stopped working....). As I waited, I was checking, and gluster volume status only showed ovirt1 and ovirt2....Apparently gluster had stopped/failed at some point on ovirt3. I assume that was the cause of the outage, still, if everything was working fine with ovirt1 gluster, and ovirt2 powers on with a very broke gluster (the volume status was showing NA for the port fileds for the gluster volumes), I would not expect to have a working gluster go stupid like that. After starting ovirt3 glusterd and checking the status, all three showed ovirt1 and ovirt3 as operational, and ovirt2 as NA. Unfortunately, recovery was still not happening, so I did some googling and found about the commands to inquire about the hosted-engine status. It appeared to be stuck "paused" and I couldn't find a way to unpause it, so I poweroff'ed it, then started it manually on engine 1, and the cluster came back up. It showed all VMs paused. I was able to unpause them and they worked again. So now I began to work the ovirt2 gluster healing problem. It didn't appear to be self-healing, but eventually I found this document: https://support.rackspace.com/how-to/recover-from-a-failed-server-in-a-glus… and from that found the magic xattr commands. After setting them, gluster volumes on ovirt2 came online. I told iso to heal, and it did but only came up about half as much data as it should have. I told it heal full, and it did finish off the remaining data, and came up to full. I then told engine to do a full heal (gluster volume heal engine full), and it transferred its data from the other gluster hosts too. However, it said it was done when it hit 9.7GB while there was 15GB on disk! It is still stuck that way; ovirt gui and gluster volume heal engine info both show the volume fully healed, but it is not: [root@ovirt1 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos_ovirt-root 20G 4.2G 16G 21% / devtmpfs 16G 0 16G 0% /dev tmpfs 16G 16K 16G 1% /dev/shm tmpfs 16G 26M 16G 1% /run tmpfs 16G 0 16G 0% /sys/fs/cgroup /dev/mapper/gluster-engine 25G 12G 14G 47% /gluster/brick1 /dev/sda1 497M 315M 183M 64% /boot /dev/mapper/gluster-data 136G 124G 13G 92% /gluster/brick2 /dev/mapper/gluster-iso 25G 7.3G 18G 29% /gluster/brick4 tmpfs 3.2G 0 3.2G 0% /run/user/0 192.168.8.11:/engine 15G 9.7G 5.4G 65% /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine 192.168.8.11:/data 136G 124G 13G 92% /rhev/data-center/mnt/glusterSD/192.168.8.11:_data 192.168.8.11:/iso 13G 7.3G 5.8G 56% /rhev/data-center/mnt/glusterSD/192.168.8.11:_iso This is from ovirt1, and before the work, both ovirt1 and ovirt2's brings had the same usage. ovirt2's bricks and the gluster mountpoints agree on iso and engine, but as you can see, not here. If I do a du -sh on /rhev/data-center/mnt/glusterSD/..../_engine, it comes back with the 12GB number (/brick1 is engine, brick2 is data and brick4 is iso). However, gluster still says its only 9.7G. I haven't figured out how to get it to finish "healing". data is in the process of healing currently. So, I think I have two main things to solve right now: 1) how do I get ovirt to see the data center/storage domain as online again? 2) How do I get engine to finish healing to ovirt2? Thanks all for reading this very long message! --Jim

2 3

Data domain on engine host
by Mitchell Smith 18 Aug '17

18 Aug '17

Hi List, I am in the process of deploying ovirt in a lab environment before implementing it in production. My lab consists of three compute hosts and one engine host. I am using the 4.2 nightly builds in this lab environment, mainly because I want to test the OVN functionality. I wanted to add a NFS data domain on the engine host to store VM templates and ISO images, however when I go in to Storage -> Domains -> New and try and add a NFS domain, it only gives me the three compute hosts to create storage on, I’m guessing because the engine host doesn’t have VDSM installed. Is there a way to do this, so I can make use of the considerable free space on the engine host, or do I have to add it as a compute host running VDSM, and if so how can I flag this host so no VMs get deployed on it, I only want to use the storage capacity. Any feedback or suggestions would be very much appreciated. Thanks in advance. Mitch

2 1

VM with attached host USB device failed to run
by Дровалев Роман 18 Aug '17

18 Aug '17

Hello, Please help! Does anyone have a working USB forwarding in a virtual machine? If "YES", how did you solve this problem? If this problem is not solved, unfortunately, the ovirt will have to completely abandon. (( oVirt Engine Version: 4.1.4.2-1.el7.centos Thanks.

3 2

glusterfs Error message constantly being reported
by Vadim 18 Aug '17

18 Aug '17

Hi, All ovirt 4.1.4 fresh install Constantly seeing this message in the logs, how to fix this: VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' VDSM kvm04 command GetGlusterVolumeAdvancedDetailsVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterVolumeStatvfs' VDSM kvm04 command GetGlusterVolumeAdvancedDetailsVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterVolumeStatvfs' VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' VDSM kvm04 command GlusterTasksListVDS failed: 'AutoProxy[instance]' object has no attribute 'glusterTasksList' -- Thanks, Vadim

3 6

install of engine
by david caughey 18 Aug '17

18 Aug '17

Hi folks, I have installed the ovirt OS on 3 dl380's. It has been deployed with gluster and all is fine up to a point. When I try to install the engine it is trying to get out to the internet but unfortunately I am behind a proxy with strict rules, is there a way to point to a local copy of the engine using the GUI install or can I scp the iso over to the first node for local install, is there a particular partition where the iso should be stored. Any help appreciated, BR/David

2 1

How to delay start ovirt service?
by Spickiy Nikita 17 Aug '17

17 Aug '17

Hi, i try install oVirt hosted-engine. For storage i use iscsi target (Infotrend DS1012RE). I make install, but have error to service ovirt-ha-broker.service and vdsmd.service: systemctl -l status ovirt-ha-broker.service ● ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker ▽ Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled; vendor preset: disabled) ▽ Active: active (running) since Thu 2017-08-17 15:28:57 +07; 48s ago ▽ Main PID: 1926 (ovirt-ha-broker) CGroup: /system.slice/ovirt-ha-broker.service └─1926 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker --no-daemon Aug 17 15:28:57 vnode10.pi.local systemd[1]: Started oVirt Hosted Engine High Availability Communications Broker. Aug 17 15:28:57 vnode10.pi.local systemd[1]: Starting oVirt Hosted Engine High Availability Communications Broker... Aug 17 15:29:08 vnode10.pi.local ovirt-ha-broker[1926]: ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'set-storage-domain FilesystemBackend dom_type=iscsi sd_uuid=817a5b54-26c1-4d7d-a711-dafdb1ea477d' Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle data) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 299, in _dispatch .set_storage_domain(client, sd_type, **options) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 66, in set_storage_domain self._backends[client].connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 470, in connect for lv in os.listdir(os.path.join("/dev", uuid)): OSError: [Errno 2] No such file or directory: '/dev/817a5b54-26c1-4d7d-a711-dafdb1ea477d’ systemctl -l status vdsmd.service ● vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2017-08-17 15:29:07 +07; 1min 30s ago Process: 3686 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS) Main PID: 3759 (vdsm) CGroup: /system.slice/vdsmd.service └─3759 /usr/bin/python2 /usr/share/vdsm/vdsm Aug 17 15:29:06 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running syslog_available Aug 17 15:29:06 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running nwfilter Aug 17 15:29:07 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running dummybr Aug 17 15:29:07 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running tune_system Aug 17 15:29:07 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running test_space Aug 17 15:29:07 vnode10.pi.local vdsmd_init_common.sh[3686]: vdsm: Running test_lo Aug 17 15:29:07 vnode10.pi.local systemd[1]: Started Virtual Desktop Server Manager. Aug 17 15:29:08 vnode10.pi.local vdsm[3759]: vdsm throttled WARN MOM not available. Aug 17 15:29:08 vnode10.pi.local vdsm[3759]: vdsm throttled WARN MOM not available, KSM stats will be missing. Aug 17 15:29:08 vnode10.pi.local vdsm[3759]: vdsm root ERROR failed to retrieve Hosted Engine HA info Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo stats = instance.get_all_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 103, in get_all_stats self._configure_broker_conn(broker) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 180, in _configure_broker_conn dom_type=dom_type) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 177, in set_storage_domain .format(sd_type, options, e)) RequestError: Failed to set storage domain FilesystemBackend, options {'dom_type': 'iscsi', 'sd_uuid': '817a5b54-26c1-4d7d-a711-dafdb1ea477d'}: Request failed: <type 'exceptions.OSError’> But device exist: ls -l /dev/817a5b54-26c1-4d7d-a711-dafdb1ea477d total 0 lrwxrwxrwx. 1 root root 8 Aug 17 16:01 17bb9c12-c496-4fd3-b53e-ee4c8d319b43 -> ../dm-10 lrwxrwxrwx. 1 root root 8 Aug 17 16:01 1e9a3147-34ac-4020-8f95-71aa5d1a5258 -> ../dm-13 lrwxrwxrwx. 1 root root 8 Aug 17 16:01 436d8816-990e-4acd-b040-7cd948f7c0af -> ../dm-11 lrwxrwxrwx. 1 root root 8 Aug 17 16:01 eeb4b310-cc23-472c-9bce-36ce27881727 -> ../dm-12 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 ids -> ../dm-5 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 inbox -> ../dm-8 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 leases -> ../dm-6 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 master -> ../dm-9 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 metadata -> ../dm-4 lrwxrwxrwx. 1 root root 7 Aug 17 16:01 outbox -> ../dm-7 I suspect the service starting before mount. I think modify After in: /usr/lib/systemd/system/vdsmd.service /usr/lib/systemd/system/ovirt-ha-broker.service I add iscsid.service, but it not help. Maybe say how to fix it?

1 0

oVirt Node with bcache
by FERNANDO FREDIANI 16 Aug '17

16 Aug '17

This is a multi-part message in MIME format. --------------4B191FE3792DDE5D00F7F478 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hello I just wanted to share a scenario with you and perhaps exchange more information with other people that may also have a similar scenario. For a couple of months I have been running a oVirt Node (CentOS 7.3 Minimal) with bcache (https://bcache.evilpiepirate.org/) for caching a SSD with HDD disks. The setup is simple and was made for a prof of concept and since them has been working better than expected. This is a standalone host with 4 disks being: 1 for Operating System, 2 x 2TB 7200 RPM in software RAID 1 and 1 x PCI-E NVMe 400GB SSD which plays the caching device for both reads and writes. The VM storage folder is mounted as a ext4 partition on the logical device created by bcache (/dev/bcache0). All this is transparent to oVirt as all it sees is a /folder to put the VMs. We monitor the IOPS on all block devices individually and see the behavior exactly as expected: random writes are all done on the SSD first and them streamed sequentially to the mechanical drives with pretty impressive performance. Also in the beginning while the total amount of data was less than 400GB ALL read used to come from the caching device and therefore didn't use IOPS from the mechanical drives leaving it free to do basically writes. Finally at sequential IOPS (as described by bcache) are intelligently passed directly to the mechanical drives (but they are not much). Although bcache is present on kernel 3.10 I had to use kernel-ml 4.12 (from Elrepo) and I had also to compile the bcache-tools as I could not find it available in any repository. Regards Fernando --------------4B191FE3792DDE5D00F7F478 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> <font face="arial, helvetica, sans-serif">Hello<br> <br> I just wanted to share a scenario with you and perhaps exchange more information with other people that may also have a similar scenario.<br> <br> For a couple of months I have been running a oVirt Node (CentOS 7.3 Minimal) with bcache (<a class="moz-txt-link-freetext" href="https://bcache.evilpiepirate.org/">https://bcache.evilpiepirate.org/</a>) for caching a SSD with HDD disks. The setup is simple and was made for a prof of concept and since them has been working better than expected.<br> This is a standalone host with 4 disks being: 1 for Operating System, 2 x 2TB 7200 RPM in software RAID 1 and 1 x PCI-E NVMe 400GB SSD which plays the caching device for both reads and writes. The VM storage folder is mounted as a ext4 partition on the logical device created by bcache (/dev/bcache0). All this is transparent to oVirt as all it sees is a /folder to put the VMs.<br> <br> We monitor the IOPS on all block devices individually and see the behavior exactly as expected: random writes are all done on the SSD first and them streamed sequentially to the mechanical drives with pretty impressive performance. Also in the beginning while the total amount of data was less than 400GB ALL read used to come from the caching device and therefore didn't use IOPS from the mechanical drives leaving it free to do basically writes. Finally at sequential IOPS (as described by bcache) are intelligently passed directly to the mechanical drives (but they are not much).<br> <br> Although bcache is present on kernel 3.10 I had to use kernel-ml 4.12 (from Elrepo) and I had also to compile the bcache-tools as I could not find it available in any repository.<br> <br> Regards<br> Fernando<br> </font> </body> </html> --------------4B191FE3792DDE5D00F7F478--

2 1