Re: [ovirt-users] [4.2.2-1.el7.centos] Image locked and unending task

--=_3c316c3018d9743cda23c6ab6b226644 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thanks for that quick answer !=0A Yes indeed I had some connectivity t= roubles on this server, a strange bonding problem I am investigating on= since yesterday. But with just one link, it is working ok, I have no si= milar errors after the ones you saw. What can I do to really remove th= e task from the list ? Manual database cleanup ? =0A=0A Le 15-Mar-2018= 11:15:40 +0100, eshenitz(a)redhat.com a crit: =0A Thank you for sending= the logs. According to the logs, it seems that you had some connectiv= ity issue while you tried to preview the snapshot. The preview operation= rolled back but according to you failed to finish. It seems like you= still have a connectivity issue with that host ('pfm-srv-virt-1.pfm-ad.= pfm.loc), try to see what happens to it. Here is the relevant part fro= m the log: 2018-03-14 17:00:48,652+01 ERROR [org.ovirt.vdsm.jsonrpc.c= lient.reactors.ReactorClient] (SSL Stomp Reactor) [] Heartbeat exceeded= for host 'pfm-srv-virt-1.pfm-ad.pfm.loc', last response arrived 2003 ms= ago. 2018-03-14 17:00:53,561+01 INFO [org.ovirt.vdsm.jsonrpc.client.rea= ctors.ReactorClient] (SSL Stomp Reactor) [] Connecting to pfm-srv-virt-1= .pfm-ad.pfm.loc/10.100.1.50 2018-03-14 17:02:21,832+01 INFO [org.ovirt.e= ngine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactor= y-engine-Thread-118906) [] transaction rolled back 2018-03-14 17:02:21= ,836+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.Audit= LogDirector] (EE-ManagedThreadFactory-engine-Thread-118906) [] EVENT_ID:= USER_TRY_BACK_TO_SNAPSHOT_FINISH_FAILURE(99), Failed to complete Snapsh= ot-Preview AFTER_INSTALL for VM pfm-ltsp-1. 2018-03-14 17:02:21,836+01 E= RROR [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThrea= dFactory-engine-Thread-118906) [] [within thread]: endAction for action= type TryBackToAllSnapshotsOfVm threw an exception.: java.lang.NullPoint= erException at org.ovirt.engine.core.bll.snapshots.SnapshotsManager.devi= ceCanBeRemoved(SnapshotsManager.java:463) [bll.jar:] at org.ovirt.engine= .core.bll.snapshots.SnapshotsManager.attempToRestoreVmConfigurationFromS= napshot(SnapshotsManager.java:415) [bll.jar:] at org.ovirt.engine.core.b= ll.snapshots.TryBackToAllSnapshotsOfVmCommand.restoreVmConfigFromSnapsho= t(TryBackToAllSnapshotsOfVmCommand.java:204) [bll.jar:] at org.ovirt.eng= ine.core.bll.snapshots.TryBackToAllSnapshotsOfVmCommand.endSuccessfully(= TryBackToAllSnapshotsOfVmCommand.java:168) [bll.jar:] at org.ovirt.engin= e.core.bll.CommandBase.internalEndSuccessfully(CommandBase.java:675) [bl= l.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionS= cope(CommandBase.java:630) [bll.jar:] at org.ovirt.engine.core.bll.Comma= ndBase.runInTransaction(CommandBase.java:1936) [bll.jar:] at org.ovirt.e= ngine.core.utils.transaction.TransactionSupport.executeInNewTransaction(= TransactionSupport.java:202) [utils.jar:] at org.ovirt.engine.core.utils= .transaction.TransactionSupport.executeInRequired(TransactionSupport.jav= a:137) [utils.jar:] at org.ovirt.engine.core.utils.transaction.Transacti= onSupport.executeInScope(TransactionSupport.java:105) [utils.jar:] at or= g.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:495) [bll= .jar:] at org.ovirt.engine.core.bll.tasks.DecoratedCommand.endAction(Dec= oratedCommand.java:17) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CoC= oAsyncTaskHelper.endAction(CoCoAsyncTaskHelper.java:353) [bll.jar:] at o= rg.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.endAction(CommandC= oordinatorImpl.java:347) [bll.jar:] at org.ovirt.engine.core.bll.tasks.C= ommandAsyncTask.endCommandAction(CommandAsyncTask.java:160) [bll.jar:] a= t org.ovirt.engine.core.bll.tasks.CommandAsyncTask.lambda$endActionIfNec= essary$0(CommandAsyncTask.java:112) [bll.jar:] at org.ovirt.engine.core.= utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUt= il.java:96) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapt= er.call(Executors.java:511) [rt.jar:1.8.0_161] at java.util.concurrent.F= utureTask.run(FutureTask.java:266) [rt.jar:1.8.0_161] at java.util.concu= rrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar= :1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Thread= PoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread= .java:748) [rt.jar:1.8.0_161] at org.glassfish.enterprise.concurrent.Man= agedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:25= 0) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.= service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronMana= gedThreadFactory.java:78) 2018-03-14 17:02:21,838+01 INFO [org.ovirt.e= ngine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-T= hread-118906) [] CommandAsyncTask::HandleEndActionResult: endAction for= action type 'TryBackToAllSnapshotsOfVm' threw an unrecoverable RuntimeE= xception the task will be cleared. 2018-03-14 17:02:21,841+01 INFO [org.= ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engin= e-Thread-118906) =0A On Thu, Mar 15, 2018 at 11:35 AM, wrote:=0A= =0A Thanks for your reply. Yesterday, I realized I was doing nothing g= ood with some the software I planed to install in a VM, so I tried to re= vert to a snapshot a took just after OS installation, as I always do. = As I had added a second disk to the VM in between, I choose to revert t= o snapshot without taking care of the second disk contents. But the pr= eview operation never ended. So I restarted the engine vm but nothing ch= anged. This morning I tried to cleanup things, using "taskcleaner" and= "unlock_entity". I could regain control over the VM, but the task is st= ill in "finalizing" state in the GUI. I even remove the second disk to= see if it was better, but nothing. You will find the engine logfile a= nd the "vdsm.log" from the server the task is running on. I am not sur= e how to check engine version precisely, so I queried the rpm database i= n the vm : ovirt-engine-4.2.2-1.el7.centos.noarch Regards =0A=0A Le= 15-Mar-2018 10:17:59 +0100, eshenitz(a)redhat.com a crit: =0A Hi, Can= you please specify the version of the engine and supply the engine.log= and the vdsm.log? Moreover, can you please specify the steps that you= did that led you to this issue? Thanks, =0A On Thu, Mar 15, 2018 at= 11:05 AM, wrote:=0A=0A Hi,=0A I tried to rollback to a snapshot on a= VM, but the preview never ended. The task has been running for about= 15 hours, with this state : {=0A "916b67fb-8808-43d2-850c-1c12650ccc4= 9": {=0A "verb": "createVolume", =0A "code": 0, =0A "state": "finished",= =0A "tag": "spm", =0A "result": {=0A "uuid": "d37ca118-820f-46a3-b99b-7= 14018ea8b42"=0A }, =0A "message": "1 jobs completed successfully", =0A "= id": "916b67fb-8808-43d2-850c-1c12650ccc49"=0A }=0A}=0A I just cancele= d it : the task list is now empty on the CLI but no change on GUI. So= I restared the engine VM, but no success. With "/usr/share/ovirt-engi= ne/setup/dbutils/unlock_entity.sh" I was able to manually unlock the ima= ge, but the task is still "finalizing". Is this a bug ? Regards =0A= =0A---------------------------------------------------------------------= ----------------------------=0AFreeMail powered by mail.fr =0A__________= _____________________________________=0A Users mailing list=0AUsers@ovir= t.org=0Ahttp://lists.ovirt.org/mailman/listinfo/users=0A=0A -- =0A Re= gards, Eyal Shenitzky =0A=0A----------------------------------------= ---------------------------------------------------------=0AFreeMail pow= ered by mail.fr =0A=0A -- =0A Regards, Eyal Shenitzky =0A=0A----= ------------------------------------------------------------------------= ---------------------=0AFreeMail powered by mail.fr --=_3c316c3018d9743cda23c6ab6b226644 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <div><span style=3D"font-family: arial, helvetica,sans-serif; font-size:= 10pt; color: #000000;">Thanks for that quick answer !<br /></span></div= a,sans-serif; font-size: 10pt; color: #000000;">Yes indeed I had some co= nnectivity troubles on this server, a strange bonding problem I am inves= tigating on since yesterday. But with just one link, it is working ok, I= have no similar errors after the ones you saw.</span></div>=0A<div>&nbs= p;</div>=0A<div><span style=3D"font-family: arial, helvetica,sans-serif;= font-size: 10pt; color: #000000;">What can I do to really remove the ta= sk from the list ? Manual database cleanup ?</span></div>=0A<div>&nbsp;<= /div>=0A<p><br /><br /> Le 15-Mar-2018 11:15:40 +0100, eshenitz(a)redhat.c= om a &eacute;crit:</p>=0A<div>&nbsp;</div>=0A<blockquote style=3D"margin= -left: 0; padding-left: 5px; border-left: 2px solid navy;">=0A<div dir= =3D"ltr">Thank you for sending the logs.=0A<div>&nbsp;</div>=0A<div>Acco= rding to the logs, it seems that you had some connectivity issue while y= ou tried to preview the snapshot.</div>=0A<div>The preview&nbsp;operatio= n rolled back but according to you failed to finish.</div>=0A<div>&nbsp;= </div>=0A<div>It seems like you still have a connectivity issue with tha= t host (<span style=3D"color: #222222; font-family: arial, sans-serif; f= ont-size: small; font-style: normal; font-weight: 400; letter-spacing: n= ormal; text-indent: 0px; text-transform: none; white-space: normal; word= -spacing: 0px; background-color: #ffff00; float: none; display: inline;"= /div>=0A<div>&nbsp;</div>=0A<div>Here is the relevant part from the log:= </div>=0A<div>&nbsp;</div>=0A<div>=0A<div><span style=3D"background-colo= r: #ffff00;">2018-03-14 17:00:48,652+01 ERROR [org.ovirt.vdsm.jsonrpc.cl= ient.reactors.ReactorClient] (SSL Stomp Reactor) [] Heartbeat exceeded f= or host 'pfm-srv-virt-1.pfm-ad.pfm.loc', last response arrived 2003 ms a= go.</span></div>=0A<div>2018-03-14 17:00:53,561+01 INFO&nbsp; [org.ovirt= .vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Conn= ecting to pfm-srv-virt-1.pfm-ad.pfm.loc/<a href=3D"http://10.100.1.50" t= arget=3D"_blank">10.100.1.50</a></div>=0A<div>2018-03-14 17:02:21,832+01= INFO&nbsp; [org.ovirt.engine.core.utils.transaction.TransactionSupport]= (EE-ManagedThreadFactory-engine-Thread-118906) []<span style=3D"backgro= und-color: #ffff00;"> transaction rolled back</span></div>=0A</div>=0A<d= iv>=0A<div>2018-03-14 17:02:21,836+01 ERROR [org.ovirt.engine.core.dal.d= bbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engi= ne-Thread-118906) [] EVENT_ID: USER_TRY_BACK_TO_SNAPSHOT_FINISH_FAILURE(= 99), Failed to complete Snapshot-Preview AFTER_INSTALL for VM pfm-ltsp-1= .</div>=0A<div>2018-03-14 17:02:21,836+01 ERROR [org.ovirt.engine.core.b= ll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-118906= ) [] [within thread]: endAction for action type TryBackToAllSnapshotsOfV= m threw an exception.: java.lang.NullPointerException</div>=0A<div>at or= g.ovirt.engine.core.bll.snapshots.SnapshotsManager.deviceCanBeRemoved(Sn= apshotsManager.java:463) [bll.jar:]</div>=0A<div>at org.ovirt.engine.cor= e.bll.snapshots.SnapshotsManager.attempToRestoreVmConfigurationFromSnaps= hot(SnapshotsManager.java:415) [bll.jar:]</div>=0A<div>at org.ovirt.engi= ne.core.bll.snapshots.TryBackToAllSnapshotsOfVmCommand.restoreVmConfigFr= omSnapshot(TryBackToAllSnapshotsOfVmCommand.java:204) [bll.jar:]</div>= =0A<div>at org.ovirt.engine.core.bll.snapshots.TryBackToAllSnapshotsOfVm= Command.endSuccessfully(TryBackToAllSnapshotsOfVmCommand.java:168) [bll.= jar:]</div>=0A<div>at org.ovirt.engine.core.bll.CommandBase.internalEndS= uccessfully(CommandBase.java:675) [bll.jar:]</div>=0A<div>at org.ovirt.e= ngine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:= 630) [bll.jar:]</div>=0A<div>at org.ovirt.engine.core.bll.CommandBase.ru= nInTransaction(CommandBase.java:1936) [bll.jar:]</div>=0A<div>at org.ovi= rt.engine.core.utils.transaction.TransactionSupport.executeInNewTransact= ion(TransactionSupport.java:202) [utils.jar:]</div>=0A<div>at org.ovirt.= engine.core.utils.transaction.TransactionSupport.executeInRequired(Trans= actionSupport.java:137) [utils.jar:]</div>=0A<div>at org.ovirt.engine.co= re.utils.transaction.TransactionSupport.executeInScope(TransactionSuppor= t.java:105) [utils.jar:]</div>=0A<div>at org.ovirt.engine.core.bll.Comma= ndBase.endAction(CommandBase.java:495) [bll.jar:]</div>=0A<div>at org.ov= irt.engine.core.bll.tasks.DecoratedCommand.endAction(DecoratedCommand.ja= va:17) [bll.jar:]</div>=0A<div>at org.ovirt.engine.core.bll.tasks.CoCoAs= yncTaskHelper.endAction(CoCoAsyncTaskHelper.java:353) [bll.jar:]</div>= =0A<div>at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.endAct= ion(CommandCoordinatorImpl.java:347) [bll.jar:]</div>=0A<div>at org.ovir= t.engine.core.bll.tasks.CommandAsyncTask.endCommandAction(CommandAsyncTa= sk.java:160) [bll.jar:]</div>=0A<div>at org.ovirt.engine.core.bll.tasks.= CommandAsyncTask.lambda$endActionIfNecessary$0(CommandAsyncTask.java:112= ) [bll.jar:]</div>=0A<div>at org.ovirt.engine.core.utils.threadpool.Thre= adPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) [utils.ja= r:]</div>=0A<div>at java.util.concurrent.Executors$RunnableAdapter.call(= Executors.java:511) [rt.jar:1.8.0_161]</div>=0A<div>at java.util.concurr= ent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_161]</div>=0A<div>= at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1149) [rt.jar:1.8.0_161]</div>=0A<div>at java.util.concurrent.Threa= dPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161]= </div>=0A<div>at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161= ]</div>=0A<div>at org.glassfish.enterprise.concurrent.ManagedThreadFacto= ryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enter= prise.concurrent-1.0.jar:]</div>=0A<div>at org.jboss.as.ee.concurrent.se= rvice.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManage= dThreadFactory.java:78)</div>=0A<div>&nbsp;</div>=0A<div>2018-03-14 17:0= 2:21,838+01 INFO&nbsp; [org.ovirt.engine.core.bll.tasks.CommandAsyncTask= ] (EE-ManagedThreadFactory-engine-Thread-118906) [] CommandAsyncTask::Ha= ndleEndActionResult: endAction for action type 'TryBackToAllSnapshotsOfV= m' threw an unrecoverable RuntimeException <span style=3D"background-col= or: #ffff00;">the task will be cleared.</span></div>=0A<div>2018-03-14 1= 7:02:21,841+01 INFO&nbsp; [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]= (EE-ManagedThreadFactory-engine-Thread-118906)&nbsp;</div>=0A</div>=0A<= div>&nbsp;</div>=0A</div>=0A<div class=3D"gmail_extra"><br />=0A<div cla= ss=3D"gmail_quote">On Thu, Mar 15, 2018 at 11:35 AM, <span dir=3D"ltr">&= lt;<a href=3D"mailto:spfma.tech@e.mail.fr" target=3D"_blank" rel=3D"nore= ferrer noopener&quot;&gt;spfma.tech(a)e.mail.fr&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br />=0A<blo= ckquote class=3D"gmail_quote" style=3D"margin: 0 0 0 .8ex; border-left:= 1px #ccc solid; padding-left: 1ex;">=0A<div>Thanks for your reply.</div= ood with some the software I planed to install in a VM, so I tried to re= vert to a snapshot a took just after OS installation, as I always do.</d= iv>=0A<div>&nbsp;</div>=0A<div>As I had added a second disk to the VM in= between, I choose to revert to snapshot without taking care of the seco= nd disk contents.</div>=0A<div>&nbsp;</div>=0A<div>But the preview opera= tion never ended. So I restarted the engine vm but nothing changed.</div= ng "taskcleaner" and "unlock_entity". I could regain control over the VM= , but the task is still in "finalizing" state in the GUI.</div>=0A<div>&= nbsp;</div>=0A<div>I even remove the second disk to see if it was better= , but nothing.</div>=0A<div>&nbsp;</div>=0A<div>You will find the engine= logfile and the "vdsm.log" from the server the task is running on.</div= recisely, so I queried the rpm database in the vm :&nbsp;ovirt-engine-4.= 2.2-1.el7.centos.noarch</div>=0A<div>&nbsp;</div>=0A<div>Regards</div>= =0A<div><span style=3D"font-family: arial, helvetica, sans-serif; font-s= ize: 10pt; color: #000000;">&nbsp;</span></div>=0A<p><br /><br /> Le 15-= Mar-2018 10:17:59 +0100, <a href=3D"mailto:eshenitz@redhat.com" target= =3D"_blank" rel=3D"noreferrer noopener&quot;&gt;eshenitz(a)redhat.com&lt;/a&gt; a &eacut= e;crit:</p>=0A<div>&nbsp;</div>=0A<blockquote style=3D"margin-left: 0; p= adding-left: 5px; border-left: 2px solid #000080;">=0A<div dir=3D"ltr">H= i,&nbsp;=0A<div>&nbsp;</div>=0A<div>Can you please specify the version o= f the engine and supply the engine.log and the vdsm.log?</div>=0A<div>&n= bsp;</div>=0A<div>Moreover, can you please specify the steps that you di= d that led you to this issue?</div>=0A<div>&nbsp;</div>=0A<div>Thanks,</= div>=0A</div>=0A<div class=3D"gmail_extra"><br />=0A<div class=3D"gmail_= quote">On Thu, Mar 15, 2018 at 11:05 AM, <span dir=3D"ltr">&lt;<a href= =3D"mailto:spfma.tech@e.mail.fr" target=3D"_blank" rel=3D"noreferrer noo= pener&quot;&gt;spfma.tech(a)e.mail.fr&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br />=0A<blockquote cl= ass=3D"gmail_quote" style=3D"margin: 0 0 0 .8ex; border-left: 1px #ccc s= olid; padding-left: 1ex;">=0A<div><span style=3D"font-family: arial, hel= vetica, sans-serif; font-size: 10pt; color: #000000;">Hi,<br /></span></= div>=0A<div>&nbsp;</div>=0A<div><span style=3D"font-family: arial, helve= tica, sans-serif; font-size: 10pt; color: #000000;">I tried to rollback= to a snapshot on a VM, but the preview never ended.</span></div>=0A<div= serif; font-size: 10pt; color: #000000;">The task has been running for a= bout 15 hours, with this state :</span></div>=0A<div>&nbsp;</div>=0A<div= pt; color: #000000;">{<br />&nbsp;&nbsp;&nbsp; "916b67fb-8808-43d2-850c-= 1c12650ccc49": {<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "verb":= "createVolume", <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "code"= : 0, <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "state": "finished= ", <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "tag": "spm", <br />= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "result": {<br />&nbsp;&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "uuid": "d37ca118= -820f-46a3-b99b-714018ea8b42"<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&= nbsp; }, <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "message": "1= jobs completed successfully", <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp= ;&nbsp; "id": "916b67fb-8808-43d2-850c-1c12650ccc49"<br />&nbsp;&nbsp;&n= bsp; }<br />}<br /></span></div>=0A<div>&nbsp;</div>=0A<div><span style= =3D"font-family: arial, helvetica, sans-serif; font-size: 10pt; color: #= 000000;">I just canceled it : the task list is now empty on the CLI but= no change on GUI.</span></div>=0A<div>&nbsp;</div>=0A<div><span style= =3D"font-family: arial, helvetica, sans-serif; font-size: 10pt; color: #= 000000;">So I restared the engine VM, but no success.</span></div>=0A<di= v>&nbsp;</div>=0A<div><span style=3D"font-family: arial, helvetica, sans= -serif; font-size: 10pt; color: #000000;">With "/usr/share/ovirt-engine/= setup/dbutils/unlock_entity.sh" I was able to manually unlock the image,= but the task is still "finalizing".</span></div>=0A<div>&nbsp;</div>=0A= <div><span style=3D"font-family: arial, helvetica, sans-serif; font-size= : 10pt; color: #000000;">Is this a bug ? </span></div>=0A<div>&nbsp;</di= v>=0A<div><span style=3D"font-family: arial, helvetica, sans-serif; font= -size: 10pt; color: #000000;">Regards</span></div>=0A<br /><hr />FreeMai= l powered by <a href=3D"https://mail.fr" target=3D"_blank" rel=3D"norefe= rrer noopener">mail.fr</a> <br />_______________________________________= ________<br /> Users mailing list<br /><a href=3D"mailto:Users@ovirt.org= " target=3D"_blank" rel=3D"noreferrer noopener&quot;&gt;Users(a)ovirt.org&lt;/a&gt;&lt;br /= ank" rel=3D"noreferrer noopener">http://lists.ovirt.org/mailman/listinfo= /users</a><br /><br /></blockquote>=0A</div>=0A<br /><br clear=3D"all" /= mail_signature">=0A<div dir=3D"ltr">Regards,=0A<div>Eyal Shenitzky</div>= =0A</div>=0A</div>=0A</div>=0A</blockquote>=0A<br /><hr />FreeMail power= ed by <a href=3D"https://mail.fr" target=3D"_blank" rel=3D"noreferrer no= opener">mail.fr</a></blockquote>=0A</div>=0A<br /><br clear=3D"all" />= =0A<div>&nbsp;</div>=0A-- <br />=0A<div class=3D"gmail_signature">=0A<di= v dir=3D"ltr">Regards,=0A<div>Eyal Shenitzky</div>=0A</div>=0A</div>=0A<= /div>=0A</blockquote>=0A <br/><hr>FreeMail powered by= <a href=3D"https://mail.fr" target=3D"_blank">mail.fr</a>=0A --=_3c316c3018d9743cda23c6ab6b226644--