engine master CI is failing on $subject, please fix or revert as soon as possible, thanks. -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas(a)redhat.com&gt; wrote: > > it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/) > I rebased https://gerrit.ovirt.org/66999 and still failing there. > > ----- Original Message ----- > > engine master CI is failing on $subject, please fix or revert as soon as > > possible, thanks. > > > > -- > > Sandro Bonazzola > > Better technology. Faster innovation. Powered by community > > collaboration. > > See how it works at redhat.com > > > > _______________________________________________ > > Devel mailing list > > Devel(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

> On 23 Nov 2016, at 17:44, Martin Sivak <msivak(a)redhat.com&gt; wrote: > > Yeah, our patches are still failing as well. It seems the patch was > not enough or there is some other conflicting db script. I think the hash changed. IIRC there was some issue in the logic when the existing script is updated (since it was merged and reverted and re-merged with the same number, but it was not the exact same content) Martin/Eli to the rescue! > > Martin > >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: >> >> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas(a)redhat.com&gt; wrote: >>> >>> it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/) >> >> >> I rebased https://gerrit.ovirt.org/66999 and still failing there. >> >> >>> >>> ----- Original Message ----- >>>> engine master CI is failing on $subject, please fix or revert as soon as >>>> possible, thanks. >>>> >>>> -- >>>> Sandro Bonazzola >>>> Better technology. Faster innovation. Powered by community >>>> collaboration. >>>> See how it works at redhat.com >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> >> >> -- >> Sandro Bonazzola >> Better technology. Faster innovation. Powered by community collaboration. >> See how it works at redhat.com >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > >

What does it mean to clean the jobs? You mean git history or temp files? On Nov 24, 2016 1:23 AM, "Martin Perina" <mperina(a)redhat.com&gt; wrote: > > So, I tested current master and also upgrade from 4.0 to master and

> > But I've found 2 issues: > > 1. Jenkins CI for some reason unknown to me doesn't use current master

> > 2. When we are reverting patches which contain db upgrade scripts which

> > a. Comment out code in db upgrade script we want to revert, but leave

> > b. Remove the db upgrade script and renumber all later db upgrade

> > Conclusion: current code in master is OK, we need clean up jenkins job

> > Martin > > > [1] https://gerrit.ovirt.org/#/c/67205/ > > > On Wed, Nov 23, 2016 at 7:17 PM, Michal Skrivanek <mskrivan(a)redhat.com&gt;

>> >> >> > On 23 Nov 2016, at 17:44, Martin Sivak <msivak(a)redhat.com&gt; wrote: >> > >> > Yeah, our patches are still failing as well. It seems the patch was >> > not enough or there is some other conflicting db script. >> >> I think the hash changed. IIRC there was some issue in the logic when

>> Martin/Eli to the rescue! >> >> > >> > Martin >> > >> >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola <

>> >> >> >> >> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas(a)redhat.com&gt;

>> >>> >> >>> it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/) >> >> >> >> >> >> I rebased https://gerrit.ovirt.org/66999 and still failing there. >> >> >> >> >> >>> >> >>> ----- Original Message ----- >> >>>> engine master CI is failing on $subject, please fix or revert as

>> >>>> possible, thanks. >> >>>> >> >>>> -- >> >>>> Sandro Bonazzola >> >>>> Better technology. Faster innovation. Powered by community >> >>>> collaboration. >> >>>> See how it works at redhat.com >> >>>> >> >>>> _______________________________________________ >> >>>> Devel mailing list >> >>>> Devel(a)ovirt.org >> >>>> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> >> >> >> >> >> >> >> -- >> >> Sandro Bonazzola >> >> Better technology. Faster innovation. Powered by community

>> >> See how it works at redhat.com >> >> >> >> _______________________________________________ >> >> Devel mailing list >> >> Devel(a)ovirt.org >> >> http://lists.ovirt.org/mailman/listinfo/devel >> > _______________________________________________ >> > Devel mailing list >> > Devel(a)ovirt.org >> > http://lists.ovirt.org/mailman/listinfo/devel >> > >> > >

Can you paste here example of such job failing now?

On Thu, Nov 24, 2016 at 9:42 AM, Martin Perina <mperina(a)redhat.com&gt; wrote: > I don't know what exactly upgrade from master job is doing, but according > to error produced by engine-setup it does not take latest patch merged to > master to prepare initial db on which we try to execute upgrade. It take > the revert patch, which is faulty and that's why upgrade job is failing. > > Martin > > On Thursday, November 24, 2016, Eyal Edri <eedri(a)redhat.com&gt; wrote: > > What does it mean to clean the jobs? You mean git history or temp > files? > > > > On Nov 24, 2016 1:23 AM, "Martin Perina" <mperina(a)redhat.com&gt; wrote: > >> > >> So, I tested current master and also upgrade from 4.0 to master and > everything works fine on my local machine. > >> > >> But I've found 2 issues: > >> > >> 1. Jenkins CI for some reason unknown to me doesn't use current master > (change-id: I59087ff8902e35b9f7c922eba0a17ba4f0791795) but it uses older > commit (most probably revert patch from Eyal [1] with Change-Id: > I8c2350aba48619dfa84624b343cfa167d0b6e0ed).as a current master when > executing upgrade job from master to patch > >> > >> 2. When we are reverting patches which contain db upgrade scripts > which are not last (for example we are dropping db script 04_01_0500 when > there's already merged script 04_01_0510) we cannot delete upgrade script > otherwise db schema upgrade script fails (revert script [1] is faulty). > Only following actions are valid in this case: > >> > >> a. Comment out code in db upgrade script we want to revert, but > leave this script in the repo > >> > >> b. Remove the db upgrade script and renumber all later db upgrade > scripts so there's no hole in db upgrade script numbering > >> > >> Conclusion: current code in master is OK, we need clean up jenkins job > to really use latest merged patch on git master branch. > >> > >> Martin > >> > >> > >> [1] https://gerrit.ovirt.org/#/c/67205/ > >> > >> > >> On Wed, Nov 23, 2016 at 7:17 PM, Michal Skrivanek <mskrivan(a)redhat.com&gt; > wrote: > >>> > >>> > >>> > On 23 Nov 2016, at 17:44, Martin Sivak <msivak(a)redhat.com&gt; wrote: > >>> > > >>> > Yeah, our patches are still failing as well. It seems the patch was > >>> > not enough or there is some other conflicting db script. > >>> > >>> I think the hash changed. IIRC there was some issue in the logic when > the existing script is updated (since it was merged and reverted and > re-merged with the same number, but it was not the exact same content) > >>> Martin/Eli to the rescue! > >>> > >>> > > >>> > Martin > >>> > > >>> >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola < > sbonazzo(a)redhat.com&gt; wrote: > >>> >> > >>> >> > >>> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas(a)redhat.com&gt; > wrote: > >>> >>> > >>> >>> it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/) > >>> >> > >>> >> > >>> >> I rebased https://gerrit.ovirt.org/66999 and still failing there. > >>> >> > >>> >> > >>> >>> > >>> >>> ----- Original Message ----- > >>> >>>> engine master CI is failing on $subject, please fix or revert as > soon as > >>> >>>> possible, thanks. > >>> >>>> > >>> >>>> -- > >>> >>>> Sandro Bonazzola > >>> >>>> Better technology. Faster innovation. Powered by community > >>> >>>> collaboration. > >>> >>>> See how it works at redhat.com > >>> >>>> > >>> >>>> _______________________________________________ > >>> >>>> Devel mailing list > >>> >>>> Devel(a)ovirt.org > >>> >>>> http://lists.ovirt.org/mailman/listinfo/devel > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Sandro Bonazzola > >>> >> Better technology. Faster innovation. Powered by community > collaboration. > >>> >> See how it works at redhat.com > >>> >> > >>> >> _______________________________________________ > >>> >> Devel mailing list > >>> >> Devel(a)ovirt.org > >>> >> http://lists.ovirt.org/mailman/listinfo/devel > >>> > _______________________________________________ > >>> > Devel mailing list > >>> > Devel(a)ovirt.org > >>> > http://lists.ovirt.org/mailman/listinfo/devel > >>> > > >>> > > >> > > > -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

On Thu, Nov 24, 2016 at 10:37 AM, Martin Perina <mperina(a)redhat.com&gt; wrote: > > > > On Thu, Nov 24, 2016 at 9:31 AM, Eyal Edri <eedri(a)redhat.com&gt; wrote: >> >> Can you paste here example of such job failing now? > > > Sure, please take a look at patch https://gerrit.ovirt.org/66999 > > I've rebased it yesterday at 23:22 and upgrade from master job failed due > to the error I've described below: > > > http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_... > > Although at that moment patch > https://gerrit.ovirt.org/67210 , which fixed faulty revert patch [1], was > already merged, so engine-setup should execute successfully. > > I've just scheduled another build on the same patch (there was no rebase > at this time), so you can compare the results. So today's build finished fine, which is strange, because patch https://gerrit.ovirt.org/67210 haven't been rebased, so upgrade from master should fail same way as yesterday -> so it seems that upgrade from master caches somewhere git hash, which is used as a startup for db upgrade job and not always using actual latest commit from master

> > Martin > >> >> On Thu, Nov 24, 2016 at 9:42 AM, Martin Perina <mperina(a)redhat.com&gt; >> wrote: >>> >>> I don't know what exactly upgrade from master job is doing, but >>> according to error produced by engine-setup it does not take latest patch >>> merged to master to prepare initial db on which we try to execute upgrade. >>> It take the revert patch, which is faulty and that's why upgrade job is >>> failing. >>> >>> Martin >>> >>> On Thursday, November 24, 2016, Eyal Edri <eedri(a)redhat.com&gt; wrote: >>> > What does it mean to clean the jobs? You mean git history or temp >>> > files? >>> > >>> > On Nov 24, 2016 1:23 AM, "Martin Perina" <mperina(a)redhat.com&gt; wrote: >>> >> >>> >> So, I tested current master and also upgrade from 4.0 to master and >>> >> everything works fine on my local machine. >>> >> >>> >> But I've found 2 issues: >>> >> >>> >> 1. Jenkins CI for some reason unknown to me doesn't use current >>> >> master (change-id: I59087ff8902e35b9f7c922eba0a17ba4f0791795) but it uses >>> >> older commit (most probably revert patch from Eyal [1] with Change-Id: >>> >> I8c2350aba48619dfa84624b343cfa167d0b6e0ed).as a current master when >>> >> executing upgrade job from master to patch >>> >> >>> >> 2. When we are reverting patches which contain db upgrade scripts >>> >> which are not last (for example we are dropping db script 04_01_0500 when >>> >> there's already merged script 04_01_0510) we cannot delete upgrade script >>> >> otherwise db schema upgrade script fails (revert script [1] is faulty). Only >>> >> following actions are valid in this case: >>> >> >>> >> a. Comment out code in db upgrade script we want to revert, but >>> >> leave this script in the repo >>> >> >>> >> b. Remove the db upgrade script and renumber all later db upgrade >>> >> scripts so there's no hole in db upgrade script numbering >>> >> >>> >> Conclusion: current code in master is OK, we need clean up jenkins >>> >> job to really use latest merged patch on git master branch. >>> >> >>> >> Martin >>> >> >>> >> >>> >> [1] https://gerrit.ovirt.org/#/c/67205/ >>> >> >>> >> >>> >> On Wed, Nov 23, 2016 at 7:17 PM, Michal Skrivanek >>> >> <mskrivan(a)redhat.com&gt; wrote: >>> >>> >>> >>> >>> >>> > On 23 Nov 2016, at 17:44, Martin Sivak <msivak(a)redhat.com&gt; wrote: >>> >>> > >>> >>> > Yeah, our patches are still failing as well. It seems the patch >>> >>> > was >>> >>> > not enough or there is some other conflicting db script. >>> >>> >>> >>> I think the hash changed. IIRC there was some issue in the logic >>> >>> when the existing script is updated (since it was merged and reverted and >>> >>> re-merged with the same number, but it was not the exact same content) >>> >>> Martin/Eli to the rescue! >>> >>> >>> >>> > >>> >>> > Martin >>> >>> > >>> >>> >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola >>> >>> >> <sbonazzo(a)redhat.com&gt; wrote: >>> >>> >> >>> >>> >> >>> >>> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas(a)redhat.com&gt; >>> >>> >>> wrote: >>> >>> >>> >>> >>> >>> it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/) >>> >>> >> >>> >>> >> >>> >>> >> I rebased https://gerrit.ovirt.org/66999 and still failing there. >>> >>> >> >>> >>> >> >>> >>> >>> >>> >>> >>> ----- Original Message ----- >>> >>> >>>> engine master CI is failing on $subject, please fix or revert >>> >>> >>>> as soon as >>> >>> >>>> possible, thanks. >>> >>> >>>> >>> >>> >>>> -- >>> >>> >>>> Sandro Bonazzola >>> >>> >>>> Better technology. Faster innovation. Powered by community >>> >>> >>>> collaboration. >>> >>> >>>> See how it works at redhat.com >>> >>> >>>> >>> >>> >>>> _______________________________________________ >>> >>> >>>> Devel mailing list >>> >>> >>>> Devel(a)ovirt.org >>> >>> >>>> http://lists.ovirt.org/mailman/listinfo/devel >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> -- >>> >>> >> Sandro Bonazzola >>> >>> >> Better technology. Faster innovation. Powered by community >>> >>> >> collaboration. >>> >>> >> See how it works at redhat.com >>> >>> >> >>> >>> >> _______________________________________________ >>> >>> >> Devel mailing list >>> >>> >> Devel(a)ovirt.org >>> >>> >> http://lists.ovirt.org/mailman/listinfo/devel >>> >>> > _______________________________________________ >>> >>> > Devel mailing list >>> >>> > Devel(a)ovirt.org >>> >>> > http://lists.ovirt.org/mailman/listinfo/devel >>> >>> > >>> >>> > >>> >> >>> > >> >> >> >> >> -- >> Eyal Edri >> Associate Manager >> RHV DevOps >> EMEA ENG Virtualization R&D >> Red Hat Israel >> >> phone: +972-9-7692018 >> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Thu, Nov 24, 2016 at 11:25 AM, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > > On Thu, Nov 24, 2016 at 12:21 PM, Martin Perina <mperina(a)redhat.com&gt; > wrote: > > > > > > On Thu, Nov 24, 2016 at 10:37 AM, Martin Perina <mperina(a)redhat.com&gt; > > wrote: > >> > >> > >> > >> On Thu, Nov 24, 2016 at 9:31 AM, Eyal Edri <eedri(a)redhat.com&gt; wrote: > >>> > >>> Can you paste here example of such job failing now? > >> > >> > >> Sure, please take a look at patch https://gerrit.ovirt.org/66999 > >> > >> I've rebased it yesterday at 23:22 and upgrade from master job failed > >> due > >> to the error I've described below: > >> > >> > >> > >> http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_... > >> > >> Although at that moment patch > >> https://gerrit.ovirt.org/67210 , which fixed faulty revert patch [1], > >> was > >> already merged, so engine-setup should execute successfully. > >> > >> I've just scheduled another build on the same patch (there was no > >> rebase > >> at this time), so you can compare the results. > > > > > > So today's build finished fine, which is strange, because > > > > patch https://gerrit.ovirt.org/67210 haven't been rebased, so upgrade > > from > > master should fail same way as yesterday -> so it seems that upgrade > > from > > master caches somewhere git hash, which is used as a startup for db > > upgrade > > job and not always using actual latest commit from master > > AFAIU the upgrade job upgrade from the latest nightly master snapshot > build, not from git. > So after you merge such a patch and its builds are finished, you have > to run the publisher. Well, it that case the logic of "upgrade from master" job is a bit misleading, because it's "upgrade from yesterday's master" actually ...

> > > > > >> > >> Martin > >> > >>> > >>> On Thu, Nov 24, 2016 at 9:42 AM, Martin Perina <mperina(a)redhat.com&gt; > >>> wrote: > >>>> > >>>> I don't know what exactly upgrade from master job is doing, but > >>>> according to error produced by engine-setup it does not take latest > >>>> patch > >>>> merged to master to prepare initial db on which we try to execute > >>>> upgrade. > >>>> It take the revert patch, which is faulty and that's why upgrade job > >>>> is > >>>> failing. > >>>> > >>>> Martin > >>>> > >>>> On Thursday, November 24, 2016, Eyal Edri <eedri(a)redhat.com&gt; wrote: > >>>> > What does it mean to clean the jobs? You mean git history or temp > >>>> > files? > >>>> > > >>>> > On Nov 24, 2016 1:23 AM, "Martin Perina" <mperina(a)redhat.com&gt; > >>>> > wrote: > >>>> >> > >>>> >> So, I tested current master and also upgrade from 4.0 to master > >>>> >> and > >>>> >> everything works fine on my local machine. > >>>> >> > >>>> >> But I've found 2 issues: > >>>> >> > >>>> >> 1. Jenkins CI for some reason unknown to me doesn't use current > >>>> >> master (change-id: I59087ff8902e35b9f7c922eba0a17ba4f0791795) but > >>>> >> it uses > >>>> >> older commit (most probably revert patch from Eyal [1] with > >>>> >> Change-Id: > >>>> >> I8c2350aba48619dfa84624b343cfa167d0b6e0ed).as a current master > >>>> >> when > >>>> >> executing upgrade job from master to patch > >>>> >> > >>>> >> 2. When we are reverting patches which contain db upgrade scripts > >>>> >> which are not last (for example we are dropping db script > >>>> >> 04_01_0500 when > >>>> >> there's already merged script 04_01_0510) we cannot delete upgrade > >>>> >> script > >>>> >> otherwise db schema upgrade script fails (revert script [1] is > >>>> >> faulty). Only > >>>> >> following actions are valid in this case: > >>>> >> > >>>> >> a. Comment out code in db upgrade script we want to revert, but > >>>> >> leave this script in the repo > >>>> >> > >>>> >> b. Remove the db upgrade script and renumber all later db > >>>> >> upgrade > >>>> >> scripts so there's no hole in db upgrade script numbering > >>>> >> > >>>> >> Conclusion: current code in master is OK, we need clean up jenkins > >>>> >> job to really use latest merged patch on git master branch. > >>>> >> > >>>> >> Martin > >>>> >> > >>>> >> > >>>> >> [1] https://gerrit.ovirt.org/#/c/67205/ > >>>> >> > >>>> >> > >>>> >> On Wed, Nov 23, 2016 at 7:17 PM, Michal Skrivanek > >>>> >> <mskrivan(a)redhat.com&gt; wrote: > >>>> >>> > >>>> >>> > >>>> >>> > On 23 Nov 2016, at 17:44, Martin Sivak <msivak(a)redhat.com&gt; > >>>> >>> > wrote: > >>>> >>> > > >>>> >>> > Yeah, our patches are still failing as well. It seems the patch > >>>> >>> > was > >>>> >>> > not enough or there is some other conflicting db script. > >>>> >>> > >>>> >>> I think the hash changed. IIRC there was some issue in the logic > >>>> >>> when the existing script is updated (since it was merged and > >>>> >>> reverted and > >>>> >>> re-merged with the same number, but it was not the exact same > >>>> >>> content) > >>>> >>> Martin/Eli to the rescue! > >>>> >>> > >>>> >>> > > >>>> >>> > Martin > >>>> >>> > > >>>> >>> >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola > >>>> >>> >> <sbonazzo(a)redhat.com&gt; wrote: > >>>> >>> >> > >>>> >>> >> > >>>> >>> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas > >>>> >>> >>> <ahadas(a)redhat.com&gt; > >>>> >>> >>> wrote: > >>>> >>> >>> > >>>> >>> >>> it should be fixed now (by > >>>> >>> >>> https://gerrit.ovirt.org/#/c/67210/) > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> I rebased https://gerrit.ovirt.org/66999 and still failing > >>>> >>> >> there. > >>>> >>> >> > >>>> >>> >> > >>>> >>> >>> > >>>> >>> >>> ----- Original Message ----- > >>>> >>> >>>> engine master CI is failing on $subject, please fix or > >>>> >>> >>>> revert > >>>> >>> >>>> as soon as > >>>> >>> >>>> possible, thanks. > >>>> >>> >>>> > >>>> >>> >>>> -- > >>>> >>> >>>> Sandro Bonazzola > >>>> >>> >>>> Better technology. Faster innovation. Powered by community > >>>> >>> >>>> collaboration. > >>>> >>> >>>> See how it works at redhat.com > >>>> >>> >>>> > >>>> >>> >>>> _______________________________________________ > >>>> >>> >>>> Devel mailing list > >>>> >>> >>>> Devel(a)ovirt.org > >>>> >>> >>>> http://lists.ovirt.org/mailman/listinfo/devel > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> -- > >>>> >>> >> Sandro Bonazzola > >>>> >>> >> Better technology. Faster innovation. Powered by community > >>>> >>> >> collaboration. > >>>> >>> >> See how it works at redhat.com > >>>> >>> >> > >>>> >>> >> _______________________________________________ > >>>> >>> >> Devel mailing list > >>>> >>> >> Devel(a)ovirt.org > >>>> >>> >> http://lists.ovirt.org/mailman/listinfo/devel > >>>> >>> > _______________________________________________ > >>>> >>> > Devel mailing list > >>>> >>> > Devel(a)ovirt.org > >>>> >>> > http://lists.ovirt.org/mailman/listinfo/devel > >>>> >>> > > >>>> >>> > > >>>> >> > >>>> > > >>> > >>> > >>> > >>> > >>> -- > >>> Eyal Edri > >>> Associate Manager > >>> RHV DevOps > >>> EMEA ENG Virtualization R&D > >>> Red Hat Israel > >>> > >>> phone: +972-9-7692018 > >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) > >> > >> > > > > > > _______________________________________________ > > Devel mailing list > > Devel(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel > > > > -- > Didi