On Thu, Nov 24, 2016 at 10:37 AM, Martin Perina <mperina@redhat.com> wrote:


On Thu, Nov 24, 2016 at 9:31 AM, Eyal Edri <eedri@redhat.com> wrote:
Can you paste here example of such job failing now?

​Sure, please take a look at patch https://gerrit.ovirt.org/66999
 
​I've rebased it yesterday at 23:22 and upgrade from master job failed due to the error I've described below:

http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_created/9295/

Although at that moment​ patch
​​
https://gerrit.ovirt.org/67210 , which fixed faulty revert patch [1], was already merged, so engine-setup should execute successfully.

I've just scheduled another build on the same patch (there was no rebase at this time), so you can compare the results.

​So today's build finished fine, which is strange, because​
 
​patch ​​ https://gerrit.ovirt.org/67210 haven't been rebased, so upgrade from master should fail same way as yesterday -> so it seems that upgrade from master caches somewhere git hash, which is used as a startup for db upgrade job and not always using actual latest commit from master


Martin


On Thu, Nov 24, 2016 at 9:42 AM, Martin Perina <mperina@redhat.com> wrote:
I don't know what exactly upgrade from master job is doing, but according to error produced by engine-setup it does not take latest patch merged to master to prepare initial db on which we try to execute upgrade. It take the revert patch, which is faulty and that's why upgrade job is failing.

Martin

On Thursday, November 24, 2016, Eyal Edri <eedri@redhat.com> wrote:
> What does it mean to clean the jobs?  You mean git history or temp files?
>
> On Nov 24, 2016 1:23 AM, "Martin Perina" <mperina@redhat.com> wrote:
>>
>> So, I tested current master and also upgrade from 4.0 to master and everything works fine on my local machine.
>>
>> But I've found 2 issues:
>>
>> 1. Jenkins CI for some reason unknown to me doesn't use current master (change-id: I59087ff8902e35b9f7c922eba0a17ba4f0791795) but it uses older commit (most probably revert patch from Eyal [1] with Change-Id: I8c2350aba48619dfa84624b343cfa167d0b6e0ed).as a current master when executing upgrade job from master to patch
>>
>> 2. When we are reverting patches which contain db upgrade scripts which are not last (for example we are dropping db script 04_01_0500 when there's already merged script 04_01_0510) we cannot delete upgrade script otherwise db schema upgrade script fails (revert script [1] is faulty). Only following actions are valid in this case:
>>
>>   a. Comment out code in db upgrade script we want to revert, but leave this script in the repo
>>
>>   b. Remove the db upgrade script and renumber all later db upgrade scripts so there's no hole in db upgrade script numbering
>>
>> Conclusion: current code in master is OK, we need clean up jenkins job to really use latest merged patch on git master branch.
>>
>> Martin
>>
>>
>> [1] https://gerrit.ovirt.org/#/c/67205/
>>
>>
>> On Wed, Nov 23, 2016 at 7:17 PM, Michal Skrivanek <mskrivan@redhat.com> wrote:
>>>
>>>
>>> > On 23 Nov 2016, at 17:44, Martin Sivak <msivak@redhat.com> wrote:
>>> >
>>> > Yeah, our patches are still failing as well. It seems the patch was
>>> > not enough or there is some other conflicting db script.
>>>
>>> I think the hash changed. IIRC there was some issue in the logic when the existing script is updated (since it was merged and reverted and re-merged with the same number, but it was not the exact same content)
>>> Martin/Eli to the rescue!
>>>
>>> >
>>> > Martin
>>> >
>>> >> On Wed, Nov 23, 2016 at 2:16 PM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
>>> >>
>>> >>
>>> >>> On Wed, Nov 23, 2016 at 2:13 PM, Arik Hadas <ahadas@redhat.com> wrote:
>>> >>>
>>> >>> it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/)
>>> >>
>>> >>
>>> >> I rebased https://gerrit.ovirt.org/66999 and still failing there.
>>> >>
>>> >>
>>> >>>
>>> >>> ----- Original Message -----
>>> >>>> engine master CI is failing on $subject, please fix or revert as soon as
>>> >>>> possible, thanks.
>>> >>>>
>>> >>>> --
>>> >>>> Sandro Bonazzola
>>> >>>> Better technology. Faster innovation. Powered by community
>>> >>>> collaboration.
>>> >>>> See how it works at redhat.com
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Devel mailing list
>>> >>>> Devel@ovirt.org
>>> >>>> http://lists.ovirt.org/mailman/listinfo/devel
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Sandro Bonazzola
>>> >> Better technology. Faster innovation. Powered by community collaboration.
>>> >> See how it works at redhat.com
>>> >>
>>> >> _______________________________________________
>>> >> Devel mailing list
>>> >> Devel@ovirt.org
>>> >> http://lists.ovirt.org/mailman/listinfo/devel
>>> > _______________________________________________
>>> > Devel mailing list
>>> > Devel@ovirt.org
>>> > http://lists.ovirt.org/mailman/listinfo/devel
>>> >
>>> >
>>
>



--
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)