<div dir="ltr">OK. so for now the patch [1] skips all storage and OVSNetwork tests<div>Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr, for the setup :\ the rest for the tests)</div><div><br><div>The output looks pretty good though :)</div><div><br></div><div>[1] <a href="https://gerrit.ovirt.org/#/c/48268/">https://gerrit.ovirt.org/#/c/48268/</a></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <span dir="ltr"><<a href="mailto:phoracek@redhat.com" target="_blank">phoracek@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped<br>
if the package is not installed) and openvswitch service running (not<br>
started automatically). NetworkTest are not able to run together with<br>
OVSNetworkTest since we are doing ugly inheritance hacking there. My<br>
apologize for network tests difficulties, it will be resolved soon. If<br>
you want your life easier, temporary skip OVSNetworkTest suite and it<br>
should be OK.<br>
<br>
Best regards,<br>
Petr<br>
<div><div class="h5"><br>
2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <<a href="mailto:ybronhei@redhat.com">ybronhei@redhat.com</a>>:<br>
> exactly - just posted a patch again that sings all fails as broken .<br>
> we'll get the report soon and I'll publish it as well. hope the run will<br>
> take less time now<br>
><br>
> On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>> wrote:<br>
>><br>
>> Nice, but we cannot enable this until all the tests pass or disabled.<br>
>><br>
>> There is no point in broken or flaky functional tests.<br>
>><br>
>> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <<a href="mailto:ybronhei@redhat.com">ybronhei@redhat.com</a>><br>
>> wrote:<br>
>> > So its not stable. It won't block merges and at least give us report<br>
>> > after<br>
>> > each merge. It takes really long time to run it (because of the tests<br>
>> > themselves. Lago things takes maximum 15minutes, but the run last for<br>
>> > more<br>
>> > than 2hrs right now and I suspect functional/storageTests.py gets stuck)<br>
>> ><br>
>> > Bellow you can see where we stand (before I added python-rtslib<br>
>> > package).<br>
>> ><br>
>> > Now, I still want to merge the patch <a href="https://gerrit.ovirt.org/#/c/48268/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/48268/</a><br>
>> > -<br>
>> > which enables this run after merges, and I still want you to consider<br>
>> > the<br>
>> > addition of Automation CI flag to our gerrit so that developer will be<br>
>> > able<br>
>> > to use it as a trigger for the check-merged.sh script run, just to see<br>
>> > if<br>
>> > their patch fixes\brakes something realted to the functional tests<br>
>> ><br>
>> > <a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/</a> -<br>
>> > is<br>
>> > an example of how the run looks like. I still work to improve the output<br>
>> ><br>
>> ><br>
>> > Please reply and let me know if the idea around the automation flag is<br>
>> > acceptable by you.. and please review the patch for comments and acks.<br>
>> > We can ask dcaro to add the flag until Friday, otherwise we'll need to<br>
>> > delay<br>
>> > this effort after the holiday..<br>
>> ><br>
>> ><br>
>> > functional.sosPluginTests.SosPluginTest<br>
>> > testSosPlugin OK<br>
>> > functional.vmRecoveryTests.RecoveryTests<br>
>> > test_vm_recovery FAIL<br>
>> > functional.vmQoSTests.VMQosTests<br>
>> > testSmallVMBallooning FAIL<br>
>> > functional.virtTests.VirtTest<br>
>> > testComplexVm FAIL<br>
>> > testHeadlessVm OK<br>
>> > testSimpleVm FAIL<br>
>> > testVmDefinitionGraphics('spice') FAIL<br>
>> > testVmDefinitionGraphics('vnc') OK<br>
>> > testVmDefinitionLegacyGraphics('qxl') FAIL<br>
>> > testVmDefinitionLegacyGraphics('vnc') OK<br>
>> > testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL<br>
>> > testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL<br>
>> > testVmWithCdrom('self') FAIL<br>
>> > testVmWithCdrom('specParams') FAIL<br>
>> > testVmWithCdrom('vmPayload') FAIL<br>
>> > testVmWithDevice('hotplugDisk') FAIL<br>
>> > testVmWithDevice('hotplugNic') FAIL<br>
>> > testVmWithDevice('smartcard') FAIL<br>
>> > testVmWithDevice('virtioNic') FAIL<br>
>> > testVmWithDevice('virtioRng') FAIL<br>
>> > testVmWithSla FAIL<br>
>> > testVmWithStorage('iscsi') SKIP:<br>
>> > python-rtslib is not installed.<br>
>> > testVmWithStorage('localfs') FAIL<br>
>> > testVmWithStorage('nfs') FAIL<br>
>> > functional.storageTests.StorageTest<br>
>> > testCreatePoolErrors OK<br>
>> > testStorage('glusterfs', 0) ERROR<br>
>> > testStorage('glusterfs', 3) ERROR<br>
>> > testStorage('iscsi', 0) SKIP:<br>
>> > python-rtslib is not installed.<br>
>> > testStorage('iscsi', 3) SKIP:<br>
>> > python-rtslib is not installed.<br>
>> > testStorage('localfs', 0) FAIL<br>
>> > testStorage('localfs', 3) FAIL<br>
>> > testStorage('nfs', 0) FAIL<br>
>> > testStorage('nfs', 3) FAIL<br>
>> > functional.networkTests.NetworkTest<br>
>> > testAddVlanedBridgeless ERROR<br>
>> > testAddVlanedBridgeless_oneCommand ERROR<br>
>> > testAfterNetworkSetupHook ERROR<br>
>> > testBeforeNetworkSetupHook ERROR<br>
>> > testBondHwAddress(False) ERROR<br>
>> > testBondHwAddress(True) ERROR<br>
>> > testBrokenNetworkReplacement(False) ERROR<br>
>> > testBrokenNetworkReplacement(True) ERROR<br>
>> > testDelNetworkBondAccumulation ERROR<br>
>> > testDelNetworkWithMTU(False) ERROR<br>
>> > testDelNetworkWithMTU(True) ERROR<br>
>> > testDelWithoutAdd ERROR<br>
>> > testDhclientLeases(4, 'default') ERROR<br>
>> > testDhclientLeases(4, 'local') ERROR<br>
>> > testDhclientLeases(6, None) ERROR<br>
>> > testDhcpReplaceNicWithBridge ERROR<br>
>> > testFailWithInvalidBondingName(False) ERROR<br>
>> > testFailWithInvalidBondingName(True) ERROR<br>
>> > testFailWithInvalidBridgeName ERROR<br>
>> > testFailWithInvalidIpConfig ERROR<br>
>> > testFailWithInvalidNic(False) ERROR<br>
>> > testFailWithInvalidNic(True) ERROR<br>
>> > testFailWithInvalidParams(False) ERROR<br>
>> > testFailWithInvalidParams(True) ERROR<br>
>> > testGetRouteDeviceTo ERROR<br>
>> > testHonorBlockingDhcp SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testIpLinkWrapper ERROR<br>
>> > testLowerMtuDoesNotOverride FAIL<br>
>> > testNoBridgeLeftovers FAIL<br>
>> > testReconfigureBrNetWithVanishedPort ERROR<br>
>> > testRedefineBondedNetworkIPs ERROR<br>
>> > testRemovingBridgeDoesNotLeaveBridge ERROR<br>
>> > testReorderBondingOptions(False) ERROR<br>
>> > testReorderBondingOptions(True) ERROR<br>
>> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR<br>
>> > testRestoreToBlockingDHCP ERROR<br>
>> > testSafeNetworkConfig(False) ERROR<br>
>> > testSafeNetworkConfig(True) ERROR<br>
>> > testSelectiveRestoreDuringUpgrade ERROR<br>
>> > testSelectiveRestoreIgnoresVdsmRegParams FAIL<br>
>> > testSetupNetworkOutboundQos(False) ERROR<br>
>> > testSetupNetworkOutboundQos(True) FAIL<br>
>> > testSetupNetworksActiveSlave ERROR<br>
>> > testSetupNetworksAddBadParams(False) ERROR<br>
>> > testSetupNetworksAddBadParams(True) FAIL<br>
>> > testSetupNetworksAddBondWithManyVlans(False) ERROR<br>
>> > testSetupNetworksAddBondWithManyVlans(True) FAIL<br>
>> > testSetupNetworksAddDelBondedNetwork(False) ERROR<br>
>> > testSetupNetworksAddDelBondedNetwork(True) FAIL<br>
>> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR<br>
>> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR<br>
>> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR<br>
>> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL<br>
>> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL<br>
>> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL<br>
>> > testSetupNetworksAddManyVlans(False) ERROR<br>
>> > testSetupNetworksAddManyVlans(True) FAIL<br>
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR<br>
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL<br>
>> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR<br>
>> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR<br>
>> > testSetupNetworksAddOverExistingBond(False) ERROR<br>
>> > testSetupNetworksAddOverExistingBond(True) ERROR<br>
>> > testSetupNetworksAddVlan(False) ERROR<br>
>> > testSetupNetworksAddVlan(True) FAIL<br>
>> > testSetupNetworksConnectivityCheck FAIL<br>
>> > testSetupNetworksConvertVlanNetBridgeness ERROR<br>
>> > testSetupNetworksDelOneOfBondNets ERROR<br>
>> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR<br>
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR<br>
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR<br>
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR<br>
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL<br>
>> > testSetupNetworksMtus(False) ERROR<br>
>> > testSetupNetworksMtus(True) SKIP:<br>
>> > This<br>
>> > test fails because the 2 different networks are getting configured with<br>
>> > the<br>
>> > same MTU. The test should assert that the reported MTUs are equal to the<br>
>> > requested ones.<br>
>> > testSetupNetworksMultiMTUsOverBond(False) ERROR<br>
>> > testSetupNetworksMultiMTUsOverBond(True) FAIL<br>
>> > testSetupNetworksMultiMTUsOverNic(False) ERROR<br>
>> > testSetupNetworksMultiMTUsOverNic(True) FAIL<br>
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR<br>
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL<br>
>> > testSetupNetworksNicless FAIL<br>
>> > testSetupNetworksNiclessBridgeless ERROR<br>
>> > testSetupNetworksOverDhcpIface ERROR<br>
>> > testSetupNetworksOverExistingBridge FAIL<br>
>> > testSetupNetworksReconfigureBridge FAIL<br>
>> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR<br>
>> > testSetupNetworksRemoveSlavelessBond ERROR<br>
>> > testSetupNetworksResizeBond(False) ERROR<br>
>> > testSetupNetworksResizeBond(True) ERROR<br>
>> > testSetupNetworksStableBond(False) ERROR<br>
>> > testSetupNetworksStableBond(True) ERROR<br>
>> > testStaticNetworkConfig((4, 6)) FAIL<br>
>> > testStaticNetworkConfig((4,)) FAIL<br>
>> > testStaticNetworkConfig((6,)) FAIL<br>
>> > testStaticSourceRouting(False) ERROR<br>
>> > testStaticSourceRouting(True) FAIL<br>
>> > testTwiceAdd(False) ERROR<br>
>> > testTwiceAdd(True) ERROR<br>
>> > testVolatileConfig(False) ERROR<br>
>> > testVolatileConfig(True) ERROR<br>
>> > test_drop_initial_bond_slaves_ip_config FAIL<br>
>> > test_getVdsStats ERROR<br>
>> > test_remove_bond_under_network FAIL<br>
>> > test_remove_initial_network_nic_ip_config FAIL<br>
>> > test_rollback FAIL<br>
>> > test_setupNetworks_bond_with_custom_option ERROR<br>
>> > test_setupNetworks_on_external_bond ERROR<br>
>> > test_setupNetworks_on_external_vlaned_bond ERROR<br>
>> > functional.networkTestsOVS.OVSNetworkTest<br>
>> > testHonorBlockingDhcp SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testLowerMtuDoesNotOverride OK<br>
>> > testNoBridgeLeftovers FAIL<br>
>> > testSelectiveRestoreIgnoresVdsmRegParams FAIL<br>
>> > testSetupNetworkOutboundQos(True) OK<br>
>> > testSetupNetworksAddBadParams(True) OK<br>
>> > testSetupNetworksAddBondWithManyVlans(True) FAIL<br>
>> > testSetupNetworksAddDelBondedNetwork(True) FAIL<br>
>> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL<br>
>> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL<br>
>> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL<br>
>> > testSetupNetworksAddManyVlans(True) OK<br>
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL<br>
>> > testSetupNetworksAddVlan(True) FAIL<br>
>> > testSetupNetworksConnectivityCheck OK<br>
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:<br>
>> > Slow<br>
>> > tests are disabled<br>
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL<br>
>> > testSetupNetworksMtus(True) SKIP:<br>
>> > This<br>
>> > test fails because the 2 different networks are getting configured with<br>
>> > the<br>
>> > same MTU. The test should assert that the reported MTUs are equal to the<br>
>> > requested ones.<br>
>> > testSetupNetworksMultiMTUsOverBond(True) FAIL<br>
>> > testSetupNetworksMultiMTUsOverNic(True) FAIL<br>
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL<br>
>> > testSetupNetworksNicless OK<br>
>> > testSetupNetworksOverExistingBridge OK<br>
>> > testSetupNetworksReconfigureBridge FAIL<br>
>> > testStaticNetworkConfig((4, 6)) FAIL<br>
>> > testStaticNetworkConfig((4,)) FAIL<br>
>> > testStaticNetworkConfig((6,)) FAIL<br>
>> > testStaticSourceRouting(True) FAIL<br>
>> > test_drop_initial_bond_slaves_ip_config FAIL<br>
>> > test_ovirtmgmtm_to_ovs FAIL<br>
>> > test_remove_bond_under_network SKIP:<br>
>> > OVS<br>
>> > does not support bridgeless networks<br>
>> > test_remove_initial_network_nic_ip_config OK<br>
>> > test_rollback FAIL<br>
>> ><br>
>> ><br>
>> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> I like the idea but I have the same feelings as Francesco. I think that<br>
>> >> we<br>
>> >> need to make sure that functional tests for each vertical are stable<br>
>> >> before<br>
>> >> enabling this process.<br>
>> >><br>
>> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>> wrote:<br>
>> >>><br>
>> >>> adding also infra team for visibility on the change in CI.<br>
>> >>> also inline.<br>
>> >>><br>
>> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <<a href="mailto:fromani@redhat.com">fromani@redhat.com</a>><br>
>> >>> wrote:<br>
>> >>>><br>
>> >>>> ----- Original Message -----<br>
>> >>>> > From: "Yaniv Bronheim" <<a href="mailto:ybronhei@redhat.com">ybronhei@redhat.com</a>><br>
>> >>>> > To: <a href="mailto:devel@ovirt.org">devel@ovirt.org</a>, "Francesco Romani" <<a href="mailto:fromani@redhat.com">fromani@redhat.com</a>>, "Nir<br>
>> >>>> > Soffer" <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>>, "Piotr Kliczewski"<br>
>> >>>> > <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>><br>
>> >>>> > Cc: "danken" <<a href="mailto:danken@redhat.com">danken@redhat.com</a>>, "David Caro"<br>
>> >>>> > <<a href="mailto:dcaroest@redhat.com">dcaroest@redhat.com</a>>,<br>
>> >>>> > "Eyal Edri" <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>><br>
>> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM<br>
>> >>>> > Subject: Automation CI for vdsm<br>
>> >>>><br>
>> >>>> [...]<br>
>> >>>> > We want to allow developers to trigger the script once reviews and<br>
>> >>>> > verification are ready (last step before merge). To do so we agreed<br>
>> >>>> > to<br>
>> >>>> > add<br>
>> >>>> > Continues Integration flag for each vdsm patch.<br>
>> >>><br>
>> >>><br>
>> >>> This flag will be called 'Workflow' or we can name it otherwise, we<br>
>> >>> just<br>
>> >>> need to choose what makes sense.<br>
>> >>> David/Yaniv - Please correct me if I'm wrong.<br>
>> >>><br>
>> >>>><br>
>> >>>> Once this flag will be<br>
>> >>>> > signed with +1 it will trigger Jenkins CI to run the check-merged<br>
>> >>>> > script<br>
>> >>>> > (adding new button to gerrit is not an option - you can image that<br>
>> >>>> > flag as<br>
>> >>>> > a trigger button), on success Jenkins CI flag will turn to +2. on<br>
>> >>>> > fail<br>
>> >>>> > we'll get -1 and once new patchset is ready the developer will<br>
>> >>>> > remove<br>
>> >>>> > the<br>
>> >>>> > +1 and add it back to the Continues Integration flag to re-trigger<br>
>> >>>> > the<br>
>> >>>> > job.<br>
>> >>>> ><br>
>> >>>> > Please ack the process before we move on with that<br>
>> >>>><br>
>> >>>> Sounds good, even though I'm a little scared (just gut feeling, no<br>
>> >>>> evidence<br>
>> >>>> whatsoever) that this could add even more complexity and fragility to<br>
>> >>>> the jenkins<br>
>> >>>> fleet.<br>
>> >>>><br>
>> >>>> In the long run, when this is reliable, it will help greatly.<br>
>> >>>> In the short term, I'm scared because this can lead to false<br>
>> >>>> positives<br>
>> >>>> and bogus<br>
>> >>>> failures.<br>
>> >>>><br>
>> >>>> Let me stress I don't have concrete item to share or specific flaws.<br>
>> >>>><br>
>> >>>> As action item on me, I will find some time next week to check virt<br>
>> >>>> functional tests,<br>
>> >>>> to see if they need some fixes, work reliably and so forth<br>
>> >>>><br>
>> >>>> > The patch for those scripts still under review and testing -<br>
>> >>>> > <a href="https://gerrit.ovirt.org/#/c/48268" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/48268</a><br>
>> >>>><br>
>> >>>> Will review asap.<br>
>> >>>><br>
>> >>>> --<br>
>> >>>> Francesco Romani<br>
>> >>>> RedHat Engineering Virtualization R & D<br>
>> >>>> Phone: 8261328<br>
>> >>>> IRC: fromani<br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> --<br>
>> >>> Eyal Edri<br>
>> >>> Supervisor, RHEV CI<br>
>> >>> EMEA ENG Virtualization R&D<br>
>> >>> Red Hat Israel<br>
>> >>><br>
>> >>> phone: <a href="tel:%2B972-9-7692018" value="+97297692018">+972-9-7692018</a><br>
>> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)<br>
>> >><br>
>> >><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> > Yaniv Bronhaim.<br>
><br>
><br>
><br>
><br>
> --<br>
> Yaniv Bronhaim.<br>
><br>
</div></div>> _______________________________________________<br>
> Devel mailing list<br>
> <a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><span style="font-size:12.8px"><b>Yaniv Bronhaim.</b></span><br></div></div></div></div></div>
</div>