[ovirt-devel] Automation CI for vdsm
Eyal Edri
eedri at redhat.com
Sun Jan 31 12:15:16 UTC 2016
Adding lago-devel.
Anyone from Lago project can help debug why it is taking so long to setup
Lago (1:17 hours?)
Yaniv, was this run done only on CI, or someone tried to run it locally on
a baremetal server / laptop?
e.
On Tue, Dec 22, 2015 at 2:50 PM, Yaniv Bronheim <ybronhei at redhat.com> wrote:
> OK. so for now the patch [1] skips all storage and OVSNetwork tests
> Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr,
> for the setup :\ the rest for the tests)
>
> The output looks pretty good though :)
>
> [1] https://gerrit.ovirt.org/#/c/48268/
>
> On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phoracek at redhat.com>
> wrote:
>
>> Hi,
>>
>> OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped
>> if the package is not installed) and openvswitch service running (not
>> started automatically). NetworkTest are not able to run together with
>> OVSNetworkTest since we are doing ugly inheritance hacking there. My
>> apologize for network tests difficulties, it will be resolved soon. If
>> you want your life easier, temporary skip OVSNetworkTest suite and it
>> should be OK.
>>
>> Best regards,
>> Petr
>>
>> 2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybronhei at redhat.com>:
>> > exactly - just posted a patch again that sings all fails as broken .
>> > we'll get the report soon and I'll publish it as well. hope the run will
>> > take less time now
>> >
>> > On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>> >>
>> >> Nice, but we cannot enable this until all the tests pass or disabled.
>> >>
>> >> There is no point in broken or flaky functional tests.
>> >>
>> >> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei at redhat.com>
>> >> wrote:
>> >> > So its not stable. It won't block merges and at least give us report
>> >> > after
>> >> > each merge. It takes really long time to run it (because of the tests
>> >> > themselves. Lago things takes maximum 15minutes, but the run last for
>> >> > more
>> >> > than 2hrs right now and I suspect functional/storageTests.py gets
>> stuck)
>> >> >
>> >> > Bellow you can see where we stand (before I added python-rtslib
>> >> > package).
>> >> >
>> >> > Now, I still want to merge the patch
>> https://gerrit.ovirt.org/#/c/48268/
>> >> > -
>> >> > which enables this run after merges, and I still want you to consider
>> >> > the
>> >> > addition of Automation CI flag to our gerrit so that developer will
>> be
>> >> > able
>> >> > to use it as a trigger for the check-merged.sh script run, just to
>> see
>> >> > if
>> >> > their patch fixes\brakes something realted to the functional tests
>> >> >
>> >> >
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ -
>> >> > is
>> >> > an example of how the run looks like. I still work to improve the
>> output
>> >> >
>> >> >
>> >> > Please reply and let me know if the idea around the automation flag
>> is
>> >> > acceptable by you.. and please review the patch for comments and
>> acks.
>> >> > We can ask dcaro to add the flag until Friday, otherwise we'll need
>> to
>> >> > delay
>> >> > this effort after the holiday..
>> >> >
>> >> >
>> >> > functional.sosPluginTests.SosPluginTest
>> >> > testSosPlugin OK
>> >> > functional.vmRecoveryTests.RecoveryTests
>> >> > test_vm_recovery FAIL
>> >> > functional.vmQoSTests.VMQosTests
>> >> > testSmallVMBallooning FAIL
>> >> > functional.virtTests.VirtTest
>> >> > testComplexVm FAIL
>> >> > testHeadlessVm OK
>> >> > testSimpleVm FAIL
>> >> > testVmDefinitionGraphics('spice') FAIL
>> >> > testVmDefinitionGraphics('vnc') OK
>> >> > testVmDefinitionLegacyGraphics('qxl') FAIL
>> >> > testVmDefinitionLegacyGraphics('vnc') OK
>> >> > testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL
>> >> > testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL
>> >> > testVmWithCdrom('self') FAIL
>> >> > testVmWithCdrom('specParams') FAIL
>> >> > testVmWithCdrom('vmPayload') FAIL
>> >> > testVmWithDevice('hotplugDisk') FAIL
>> >> > testVmWithDevice('hotplugNic') FAIL
>> >> > testVmWithDevice('smartcard') FAIL
>> >> > testVmWithDevice('virtioNic') FAIL
>> >> > testVmWithDevice('virtioRng') FAIL
>> >> > testVmWithSla FAIL
>> >> > testVmWithStorage('iscsi') SKIP:
>> >> > python-rtslib is not installed.
>> >> > testVmWithStorage('localfs') FAIL
>> >> > testVmWithStorage('nfs') FAIL
>> >> > functional.storageTests.StorageTest
>> >> > testCreatePoolErrors OK
>> >> > testStorage('glusterfs', 0) ERROR
>> >> > testStorage('glusterfs', 3) ERROR
>> >> > testStorage('iscsi', 0) SKIP:
>> >> > python-rtslib is not installed.
>> >> > testStorage('iscsi', 3) SKIP:
>> >> > python-rtslib is not installed.
>> >> > testStorage('localfs', 0) FAIL
>> >> > testStorage('localfs', 3) FAIL
>> >> > testStorage('nfs', 0) FAIL
>> >> > testStorage('nfs', 3) FAIL
>> >> > functional.networkTests.NetworkTest
>> >> > testAddVlanedBridgeless ERROR
>> >> > testAddVlanedBridgeless_oneCommand ERROR
>> >> > testAfterNetworkSetupHook ERROR
>> >> > testBeforeNetworkSetupHook ERROR
>> >> > testBondHwAddress(False) ERROR
>> >> > testBondHwAddress(True) ERROR
>> >> > testBrokenNetworkReplacement(False) ERROR
>> >> > testBrokenNetworkReplacement(True) ERROR
>> >> > testDelNetworkBondAccumulation ERROR
>> >> > testDelNetworkWithMTU(False) ERROR
>> >> > testDelNetworkWithMTU(True) ERROR
>> >> > testDelWithoutAdd ERROR
>> >> > testDhclientLeases(4, 'default') ERROR
>> >> > testDhclientLeases(4, 'local') ERROR
>> >> > testDhclientLeases(6, None) ERROR
>> >> > testDhcpReplaceNicWithBridge ERROR
>> >> > testFailWithInvalidBondingName(False) ERROR
>> >> > testFailWithInvalidBondingName(True) ERROR
>> >> > testFailWithInvalidBridgeName ERROR
>> >> > testFailWithInvalidIpConfig ERROR
>> >> > testFailWithInvalidNic(False) ERROR
>> >> > testFailWithInvalidNic(True) ERROR
>> >> > testFailWithInvalidParams(False) ERROR
>> >> > testFailWithInvalidParams(True) ERROR
>> >> > testGetRouteDeviceTo ERROR
>> >> > testHonorBlockingDhcp SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testIpLinkWrapper ERROR
>> >> > testLowerMtuDoesNotOverride FAIL
>> >> > testNoBridgeLeftovers FAIL
>> >> > testReconfigureBrNetWithVanishedPort ERROR
>> >> > testRedefineBondedNetworkIPs ERROR
>> >> > testRemovingBridgeDoesNotLeaveBridge ERROR
>> >> > testReorderBondingOptions(False) ERROR
>> >> > testReorderBondingOptions(True) ERROR
>> >> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
>> >> > testRestoreToBlockingDHCP ERROR
>> >> > testSafeNetworkConfig(False) ERROR
>> >> > testSafeNetworkConfig(True) ERROR
>> >> > testSelectiveRestoreDuringUpgrade ERROR
>> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
>> >> > testSetupNetworkOutboundQos(False) ERROR
>> >> > testSetupNetworkOutboundQos(True) FAIL
>> >> > testSetupNetworksActiveSlave ERROR
>> >> > testSetupNetworksAddBadParams(False) ERROR
>> >> > testSetupNetworksAddBadParams(True) FAIL
>> >> > testSetupNetworksAddBondWithManyVlans(False) ERROR
>> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL
>> >> > testSetupNetworksAddDelBondedNetwork(False) ERROR
>> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL
>> >> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
>> >> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR
>> >> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR
>> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
>> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
>> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
>> >> > testSetupNetworksAddManyVlans(False) ERROR
>> >> > testSetupNetworksAddManyVlans(True) FAIL
>> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
>> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
>> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
>> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
>> >> > testSetupNetworksAddOverExistingBond(False) ERROR
>> >> > testSetupNetworksAddOverExistingBond(True) ERROR
>> >> > testSetupNetworksAddVlan(False) ERROR
>> >> > testSetupNetworksAddVlan(True) FAIL
>> >> > testSetupNetworksConnectivityCheck FAIL
>> >> > testSetupNetworksConvertVlanNetBridgeness ERROR
>> >> > testSetupNetworksDelOneOfBondNets ERROR
>> >> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
>> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
>> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
>> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
>> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
>> >> > testSetupNetworksMtus(False) ERROR
>> >> > testSetupNetworksMtus(True) SKIP:
>> >> > This
>> >> > test fails because the 2 different networks are getting configured
>> with
>> >> > the
>> >> > same MTU. The test should assert that the reported MTUs are equal to
>> the
>> >> > requested ones.
>> >> > testSetupNetworksMultiMTUsOverBond(False) ERROR
>> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL
>> >> > testSetupNetworksMultiMTUsOverNic(False) ERROR
>> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL
>> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
>> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
>> >> > testSetupNetworksNicless FAIL
>> >> > testSetupNetworksNiclessBridgeless ERROR
>> >> > testSetupNetworksOverDhcpIface ERROR
>> >> > testSetupNetworksOverExistingBridge FAIL
>> >> > testSetupNetworksReconfigureBridge FAIL
>> >> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
>> >> > testSetupNetworksRemoveSlavelessBond ERROR
>> >> > testSetupNetworksResizeBond(False) ERROR
>> >> > testSetupNetworksResizeBond(True) ERROR
>> >> > testSetupNetworksStableBond(False) ERROR
>> >> > testSetupNetworksStableBond(True) ERROR
>> >> > testStaticNetworkConfig((4, 6)) FAIL
>> >> > testStaticNetworkConfig((4,)) FAIL
>> >> > testStaticNetworkConfig((6,)) FAIL
>> >> > testStaticSourceRouting(False) ERROR
>> >> > testStaticSourceRouting(True) FAIL
>> >> > testTwiceAdd(False) ERROR
>> >> > testTwiceAdd(True) ERROR
>> >> > testVolatileConfig(False) ERROR
>> >> > testVolatileConfig(True) ERROR
>> >> > test_drop_initial_bond_slaves_ip_config FAIL
>> >> > test_getVdsStats ERROR
>> >> > test_remove_bond_under_network FAIL
>> >> > test_remove_initial_network_nic_ip_config FAIL
>> >> > test_rollback FAIL
>> >> > test_setupNetworks_bond_with_custom_option ERROR
>> >> > test_setupNetworks_on_external_bond ERROR
>> >> > test_setupNetworks_on_external_vlaned_bond ERROR
>> >> > functional.networkTestsOVS.OVSNetworkTest
>> >> > testHonorBlockingDhcp SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testLowerMtuDoesNotOverride OK
>> >> > testNoBridgeLeftovers FAIL
>> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
>> >> > testSetupNetworkOutboundQos(True) OK
>> >> > testSetupNetworksAddBadParams(True) OK
>> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL
>> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL
>> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
>> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
>> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
>> >> > testSetupNetworksAddManyVlans(True) OK
>> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
>> >> > testSetupNetworksAddVlan(True) FAIL
>> >> > testSetupNetworksConnectivityCheck OK
>> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
>> >> > testSetupNetworksMtus(True) SKIP:
>> >> > This
>> >> > test fails because the 2 different networks are getting configured
>> with
>> >> > the
>> >> > same MTU. The test should assert that the reported MTUs are equal to
>> the
>> >> > requested ones.
>> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL
>> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL
>> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
>> >> > testSetupNetworksNicless OK
>> >> > testSetupNetworksOverExistingBridge OK
>> >> > testSetupNetworksReconfigureBridge FAIL
>> >> > testStaticNetworkConfig((4, 6)) FAIL
>> >> > testStaticNetworkConfig((4,)) FAIL
>> >> > testStaticNetworkConfig((6,)) FAIL
>> >> > testStaticSourceRouting(True) FAIL
>> >> > test_drop_initial_bond_slaves_ip_config FAIL
>> >> > test_ovirtmgmtm_to_ovs FAIL
>> >> > test_remove_bond_under_network SKIP:
>> >> > OVS
>> >> > does not support bridgeless networks
>> >> > test_remove_initial_network_nic_ip_config OK
>> >> > test_rollback FAIL
>> >> >
>> >> >
>> >> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <
>> pkliczew at redhat.com>
>> >> > wrote:
>> >> >>
>> >> >> I like the idea but I have the same feelings as Francesco. I think
>> that
>> >> >> we
>> >> >> need to make sure that functional tests for each vertical are stable
>> >> >> before
>> >> >> enabling this process.
>> >> >>
>> >> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri at redhat.com>
>> wrote:
>> >> >>>
>> >> >>> adding also infra team for visibility on the change in CI.
>> >> >>> also inline.
>> >> >>>
>> >> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <
>> fromani at redhat.com>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> ----- Original Message -----
>> >> >>>> > From: "Yaniv Bronheim" <ybronhei at redhat.com>
>> >> >>>> > To: devel at ovirt.org, "Francesco Romani" <fromani at redhat.com>,
>> "Nir
>> >> >>>> > Soffer" <nsoffer at redhat.com>, "Piotr Kliczewski"
>> >> >>>> > <pkliczew at redhat.com>
>> >> >>>> > Cc: "danken" <danken at redhat.com>, "David Caro"
>> >> >>>> > <dcaroest at redhat.com>,
>> >> >>>> > "Eyal Edri" <eedri at redhat.com>
>> >> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>> >> >>>> > Subject: Automation CI for vdsm
>> >> >>>>
>> >> >>>> [...]
>> >> >>>> > We want to allow developers to trigger the script once reviews
>> and
>> >> >>>> > verification are ready (last step before merge). To do so we
>> agreed
>> >> >>>> > to
>> >> >>>> > add
>> >> >>>> > Continues Integration flag for each vdsm patch.
>> >> >>>
>> >> >>>
>> >> >>> This flag will be called 'Workflow' or we can name it otherwise, we
>> >> >>> just
>> >> >>> need to choose what makes sense.
>> >> >>> David/Yaniv - Please correct me if I'm wrong.
>> >> >>>
>> >> >>>>
>> >> >>>> Once this flag will be
>> >> >>>> > signed with +1 it will trigger Jenkins CI to run the
>> check-merged
>> >> >>>> > script
>> >> >>>> > (adding new button to gerrit is not an option - you can image
>> that
>> >> >>>> > flag as
>> >> >>>> > a trigger button), on success Jenkins CI flag will turn to +2.
>> on
>> >> >>>> > fail
>> >> >>>> > we'll get -1 and once new patchset is ready the developer will
>> >> >>>> > remove
>> >> >>>> > the
>> >> >>>> > +1 and add it back to the Continues Integration flag to
>> re-trigger
>> >> >>>> > the
>> >> >>>> > job.
>> >> >>>> >
>> >> >>>> > Please ack the process before we move on with that
>> >> >>>>
>> >> >>>> Sounds good, even though I'm a little scared (just gut feeling, no
>> >> >>>> evidence
>> >> >>>> whatsoever) that this could add even more complexity and
>> fragility to
>> >> >>>> the jenkins
>> >> >>>> fleet.
>> >> >>>>
>> >> >>>> In the long run, when this is reliable, it will help greatly.
>> >> >>>> In the short term, I'm scared because this can lead to false
>> >> >>>> positives
>> >> >>>> and bogus
>> >> >>>> failures.
>> >> >>>>
>> >> >>>> Let me stress I don't have concrete item to share or specific
>> flaws.
>> >> >>>>
>> >> >>>> As action item on me, I will find some time next week to check
>> virt
>> >> >>>> functional tests,
>> >> >>>> to see if they need some fixes, work reliably and so forth
>> >> >>>>
>> >> >>>> > The patch for those scripts still under review and testing -
>> >> >>>> > https://gerrit.ovirt.org/#/c/48268
>> >> >>>>
>> >> >>>> Will review asap.
>> >> >>>>
>> >> >>>> --
>> >> >>>> Francesco Romani
>> >> >>>> RedHat Engineering Virtualization R & D
>> >> >>>> Phone: 8261328
>> >> >>>> IRC: fromani
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Eyal Edri
>> >> >>> Supervisor, RHEV CI
>> >> >>> EMEA ENG Virtualization R&D
>> >> >>> Red Hat Israel
>> >> >>>
>> >> >>> phone: +972-9-7692018
>> >> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Yaniv Bronhaim.
>> >
>> >
>> >
>> >
>> > --
>> > Yaniv Bronhaim.
>> >
>> > _______________________________________________
>> > Devel mailing list
>> > Devel at ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> *Yaniv Bronhaim.*
>
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>
>
--
Eyal Edri
Associate Manager
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20160131/044a6afd/attachment.html>
More information about the Infra
mailing list