[ovirt-devel] Automation CI for vdsm

Eyal Edri eedri at redhat.com
Sun Jan 31 12:15:16 UTC 2016


Adding lago-devel.
Anyone from Lago project can help debug why it is taking so long to setup
Lago (1:17 hours?)

Yaniv, was this run done only on CI, or someone tried to run it locally on
a baremetal server / laptop?

e.

On Tue, Dec 22, 2015 at 2:50 PM, Yaniv Bronheim <ybronhei at redhat.com> wrote:

> OK. so for now the patch [1] skips all storage and OVSNetwork tests
> Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr,
> for the setup :\ the rest for the tests)
>
> The output looks pretty good though :)
>
> [1] https://gerrit.ovirt.org/#/c/48268/
>
> On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phoracek at redhat.com>
> wrote:
>
>> Hi,
>>
>> OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped
>> if the package is not installed) and openvswitch service running (not
>> started automatically). NetworkTest are not able to run together with
>> OVSNetworkTest since we are doing ugly inheritance hacking there. My
>> apologize for network tests difficulties, it will be resolved soon. If
>> you want your life easier, temporary skip OVSNetworkTest suite and it
>> should be OK.
>>
>> Best regards,
>> Petr
>>
>> 2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybronhei at redhat.com>:
>> > exactly - just posted a patch again that sings all fails as broken .
>> > we'll get the report soon and I'll publish it as well. hope the run will
>> > take less time now
>> >
>> > On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>> >>
>> >> Nice, but we cannot enable this until all the tests pass or disabled.
>> >>
>> >> There is no point in broken or flaky functional tests.
>> >>
>> >> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei at redhat.com>
>> >> wrote:
>> >> > So its not stable. It won't block merges and at least give us report
>> >> > after
>> >> > each merge. It takes really long time to run it (because of the tests
>> >> > themselves. Lago things takes maximum 15minutes, but the run last for
>> >> > more
>> >> > than 2hrs right now and I suspect functional/storageTests.py gets
>> stuck)
>> >> >
>> >> > Bellow you can see where we stand (before I added python-rtslib
>> >> > package).
>> >> >
>> >> > Now, I still want to merge the patch
>> https://gerrit.ovirt.org/#/c/48268/
>> >> > -
>> >> > which enables this run after merges, and I still want you to consider
>> >> > the
>> >> > addition of Automation CI flag to our gerrit so that developer will
>> be
>> >> > able
>> >> > to use it as a trigger for the check-merged.sh script run, just to
>> see
>> >> > if
>> >> > their patch fixes\brakes something realted to the functional tests
>> >> >
>> >> >
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ -
>> >> > is
>> >> > an example of how the run looks like. I still work to improve the
>> output
>> >> >
>> >> >
>> >> > Please reply and let me know if the idea around the automation flag
>> is
>> >> > acceptable by you.. and please review the patch for comments and
>> acks.
>> >> > We can ask dcaro to add the flag until Friday, otherwise we'll need
>> to
>> >> > delay
>> >> > this effort after the holiday..
>> >> >
>> >> >
>> >> > functional.sosPluginTests.SosPluginTest
>> >> >     testSosPlugin                                               OK
>> >> > functional.vmRecoveryTests.RecoveryTests
>> >> >     test_vm_recovery                                            FAIL
>> >> > functional.vmQoSTests.VMQosTests
>> >> >     testSmallVMBallooning                                       FAIL
>> >> > functional.virtTests.VirtTest
>> >> >     testComplexVm                                               FAIL
>> >> >     testHeadlessVm                                              OK
>> >> >     testSimpleVm                                                FAIL
>> >> >     testVmDefinitionGraphics('spice')                           FAIL
>> >> >     testVmDefinitionGraphics('vnc')                             OK
>> >> >     testVmDefinitionLegacyGraphics('qxl')                       FAIL
>> >> >     testVmDefinitionLegacyGraphics('vnc')                       OK
>> >> >     testVmDefinitionMultipleGraphics('spice', 'vnc')            FAIL
>> >> >     testVmDefinitionMultipleGraphics('vnc', 'spice')            FAIL
>> >> >     testVmWithCdrom('self')                                     FAIL
>> >> >     testVmWithCdrom('specParams')                               FAIL
>> >> >     testVmWithCdrom('vmPayload')                                FAIL
>> >> >     testVmWithDevice('hotplugDisk')                             FAIL
>> >> >     testVmWithDevice('hotplugNic')                              FAIL
>> >> >     testVmWithDevice('smartcard')                               FAIL
>> >> >     testVmWithDevice('virtioNic')                               FAIL
>> >> >     testVmWithDevice('virtioRng')                               FAIL
>> >> >     testVmWithSla                                               FAIL
>> >> >     testVmWithStorage('iscsi')                                  SKIP:
>> >> > python-rtslib is not installed.
>> >> >     testVmWithStorage('localfs')                                FAIL
>> >> >     testVmWithStorage('nfs')                                    FAIL
>> >> > functional.storageTests.StorageTest
>> >> >     testCreatePoolErrors                                        OK
>> >> >     testStorage('glusterfs', 0)                                 ERROR
>> >> >     testStorage('glusterfs', 3)                                 ERROR
>> >> >     testStorage('iscsi', 0)                                     SKIP:
>> >> > python-rtslib is not installed.
>> >> >     testStorage('iscsi', 3)                                     SKIP:
>> >> > python-rtslib is not installed.
>> >> >     testStorage('localfs', 0)                                   FAIL
>> >> >     testStorage('localfs', 3)                                   FAIL
>> >> >     testStorage('nfs', 0)                                       FAIL
>> >> >     testStorage('nfs', 3)                                       FAIL
>> >> > functional.networkTests.NetworkTest
>> >> >     testAddVlanedBridgeless                                     ERROR
>> >> >     testAddVlanedBridgeless_oneCommand                          ERROR
>> >> >     testAfterNetworkSetupHook                                   ERROR
>> >> >     testBeforeNetworkSetupHook                                  ERROR
>> >> >     testBondHwAddress(False)                                    ERROR
>> >> >     testBondHwAddress(True)                                     ERROR
>> >> >     testBrokenNetworkReplacement(False)                         ERROR
>> >> >     testBrokenNetworkReplacement(True)                          ERROR
>> >> >     testDelNetworkBondAccumulation                              ERROR
>> >> >     testDelNetworkWithMTU(False)                                ERROR
>> >> >     testDelNetworkWithMTU(True)                                 ERROR
>> >> >     testDelWithoutAdd                                           ERROR
>> >> >     testDhclientLeases(4, 'default')                            ERROR
>> >> >     testDhclientLeases(4, 'local')                              ERROR
>> >> >     testDhclientLeases(6, None)                                 ERROR
>> >> >     testDhcpReplaceNicWithBridge                                ERROR
>> >> >     testFailWithInvalidBondingName(False)                       ERROR
>> >> >     testFailWithInvalidBondingName(True)                        ERROR
>> >> >     testFailWithInvalidBridgeName                               ERROR
>> >> >     testFailWithInvalidIpConfig                                 ERROR
>> >> >     testFailWithInvalidNic(False)                               ERROR
>> >> >     testFailWithInvalidNic(True)                                ERROR
>> >> >     testFailWithInvalidParams(False)                            ERROR
>> >> >     testFailWithInvalidParams(True)                             ERROR
>> >> >     testGetRouteDeviceTo                                        ERROR
>> >> >     testHonorBlockingDhcp                                       SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testIpLinkWrapper                                           ERROR
>> >> >     testLowerMtuDoesNotOverride                                 FAIL
>> >> >     testNoBridgeLeftovers                                       FAIL
>> >> >     testReconfigureBrNetWithVanishedPort                        ERROR
>> >> >     testRedefineBondedNetworkIPs                                ERROR
>> >> >     testRemovingBridgeDoesNotLeaveBridge                        ERROR
>> >> >     testReorderBondingOptions(False)                            ERROR
>> >> >     testReorderBondingOptions(True)                             ERROR
>> >> >     testRestoreNetworksOnlyRestoreUnchangedDevices              ERROR
>> >> >     testRestoreToBlockingDHCP                                   ERROR
>> >> >     testSafeNetworkConfig(False)                                ERROR
>> >> >     testSafeNetworkConfig(True)                                 ERROR
>> >> >     testSelectiveRestoreDuringUpgrade                           ERROR
>> >> >     testSelectiveRestoreIgnoresVdsmRegParams                    FAIL
>> >> >     testSetupNetworkOutboundQos(False)                          ERROR
>> >> >     testSetupNetworkOutboundQos(True)                           FAIL
>> >> >     testSetupNetworksActiveSlave                                ERROR
>> >> >     testSetupNetworksAddBadParams(False)                        ERROR
>> >> >     testSetupNetworksAddBadParams(True)                         FAIL
>> >> >     testSetupNetworksAddBondWithManyVlans(False)                ERROR
>> >> >     testSetupNetworksAddBondWithManyVlans(True)                 FAIL
>> >> >     testSetupNetworksAddDelBondedNetwork(False)                 ERROR
>> >> >     testSetupNetworksAddDelBondedNetwork(True)                  FAIL
>> >> >     testSetupNetworksAddDelDhcp(False, (4, 6))                  ERROR
>> >> >     testSetupNetworksAddDelDhcp(False, (4,))                    ERROR
>> >> >     testSetupNetworksAddDelDhcp(False, (6,))                    ERROR
>> >> >     testSetupNetworksAddDelDhcp(True, (4, 6))                   FAIL
>> >> >     testSetupNetworksAddDelDhcp(True, (4,))                     FAIL
>> >> >     testSetupNetworksAddDelDhcp(True, (6,))                     FAIL
>> >> >     testSetupNetworksAddManyVlans(False)                        ERROR
>> >> >     testSetupNetworksAddManyVlans(True)                         FAIL
>> >> >     testSetupNetworksAddNetworkToNicAfterBondBreaking(False)    ERROR
>> >> >     testSetupNetworksAddNetworkToNicAfterBondBreaking(True)     FAIL
>> >> >     testSetupNetworksAddNetworkToNicAfterBondResizing(False)    ERROR
>> >> >     testSetupNetworksAddNetworkToNicAfterBondResizing(True)     ERROR
>> >> >     testSetupNetworksAddOverExistingBond(False)                 ERROR
>> >> >     testSetupNetworksAddOverExistingBond(True)                  ERROR
>> >> >     testSetupNetworksAddVlan(False)                             ERROR
>> >> >     testSetupNetworksAddVlan(True)                              FAIL
>> >> >     testSetupNetworksConnectivityCheck                          FAIL
>> >> >     testSetupNetworksConvertVlanNetBridgeness                   ERROR
>> >> >     testSetupNetworksDelOneOfBondNets                           ERROR
>> >> >     testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured   ERROR
>> >> >     testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
>> >> >     testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
>> >> >     testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testSetupNetworksKeepNetworkOnBondAfterBondResizing(False)  ERROR
>> >> >     testSetupNetworksKeepNetworkOnBondAfterBondResizing(True)   FAIL
>> >> >     testSetupNetworksMtus(False)                                ERROR
>> >> >     testSetupNetworksMtus(True)                                 SKIP:
>> >> > This
>> >> > test fails because the 2 different networks are getting configured
>> with
>> >> > the
>> >> > same MTU. The test should assert that the reported MTUs are equal to
>> the
>> >> > requested ones.
>> >> >     testSetupNetworksMultiMTUsOverBond(False)                   ERROR
>> >> >     testSetupNetworksMultiMTUsOverBond(True)                    FAIL
>> >> >     testSetupNetworksMultiMTUsOverNic(False)                    ERROR
>> >> >     testSetupNetworksMultiMTUsOverNic(True)                     FAIL
>> >> >     testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
>> >> >     testSetupNetworksNetCompatibilityMultipleNetsSameNic(True)  FAIL
>> >> >     testSetupNetworksNicless                                    FAIL
>> >> >     testSetupNetworksNiclessBridgeless                          ERROR
>> >> >     testSetupNetworksOverDhcpIface                              ERROR
>> >> >     testSetupNetworksOverExistingBridge                         FAIL
>> >> >     testSetupNetworksReconfigureBridge                          FAIL
>> >> >     testSetupNetworksRemoveBondWithKilledEnslavedNics           ERROR
>> >> >     testSetupNetworksRemoveSlavelessBond                        ERROR
>> >> >     testSetupNetworksResizeBond(False)                          ERROR
>> >> >     testSetupNetworksResizeBond(True)                           ERROR
>> >> >     testSetupNetworksStableBond(False)                          ERROR
>> >> >     testSetupNetworksStableBond(True)                           ERROR
>> >> >     testStaticNetworkConfig((4, 6))                             FAIL
>> >> >     testStaticNetworkConfig((4,))                               FAIL
>> >> >     testStaticNetworkConfig((6,))                               FAIL
>> >> >     testStaticSourceRouting(False)                              ERROR
>> >> >     testStaticSourceRouting(True)                               FAIL
>> >> >     testTwiceAdd(False)                                         ERROR
>> >> >     testTwiceAdd(True)                                          ERROR
>> >> >     testVolatileConfig(False)                                   ERROR
>> >> >     testVolatileConfig(True)                                    ERROR
>> >> >     test_drop_initial_bond_slaves_ip_config                     FAIL
>> >> >     test_getVdsStats                                            ERROR
>> >> >     test_remove_bond_under_network                              FAIL
>> >> >     test_remove_initial_network_nic_ip_config                   FAIL
>> >> >     test_rollback                                               FAIL
>> >> >     test_setupNetworks_bond_with_custom_option                  ERROR
>> >> >     test_setupNetworks_on_external_bond                         ERROR
>> >> >     test_setupNetworks_on_external_vlaned_bond                  ERROR
>> >> > functional.networkTestsOVS.OVSNetworkTest
>> >> >     testHonorBlockingDhcp                                       SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testLowerMtuDoesNotOverride                                 OK
>> >> >     testNoBridgeLeftovers                                       FAIL
>> >> >     testSelectiveRestoreIgnoresVdsmRegParams                    FAIL
>> >> >     testSetupNetworkOutboundQos(True)                           OK
>> >> >     testSetupNetworksAddBadParams(True)                         OK
>> >> >     testSetupNetworksAddBondWithManyVlans(True)                 FAIL
>> >> >     testSetupNetworksAddDelBondedNetwork(True)                  FAIL
>> >> >     testSetupNetworksAddDelDhcp(True, (4, 6))                   FAIL
>> >> >     testSetupNetworksAddDelDhcp(True, (4,))                     FAIL
>> >> >     testSetupNetworksAddDelDhcp(True, (6,))                     FAIL
>> >> >     testSetupNetworksAddManyVlans(True)                         OK
>> >> >     testSetupNetworksAddNetworkToNicAfterBondBreaking(True)     FAIL
>> >> >     testSetupNetworksAddVlan(True)                              FAIL
>> >> >     testSetupNetworksConnectivityCheck                          OK
>> >> >     testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> >> > Slow
>> >> > tests are disabled
>> >> >     testSetupNetworksKeepNetworkOnBondAfterBondResizing(True)   FAIL
>> >> >     testSetupNetworksMtus(True)                                 SKIP:
>> >> > This
>> >> > test fails because the 2 different networks are getting configured
>> with
>> >> > the
>> >> > same MTU. The test should assert that the reported MTUs are equal to
>> the
>> >> > requested ones.
>> >> >     testSetupNetworksMultiMTUsOverBond(True)                    FAIL
>> >> >     testSetupNetworksMultiMTUsOverNic(True)                     FAIL
>> >> >     testSetupNetworksNetCompatibilityMultipleNetsSameNic(True)  FAIL
>> >> >     testSetupNetworksNicless                                    OK
>> >> >     testSetupNetworksOverExistingBridge                         OK
>> >> >     testSetupNetworksReconfigureBridge                          FAIL
>> >> >     testStaticNetworkConfig((4, 6))                             FAIL
>> >> >     testStaticNetworkConfig((4,))                               FAIL
>> >> >     testStaticNetworkConfig((6,))                               FAIL
>> >> >     testStaticSourceRouting(True)                               FAIL
>> >> >     test_drop_initial_bond_slaves_ip_config                     FAIL
>> >> >     test_ovirtmgmtm_to_ovs                                      FAIL
>> >> >     test_remove_bond_under_network                              SKIP:
>> >> > OVS
>> >> > does not support bridgeless networks
>> >> >     test_remove_initial_network_nic_ip_config                   OK
>> >> >     test_rollback                                               FAIL
>> >> >
>> >> >
>> >> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <
>> pkliczew at redhat.com>
>> >> > wrote:
>> >> >>
>> >> >> I like the idea but I have the same feelings as Francesco. I think
>> that
>> >> >> we
>> >> >> need to make sure that functional tests for each vertical are stable
>> >> >> before
>> >> >> enabling this process.
>> >> >>
>> >> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri at redhat.com>
>> wrote:
>> >> >>>
>> >> >>> adding also infra team for visibility on the change in CI.
>> >> >>> also inline.
>> >> >>>
>> >> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <
>> fromani at redhat.com>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> ----- Original Message -----
>> >> >>>> > From: "Yaniv Bronheim" <ybronhei at redhat.com>
>> >> >>>> > To: devel at ovirt.org, "Francesco Romani" <fromani at redhat.com>,
>> "Nir
>> >> >>>> > Soffer" <nsoffer at redhat.com>, "Piotr Kliczewski"
>> >> >>>> > <pkliczew at redhat.com>
>> >> >>>> > Cc: "danken" <danken at redhat.com>, "David Caro"
>> >> >>>> > <dcaroest at redhat.com>,
>> >> >>>> > "Eyal Edri" <eedri at redhat.com>
>> >> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>> >> >>>> > Subject: Automation CI for vdsm
>> >> >>>>
>> >> >>>> [...]
>> >> >>>> > We want to allow developers to trigger the script once reviews
>> and
>> >> >>>> > verification are ready (last step before merge). To do so we
>> agreed
>> >> >>>> > to
>> >> >>>> > add
>> >> >>>> > Continues Integration flag for each vdsm patch.
>> >> >>>
>> >> >>>
>> >> >>> This flag will be called 'Workflow' or we can name it otherwise, we
>> >> >>> just
>> >> >>> need to choose what makes sense.
>> >> >>> David/Yaniv - Please correct me if I'm wrong.
>> >> >>>
>> >> >>>>
>> >> >>>> Once this flag will be
>> >> >>>> > signed with +1 it will trigger Jenkins CI to run the
>> check-merged
>> >> >>>> > script
>> >> >>>> > (adding new button to gerrit is not an option - you can image
>> that
>> >> >>>> > flag as
>> >> >>>> > a trigger button), on success Jenkins CI flag will turn to +2.
>> on
>> >> >>>> > fail
>> >> >>>> > we'll get -1 and once new patchset is ready the developer will
>> >> >>>> > remove
>> >> >>>> > the
>> >> >>>> > +1 and add it back to the Continues Integration flag to
>> re-trigger
>> >> >>>> > the
>> >> >>>> > job.
>> >> >>>> >
>> >> >>>> > Please ack the process before we move on with that
>> >> >>>>
>> >> >>>> Sounds good, even though I'm a little scared (just gut feeling, no
>> >> >>>> evidence
>> >> >>>> whatsoever) that this could add even more complexity and
>> fragility to
>> >> >>>> the jenkins
>> >> >>>> fleet.
>> >> >>>>
>> >> >>>> In the long run, when this is reliable, it will help greatly.
>> >> >>>> In the short term, I'm scared because this can lead to false
>> >> >>>> positives
>> >> >>>> and bogus
>> >> >>>> failures.
>> >> >>>>
>> >> >>>> Let me stress I don't have concrete item to share or specific
>> flaws.
>> >> >>>>
>> >> >>>> As action item on me, I will find some time next week to check
>> virt
>> >> >>>> functional tests,
>> >> >>>> to see if they need some fixes, work reliably and so forth
>> >> >>>>
>> >> >>>> > The patch for those scripts still under review and testing -
>> >> >>>> > https://gerrit.ovirt.org/#/c/48268
>> >> >>>>
>> >> >>>> Will review asap.
>> >> >>>>
>> >> >>>> --
>> >> >>>> Francesco Romani
>> >> >>>> RedHat Engineering Virtualization R & D
>> >> >>>> Phone: 8261328
>> >> >>>> IRC: fromani
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Eyal Edri
>> >> >>> Supervisor, RHEV CI
>> >> >>> EMEA ENG Virtualization R&D
>> >> >>> Red Hat Israel
>> >> >>>
>> >> >>> phone: +972-9-7692018
>> >> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Yaniv Bronhaim.
>> >
>> >
>> >
>> >
>> > --
>> > Yaniv Bronhaim.
>> >
>> > _______________________________________________
>> > Devel mailing list
>> > Devel at ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> *Yaniv Bronhaim.*
>
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>
>


-- 
Eyal Edri
Associate Manager
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20160131/044a6afd/attachment-0001.html>


More information about the Devel mailing list