11:46 a.m.
Hi all,
We want to run functional tests as part of Vdsm CI for each patch before
merge. Therefore we need to declare how to automate this process without
overloading our jenkins machines.
The functional tests will run using lago (https://github.com/ovirt/lago) -
It will initiate multiply vms, install vdsm and manipulate it by nosetests
or other procedures such as upgrade, removal and so on.
Currently standard CI provides check-patch and check-merged scripts (
http://ovirt-infra-docs.readthedocs.org/en/latest/CI/Build_and_test_stand...)
the problem with check-merged is that it will run after merge which doesn't
help if something fails.
We want to allow developers to trigger the script once reviews and
verification are ready (last step before merge). To do so we agreed to add
Continues Integration flag for each vdsm patch. Once this flag will be
signed with +1 it will trigger Jenkins CI to run the check-merged script
(adding new button to gerrit is not an option - you can image that flag as
a trigger button), on success Jenkins CI flag will turn to +2. on fail
we'll get -1 and once new patchset is ready the developer will remove the
+1 and add it back to the Continues Integration flag to re-trigger the job.
Please ack the process before we move on with that
The patch for those scripts still under review and testing -
https://gerrit.ovirt.org/#/c/48268
Thanks
--
*Yaniv Bronhaim.*
8:19 a.m.
----- Original Message -----
From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
To: devel(a)ovirt.org, "Francesco Romani" <fromani(a)redhat.com>, "Nir
Soffer" <nsoffer(a)redhat.com>, "Piotr Kliczewski"
<pkliczew(a)redhat.com>
Cc: "danken" <danken(a)redhat.com>, "David Caro"
<dcaroest(a)redhat.com>, "Eyal Edri" <eedri(a)redhat.com>
Sent: Thursday, December 10, 2015 6:46:37 PM
Subject: Automation CI for vdsm
[...]
We want to allow developers to trigger the script once reviews and
verification are ready (last step before merge). To do so we agreed to add
Continues Integration flag for each vdsm patch. Once this flag will be
signed with +1 it will trigger Jenkins CI to run the check-merged script
(adding new button to gerrit is not an option - you can image that flag as
a trigger button), on success Jenkins CI flag will turn to +2. on fail
we'll get -1 and once new patchset is ready the developer will remove the
+1 and add it back to the Continues Integration flag to re-trigger the job.
Please ack the process before we move on with that
Sounds good, even though I'm a little scared (just gut feeling, no evidence
whatsoever) that this could add even more complexity and fragility to the jenkins
fleet.
In the long run, when this is reliable, it will help greatly.
In the short term, I'm scared because this can lead to false positives and bogus
failures.
Let me stress I don't have concrete item to share or specific flaws.
As action item on me, I will find some time next week to check virt functional tests,
to see if they need some fixes, work reliably and so forth
The patch for those scripts still under review and testing -
https://gerrit.ovirt.org/#/c/48268
Will review asap.
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
1:34 a.m.
adding also infra team for visibility on the change in CI.
also inline.
On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <fromani(a)redhat.com>
wrote:
----- Original Message -----
> From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
> To: devel(a)ovirt.org, "Francesco Romani" <fromani(a)redhat.com>,
"Nir
Soffer" <nsoffer(a)redhat.com>, "Piotr Kliczewski"
> <pkliczew(a)redhat.com>
> Cc: "danken" <danken(a)redhat.com>, "David Caro"
<dcaroest(a)redhat.com>,
"Eyal Edri" <eedri(a)redhat.com>
> Sent: Thursday, December 10, 2015 6:46:37 PM
> Subject: Automation CI for vdsm
[...]
> We want to allow developers to trigger the script once reviews and
> verification are ready (last step before merge). To do so we agreed to
add
> Continues Integration flag for each vdsm patch.
This flag will be called 'Workflow' or we can name it otherwise, we just
need to choose what makes sense.
David/Yaniv - Please correct me if I'm wrong.
Once this flag will be
> signed with +1 it will trigger Jenkins CI to run the check-merged script
> (adding new button to gerrit is not an option - you can image that flag
as
> a trigger button), on success Jenkins CI flag will turn to +2. on fail
> we'll get -1 and once new patchset is ready the developer will remove the
> +1 and add it back to the Continues Integration flag to re-trigger the
job.
>
> Please ack the process before we move on with that
Sounds good, even though I'm a little scared (just gut feeling, no evidence
whatsoever) that this could add even more complexity and fragility to the
jenkins
fleet.
In the long run, when this is reliable, it will help greatly.
In the short term, I'm scared because this can lead to false positives and
bogus
failures.
Let me stress I don't have concrete item to share or specific flaws.
As action item on me, I will find some time next week to check virt
functional tests,
to see if they need some fixes, work reliably and so forth
> The patch for those scripts still under review and testing -
> https://gerrit.ovirt.org/#/c/48268
Will review asap.
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
--
Eyal Edri
Supervisor, RHEV CI
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
2:07 p.m.
I like the idea but I have the same feelings as Francesco. I think that we
need to make sure that functional tests for each vertical are stable before
enabling this process.
On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com> wrote:
adding also infra team for visibility on the change in CI.
also inline.
On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <fromani(a)redhat.com>
wrote:
> ----- Original Message -----
> > From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
> > To: devel(a)ovirt.org, "Francesco Romani" <fromani(a)redhat.com>,
"Nir
> Soffer" <nsoffer(a)redhat.com>, "Piotr Kliczewski"
> > <pkliczew(a)redhat.com>
> > Cc: "danken" <danken(a)redhat.com>, "David Caro"
<dcaroest(a)redhat.com>,
> "Eyal Edri" <eedri(a)redhat.com>
> > Sent: Thursday, December 10, 2015 6:46:37 PM
> > Subject: Automation CI for vdsm
>
> [...]
> > We want to allow developers to trigger the script once reviews and
> > verification are ready (last step before merge). To do so we agreed to
> add
> > Continues Integration flag for each vdsm patch.
This flag will be called 'Workflow' or we can name it otherwise, we just
need to choose what makes sense.
David/Yaniv - Please correct me if I'm wrong.
> Once this flag will be
> > signed with +1 it will trigger Jenkins CI to run the check-merged script
> > (adding new button to gerrit is not an option - you can image that flag
> as
> > a trigger button), on success Jenkins CI flag will turn to +2. on fail
> > we'll get -1 and once new patchset is ready the developer will remove
> the
> > +1 and add it back to the Continues Integration flag to re-trigger the
> job.
> >
> > Please ack the process before we move on with that
>
> Sounds good, even though I'm a little scared (just gut feeling, no
> evidence
> whatsoever) that this could add even more complexity and fragility to the
> jenkins
> fleet.
>
> In the long run, when this is reliable, it will help greatly.
> In the short term, I'm scared because this can lead to false positives
> and bogus
> failures.
>
> Let me stress I don't have concrete item to share or specific flaws.
>
> As action item on me, I will find some time next week to check virt
> functional tests,
> to see if they need some fixes, work reliably and so forth
>
> > The patch for those scripts still under review and testing -
> > https://gerrit.ovirt.org/#/c/48268
>
> Will review asap.
>
> --
> Francesco Romani
> RedHat Engineering Virtualization R & D
> Phone: 8261328
> IRC: fromani
>
--
Eyal Edri
Supervisor, RHEV CI
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
9:34 a.m.
So its not stable. It won't block merges and at least give us report after
each merge. It takes really long time to run it (because of the tests
themselves. Lago things takes maximum 15minutes, but the run last for more
than 2hrs right now and I suspect functional/storageTests.py gets stuck)
Bellow you can see where we stand (before I added python-rtslib package).
Now, I still want to merge the patch https://gerrit.ovirt.org/#/c/48268/ -
which enables this run after merges, and I still want you to consider the
addition of Automation CI flag to our gerrit so that developer will be able
to use it as a trigger for the check-merged.sh script run, just to see if
their patch fixes\brakes something realted to the functional tests
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ - is
an example of how the run looks like. I still work to improve the output
Please reply and let me know if the idea around the automation flag is
acceptable by you.. and please review the patch for comments and acks.
We can ask dcaro to add the flag until Friday, otherwise we'll need to
delay this effort after the holiday..
functional.sosPluginTests.SosPluginTest
testSosPlugin OK
functional.vmRecoveryTests.RecoveryTests
test_vm_recovery FAIL
functional.vmQoSTests.VMQosTests
testSmallVMBallooning FAIL
functional.virtTests.VirtTest
testComplexVm FAIL
testHeadlessVm OK
testSimpleVm FAIL
testVmDefinitionGraphics('spice') FAIL
testVmDefinitionGraphics('vnc') OK
testVmDefinitionLegacyGraphics('qxl') FAIL
testVmDefinitionLegacyGraphics('vnc') OK
testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL
testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL
testVmWithCdrom('self') FAIL
testVmWithCdrom('specParams') FAIL
testVmWithCdrom('vmPayload') FAIL
testVmWithDevice('hotplugDisk') FAIL
testVmWithDevice('hotplugNic') FAIL
testVmWithDevice('smartcard') FAIL
testVmWithDevice('virtioNic') FAIL
testVmWithDevice('virtioRng') FAIL
testVmWithSla FAIL
testVmWithStorage('iscsi') SKIP:
python-rtslib is not installed.
testVmWithStorage('localfs') FAIL
testVmWithStorage('nfs') FAIL
functional.storageTests.StorageTest
testCreatePoolErrors OK
testStorage('glusterfs', 0) ERROR
testStorage('glusterfs', 3) ERROR
testStorage('iscsi', 0) SKIP:
python-rtslib is not installed.
testStorage('iscsi', 3) SKIP:
python-rtslib is not installed.
testStorage('localfs', 0) FAIL
testStorage('localfs', 3) FAIL
testStorage('nfs', 0) FAIL
testStorage('nfs', 3) FAIL
functional.networkTests.NetworkTest
testAddVlanedBridgeless ERROR
testAddVlanedBridgeless_oneCommand ERROR
testAfterNetworkSetupHook ERROR
testBeforeNetworkSetupHook ERROR
testBondHwAddress(False) ERROR
testBondHwAddress(True) ERROR
testBrokenNetworkReplacement(False) ERROR
testBrokenNetworkReplacement(True) ERROR
testDelNetworkBondAccumulation ERROR
testDelNetworkWithMTU(False) ERROR
testDelNetworkWithMTU(True) ERROR
testDelWithoutAdd ERROR
testDhclientLeases(4, 'default') ERROR
testDhclientLeases(4, 'local') ERROR
testDhclientLeases(6, None) ERROR
testDhcpReplaceNicWithBridge ERROR
testFailWithInvalidBondingName(False) ERROR
testFailWithInvalidBondingName(True) ERROR
testFailWithInvalidBridgeName ERROR
testFailWithInvalidIpConfig ERROR
testFailWithInvalidNic(False) ERROR
testFailWithInvalidNic(True) ERROR
testFailWithInvalidParams(False) ERROR
testFailWithInvalidParams(True) ERROR
testGetRouteDeviceTo ERROR
testHonorBlockingDhcp SKIP: Slow
tests are disabled
testIpLinkWrapper ERROR
testLowerMtuDoesNotOverride FAIL
testNoBridgeLeftovers FAIL
testReconfigureBrNetWithVanishedPort ERROR
testRedefineBondedNetworkIPs ERROR
testRemovingBridgeDoesNotLeaveBridge ERROR
testReorderBondingOptions(False) ERROR
testReorderBondingOptions(True) ERROR
testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
testRestoreToBlockingDHCP ERROR
testSafeNetworkConfig(False) ERROR
testSafeNetworkConfig(True) ERROR
testSelectiveRestoreDuringUpgrade ERROR
testSelectiveRestoreIgnoresVdsmRegParams FAIL
testSetupNetworkOutboundQos(False) ERROR
testSetupNetworkOutboundQos(True) FAIL
testSetupNetworksActiveSlave ERROR
testSetupNetworksAddBadParams(False) ERROR
testSetupNetworksAddBadParams(True) FAIL
testSetupNetworksAddBondWithManyVlans(False) ERROR
testSetupNetworksAddBondWithManyVlans(True) FAIL
testSetupNetworksAddDelBondedNetwork(False) ERROR
testSetupNetworksAddDelBondedNetwork(True) FAIL
testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
testSetupNetworksAddDelDhcp(False, (4,)) ERROR
testSetupNetworksAddDelDhcp(False, (6,)) ERROR
testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
testSetupNetworksAddDelDhcp(True, (4,)) FAIL
testSetupNetworksAddDelDhcp(True, (6,)) FAIL
testSetupNetworksAddManyVlans(False) ERROR
testSetupNetworksAddManyVlans(True) FAIL
testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
testSetupNetworksAddOverExistingBond(False) ERROR
testSetupNetworksAddOverExistingBond(True) ERROR
testSetupNetworksAddVlan(False) ERROR
testSetupNetworksAddVlan(True) FAIL
testSetupNetworksConnectivityCheck FAIL
testSetupNetworksConvertVlanNetBridgeness ERROR
testSetupNetworksDelOneOfBondNets ERROR
testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
testSetupNetworksMtus(False) ERROR
testSetupNetworksMtus(True) SKIP: This
test fails because the 2 different networks are getting configured with the
same MTU. The test should assert that the reported MTUs are equal to the
requested ones.
testSetupNetworksMultiMTUsOverBond(False) ERROR
testSetupNetworksMultiMTUsOverBond(True) FAIL
testSetupNetworksMultiMTUsOverNic(False) ERROR
testSetupNetworksMultiMTUsOverNic(True) FAIL
testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
testSetupNetworksNicless FAIL
testSetupNetworksNiclessBridgeless ERROR
testSetupNetworksOverDhcpIface ERROR
testSetupNetworksOverExistingBridge FAIL
testSetupNetworksReconfigureBridge FAIL
testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
testSetupNetworksRemoveSlavelessBond ERROR
testSetupNetworksResizeBond(False) ERROR
testSetupNetworksResizeBond(True) ERROR
testSetupNetworksStableBond(False) ERROR
testSetupNetworksStableBond(True) ERROR
testStaticNetworkConfig((4, 6)) FAIL
testStaticNetworkConfig((4,)) FAIL
testStaticNetworkConfig((6,)) FAIL
testStaticSourceRouting(False) ERROR
testStaticSourceRouting(True) FAIL
testTwiceAdd(False) ERROR
testTwiceAdd(True) ERROR
testVolatileConfig(False) ERROR
testVolatileConfig(True) ERROR
test_drop_initial_bond_slaves_ip_config FAIL
test_getVdsStats ERROR
test_remove_bond_under_network FAIL
test_remove_initial_network_nic_ip_config FAIL
test_rollback FAIL
test_setupNetworks_bond_with_custom_option ERROR
test_setupNetworks_on_external_bond ERROR
test_setupNetworks_on_external_vlaned_bond ERROR
functional.networkTestsOVS.OVSNetworkTest
testHonorBlockingDhcp SKIP: Slow
tests are disabled
testLowerMtuDoesNotOverride OK
testNoBridgeLeftovers FAIL
testSelectiveRestoreIgnoresVdsmRegParams FAIL
testSetupNetworkOutboundQos(True) OK
testSetupNetworksAddBadParams(True) OK
testSetupNetworksAddBondWithManyVlans(True) FAIL
testSetupNetworksAddDelBondedNetwork(True) FAIL
testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
testSetupNetworksAddDelDhcp(True, (4,)) FAIL
testSetupNetworksAddDelDhcp(True, (6,)) FAIL
testSetupNetworksAddManyVlans(True) OK
testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
testSetupNetworksAddVlan(True) FAIL
testSetupNetworksConnectivityCheck OK
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
testSetupNetworksMtus(True) SKIP: This
test fails because the 2 different networks are getting configured with the
same MTU. The test should assert that the reported MTUs are equal to the
requested ones.
testSetupNetworksMultiMTUsOverBond(True) FAIL
testSetupNetworksMultiMTUsOverNic(True) FAIL
testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
testSetupNetworksNicless OK
testSetupNetworksOverExistingBridge OK
testSetupNetworksReconfigureBridge FAIL
testStaticNetworkConfig((4, 6)) FAIL
testStaticNetworkConfig((4,)) FAIL
testStaticNetworkConfig((6,)) FAIL
testStaticSourceRouting(True) FAIL
test_drop_initial_bond_slaves_ip_config FAIL
test_ovirtmgmtm_to_ovs FAIL
test_remove_bond_under_network SKIP: OVS
does not support bridgeless networks
test_remove_initial_network_nic_ip_config OK
test_rollback FAIL
On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <pkliczew(a)redhat.com>
wrote:
I like the idea but I have the same feelings as Francesco. I think
that we
need to make sure that functional tests for each vertical are stable before
enabling this process.
On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com> wrote:
> adding also infra team for visibility on the change in CI.
> also inline.
>
> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <fromani(a)redhat.com>
> wrote:
>
>> ----- Original Message -----
>> > From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>, "Nir
>> Soffer" <nsoffer(a)redhat.com>, "Piotr Kliczewski"
>> > <pkliczew(a)redhat.com>
>> > Cc: "danken" <danken(a)redhat.com>, "David Caro"
<dcaroest(a)redhat.com>,
>> "Eyal Edri" <eedri(a)redhat.com>
>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>> > Subject: Automation CI for vdsm
>>
>> [...]
>> > We want to allow developers to trigger the script once reviews and
>> > verification are ready (last step before merge). To do so we agreed to
>> add
>> > Continues Integration flag for each vdsm patch.
>
>
> This flag will be called 'Workflow' or we can name it otherwise, we just
> need to choose what makes sense.
> David/Yaniv - Please correct me if I'm wrong.
>
>
>> Once this flag will be
>> > signed with +1 it will trigger Jenkins CI to run the check-merged
>> script
>> > (adding new button to gerrit is not an option - you can image that
>> flag as
>> > a trigger button), on success Jenkins CI flag will turn to +2. on fail
>> > we'll get -1 and once new patchset is ready the developer will remove
>> the
>> > +1 and add it back to the Continues Integration flag to re-trigger the
>> job.
>> >
>> > Please ack the process before we move on with that
>>
>> Sounds good, even though I'm a little scared (just gut feeling, no
>> evidence
>> whatsoever) that this could add even more complexity and fragility to
>> the jenkins
>> fleet.
>>
>> In the long run, when this is reliable, it will help greatly.
>> In the short term, I'm scared because this can lead to false positives
>> and bogus
>> failures.
>>
>> Let me stress I don't have concrete item to share or specific flaws.
>>
>> As action item on me, I will find some time next week to check virt
>> functional tests,
>> to see if they need some fixes, work reliably and so forth
>>
>> > The patch for those scripts still under review and testing -
>> > https://gerrit.ovirt.org/#/c/48268
>>
>> Will review asap.
>>
>> --
>> Francesco Romani
>> RedHat Engineering Virtualization R & D
>> Phone: 8261328
>> IRC: fromani
>>
>
>
>
> --
> Eyal Edri
> Supervisor, RHEV CI
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>
--
*Yaniv Bronhaim.*
10:19 a.m.
Nice, but we cannot enable this until all the tests pass or disabled.
There is no point in broken or flaky functional tests.
On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei(a)redhat.com> wrote:
So its not stable. It won't block merges and at least give us
report after
each merge. It takes really long time to run it (because of the tests
themselves. Lago things takes maximum 15minutes, but the run last for more
than 2hrs right now and I suspect functional/storageTests.py gets stuck)
Bellow you can see where we stand (before I added python-rtslib package).
Now, I still want to merge the patch https://gerrit.ovirt.org/#/c/48268/ -
which enables this run after merges, and I still want you to consider the
addition of Automation CI flag to our gerrit so that developer will be able
to use it as a trigger for the check-merged.sh script run, just to see if
their patch fixes\brakes something realted to the functional tests
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ - is
an example of how the run looks like. I still work to improve the output
Please reply and let me know if the idea around the automation flag is
acceptable by you.. and please review the patch for comments and acks.
We can ask dcaro to add the flag until Friday, otherwise we'll need to delay
this effort after the holiday..
functional.sosPluginTests.SosPluginTest
testSosPlugin OK
functional.vmRecoveryTests.RecoveryTests
test_vm_recovery FAIL
functional.vmQoSTests.VMQosTests
testSmallVMBallooning FAIL
functional.virtTests.VirtTest
testComplexVm FAIL
testHeadlessVm OK
testSimpleVm FAIL
testVmDefinitionGraphics('spice') FAIL
testVmDefinitionGraphics('vnc') OK
testVmDefinitionLegacyGraphics('qxl') FAIL
testVmDefinitionLegacyGraphics('vnc') OK
testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL
testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL
testVmWithCdrom('self') FAIL
testVmWithCdrom('specParams') FAIL
testVmWithCdrom('vmPayload') FAIL
testVmWithDevice('hotplugDisk') FAIL
testVmWithDevice('hotplugNic') FAIL
testVmWithDevice('smartcard') FAIL
testVmWithDevice('virtioNic') FAIL
testVmWithDevice('virtioRng') FAIL
testVmWithSla FAIL
testVmWithStorage('iscsi') SKIP:
python-rtslib is not installed.
testVmWithStorage('localfs') FAIL
testVmWithStorage('nfs') FAIL
functional.storageTests.StorageTest
testCreatePoolErrors OK
testStorage('glusterfs', 0) ERROR
testStorage('glusterfs', 3) ERROR
testStorage('iscsi', 0) SKIP:
python-rtslib is not installed.
testStorage('iscsi', 3) SKIP:
python-rtslib is not installed.
testStorage('localfs', 0) FAIL
testStorage('localfs', 3) FAIL
testStorage('nfs', 0) FAIL
testStorage('nfs', 3) FAIL
functional.networkTests.NetworkTest
testAddVlanedBridgeless ERROR
testAddVlanedBridgeless_oneCommand ERROR
testAfterNetworkSetupHook ERROR
testBeforeNetworkSetupHook ERROR
testBondHwAddress(False) ERROR
testBondHwAddress(True) ERROR
testBrokenNetworkReplacement(False) ERROR
testBrokenNetworkReplacement(True) ERROR
testDelNetworkBondAccumulation ERROR
testDelNetworkWithMTU(False) ERROR
testDelNetworkWithMTU(True) ERROR
testDelWithoutAdd ERROR
testDhclientLeases(4, 'default') ERROR
testDhclientLeases(4, 'local') ERROR
testDhclientLeases(6, None) ERROR
testDhcpReplaceNicWithBridge ERROR
testFailWithInvalidBondingName(False) ERROR
testFailWithInvalidBondingName(True) ERROR
testFailWithInvalidBridgeName ERROR
testFailWithInvalidIpConfig ERROR
testFailWithInvalidNic(False) ERROR
testFailWithInvalidNic(True) ERROR
testFailWithInvalidParams(False) ERROR
testFailWithInvalidParams(True) ERROR
testGetRouteDeviceTo ERROR
testHonorBlockingDhcp SKIP: Slow
tests are disabled
testIpLinkWrapper ERROR
testLowerMtuDoesNotOverride FAIL
testNoBridgeLeftovers FAIL
testReconfigureBrNetWithVanishedPort ERROR
testRedefineBondedNetworkIPs ERROR
testRemovingBridgeDoesNotLeaveBridge ERROR
testReorderBondingOptions(False) ERROR
testReorderBondingOptions(True) ERROR
testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
testRestoreToBlockingDHCP ERROR
testSafeNetworkConfig(False) ERROR
testSafeNetworkConfig(True) ERROR
testSelectiveRestoreDuringUpgrade ERROR
testSelectiveRestoreIgnoresVdsmRegParams FAIL
testSetupNetworkOutboundQos(False) ERROR
testSetupNetworkOutboundQos(True) FAIL
testSetupNetworksActiveSlave ERROR
testSetupNetworksAddBadParams(False) ERROR
testSetupNetworksAddBadParams(True) FAIL
testSetupNetworksAddBondWithManyVlans(False) ERROR
testSetupNetworksAddBondWithManyVlans(True) FAIL
testSetupNetworksAddDelBondedNetwork(False) ERROR
testSetupNetworksAddDelBondedNetwork(True) FAIL
testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
testSetupNetworksAddDelDhcp(False, (4,)) ERROR
testSetupNetworksAddDelDhcp(False, (6,)) ERROR
testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
testSetupNetworksAddDelDhcp(True, (4,)) FAIL
testSetupNetworksAddDelDhcp(True, (6,)) FAIL
testSetupNetworksAddManyVlans(False) ERROR
testSetupNetworksAddManyVlans(True) FAIL
testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
testSetupNetworksAddOverExistingBond(False) ERROR
testSetupNetworksAddOverExistingBond(True) ERROR
testSetupNetworksAddVlan(False) ERROR
testSetupNetworksAddVlan(True) FAIL
testSetupNetworksConnectivityCheck FAIL
testSetupNetworksConvertVlanNetBridgeness ERROR
testSetupNetworksDelOneOfBondNets ERROR
testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
testSetupNetworksMtus(False) ERROR
testSetupNetworksMtus(True) SKIP: This
test fails because the 2 different networks are getting configured with the
same MTU. The test should assert that the reported MTUs are equal to the
requested ones.
testSetupNetworksMultiMTUsOverBond(False) ERROR
testSetupNetworksMultiMTUsOverBond(True) FAIL
testSetupNetworksMultiMTUsOverNic(False) ERROR
testSetupNetworksMultiMTUsOverNic(True) FAIL
testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
testSetupNetworksNicless FAIL
testSetupNetworksNiclessBridgeless ERROR
testSetupNetworksOverDhcpIface ERROR
testSetupNetworksOverExistingBridge FAIL
testSetupNetworksReconfigureBridge FAIL
testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
testSetupNetworksRemoveSlavelessBond ERROR
testSetupNetworksResizeBond(False) ERROR
testSetupNetworksResizeBond(True) ERROR
testSetupNetworksStableBond(False) ERROR
testSetupNetworksStableBond(True) ERROR
testStaticNetworkConfig((4, 6)) FAIL
testStaticNetworkConfig((4,)) FAIL
testStaticNetworkConfig((6,)) FAIL
testStaticSourceRouting(False) ERROR
testStaticSourceRouting(True) FAIL
testTwiceAdd(False) ERROR
testTwiceAdd(True) ERROR
testVolatileConfig(False) ERROR
testVolatileConfig(True) ERROR
test_drop_initial_bond_slaves_ip_config FAIL
test_getVdsStats ERROR
test_remove_bond_under_network FAIL
test_remove_initial_network_nic_ip_config FAIL
test_rollback FAIL
test_setupNetworks_bond_with_custom_option ERROR
test_setupNetworks_on_external_bond ERROR
test_setupNetworks_on_external_vlaned_bond ERROR
functional.networkTestsOVS.OVSNetworkTest
testHonorBlockingDhcp SKIP: Slow
tests are disabled
testLowerMtuDoesNotOverride OK
testNoBridgeLeftovers FAIL
testSelectiveRestoreIgnoresVdsmRegParams FAIL
testSetupNetworkOutboundQos(True) OK
testSetupNetworksAddBadParams(True) OK
testSetupNetworksAddBondWithManyVlans(True) FAIL
testSetupNetworksAddDelBondedNetwork(True) FAIL
testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
testSetupNetworksAddDelDhcp(True, (4,)) FAIL
testSetupNetworksAddDelDhcp(True, (6,)) FAIL
testSetupNetworksAddManyVlans(True) OK
testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
testSetupNetworksAddVlan(True) FAIL
testSetupNetworksConnectivityCheck OK
testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: Slow
tests are disabled
testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
testSetupNetworksMtus(True) SKIP: This
test fails because the 2 different networks are getting configured with the
same MTU. The test should assert that the reported MTUs are equal to the
requested ones.
testSetupNetworksMultiMTUsOverBond(True) FAIL
testSetupNetworksMultiMTUsOverNic(True) FAIL
testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
testSetupNetworksNicless OK
testSetupNetworksOverExistingBridge OK
testSetupNetworksReconfigureBridge FAIL
testStaticNetworkConfig((4, 6)) FAIL
testStaticNetworkConfig((4,)) FAIL
testStaticNetworkConfig((6,)) FAIL
testStaticSourceRouting(True) FAIL
test_drop_initial_bond_slaves_ip_config FAIL
test_ovirtmgmtm_to_ovs FAIL
test_remove_bond_under_network SKIP: OVS
does not support bridgeless networks
test_remove_initial_network_nic_ip_config OK
test_rollback FAIL
On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <pkliczew(a)redhat.com>
wrote:
>
> I like the idea but I have the same feelings as Francesco. I think that we
> need to make sure that functional tests for each vertical are stable before
> enabling this process.
>
> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com> wrote:
>>
>> adding also infra team for visibility on the change in CI.
>> also inline.
>>
>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <fromani(a)redhat.com>
>> wrote:
>>>
>>> ----- Original Message -----
>>> > From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
>>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>, "Nir
>>> > Soffer" <nsoffer(a)redhat.com>, "Piotr Kliczewski"
>>> > <pkliczew(a)redhat.com>
>>> > Cc: "danken" <danken(a)redhat.com>, "David Caro"
<dcaroest(a)redhat.com>,
>>> > "Eyal Edri" <eedri(a)redhat.com>
>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>>> > Subject: Automation CI for vdsm
>>>
>>> [...]
>>> > We want to allow developers to trigger the script once reviews and
>>> > verification are ready (last step before merge). To do so we agreed to
>>> > add
>>> > Continues Integration flag for each vdsm patch.
>>
>>
>> This flag will be called 'Workflow' or we can name it otherwise, we just
>> need to choose what makes sense.
>> David/Yaniv - Please correct me if I'm wrong.
>>
>>>
>>> Once this flag will be
>>> > signed with +1 it will trigger Jenkins CI to run the check-merged
>>> > script
>>> > (adding new button to gerrit is not an option - you can image that
>>> > flag as
>>> > a trigger button), on success Jenkins CI flag will turn to +2. on fail
>>> > we'll get -1 and once new patchset is ready the developer will
remove
>>> > the
>>> > +1 and add it back to the Continues Integration flag to re-trigger the
>>> > job.
>>> >
>>> > Please ack the process before we move on with that
>>>
>>> Sounds good, even though I'm a little scared (just gut feeling, no
>>> evidence
>>> whatsoever) that this could add even more complexity and fragility to
>>> the jenkins
>>> fleet.
>>>
>>> In the long run, when this is reliable, it will help greatly.
>>> In the short term, I'm scared because this can lead to false positives
>>> and bogus
>>> failures.
>>>
>>> Let me stress I don't have concrete item to share or specific flaws.
>>>
>>> As action item on me, I will find some time next week to check virt
>>> functional tests,
>>> to see if they need some fixes, work reliably and so forth
>>>
>>> > The patch for those scripts still under review and testing -
>>> > https://gerrit.ovirt.org/#/c/48268
>>>
>>> Will review asap.
>>>
>>> --
>>> Francesco Romani
>>> RedHat Engineering Virtualization R & D
>>> Phone: 8261328
>>> IRC: fromani
>>
>>
>>
>>
>> --
>> Eyal Edri
>> Supervisor, RHEV CI
>> EMEA ENG Virtualization R&D
>> Red Hat Israel
>>
>> phone: +972-9-7692018
>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>
>
--
Yaniv Bronhaim.
10:25 a.m.
exactly - just posted a patch again that sings all fails as broken .
we'll get the report soon and I'll publish it as well. hope the run will
take less time now
On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
Nice, but we cannot enable this until all the tests pass or
disabled.
There is no point in broken or flaky functional tests.
On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei(a)redhat.com>
wrote:
> So its not stable. It won't block merges and at least give us report
after
> each merge. It takes really long time to run it (because of the tests
> themselves. Lago things takes maximum 15minutes, but the run last for
more
> than 2hrs right now and I suspect functional/storageTests.py gets stuck)
>
> Bellow you can see where we stand (before I added python-rtslib package).
>
> Now, I still want to merge the patch https://gerrit.ovirt.org/#/c/48268/
-
> which enables this run after merges, and I still want you to consider the
> addition of Automation CI flag to our gerrit so that developer will be
able
> to use it as a trigger for the check-merged.sh script run, just to see if
> their patch fixes\brakes something realted to the functional tests
>
> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/
- is
> an example of how the run looks like. I still work to improve the output
>
>
> Please reply and let me know if the idea around the automation flag is
> acceptable by you.. and please review the patch for comments and acks.
> We can ask dcaro to add the flag until Friday, otherwise we'll need to
delay
> this effort after the holiday..
>
>
> functional.sosPluginTests.SosPluginTest
> testSosPlugin OK
> functional.vmRecoveryTests.RecoveryTests
> test_vm_recovery FAIL
> functional.vmQoSTests.VMQosTests
> testSmallVMBallooning FAIL
> functional.virtTests.VirtTest
> testComplexVm FAIL
> testHeadlessVm OK
> testSimpleVm FAIL
> testVmDefinitionGraphics('spice') FAIL
> testVmDefinitionGraphics('vnc') OK
> testVmDefinitionLegacyGraphics('qxl') FAIL
> testVmDefinitionLegacyGraphics('vnc') OK
> testVmDefinitionMultipleGraphics('spice', 'vnc')
FAIL
> testVmDefinitionMultipleGraphics('vnc', 'spice')
FAIL
> testVmWithCdrom('self') FAIL
> testVmWithCdrom('specParams') FAIL
> testVmWithCdrom('vmPayload') FAIL
> testVmWithDevice('hotplugDisk') FAIL
> testVmWithDevice('hotplugNic') FAIL
> testVmWithDevice('smartcard') FAIL
> testVmWithDevice('virtioNic') FAIL
> testVmWithDevice('virtioRng') FAIL
> testVmWithSla FAIL
> testVmWithStorage('iscsi') SKIP:
> python-rtslib is not installed.
> testVmWithStorage('localfs') FAIL
> testVmWithStorage('nfs') FAIL
> functional.storageTests.StorageTest
> testCreatePoolErrors OK
> testStorage('glusterfs', 0) ERROR
> testStorage('glusterfs', 3) ERROR
> testStorage('iscsi', 0) SKIP:
> python-rtslib is not installed.
> testStorage('iscsi', 3) SKIP:
> python-rtslib is not installed.
> testStorage('localfs', 0) FAIL
> testStorage('localfs', 3) FAIL
> testStorage('nfs', 0) FAIL
> testStorage('nfs', 3) FAIL
> functional.networkTests.NetworkTest
> testAddVlanedBridgeless ERROR
> testAddVlanedBridgeless_oneCommand ERROR
> testAfterNetworkSetupHook ERROR
> testBeforeNetworkSetupHook ERROR
> testBondHwAddress(False) ERROR
> testBondHwAddress(True) ERROR
> testBrokenNetworkReplacement(False) ERROR
> testBrokenNetworkReplacement(True) ERROR
> testDelNetworkBondAccumulation ERROR
> testDelNetworkWithMTU(False) ERROR
> testDelNetworkWithMTU(True) ERROR
> testDelWithoutAdd ERROR
> testDhclientLeases(4, 'default') ERROR
> testDhclientLeases(4, 'local') ERROR
> testDhclientLeases(6, None) ERROR
> testDhcpReplaceNicWithBridge ERROR
> testFailWithInvalidBondingName(False) ERROR
> testFailWithInvalidBondingName(True) ERROR
> testFailWithInvalidBridgeName ERROR
> testFailWithInvalidIpConfig ERROR
> testFailWithInvalidNic(False) ERROR
> testFailWithInvalidNic(True) ERROR
> testFailWithInvalidParams(False) ERROR
> testFailWithInvalidParams(True) ERROR
> testGetRouteDeviceTo ERROR
> testHonorBlockingDhcp SKIP:
Slow
> tests are disabled
> testIpLinkWrapper ERROR
> testLowerMtuDoesNotOverride FAIL
> testNoBridgeLeftovers FAIL
> testReconfigureBrNetWithVanishedPort ERROR
> testRedefineBondedNetworkIPs ERROR
> testRemovingBridgeDoesNotLeaveBridge ERROR
> testReorderBondingOptions(False) ERROR
> testReorderBondingOptions(True) ERROR
> testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
> testRestoreToBlockingDHCP ERROR
> testSafeNetworkConfig(False) ERROR
> testSafeNetworkConfig(True) ERROR
> testSelectiveRestoreDuringUpgrade ERROR
> testSelectiveRestoreIgnoresVdsmRegParams FAIL
> testSetupNetworkOutboundQos(False) ERROR
> testSetupNetworkOutboundQos(True) FAIL
> testSetupNetworksActiveSlave ERROR
> testSetupNetworksAddBadParams(False) ERROR
> testSetupNetworksAddBadParams(True) FAIL
> testSetupNetworksAddBondWithManyVlans(False) ERROR
> testSetupNetworksAddBondWithManyVlans(True) FAIL
> testSetupNetworksAddDelBondedNetwork(False) ERROR
> testSetupNetworksAddDelBondedNetwork(True) FAIL
> testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
> testSetupNetworksAddDelDhcp(False, (4,)) ERROR
> testSetupNetworksAddDelDhcp(False, (6,)) ERROR
> testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> testSetupNetworksAddManyVlans(False) ERROR
> testSetupNetworksAddManyVlans(True) FAIL
> testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
> testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
> testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
> testSetupNetworksAddOverExistingBond(False) ERROR
> testSetupNetworksAddOverExistingBond(True) ERROR
> testSetupNetworksAddVlan(False) ERROR
> testSetupNetworksAddVlan(True) FAIL
> testSetupNetworksConnectivityCheck FAIL
> testSetupNetworksConvertVlanNetBridgeness ERROR
> testSetupNetworksDelOneOfBondNets ERROR
> testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
> testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
> testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
Slow
> tests are disabled
> testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
> testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
Slow
> tests are disabled
> testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
> testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> testSetupNetworksMtus(False) ERROR
> testSetupNetworksMtus(True) SKIP:
This
> test fails because the 2 different networks are getting configured with
the
> same MTU. The test should assert that the reported MTUs are equal to the
> requested ones.
> testSetupNetworksMultiMTUsOverBond(False) ERROR
> testSetupNetworksMultiMTUsOverBond(True) FAIL
> testSetupNetworksMultiMTUsOverNic(False) ERROR
> testSetupNetworksMultiMTUsOverNic(True) FAIL
> testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
> testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> testSetupNetworksNicless FAIL
> testSetupNetworksNiclessBridgeless ERROR
> testSetupNetworksOverDhcpIface ERROR
> testSetupNetworksOverExistingBridge FAIL
> testSetupNetworksReconfigureBridge FAIL
> testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
> testSetupNetworksRemoveSlavelessBond ERROR
> testSetupNetworksResizeBond(False) ERROR
> testSetupNetworksResizeBond(True) ERROR
> testSetupNetworksStableBond(False) ERROR
> testSetupNetworksStableBond(True) ERROR
> testStaticNetworkConfig((4, 6)) FAIL
> testStaticNetworkConfig((4,)) FAIL
> testStaticNetworkConfig((6,)) FAIL
> testStaticSourceRouting(False) ERROR
> testStaticSourceRouting(True) FAIL
> testTwiceAdd(False) ERROR
> testTwiceAdd(True) ERROR
> testVolatileConfig(False) ERROR
> testVolatileConfig(True) ERROR
> test_drop_initial_bond_slaves_ip_config FAIL
> test_getVdsStats ERROR
> test_remove_bond_under_network FAIL
> test_remove_initial_network_nic_ip_config FAIL
> test_rollback FAIL
> test_setupNetworks_bond_with_custom_option ERROR
> test_setupNetworks_on_external_bond ERROR
> test_setupNetworks_on_external_vlaned_bond ERROR
> functional.networkTestsOVS.OVSNetworkTest
> testHonorBlockingDhcp SKIP:
Slow
> tests are disabled
> testLowerMtuDoesNotOverride OK
> testNoBridgeLeftovers FAIL
> testSelectiveRestoreIgnoresVdsmRegParams FAIL
> testSetupNetworkOutboundQos(True) OK
> testSetupNetworksAddBadParams(True) OK
> testSetupNetworksAddBondWithManyVlans(True) FAIL
> testSetupNetworksAddDelBondedNetwork(True) FAIL
> testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> testSetupNetworksAddManyVlans(True) OK
> testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> testSetupNetworksAddVlan(True) FAIL
> testSetupNetworksConnectivityCheck OK
> testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
Slow
> tests are disabled
> testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
Slow
> tests are disabled
> testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> testSetupNetworksMtus(True) SKIP:
This
> test fails because the 2 different networks are getting configured with
the
> same MTU. The test should assert that the reported MTUs are equal to the
> requested ones.
> testSetupNetworksMultiMTUsOverBond(True) FAIL
> testSetupNetworksMultiMTUsOverNic(True) FAIL
> testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> testSetupNetworksNicless OK
> testSetupNetworksOverExistingBridge OK
> testSetupNetworksReconfigureBridge FAIL
> testStaticNetworkConfig((4, 6)) FAIL
> testStaticNetworkConfig((4,)) FAIL
> testStaticNetworkConfig((6,)) FAIL
> testStaticSourceRouting(True) FAIL
> test_drop_initial_bond_slaves_ip_config FAIL
> test_ovirtmgmtm_to_ovs FAIL
> test_remove_bond_under_network SKIP: OVS
> does not support bridgeless networks
> test_remove_initial_network_nic_ip_config OK
> test_rollback FAIL
>
>
> On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <pkliczew(a)redhat.com>
> wrote:
>>
>> I like the idea but I have the same feelings as Francesco. I think that
we
>> need to make sure that functional tests for each vertical are stable
before
>> enabling this process.
>>
>> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com> wrote:
>>>
>>> adding also infra team for visibility on the change in CI.
>>> also inline.
>>>
>>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani
<fromani(a)redhat.com>
>>> wrote:
>>>>
>>>> ----- Original Message -----
>>>> > From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
>>>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>, "Nir
>>>> > Soffer" <nsoffer(a)redhat.com>, "Piotr
Kliczewski"
>>>> > <pkliczew(a)redhat.com>
>>>> > Cc: "danken" <danken(a)redhat.com>, "David
Caro" <dcaroest(a)redhat.com
>,
>>>> > "Eyal Edri" <eedri(a)redhat.com>
>>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>>>> > Subject: Automation CI for vdsm
>>>>
>>>> [...]
>>>> > We want to allow developers to trigger the script once reviews and
>>>> > verification are ready (last step before merge). To do so we
agreed
to
>>>> > add
>>>> > Continues Integration flag for each vdsm patch.
>>>
>>>
>>> This flag will be called 'Workflow' or we can name it otherwise, we
just
>>> need to choose what makes sense.
>>> David/Yaniv - Please correct me if I'm wrong.
>>>
>>>>
>>>> Once this flag will be
>>>> > signed with +1 it will trigger Jenkins CI to run the check-merged
>>>> > script
>>>> > (adding new button to gerrit is not an option - you can image that
>>>> > flag as
>>>> > a trigger button), on success Jenkins CI flag will turn to +2. on
fail
>>>> > we'll get -1 and once new patchset is ready the developer will
remove
>>>> > the
>>>> > +1 and add it back to the Continues Integration flag to re-trigger
the
>>>> > job.
>>>> >
>>>> > Please ack the process before we move on with that
>>>>
>>>> Sounds good, even though I'm a little scared (just gut feeling, no
>>>> evidence
>>>> whatsoever) that this could add even more complexity and fragility to
>>>> the jenkins
>>>> fleet.
>>>>
>>>> In the long run, when this is reliable, it will help greatly.
>>>> In the short term, I'm scared because this can lead to false
positives
>>>> and bogus
>>>> failures.
>>>>
>>>> Let me stress I don't have concrete item to share or specific
flaws.
>>>>
>>>> As action item on me, I will find some time next week to check virt
>>>> functional tests,
>>>> to see if they need some fixes, work reliably and so forth
>>>>
>>>> > The patch for those scripts still under review and testing -
>>>> > https://gerrit.ovirt.org/#/c/48268
>>>>
>>>> Will review asap.
>>>>
>>>> --
>>>> Francesco Romani
>>>> RedHat Engineering Virtualization R & D
>>>> Phone: 8261328
>>>> IRC: fromani
>>>
>>>
>>>
>>>
>>> --
>>> Eyal Edri
>>> Supervisor, RHEV CI
>>> EMEA ENG Virtualization R&D
>>> Red Hat Israel
>>>
>>> phone: +972-9-7692018
>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>
>>
>
>
>
> --
> Yaniv Bronhaim.
--
*Yaniv Bronhaim.*
4:13 a.m.
Hi,
OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped
if the package is not installed) and openvswitch service running (not
started automatically). NetworkTest are not able to run together with
OVSNetworkTest since we are doing ugly inheritance hacking there. My
apologize for network tests difficulties, it will be resolved soon. If
you want your life easier, temporary skip OVSNetworkTest suite and it
should be OK.
Best regards,
Petr
2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybronhei(a)redhat.com>:
exactly - just posted a patch again that sings all fails as broken .
we'll get the report soon and I'll publish it as well. hope the run will
take less time now
On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>
> Nice, but we cannot enable this until all the tests pass or disabled.
>
> There is no point in broken or flaky functional tests.
>
> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei(a)redhat.com>
> wrote:
> > So its not stable. It won't block merges and at least give us report
> > after
> > each merge. It takes really long time to run it (because of the tests
> > themselves. Lago things takes maximum 15minutes, but the run last for
> > more
> > than 2hrs right now and I suspect functional/storageTests.py gets stuck)
> >
> > Bellow you can see where we stand (before I added python-rtslib
> > package).
> >
> > Now, I still want to merge the patch https://gerrit.ovirt.org/#/c/48268/
> > -
> > which enables this run after merges, and I still want you to consider
> > the
> > addition of Automation CI flag to our gerrit so that developer will be
> > able
> > to use it as a trigger for the check-merged.sh script run, just to see
> > if
> > their patch fixes\brakes something realted to the functional tests
> >
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ -
> > is
> > an example of how the run looks like. I still work to improve the output
> >
> >
> > Please reply and let me know if the idea around the automation flag is
> > acceptable by you.. and please review the patch for comments and acks.
> > We can ask dcaro to add the flag until Friday, otherwise we'll need to
> > delay
> > this effort after the holiday..
> >
> >
> > functional.sosPluginTests.SosPluginTest
> > testSosPlugin OK
> > functional.vmRecoveryTests.RecoveryTests
> > test_vm_recovery FAIL
> > functional.vmQoSTests.VMQosTests
> > testSmallVMBallooning FAIL
> > functional.virtTests.VirtTest
> > testComplexVm FAIL
> > testHeadlessVm OK
> > testSimpleVm FAIL
> > testVmDefinitionGraphics('spice') FAIL
> > testVmDefinitionGraphics('vnc') OK
> > testVmDefinitionLegacyGraphics('qxl') FAIL
> > testVmDefinitionLegacyGraphics('vnc') OK
> > testVmDefinitionMultipleGraphics('spice', 'vnc')
FAIL
> > testVmDefinitionMultipleGraphics('vnc', 'spice')
FAIL
> > testVmWithCdrom('self') FAIL
> > testVmWithCdrom('specParams') FAIL
> > testVmWithCdrom('vmPayload') FAIL
> > testVmWithDevice('hotplugDisk') FAIL
> > testVmWithDevice('hotplugNic') FAIL
> > testVmWithDevice('smartcard') FAIL
> > testVmWithDevice('virtioNic') FAIL
> > testVmWithDevice('virtioRng') FAIL
> > testVmWithSla FAIL
> > testVmWithStorage('iscsi') SKIP:
> > python-rtslib is not installed.
> > testVmWithStorage('localfs') FAIL
> > testVmWithStorage('nfs') FAIL
> > functional.storageTests.StorageTest
> > testCreatePoolErrors OK
> > testStorage('glusterfs', 0) ERROR
> > testStorage('glusterfs', 3) ERROR
> > testStorage('iscsi', 0) SKIP:
> > python-rtslib is not installed.
> > testStorage('iscsi', 3) SKIP:
> > python-rtslib is not installed.
> > testStorage('localfs', 0) FAIL
> > testStorage('localfs', 3) FAIL
> > testStorage('nfs', 0) FAIL
> > testStorage('nfs', 3) FAIL
> > functional.networkTests.NetworkTest
> > testAddVlanedBridgeless ERROR
> > testAddVlanedBridgeless_oneCommand ERROR
> > testAfterNetworkSetupHook ERROR
> > testBeforeNetworkSetupHook ERROR
> > testBondHwAddress(False) ERROR
> > testBondHwAddress(True) ERROR
> > testBrokenNetworkReplacement(False) ERROR
> > testBrokenNetworkReplacement(True) ERROR
> > testDelNetworkBondAccumulation ERROR
> > testDelNetworkWithMTU(False) ERROR
> > testDelNetworkWithMTU(True) ERROR
> > testDelWithoutAdd ERROR
> > testDhclientLeases(4, 'default') ERROR
> > testDhclientLeases(4, 'local') ERROR
> > testDhclientLeases(6, None) ERROR
> > testDhcpReplaceNicWithBridge ERROR
> > testFailWithInvalidBondingName(False) ERROR
> > testFailWithInvalidBondingName(True) ERROR
> > testFailWithInvalidBridgeName ERROR
> > testFailWithInvalidIpConfig ERROR
> > testFailWithInvalidNic(False) ERROR
> > testFailWithInvalidNic(True) ERROR
> > testFailWithInvalidParams(False) ERROR
> > testFailWithInvalidParams(True) ERROR
> > testGetRouteDeviceTo ERROR
> > testHonorBlockingDhcp SKIP:
> > Slow
> > tests are disabled
> > testIpLinkWrapper ERROR
> > testLowerMtuDoesNotOverride FAIL
> > testNoBridgeLeftovers FAIL
> > testReconfigureBrNetWithVanishedPort ERROR
> > testRedefineBondedNetworkIPs ERROR
> > testRemovingBridgeDoesNotLeaveBridge ERROR
> > testReorderBondingOptions(False) ERROR
> > testReorderBondingOptions(True) ERROR
> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
> > testRestoreToBlockingDHCP ERROR
> > testSafeNetworkConfig(False) ERROR
> > testSafeNetworkConfig(True) ERROR
> > testSelectiveRestoreDuringUpgrade ERROR
> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
> > testSetupNetworkOutboundQos(False) ERROR
> > testSetupNetworkOutboundQos(True) FAIL
> > testSetupNetworksActiveSlave ERROR
> > testSetupNetworksAddBadParams(False) ERROR
> > testSetupNetworksAddBadParams(True) FAIL
> > testSetupNetworksAddBondWithManyVlans(False) ERROR
> > testSetupNetworksAddBondWithManyVlans(True) FAIL
> > testSetupNetworksAddDelBondedNetwork(False) ERROR
> > testSetupNetworksAddDelBondedNetwork(True) FAIL
> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR
> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR
> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> > testSetupNetworksAddManyVlans(False) ERROR
> > testSetupNetworksAddManyVlans(True) FAIL
> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
> > testSetupNetworksAddOverExistingBond(False) ERROR
> > testSetupNetworksAddOverExistingBond(True) ERROR
> > testSetupNetworksAddVlan(False) ERROR
> > testSetupNetworksAddVlan(True) FAIL
> > testSetupNetworksConnectivityCheck FAIL
> > testSetupNetworksConvertVlanNetBridgeness ERROR
> > testSetupNetworksDelOneOfBondNets ERROR
> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
> > Slow
> > tests are disabled
> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
> > Slow
> > tests are disabled
> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> > testSetupNetworksMtus(False) ERROR
> > testSetupNetworksMtus(True) SKIP:
> > This
> > test fails because the 2 different networks are getting configured with
> > the
> > same MTU. The test should assert that the reported MTUs are equal to the
> > requested ones.
> > testSetupNetworksMultiMTUsOverBond(False) ERROR
> > testSetupNetworksMultiMTUsOverBond(True) FAIL
> > testSetupNetworksMultiMTUsOverNic(False) ERROR
> > testSetupNetworksMultiMTUsOverNic(True) FAIL
> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> > testSetupNetworksNicless FAIL
> > testSetupNetworksNiclessBridgeless ERROR
> > testSetupNetworksOverDhcpIface ERROR
> > testSetupNetworksOverExistingBridge FAIL
> > testSetupNetworksReconfigureBridge FAIL
> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
> > testSetupNetworksRemoveSlavelessBond ERROR
> > testSetupNetworksResizeBond(False) ERROR
> > testSetupNetworksResizeBond(True) ERROR
> > testSetupNetworksStableBond(False) ERROR
> > testSetupNetworksStableBond(True) ERROR
> > testStaticNetworkConfig((4, 6)) FAIL
> > testStaticNetworkConfig((4,)) FAIL
> > testStaticNetworkConfig((6,)) FAIL
> > testStaticSourceRouting(False) ERROR
> > testStaticSourceRouting(True) FAIL
> > testTwiceAdd(False) ERROR
> > testTwiceAdd(True) ERROR
> > testVolatileConfig(False) ERROR
> > testVolatileConfig(True) ERROR
> > test_drop_initial_bond_slaves_ip_config FAIL
> > test_getVdsStats ERROR
> > test_remove_bond_under_network FAIL
> > test_remove_initial_network_nic_ip_config FAIL
> > test_rollback FAIL
> > test_setupNetworks_bond_with_custom_option ERROR
> > test_setupNetworks_on_external_bond ERROR
> > test_setupNetworks_on_external_vlaned_bond ERROR
> > functional.networkTestsOVS.OVSNetworkTest
> > testHonorBlockingDhcp SKIP:
> > Slow
> > tests are disabled
> > testLowerMtuDoesNotOverride OK
> > testNoBridgeLeftovers FAIL
> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
> > testSetupNetworkOutboundQos(True) OK
> > testSetupNetworksAddBadParams(True) OK
> > testSetupNetworksAddBondWithManyVlans(True) FAIL
> > testSetupNetworksAddDelBondedNetwork(True) FAIL
> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> > testSetupNetworksAddManyVlans(True) OK
> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> > testSetupNetworksAddVlan(True) FAIL
> > testSetupNetworksConnectivityCheck OK
> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
> > Slow
> > tests are disabled
> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
> > Slow
> > tests are disabled
> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> > testSetupNetworksMtus(True) SKIP:
> > This
> > test fails because the 2 different networks are getting configured with
> > the
> > same MTU. The test should assert that the reported MTUs are equal to the
> > requested ones.
> > testSetupNetworksMultiMTUsOverBond(True) FAIL
> > testSetupNetworksMultiMTUsOverNic(True) FAIL
> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> > testSetupNetworksNicless OK
> > testSetupNetworksOverExistingBridge OK
> > testSetupNetworksReconfigureBridge FAIL
> > testStaticNetworkConfig((4, 6)) FAIL
> > testStaticNetworkConfig((4,)) FAIL
> > testStaticNetworkConfig((6,)) FAIL
> > testStaticSourceRouting(True) FAIL
> > test_drop_initial_bond_slaves_ip_config FAIL
> > test_ovirtmgmtm_to_ovs FAIL
> > test_remove_bond_under_network SKIP:
> > OVS
> > does not support bridgeless networks
> > test_remove_initial_network_nic_ip_config OK
> > test_rollback FAIL
> >
> >
> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <pkliczew(a)redhat.com>
> > wrote:
> >>
> >> I like the idea but I have the same feelings as Francesco. I think that
> >> we
> >> need to make sure that functional tests for each vertical are stable
> >> before
> >> enabling this process.
> >>
> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com> wrote:
> >>>
> >>> adding also infra team for visibility on the change in CI.
> >>> also inline.
> >>>
> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani
<fromani(a)redhat.com>
> >>> wrote:
> >>>>
> >>>> ----- Original Message -----
> >>>> > From: "Yaniv Bronheim" <ybronhei(a)redhat.com>
> >>>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>, "Nir
> >>>> > Soffer" <nsoffer(a)redhat.com>, "Piotr
Kliczewski"
> >>>> > <pkliczew(a)redhat.com>
> >>>> > Cc: "danken" <danken(a)redhat.com>, "David
Caro"
> >>>> > <dcaroest(a)redhat.com>,
> >>>> > "Eyal Edri" <eedri(a)redhat.com>
> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
> >>>> > Subject: Automation CI for vdsm
> >>>>
> >>>> [...]
> >>>> > We want to allow developers to trigger the script once reviews
and
> >>>> > verification are ready (last step before merge). To do so we
agreed
> >>>> > to
> >>>> > add
> >>>> > Continues Integration flag for each vdsm patch.
> >>>
> >>>
> >>> This flag will be called 'Workflow' or we can name it otherwise,
we
> >>> just
> >>> need to choose what makes sense.
> >>> David/Yaniv - Please correct me if I'm wrong.
> >>>
> >>>>
> >>>> Once this flag will be
> >>>> > signed with +1 it will trigger Jenkins CI to run the
check-merged
> >>>> > script
> >>>> > (adding new button to gerrit is not an option - you can image
that
> >>>> > flag as
> >>>> > a trigger button), on success Jenkins CI flag will turn to +2.
on
> >>>> > fail
> >>>> > we'll get -1 and once new patchset is ready the developer
will
> >>>> > remove
> >>>> > the
> >>>> > +1 and add it back to the Continues Integration flag to
re-trigger
> >>>> > the
> >>>> > job.
> >>>> >
> >>>> > Please ack the process before we move on with that
> >>>>
> >>>> Sounds good, even though I'm a little scared (just gut feeling,
no
> >>>> evidence
> >>>> whatsoever) that this could add even more complexity and fragility
to
> >>>> the jenkins
> >>>> fleet.
> >>>>
> >>>> In the long run, when this is reliable, it will help greatly.
> >>>> In the short term, I'm scared because this can lead to false
> >>>> positives
> >>>> and bogus
> >>>> failures.
> >>>>
> >>>> Let me stress I don't have concrete item to share or specific
flaws.
> >>>>
> >>>> As action item on me, I will find some time next week to check virt
> >>>> functional tests,
> >>>> to see if they need some fixes, work reliably and so forth
> >>>>
> >>>> > The patch for those scripts still under review and testing -
> >>>> > https://gerrit.ovirt.org/#/c/48268
> >>>>
> >>>> Will review asap.
> >>>>
> >>>> --
> >>>> Francesco Romani
> >>>> RedHat Engineering Virtualization R & D
> >>>> Phone: 8261328
> >>>> IRC: fromani
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Eyal Edri
> >>> Supervisor, RHEV CI
> >>> EMEA ENG Virtualization R&D
> >>> Red Hat Israel
> >>>
> >>> phone: +972-9-7692018
> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
> >>
> >>
> >
> >
> >
> > --
> > Yaniv Bronhaim.
--
Yaniv Bronhaim.
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
6:50 a.m.
OK. so for now the patch [1] skips all storage and OVSNetwork tests
Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr,
for the setup :\ the rest for the tests)
The output looks pretty good though :)
[1] https://gerrit.ovirt.org/#/c/48268/
On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phoracek(a)redhat.com> wrote:
Hi,
OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped
if the package is not installed) and openvswitch service running (not
started automatically). NetworkTest are not able to run together with
OVSNetworkTest since we are doing ugly inheritance hacking there. My
apologize for network tests difficulties, it will be resolved soon. If
you want your life easier, temporary skip OVSNetworkTest suite and it
should be OK.
Best regards,
Petr
2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybronhei(a)redhat.com>:
> exactly - just posted a patch again that sings all fails as broken .
> we'll get the report soon and I'll publish it as well. hope the run will
> take less time now
>
> On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>>
>> Nice, but we cannot enable this until all the tests pass or disabled.
>>
>> There is no point in broken or flaky functional tests.
>>
>> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei(a)redhat.com>
>> wrote:
>> > So its not stable. It won't block merges and at least give us report
>> > after
>> > each merge. It takes really long time to run it (because of the tests
>> > themselves. Lago things takes maximum 15minutes, but the run last for
>> > more
>> > than 2hrs right now and I suspect functional/storageTests.py gets
stuck)
>> >
>> > Bellow you can see where we stand (before I added python-rtslib
>> > package).
>> >
>> > Now, I still want to merge the patch
https://gerrit.ovirt.org/#/c/48268/
>> > -
>> > which enables this run after merges, and I still want you to consider
>> > the
>> > addition of Automation CI flag to our gerrit so that developer will be
>> > able
>> > to use it as a trigger for the check-merged.sh script run, just to see
>> > if
>> > their patch fixes\brakes something realted to the functional tests
>> >
>> >
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ -
>> > is
>> > an example of how the run looks like. I still work to improve the
output
>> >
>> >
>> > Please reply and let me know if the idea around the automation flag is
>> > acceptable by you.. and please review the patch for comments and acks.
>> > We can ask dcaro to add the flag until Friday, otherwise we'll need to
>> > delay
>> > this effort after the holiday..
>> >
>> >
>> > functional.sosPluginTests.SosPluginTest
>> > testSosPlugin OK
>> > functional.vmRecoveryTests.RecoveryTests
>> > test_vm_recovery FAIL
>> > functional.vmQoSTests.VMQosTests
>> > testSmallVMBallooning FAIL
>> > functional.virtTests.VirtTest
>> > testComplexVm FAIL
>> > testHeadlessVm OK
>> > testSimpleVm FAIL
>> > testVmDefinitionGraphics('spice')
FAIL
>> > testVmDefinitionGraphics('vnc') OK
>> > testVmDefinitionLegacyGraphics('qxl')
FAIL
>> > testVmDefinitionLegacyGraphics('vnc') OK
>> > testVmDefinitionMultipleGraphics('spice', 'vnc')
FAIL
>> > testVmDefinitionMultipleGraphics('vnc', 'spice')
FAIL
>> > testVmWithCdrom('self')
FAIL
>> > testVmWithCdrom('specParams')
FAIL
>> > testVmWithCdrom('vmPayload')
FAIL
>> > testVmWithDevice('hotplugDisk')
FAIL
>> > testVmWithDevice('hotplugNic')
FAIL
>> > testVmWithDevice('smartcard')
FAIL
>> > testVmWithDevice('virtioNic')
FAIL
>> > testVmWithDevice('virtioRng')
FAIL
>> > testVmWithSla FAIL
>> > testVmWithStorage('iscsi')
SKIP:
>> > python-rtslib is not installed.
>> > testVmWithStorage('localfs')
FAIL
>> > testVmWithStorage('nfs')
FAIL
>> > functional.storageTests.StorageTest
>> > testCreatePoolErrors OK
>> > testStorage('glusterfs', 0)
ERROR
>> > testStorage('glusterfs', 3)
ERROR
>> > testStorage('iscsi', 0)
SKIP:
>> > python-rtslib is not installed.
>> > testStorage('iscsi', 3)
SKIP:
>> > python-rtslib is not installed.
>> > testStorage('localfs', 0)
FAIL
>> > testStorage('localfs', 3)
FAIL
>> > testStorage('nfs', 0)
FAIL
>> > testStorage('nfs', 3)
FAIL
>> > functional.networkTests.NetworkTest
>> > testAddVlanedBridgeless ERROR
>> > testAddVlanedBridgeless_oneCommand ERROR
>> > testAfterNetworkSetupHook ERROR
>> > testBeforeNetworkSetupHook ERROR
>> > testBondHwAddress(False) ERROR
>> > testBondHwAddress(True) ERROR
>> > testBrokenNetworkReplacement(False) ERROR
>> > testBrokenNetworkReplacement(True) ERROR
>> > testDelNetworkBondAccumulation ERROR
>> > testDelNetworkWithMTU(False) ERROR
>> > testDelNetworkWithMTU(True) ERROR
>> > testDelWithoutAdd ERROR
>> > testDhclientLeases(4, 'default')
ERROR
>> > testDhclientLeases(4, 'local')
ERROR
>> > testDhclientLeases(6, None) ERROR
>> > testDhcpReplaceNicWithBridge ERROR
>> > testFailWithInvalidBondingName(False) ERROR
>> > testFailWithInvalidBondingName(True) ERROR
>> > testFailWithInvalidBridgeName ERROR
>> > testFailWithInvalidIpConfig ERROR
>> > testFailWithInvalidNic(False) ERROR
>> > testFailWithInvalidNic(True) ERROR
>> > testFailWithInvalidParams(False) ERROR
>> > testFailWithInvalidParams(True) ERROR
>> > testGetRouteDeviceTo ERROR
>> > testHonorBlockingDhcp SKIP:
>> > Slow
>> > tests are disabled
>> > testIpLinkWrapper ERROR
>> > testLowerMtuDoesNotOverride FAIL
>> > testNoBridgeLeftovers FAIL
>> > testReconfigureBrNetWithVanishedPort ERROR
>> > testRedefineBondedNetworkIPs ERROR
>> > testRemovingBridgeDoesNotLeaveBridge ERROR
>> > testReorderBondingOptions(False) ERROR
>> > testReorderBondingOptions(True) ERROR
>> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
>> > testRestoreToBlockingDHCP ERROR
>> > testSafeNetworkConfig(False) ERROR
>> > testSafeNetworkConfig(True) ERROR
>> > testSelectiveRestoreDuringUpgrade ERROR
>> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
>> > testSetupNetworkOutboundQos(False) ERROR
>> > testSetupNetworkOutboundQos(True) FAIL
>> > testSetupNetworksActiveSlave ERROR
>> > testSetupNetworksAddBadParams(False) ERROR
>> > testSetupNetworksAddBadParams(True) FAIL
>> > testSetupNetworksAddBondWithManyVlans(False) ERROR
>> > testSetupNetworksAddBondWithManyVlans(True) FAIL
>> > testSetupNetworksAddDelBondedNetwork(False) ERROR
>> > testSetupNetworksAddDelBondedNetwork(True) FAIL
>> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
>> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR
>> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR
>> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
>> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
>> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
>> > testSetupNetworksAddManyVlans(False) ERROR
>> > testSetupNetworksAddManyVlans(True) FAIL
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
>> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
>> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
>> > testSetupNetworksAddOverExistingBond(False) ERROR
>> > testSetupNetworksAddOverExistingBond(True) ERROR
>> > testSetupNetworksAddVlan(False) ERROR
>> > testSetupNetworksAddVlan(True) FAIL
>> > testSetupNetworksConnectivityCheck FAIL
>> > testSetupNetworksConvertVlanNetBridgeness ERROR
>> > testSetupNetworksDelOneOfBondNets ERROR
>> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> > Slow
>> > tests are disabled
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> > Slow
>> > tests are disabled
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
>> > testSetupNetworksMtus(False) ERROR
>> > testSetupNetworksMtus(True) SKIP:
>> > This
>> > test fails because the 2 different networks are getting configured
with
>> > the
>> > same MTU. The test should assert that the reported MTUs are equal to
the
>> > requested ones.
>> > testSetupNetworksMultiMTUsOverBond(False) ERROR
>> > testSetupNetworksMultiMTUsOverBond(True) FAIL
>> > testSetupNetworksMultiMTUsOverNic(False) ERROR
>> > testSetupNetworksMultiMTUsOverNic(True) FAIL
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
>> > testSetupNetworksNicless FAIL
>> > testSetupNetworksNiclessBridgeless ERROR
>> > testSetupNetworksOverDhcpIface ERROR
>> > testSetupNetworksOverExistingBridge FAIL
>> > testSetupNetworksReconfigureBridge FAIL
>> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
>> > testSetupNetworksRemoveSlavelessBond ERROR
>> > testSetupNetworksResizeBond(False) ERROR
>> > testSetupNetworksResizeBond(True) ERROR
>> > testSetupNetworksStableBond(False) ERROR
>> > testSetupNetworksStableBond(True) ERROR
>> > testStaticNetworkConfig((4, 6)) FAIL
>> > testStaticNetworkConfig((4,)) FAIL
>> > testStaticNetworkConfig((6,)) FAIL
>> > testStaticSourceRouting(False) ERROR
>> > testStaticSourceRouting(True) FAIL
>> > testTwiceAdd(False) ERROR
>> > testTwiceAdd(True) ERROR
>> > testVolatileConfig(False) ERROR
>> > testVolatileConfig(True) ERROR
>> > test_drop_initial_bond_slaves_ip_config FAIL
>> > test_getVdsStats ERROR
>> > test_remove_bond_under_network FAIL
>> > test_remove_initial_network_nic_ip_config FAIL
>> > test_rollback FAIL
>> > test_setupNetworks_bond_with_custom_option ERROR
>> > test_setupNetworks_on_external_bond ERROR
>> > test_setupNetworks_on_external_vlaned_bond ERROR
>> > functional.networkTestsOVS.OVSNetworkTest
>> > testHonorBlockingDhcp SKIP:
>> > Slow
>> > tests are disabled
>> > testLowerMtuDoesNotOverride OK
>> > testNoBridgeLeftovers FAIL
>> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
>> > testSetupNetworkOutboundQos(True) OK
>> > testSetupNetworksAddBadParams(True) OK
>> > testSetupNetworksAddBondWithManyVlans(True) FAIL
>> > testSetupNetworksAddDelBondedNetwork(True) FAIL
>> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
>> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
>> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
>> > testSetupNetworksAddManyVlans(True) OK
>> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
>> > testSetupNetworksAddVlan(True) FAIL
>> > testSetupNetworksConnectivityCheck OK
>> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
>> > Slow
>> > tests are disabled
>> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
>> > Slow
>> > tests are disabled
>> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
>> > testSetupNetworksMtus(True) SKIP:
>> > This
>> > test fails because the 2 different networks are getting configured
with
>> > the
>> > same MTU. The test should assert that the reported MTUs are equal to
the
>> > requested ones.
>> > testSetupNetworksMultiMTUsOverBond(True) FAIL
>> > testSetupNetworksMultiMTUsOverNic(True) FAIL
>> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
>> > testSetupNetworksNicless OK
>> > testSetupNetworksOverExistingBridge OK
>> > testSetupNetworksReconfigureBridge FAIL
>> > testStaticNetworkConfig((4, 6)) FAIL
>> > testStaticNetworkConfig((4,)) FAIL
>> > testStaticNetworkConfig((6,)) FAIL
>> > testStaticSourceRouting(True) FAIL
>> > test_drop_initial_bond_slaves_ip_config FAIL
>> > test_ovirtmgmtm_to_ovs FAIL
>> > test_remove_bond_under_network SKIP:
>> > OVS
>> > does not support bridgeless networks
>> > test_remove_initial_network_nic_ip_config OK
>> > test_rollback FAIL
>> >
>> >
>> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <
pkliczew(a)redhat.com>
>> > wrote:
>> >>
>> >> I like the idea but I have the same feelings as Francesco. I think
that
>> >> we
>> >> need to make sure that functional tests for each vertical are stable
>> >> before
>> >> enabling this process.
>> >>
>> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <eedri(a)redhat.com>
wrote:
>> >>>
>> >>> adding also infra team for visibility on the change in CI.
>> >>> also inline.
>> >>>
>> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <
fromani(a)redhat.com>
>> >>> wrote:
>> >>>>
>> >>>> ----- Original Message -----
>> >>>> > From: "Yaniv Bronheim"
<ybronhei(a)redhat.com>
>> >>>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>,
"Nir
>> >>>> > Soffer" <nsoffer(a)redhat.com>, "Piotr
Kliczewski"
>> >>>> > <pkliczew(a)redhat.com>
>> >>>> > Cc: "danken" <danken(a)redhat.com>,
"David Caro"
>> >>>> > <dcaroest(a)redhat.com>,
>> >>>> > "Eyal Edri" <eedri(a)redhat.com>
>> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
>> >>>> > Subject: Automation CI for vdsm
>> >>>>
>> >>>> [...]
>> >>>> > We want to allow developers to trigger the script once
reviews
and
>> >>>> > verification are ready (last step before merge). To do so
we
agreed
>> >>>> > to
>> >>>> > add
>> >>>> > Continues Integration flag for each vdsm patch.
>> >>>
>> >>>
>> >>> This flag will be called 'Workflow' or we can name it
otherwise, we
>> >>> just
>> >>> need to choose what makes sense.
>> >>> David/Yaniv - Please correct me if I'm wrong.
>> >>>
>> >>>>
>> >>>> Once this flag will be
>> >>>> > signed with +1 it will trigger Jenkins CI to run the
check-merged
>> >>>> > script
>> >>>> > (adding new button to gerrit is not an option - you can
image
that
>> >>>> > flag as
>> >>>> > a trigger button), on success Jenkins CI flag will turn to
+2. on
>> >>>> > fail
>> >>>> > we'll get -1 and once new patchset is ready the
developer will
>> >>>> > remove
>> >>>> > the
>> >>>> > +1 and add it back to the Continues Integration flag to
re-trigger
>> >>>> > the
>> >>>> > job.
>> >>>> >
>> >>>> > Please ack the process before we move on with that
>> >>>>
>> >>>> Sounds good, even though I'm a little scared (just gut
feeling, no
>> >>>> evidence
>> >>>> whatsoever) that this could add even more complexity and
fragility
to
>> >>>> the jenkins
>> >>>> fleet.
>> >>>>
>> >>>> In the long run, when this is reliable, it will help greatly.
>> >>>> In the short term, I'm scared because this can lead to
false
>> >>>> positives
>> >>>> and bogus
>> >>>> failures.
>> >>>>
>> >>>> Let me stress I don't have concrete item to share or
specific
flaws.
>> >>>>
>> >>>> As action item on me, I will find some time next week to check
virt
>> >>>> functional tests,
>> >>>> to see if they need some fixes, work reliably and so forth
>> >>>>
>> >>>> > The patch for those scripts still under review and testing
-
>> >>>> > https://gerrit.ovirt.org/#/c/48268
>> >>>>
>> >>>> Will review asap.
>> >>>>
>> >>>> --
>> >>>> Francesco Romani
>> >>>> RedHat Engineering Virtualization R & D
>> >>>> Phone: 8261328
>> >>>> IRC: fromani
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Eyal Edri
>> >>> Supervisor, RHEV CI
>> >>> EMEA ENG Virtualization R&D
>> >>> Red Hat Israel
>> >>>
>> >>> phone: +972-9-7692018
>> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Yaniv Bronhaim.
>
>
>
>
> --
> Yaniv Bronhaim.
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
--
*Yaniv Bronhaim.*
6:15 a.m.
Adding lago-devel.
Anyone from Lago project can help debug why it is taking so long to setup
Lago (1:17 hours?)
Yaniv, was this run done only on CI, or someone tried to run it locally on
a baremetal server / laptop?
e.
On Tue, Dec 22, 2015 at 2:50 PM, Yaniv Bronheim <ybronhei(a)redhat.com> wrote:
OK. so for now the patch [1] skips all storage and OVSNetwork tests
Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr,
for the setup :\ the rest for the tests)
The output looks pretty good though :)
[1] https://gerrit.ovirt.org/#/c/48268/
On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phoracek(a)redhat.com>
wrote:
> Hi,
>
> OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped
> if the package is not installed) and openvswitch service running (not
> started automatically). NetworkTest are not able to run together with
> OVSNetworkTest since we are doing ugly inheritance hacking there. My
> apologize for network tests difficulties, it will be resolved soon. If
> you want your life easier, temporary skip OVSNetworkTest suite and it
> should be OK.
>
> Best regards,
> Petr
>
> 2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybronhei(a)redhat.com>:
> > exactly - just posted a patch again that sings all fails as broken .
> > we'll get the report soon and I'll publish it as well. hope the run
will
> > take less time now
> >
> > On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> >>
> >> Nice, but we cannot enable this until all the tests pass or disabled.
> >>
> >> There is no point in broken or flaky functional tests.
> >>
> >> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybronhei(a)redhat.com>
> >> wrote:
> >> > So its not stable. It won't block merges and at least give us
report
> >> > after
> >> > each merge. It takes really long time to run it (because of the tests
> >> > themselves. Lago things takes maximum 15minutes, but the run last for
> >> > more
> >> > than 2hrs right now and I suspect functional/storageTests.py gets
> stuck)
> >> >
> >> > Bellow you can see where we stand (before I added python-rtslib
> >> > package).
> >> >
> >> > Now, I still want to merge the patch
> https://gerrit.ovirt.org/#/c/48268/
> >> > -
> >> > which enables this run after merges, and I still want you to consider
> >> > the
> >> > addition of Automation CI flag to our gerrit so that developer will
> be
> >> > able
> >> > to use it as a trigger for the check-merged.sh script run, just to
> see
> >> > if
> >> > their patch fixes\brakes something realted to the functional tests
> >> >
> >> >
> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ -
> >> > is
> >> > an example of how the run looks like. I still work to improve the
> output
> >> >
> >> >
> >> > Please reply and let me know if the idea around the automation flag
> is
> >> > acceptable by you.. and please review the patch for comments and
> acks.
> >> > We can ask dcaro to add the flag until Friday, otherwise we'll
need
> to
> >> > delay
> >> > this effort after the holiday..
> >> >
> >> >
> >> > functional.sosPluginTests.SosPluginTest
> >> > testSosPlugin OK
> >> > functional.vmRecoveryTests.RecoveryTests
> >> > test_vm_recovery FAIL
> >> > functional.vmQoSTests.VMQosTests
> >> > testSmallVMBallooning FAIL
> >> > functional.virtTests.VirtTest
> >> > testComplexVm FAIL
> >> > testHeadlessVm OK
> >> > testSimpleVm FAIL
> >> > testVmDefinitionGraphics('spice')
FAIL
> >> > testVmDefinitionGraphics('vnc')
OK
> >> > testVmDefinitionLegacyGraphics('qxl')
FAIL
> >> > testVmDefinitionLegacyGraphics('vnc')
OK
> >> > testVmDefinitionMultipleGraphics('spice', 'vnc')
FAIL
> >> > testVmDefinitionMultipleGraphics('vnc', 'spice')
FAIL
> >> > testVmWithCdrom('self')
FAIL
> >> > testVmWithCdrom('specParams')
FAIL
> >> > testVmWithCdrom('vmPayload')
FAIL
> >> > testVmWithDevice('hotplugDisk')
FAIL
> >> > testVmWithDevice('hotplugNic')
FAIL
> >> > testVmWithDevice('smartcard')
FAIL
> >> > testVmWithDevice('virtioNic')
FAIL
> >> > testVmWithDevice('virtioRng')
FAIL
> >> > testVmWithSla FAIL
> >> > testVmWithStorage('iscsi')
SKIP:
> >> > python-rtslib is not installed.
> >> > testVmWithStorage('localfs')
FAIL
> >> > testVmWithStorage('nfs')
FAIL
> >> > functional.storageTests.StorageTest
> >> > testCreatePoolErrors OK
> >> > testStorage('glusterfs', 0)
ERROR
> >> > testStorage('glusterfs', 3)
ERROR
> >> > testStorage('iscsi', 0)
SKIP:
> >> > python-rtslib is not installed.
> >> > testStorage('iscsi', 3)
SKIP:
> >> > python-rtslib is not installed.
> >> > testStorage('localfs', 0)
FAIL
> >> > testStorage('localfs', 3)
FAIL
> >> > testStorage('nfs', 0)
FAIL
> >> > testStorage('nfs', 3)
FAIL
> >> > functional.networkTests.NetworkTest
> >> > testAddVlanedBridgeless ERROR
> >> > testAddVlanedBridgeless_oneCommand ERROR
> >> > testAfterNetworkSetupHook ERROR
> >> > testBeforeNetworkSetupHook ERROR
> >> > testBondHwAddress(False) ERROR
> >> > testBondHwAddress(True) ERROR
> >> > testBrokenNetworkReplacement(False) ERROR
> >> > testBrokenNetworkReplacement(True) ERROR
> >> > testDelNetworkBondAccumulation ERROR
> >> > testDelNetworkWithMTU(False) ERROR
> >> > testDelNetworkWithMTU(True) ERROR
> >> > testDelWithoutAdd ERROR
> >> > testDhclientLeases(4, 'default')
ERROR
> >> > testDhclientLeases(4, 'local')
ERROR
> >> > testDhclientLeases(6, None) ERROR
> >> > testDhcpReplaceNicWithBridge ERROR
> >> > testFailWithInvalidBondingName(False) ERROR
> >> > testFailWithInvalidBondingName(True) ERROR
> >> > testFailWithInvalidBridgeName ERROR
> >> > testFailWithInvalidIpConfig ERROR
> >> > testFailWithInvalidNic(False) ERROR
> >> > testFailWithInvalidNic(True) ERROR
> >> > testFailWithInvalidParams(False) ERROR
> >> > testFailWithInvalidParams(True) ERROR
> >> > testGetRouteDeviceTo ERROR
> >> > testHonorBlockingDhcp SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testIpLinkWrapper ERROR
> >> > testLowerMtuDoesNotOverride FAIL
> >> > testNoBridgeLeftovers FAIL
> >> > testReconfigureBrNetWithVanishedPort ERROR
> >> > testRedefineBondedNetworkIPs ERROR
> >> > testRemovingBridgeDoesNotLeaveBridge ERROR
> >> > testReorderBondingOptions(False) ERROR
> >> > testReorderBondingOptions(True) ERROR
> >> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR
> >> > testRestoreToBlockingDHCP ERROR
> >> > testSafeNetworkConfig(False) ERROR
> >> > testSafeNetworkConfig(True) ERROR
> >> > testSelectiveRestoreDuringUpgrade ERROR
> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
> >> > testSetupNetworkOutboundQos(False) ERROR
> >> > testSetupNetworkOutboundQos(True) FAIL
> >> > testSetupNetworksActiveSlave ERROR
> >> > testSetupNetworksAddBadParams(False) ERROR
> >> > testSetupNetworksAddBadParams(True) FAIL
> >> > testSetupNetworksAddBondWithManyVlans(False) ERROR
> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL
> >> > testSetupNetworksAddDelBondedNetwork(False) ERROR
> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL
> >> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR
> >> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR
> >> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR
> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> >> > testSetupNetworksAddManyVlans(False) ERROR
> >> > testSetupNetworksAddManyVlans(True) FAIL
> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR
> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR
> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR
> >> > testSetupNetworksAddOverExistingBond(False) ERROR
> >> > testSetupNetworksAddOverExistingBond(True) ERROR
> >> > testSetupNetworksAddVlan(False) ERROR
> >> > testSetupNetworksAddVlan(True) FAIL
> >> > testSetupNetworksConnectivityCheck FAIL
> >> > testSetupNetworksConvertVlanNetBridgeness ERROR
> >> > testSetupNetworksDelOneOfBondNets ERROR
> >> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR
> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR
> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR
> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR
> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> >> > testSetupNetworksMtus(False) ERROR
> >> > testSetupNetworksMtus(True) SKIP:
> >> > This
> >> > test fails because the 2 different networks are getting configured
> with
> >> > the
> >> > same MTU. The test should assert that the reported MTUs are equal to
> the
> >> > requested ones.
> >> > testSetupNetworksMultiMTUsOverBond(False) ERROR
> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL
> >> > testSetupNetworksMultiMTUsOverNic(False) ERROR
> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL
> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR
> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> >> > testSetupNetworksNicless FAIL
> >> > testSetupNetworksNiclessBridgeless ERROR
> >> > testSetupNetworksOverDhcpIface ERROR
> >> > testSetupNetworksOverExistingBridge FAIL
> >> > testSetupNetworksReconfigureBridge FAIL
> >> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR
> >> > testSetupNetworksRemoveSlavelessBond ERROR
> >> > testSetupNetworksResizeBond(False) ERROR
> >> > testSetupNetworksResizeBond(True) ERROR
> >> > testSetupNetworksStableBond(False) ERROR
> >> > testSetupNetworksStableBond(True) ERROR
> >> > testStaticNetworkConfig((4, 6)) FAIL
> >> > testStaticNetworkConfig((4,)) FAIL
> >> > testStaticNetworkConfig((6,)) FAIL
> >> > testStaticSourceRouting(False) ERROR
> >> > testStaticSourceRouting(True) FAIL
> >> > testTwiceAdd(False) ERROR
> >> > testTwiceAdd(True) ERROR
> >> > testVolatileConfig(False) ERROR
> >> > testVolatileConfig(True) ERROR
> >> > test_drop_initial_bond_slaves_ip_config FAIL
> >> > test_getVdsStats ERROR
> >> > test_remove_bond_under_network FAIL
> >> > test_remove_initial_network_nic_ip_config FAIL
> >> > test_rollback FAIL
> >> > test_setupNetworks_bond_with_custom_option ERROR
> >> > test_setupNetworks_on_external_bond ERROR
> >> > test_setupNetworks_on_external_vlaned_bond ERROR
> >> > functional.networkTestsOVS.OVSNetworkTest
> >> > testHonorBlockingDhcp SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testLowerMtuDoesNotOverride OK
> >> > testNoBridgeLeftovers FAIL
> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL
> >> > testSetupNetworkOutboundQos(True) OK
> >> > testSetupNetworksAddBadParams(True) OK
> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL
> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL
> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL
> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL
> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL
> >> > testSetupNetworksAddManyVlans(True) OK
> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL
> >> > testSetupNetworksAddVlan(True) FAIL
> >> > testSetupNetworksConnectivityCheck OK
> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP:
> >> > Slow
> >> > tests are disabled
> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL
> >> > testSetupNetworksMtus(True) SKIP:
> >> > This
> >> > test fails because the 2 different networks are getting configured
> with
> >> > the
> >> > same MTU. The test should assert that the reported MTUs are equal to
> the
> >> > requested ones.
> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL
> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL
> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL
> >> > testSetupNetworksNicless OK
> >> > testSetupNetworksOverExistingBridge OK
> >> > testSetupNetworksReconfigureBridge FAIL
> >> > testStaticNetworkConfig((4, 6)) FAIL
> >> > testStaticNetworkConfig((4,)) FAIL
> >> > testStaticNetworkConfig((6,)) FAIL
> >> > testStaticSourceRouting(True) FAIL
> >> > test_drop_initial_bond_slaves_ip_config FAIL
> >> > test_ovirtmgmtm_to_ovs FAIL
> >> > test_remove_bond_under_network SKIP:
> >> > OVS
> >> > does not support bridgeless networks
> >> > test_remove_initial_network_nic_ip_config OK
> >> > test_rollback FAIL
> >> >
> >> >
> >> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <
> pkliczew(a)redhat.com>
> >> > wrote:
> >> >>
> >> >> I like the idea but I have the same feelings as Francesco. I think
> that
> >> >> we
> >> >> need to make sure that functional tests for each vertical are
stable
> >> >> before
> >> >> enabling this process.
> >> >>
> >> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri
<eedri(a)redhat.com>
> wrote:
> >> >>>
> >> >>> adding also infra team for visibility on the change in CI.
> >> >>> also inline.
> >> >>>
> >> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <
> fromani(a)redhat.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> ----- Original Message -----
> >> >>>> > From: "Yaniv Bronheim"
<ybronhei(a)redhat.com>
> >> >>>> > To: devel(a)ovirt.org, "Francesco Romani"
<fromani(a)redhat.com>,
> "Nir
> >> >>>> > Soffer" <nsoffer(a)redhat.com>, "Piotr
Kliczewski"
> >> >>>> > <pkliczew(a)redhat.com>
> >> >>>> > Cc: "danken" <danken(a)redhat.com>,
"David Caro"
> >> >>>> > <dcaroest(a)redhat.com>,
> >> >>>> > "Eyal Edri" <eedri(a)redhat.com>
> >> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM
> >> >>>> > Subject: Automation CI for vdsm
> >> >>>>
> >> >>>> [...]
> >> >>>> > We want to allow developers to trigger the script once
reviews
> and
> >> >>>> > verification are ready (last step before merge). To do
so we
> agreed
> >> >>>> > to
> >> >>>> > add
> >> >>>> > Continues Integration flag for each vdsm patch.
> >> >>>
> >> >>>
> >> >>> This flag will be called 'Workflow' or we can name it
otherwise, we
> >> >>> just
> >> >>> need to choose what makes sense.
> >> >>> David/Yaniv - Please correct me if I'm wrong.
> >> >>>
> >> >>>>
> >> >>>> Once this flag will be
> >> >>>> > signed with +1 it will trigger Jenkins CI to run the
> check-merged
> >> >>>> > script
> >> >>>> > (adding new button to gerrit is not an option - you
can image
> that
> >> >>>> > flag as
> >> >>>> > a trigger button), on success Jenkins CI flag will
turn to +2.
> on
> >> >>>> > fail
> >> >>>> > we'll get -1 and once new patchset is ready the
developer will
> >> >>>> > remove
> >> >>>> > the
> >> >>>> > +1 and add it back to the Continues Integration flag
to
> re-trigger
> >> >>>> > the
> >> >>>> > job.
> >> >>>> >
> >> >>>> > Please ack the process before we move on with that
> >> >>>>
> >> >>>> Sounds good, even though I'm a little scared (just gut
feeling, no
> >> >>>> evidence
> >> >>>> whatsoever) that this could add even more complexity and
> fragility to
> >> >>>> the jenkins
> >> >>>> fleet.
> >> >>>>
> >> >>>> In the long run, when this is reliable, it will help
greatly.
> >> >>>> In the short term, I'm scared because this can lead to
false
> >> >>>> positives
> >> >>>> and bogus
> >> >>>> failures.
> >> >>>>
> >> >>>> Let me stress I don't have concrete item to share or
specific
> flaws.
> >> >>>>
> >> >>>> As action item on me, I will find some time next week to
check
> virt
> >> >>>> functional tests,
> >> >>>> to see if they need some fixes, work reliably and so forth
> >> >>>>
> >> >>>> > The patch for those scripts still under review and
testing -
> >> >>>> > https://gerrit.ovirt.org/#/c/48268
> >> >>>>
> >> >>>> Will review asap.
> >> >>>>
> >> >>>> --
> >> >>>> Francesco Romani
> >> >>>> RedHat Engineering Virtualization R & D
> >> >>>> Phone: 8261328
> >> >>>> IRC: fromani
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Eyal Edri
> >> >>> Supervisor, RHEV CI
> >> >>> EMEA ENG Virtualization R&D
> >> >>> Red Hat Israel
> >> >>>
> >> >>> phone: +972-9-7692018
> >> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Yaniv Bronhaim.
> >
> >
> >
> >
> > --
> > Yaniv Bronhaim.
> >
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
>
--
*Yaniv Bronhaim.*
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
--
Eyal Edri
Associate Manager
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
10:53 a.m.
On 31 January 2016 at 14:15, Eyal Edri <eedri(a)redhat.com> wrote:
Adding lago-devel.
Anyone from Lago project can help debug why it is taking so long to setup
Lago (1:17 hours?)
Simple, Lago downloads some big images and does a full sync of some
large package repos...
Most of that should be a lot faster the 2nd time around, and also
there is some work being done to shrink down the images.
@Yaniv, you did the test on a machine that didn't run Lago before and
was in TLV while using the repo in PHX right?
--
Barak Korren
bkorren(a)redhat.com
RHEV-CI Team
3:07 p.m.
----_com.boxer.email_701005613951540
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
SSBoYXZlIGFuIHVuc3RhYmxlIHRlbXBvcmFyeSByZXBvIHdpdGggZWw2LCBlbDcsIGZjMjIgYW5k
IGZjMjMgaW1hZ2VzIHRoYXQgd2VpZ2h0IH4gMzAwIG1iIGFuZCBhcmUgdXBkYXRlZCB0byBsYXRl
c3QgcGFja2FnZXMsIHdvcmtpbmcgb24gZ2V0IGl0IHByb3Blcmx5IGdlbmVyYXRlZCBhbmQgYmUg
YWJsZSB0byByZXBsYWNlIGFsbCB0aGUgY3VycmVudCBpbWFnZXMsIHRoYXQgc2hvdWxkIGVhc2Ug
dGhlIGZpcnN0IGluaXQsIGJ1dCB0aGUgcmVwb3NldHVwIGlzIHN0aWxsIGFuIGlzc3VlLCB5b3Ug
bWlnaHQgY29uc2lkZXIgbm90IGRvaW5nIGl0IGlmIHlvdSBoYXZlIGludGVybmV0IGNvbm5lY3Rp
b24gYW5kIGRvbid0IGNhcmUgc2xvd2luZyBldmVyIHJ1biBhIGJpdCAod2hhdCBpdCB0YWtlcyB0
byBkb3dubG9hZCBmcm9uIHRoZSByZXBvcyB3aGF0ZXZlciB5b3Ugd2FudCB0byBpbnN0YWxsKS4K
CldlIGFyZSBhbHNvIGxvb2tpbmcgaW50IGJlaW5nIGFibGUgdG8gZ2VuZXJhdGUgdGVtcGxhdGVz
IGxvY2FsbHkgYW5kL29yIHVwbG9hZCB0aGVtIHRvIHRoZSByZXBvcyAoc28geW91IGluc3RhbGwg
ZXZlcnl0aGluZyB5b3UgbmVlZCBvbmNlLCBhbmQgdGhlbiBqdXN0IHJldXNlIHRoZSBsb2NhbGx5
IGdlbmVyYXRlZCBpbWFnZSkKCkRhdmlkIENhcm8KCkVsIDMxLzEvMjAxNiAxNzo1NCwgQmFyYWsg
S29ycmVuIDxia29ycmVuQHJlZGhhdC5jb20+IGVzY3JpYmnDszoKCk9uIDMxIEphbnVhcnkgMjAx
NiBhdCAxNDoxNSwgRXlhbCBFZHJpIDxlZWRyaUByZWRoYXQuY29tPiB3cm90ZToKPiBBZGRpbmcg
bGFnby1kZXZlbC4KPiBBbnlvbmUgZnJvbSBMYWdvIHByb2plY3QgY2FuIGhlbHAgZGVidWcgd2h5
IGl0IGlzIHRha2luZyBzbyBsb25nIHRvIHNldHVwCj4gTGFnbyAoMToxNyBob3Vycz8pCj4KU2lt
cGxlLCBMYWdvIGRvd25sb2FkcyBzb21lIGJpZyBpbWFnZXMgYW5kIGRvZXMgYSBmdWxsIHN5bmMg
b2Ygc29tZQpsYXJnZSBwYWNrYWdlIHJlcG9zLi4uCk1vc3Qgb2YgdGhhdCBzaG91bGQgYmUgYSBs
b3QgZmFzdGVyIHRoZSAybmQgdGltZSBhcm91bmQsIGFuZCBhbHNvCnRoZXJlIGlzIHNvbWUgd29y
ayBiZWluZyBkb25lIHRvIHNocmluayBkb3duIHRoZSBpbWFnZXMuCgpAWWFuaXYsIHlvdSBkaWQg
dGhlIHRlc3Qgb24gYSBtYWNoaW5lIHRoYXQgZGlkbid0IHJ1biBMYWdvIGJlZm9yZSBhbmQKd2Fz
IGluIFRMViB3aGlsZSB1c2luZyB0aGUgcmVwbyBpbiBQSFggcmlnaHQ/CgotLSAKQmFyYWsgS29y
cmVuCmJrb3JyZW5AcmVkaGF0LmNvbQpSSEVWLUNJIFRlYW0KX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX18KSW5mcmEgbWFpbGluZyBsaXN0CkluZnJhQG92aXJ0
Lm9yZwpodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vaW5mcmEKCg==
----_com.boxer.email_701005613951540
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
PHAgZGlyPSJsdHIiPkkgaGF2ZSBhbiB1bnN0YWJsZSB0ZW1wb3JhcnkgcmVwbyB3aXRoIGVsNiwg
ZWw3LCBmYzIyIGFuZCBmYzIzIGltYWdlcyB0aGF0IHdlaWdodCB+IDMwMCBtYiBhbmQgYXJlIHVw
ZGF0ZWQgdG8gbGF0ZXN0IHBhY2thZ2VzLCB3b3JraW5nIG9uIGdldCBpdCBwcm9wZXJseSBnZW5l
cmF0ZWQgYW5kIGJlIGFibGUgdG8gcmVwbGFjZSBhbGwgdGhlIGN1cnJlbnQgaW1hZ2VzLCB0aGF0
IHNob3VsZCBlYXNlIHRoZSBmaXJzdCBpbml0LCBidXQgdGhlIHJlcG9zZXR1cCBpcyBzdGlsbCBh
biBpc3N1ZSwgeW91IG1pZ2h0IGNvbnNpZGVyIG5vdCBkb2luZyBpdCBpZiB5b3UgaGF2ZSBpbnRl
cm5ldCBjb25uZWN0aW9uIGFuZCBkb24ndCBjYXJlIHNsb3dpbmcgZXZlciBydW4gYSBiaXQgKHdo
YXQgaXQgdGFrZXMgdG8gZG93bmxvYWQgZnJvbiB0aGUgcmVwb3Mgd2hhdGV2ZXIgeW91IHdhbnQg
dG8gaW5zdGFsbCkuPC9wPgo8cCBkaXI9Imx0ciI+V2UgYXJlIGFsc28gbG9va2luZyBpbnQgYmVp
bmcgYWJsZSB0byBnZW5lcmF0ZSB0ZW1wbGF0ZXMgbG9jYWxseSBhbmQvb3IgdXBsb2FkIHRoZW0g
dG8gdGhlIHJlcG9zIChzbyB5b3UgaW5zdGFsbCBldmVyeXRoaW5nIHlvdSBuZWVkIG9uY2UsIGFu
ZCB0aGVuIGp1c3QgcmV1c2UgdGhlIGxvY2FsbHkgZ2VuZXJhdGVkIGltYWdlKTwvcD4KPHAgZGly
PSJsdHIiPkRhdmlkIENhcm88L3A+CjxkaXYgY2xhc3M9InF1b3RlIj5FbCAzMS8xLzIwMTYgMTc6
NTQsIEJhcmFrIEtvcnJlbiAmbHQ7YmtvcnJlbkByZWRoYXQuY29tJmd0OyBlc2NyaWJpw7M6PGJy
IHR5cGU9J2F0dHJpYnV0aW9uJz48YmxvY2txdW90ZSBjbGFzcz0icXVvdGUiIHN0eWxlPSJtYXJn
aW46MCAwIDAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4
Ij48cCBkaXI9Imx0ciI+T24gMzEgSmFudWFyeSAyMDE2IGF0IDE0OjE1LCBFeWFsIEVkcmkgJmx0
O2VlZHJpQHJlZGhhdC5jb20mZ3Q7IHdyb3RlOiYjMTM7PGJyPgomZ3Q7IEFkZGluZyBsYWdvLWRl
dmVsLiYjMTM7PGJyPgomZ3Q7IEFueW9uZSBmcm9tIExhZ28gcHJvamVjdCBjYW4gaGVscCBkZWJ1
ZyB3aHkgaXQgaXMgdGFraW5nIHNvIGxvbmcgdG8gc2V0dXAmIzEzOzxicj4KJmd0OyBMYWdvICgx
OjE3IGhvdXJzPykmIzEzOzxicj4KJmd0OyYjMTM7PGJyPgpTaW1wbGUsIExhZ28gZG93bmxvYWRz
IHNvbWUgYmlnIGltYWdlcyBhbmQgZG9lcyBhIGZ1bGwgc3luYyBvZiBzb21lJiMxMzs8YnI+Cmxh
cmdlIHBhY2thZ2UgcmVwb3MuLi4mIzEzOzxicj4KTW9zdCBvZiB0aGF0IHNob3VsZCBiZSBhIGxv
dCBmYXN0ZXIgdGhlIDJuZCB0aW1lIGFyb3VuZCwgYW5kIGFsc28mIzEzOzxicj4KdGhlcmUgaXMg
c29tZSB3b3JrIGJlaW5nIGRvbmUgdG8gc2hyaW5rIGRvd24gdGhlIGltYWdlcy4mIzEzOzxicj4K
JiMxMzs8YnI+CkBZYW5pdiwgeW91IGRpZCB0aGUgdGVzdCBvbiBhIG1hY2hpbmUgdGhhdCBkaWRu
J3QgcnVuIExhZ28gYmVmb3JlIGFuZCYjMTM7PGJyPgp3YXMgaW4gVExWIHdoaWxlIHVzaW5nIHRo
ZSByZXBvIGluIFBIWCByaWdodD8mIzEzOzxicj4KJiMxMzs8YnI+Ci0tICYjMTM7PGJyPgpCYXJh
ayBLb3JyZW4mIzEzOzxicj4KYmtvcnJlbkByZWRoYXQuY29tJiMxMzs8YnI+ClJIRVYtQ0kgVGVh
bSYjMTM7PGJyPgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
XyYjMTM7PGJyPgpJbmZyYSBtYWlsaW5nIGxpc3QmIzEzOzxicj4KSW5mcmFAb3ZpcnQub3JnJiMx
Mzs8YnI+Cmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby9pbmZyYSYjMTM7
PGJyPgo8L3A+CjwvYmxvY2txdW90ZT48L2Rpdj4=
----_com.boxer.email_701005613951540--
3007
days inactive
3059
days old
11 comments
8 participants
participants (8)
-
Barak Korren -
David Caro Estevez -
Eyal Edri -
Francesco Romani -
Nir Soffer -
Petr Horacek -
Piotr Kliczewski -
Yaniv Bronheim