- Devel - oVirt List Archives

[ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ]
by Daniel Belenky 15 Mar '18

15 Mar '18

Hi, The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/88738/2 Link to failed build: here <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1181/> Link to all logs: here <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1181/artifact/ex…> Error snippet from engine.log <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1181/artifact/ex…> : 2018-03-11 06:10:58,521-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is not supported.] Thanks, -- DANIEL BELENKY RHV DEVOPS

6 16

[ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ]
by Dafna Ron 14 Mar '18

14 Mar '18

Hi, CQ reported failure on both basic and upgrade suites in otpi project. *Link and headline of suspected patches: * *core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ <https://gerrit.ovirt.org/#/c/87276/>Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/>Link to all logs:* http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/artifact… *(Relevant) error snippet from the log: <error> at org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.lambda$executeCommand$2(AddVdsCommand.java:217) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_161] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)2018-03-13 09:22:40,287-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [7e847b89] Error during deploy dialog2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] Entered SsoRestApiAuthFilter2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating with sso2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter successfully authenticated using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] Entered SsoRestApiNegotiationFilter2018-03-13 09:22:40,292-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth2018-03-13 09:22:40,297-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-basic-suite-master-host-1: Unexpected error during execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory.2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install, preferring first exception: Unexpected connection termination2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', 'lago-basic-suite-master-host-1': Unexpected connection termination</error>*

4 4

ovirt-system-tests hackathon report
by Sandro Bonazzola 13 Mar '18

13 Mar '18

4 people accepted calendar invite: - Devin A. Bougie - Francesco Romani - Jiri Belka - suporte, logicworks 4 people tentatively accepted calendar invite: - Amnon Maimon - Andreas Bleischwitz - Arnaud Lauriou - Stephen Pesini 2 mailing lists accepted calendar invite: users(a)ovirt.org, devel(a)ovirt.org (don't ask me how) so I may have missed someone in above list 4 patches got merged: Add check for host update to the 1st host. <https://gerrit.ovirt.org/88767> Merged Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (add_upgrade_check) <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branc…> 4:10 PM basic-suite-master: add vnic_profile_mappings to register vm <https://gerrit.ovirt.org/87438> Merged Eitan Raviv <https://gerrit.ovirt.org/#/q/owner:eraviv%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (register-template-vnic-mapping) <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branc…> 2:50 PM Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" <https://gerrit.ovirt.org/88881> Merged Eyal Edri <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branc…> 11:36 AM seperate 4.2 tests and utils from master <https://gerrit.ovirt.org/88878> Merged Eyal Edri <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branc…> 11:35 AM 13 patches has been pushed / reviewed / rebased Add gdeploy to ovirt-4.2.repo <https://gerrit.ovirt.org/88929> Daniel Belenky <https://gerrit.ovirt.org/#/q/owner:dbelenky%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 4:53 PM Cleanup of test code - next() replaced with any() <https://gerrit.ovirt.org/88928> Martin Sivák <https://gerrit.ovirt.org/#/q/owner:msivak%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 4:51 PM Add network queues custom property and use it in the vnic profile for VM0 <https://gerrit.ovirt.org/88829> Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (multi_queue_config) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 4:49 PM new suite: he-basic-iscsi-suite-master <https://gerrit.ovirt.org/85838> Yuval Turgeman <https://gerrit.ovirt.org/#/q/owner:yturgema%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (he-basic-iscsi-suite-master) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 4:47 PM Collect host-deploy bundle from the engine <https://gerrit.ovirt.org/88925> Yedidyah Bar David <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 4:41 PM network-suite-master: Make openstack_client_config fixture available to all ... <https://gerrit.ovirt.org/88029> Merge Conflict Marcin Mirecki <https://gerrit.ovirt.org/#/q/owner:mmirecki%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 3:39 PM new suite: he-basic-ng-ansible-suite-master <https://gerrit.ovirt.org/88901> Sandro Bonazzola <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (he-basic-ng-ansible-suite-master) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 3:37 PM Enable and move additional tests to 002 <https://gerrit.ovirt.org/88711> Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master (move_more_to_002) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 3:08 PM common: ovirt-4.2.repo <https://gerrit.ovirt.org/88904> Sandro Bonazzola <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 2:34 PM networking: Introducing test_stateless_vm_duplicate_mac_addr_vnic_does_not_be... <https://gerrit.ovirt.org/86946> Leon Goldberg <https://gerrit.ovirt.org/#/q/owner:lgoldber%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 2:08 PM hc: Updating gdeploy conf to create vdo volumes <https://gerrit.ovirt.org/88899> Sahina Bose <https://gerrit.ovirt.org/#/q/owner:sabose%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 12:55 PM master: add USB to the sanity VM <https://gerrit.ovirt.org/88731> Michal Skrivanek <https://gerrit.ovirt.org/#/q/owner:michal.skrivanek%2540redhat.com+status:o…> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 12:54 PM Test hosted-engine cleanup <https://gerrit.ovirt.org/88331> Yedidyah Bar David <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:…> 9:39 AM Feedback from the event: - "if we want to add many more tests to OST, and I think we do, we need to do some change there to allow that. Current framework is simply not scalable enough" - not joining the hackathon because "I'd be like an elephant in a porcelain shop" - "I'm not sure I'm OK with the flood of suites that we have - the more we have, the harder it is to sync and maintain but more importantly - to run." - "We can't keep adding new suite for each parameter we want to test, it adds overhead to monitoring, resources and maintenance." - invite wasn't clear enough. I found people on #ovirt on Freenode and on Red Hat IRC servers and redirected them to OFTC IRC server (my fault, hopefully managed to workaround it by talking to people) Lessons learned: - Calendar invites to mailing lists doesn't work well, need a different way to track mailing list members joining the events. - Invites needs to be pedantic on how to join the event, not leaving space for interpretation and misunderstanding. - We need a contribution guide to ovirt-system-test: we need to make people comfortable in trying to add a new test and we need to ensure that we won't reject a day of work because the patch doesn't match core contributors plannings on number of suites, resources and so on - The ovirt-system-tests check patch script is not good enough. It triggers too many sequential suites on every single patch pushed, and fails due to timeout taking more than 6 hours to complete. - The way ovirt-system-test collects rpms from defined repos is not smart enough: it doesn't take the latest version of a given package, just the first found in sequential order of the repos, Thanks everyone who participated to the event! if you have time please continue improving ovirt-system-test even if today event is almost completed! -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

4 3

Heads up: dropping fcraw jobs from jenkins
by Sandro Bonazzola 13 Mar '18

13 Mar '18

Following today call, I'm going to push a patch for dropping all rawhide jobs from jenkins. This should help avoiding distractions and reducing patches blocked by rawhide issues. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

5 8

qemu-kvm-ev-2.9.0-16.el7_4.14.1 has been released
by Sandro Bonazzola 12 Mar '18

12 Mar '18

Hi, qemu-kvm-ev-2.9.0-16.el7_4.14.1 <http://cbs.centos.org/koji/buildinfo?buildID=22232> has been tagged for release and should land on mirrors.centos.org on Monday, March 12th 2018. Here's the ChangeLog: * Thu Mar 08 2018 Sandro Bonazzola <sbonazzo(a)redhat.com> - ev-2.9.0-16.el7_4.14.1 - Removing RH branding from package name * Thu Jan 18 2018 Miroslav Rezanina <mrezanin(a)redhat.com> - rhev-2.9.0-16.el7_4.14 - kvm-fw_cfg-fix-memory-corruption-when-all-fw_cfg-slots-a.patch [bz#1534649] - kvm-mirror-Fix-inconsistent-backing-AioContext-for-after.patch [bz#1535125] - Resolves: bz#1534649 (Qemu crashes when all fw_cfg slots are used [rhel-7.4.z]) - Resolves: bz#1535125 (Mirror jobs for drives with iothreads make QEMU to abort with "block.c:1895: bdrv_attach_child: Assertion `bdrv_get_aio_context(parent_bs) == bdrv_get_aio_context(child_bs)' failed." [rhel-7.4.z]) Regards, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

OST Failure - Weekly update [03/03/2018-09/03/2018]
by Dafna Ron 09 Mar '18

09 Mar '18

Hello, I would like to update on this week's failures and OST current status. We had a few failure this week: 1. A change in OST caused a failure in one of the tests. this was due to using a 4.3 SDK feature which is not available in the current SDK version. The change was https://gerrit.ovirt.org/#/c/84338 - https://gerrit.ovirt.org/#/c/84338/ We reverted the change: https://gerrit.ovirt.org/#/c/88441/ However, this merge actually exposed an issue in the vdsm logs which report guest agent errors. bug was opened: https://bugzilla.redhat.com/show_bug.cgi?id=1553893 2. We discovered that there was a syntax error in 'release_branches' in ovirt-ansible. we re-merged after fixing the issue - Thanks you Barak/Sandro and Ondra for investigating and fixing the issue. This solved issues that were caused by running tests from stable older ansible packages. 3. fcraw build-artifacts jobs were removed since they were causing changes to fail running on cq. it was decided that its better to disable those jobs for now. 4. we have a second failure in build-artifacts for fc27 which caused several changes that were merged in cockpit-ovirt to fail. Thank you Douglas for acting quickly to resolve the issue. *Below you can see the chart for this week's resolved issues but cause of failure: * Code = regression of working components/functionalities Configurations = package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna

2 1

[ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ]
by Daniel Belenky 09 Mar '18

09 Mar '18

Hi, The following test failed OST: 004_basic_sanity.disk_operations. Link to suspected patch: https://gerrit.ovirt.org/c/88404/ Link to the failed job: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/ Link to all test logs: - engine <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/artifact/ex…> - host 0 <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/artifact/ex…> - host 1 <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/artifact/ex…> Error snippet from engine: 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. Error snippet from host: Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected Thanks, -- DANIEL BELENKY RHV DEVOPS

6 7

Updated invitation: oVirt System Test Hackathon @ Tue Mar 13, 2018 (devel@ovirt.org)
by sbonazzo＠redhat.com 08 Mar '18

08 Mar '18

This event has been changed. Title: oVirt System Test Hackathon Please join us in an ovirt-system-tests hackathon pushing new tests and improving existing ones for testing Hosted Engine.Git repo is available: https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=summaryIntegrati…, Node and CI team will be available for helping in the effort and reviewing patches.Here's a public trello board tracking the efforts: https://trello.com/b/Pp76YoRL (changed) When: Tue Mar 13, 2018 Where: #ovirt IRC channel Calendar: devel(a)ovirt.org Who: * sbonazzo(a)redhat.com - organizer * devel(a)ovirt.org * users(a)ovirt.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=MDhuZTY5bXRxbGo0cnNmM… Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account devel(a)ovirt.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding

1 0

Invitation: oVirt System Test Hackathon @ Tue Mar 13, 2018 (devel@ovirt.org)
by sbonazzo＠redhat.com 08 Mar '18

08 Mar '18

You have been invited to the following event. Title: oVirt System Test Hackathon Please join us in an ovirt-system-tests hackathon pushing new tests and improving existing ones for testing Hosted Engine. Git repo is available: https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=summary Integration, Node and CI team will be available for helping in the effort and reviewing patches. When: Tue Mar 13, 2018 Where: #ovirt IRC channel Calendar: devel(a)ovirt.org Who: * sbonazzo(a)redhat.com - organizer * devel(a)ovirt.org * users(a)ovirt.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=MDhuZTY5bXRxbGo0cnNmM… Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account devel(a)ovirt.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding

1 0

he-basic-ansible-suite-master and he-basic-suite-master
by Sandro Bonazzola 08 Mar '18

08 Mar '18

Hi, I'm looking at diffs between he-basic-ansible-suite-master and he-basic-suite-master in oVirt Sytem Test. Since we are now defaulting to the ansible deployment, aren't these 2 suites substantially testing the same flow? Can we drop one of them? Or should we change the non ansible suite to enforce using legacy otopi flow? Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

3 2