- Devel - oVirt List Archives

[Ovirt] [CQ weekly status] [08-03-2019]
by Dafna Ron 09 Mar '19

09 Mar '19

Hi, This mail is to provide the current status of CQ and allow people to review status before and after the weekend. Please refer to below colour map for further information on the meaning of the colours. *CQ-4.2*: GREEN (#1) Last CQ job failure on 4.2 was 06-03-2019 on project ovirt-ansible-vm-infra due to missing jtk2 package. this issue is now fixed with patch: https://gerrit.ovirt.org/#/c/98304/ *CQ-4.3*: RED (#1) Last job to fail is for project imgbased. the failure is due to the random failures we have been having from test add_vm2_lease. this issue is currently debated in the list. New repo and packages were updated by Galit due to failure in CQ for missing packages: https://gerrit.ovirt.org/#/c/98312/ *CQ-Master:* RED (#1) We were failing this week due to missing packages which was fixed by: https://gerrit.ovirt.org/#/c/98304/ We also had two regressions which were fixed by: https://gerrit.ovirt.org/#/c/98317/ and by https://gerrit.ovirt.org/#/c/98371/ All projects are currently passing Current running jobs for 4.2 [1] and master [2] can be found here: [1] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-qu… [2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change… Happy week! Dafna ------------------------------------------------------------------------------------------------------------------- COLOUR MAP Green = job has been passing successfully ** green for more than 3 days may suggest we need a review of our test coverage 1. 1-3 days GREEN (#1) 2. 4-7 days GREEN (#2) 3. Over 7 days GREEN (#3) Yellow = intermittent failures for different projects but no lasting or current regressions ** intermittent would be a healthy project as we expect a number of failures during the week ** I will not report any of the solved failures or regressions. 1. Solved job failures YELLOW (#1) 2. Solved regressions YELLOW (#2) Red = job has been failing ** Active Failures. The colour will change based on the amount of time the project/s has been broken. Only active regressions would be reported. 1. 1-3 days RED (#1) 2. 4-7 days RED (#2) 3. Over 7 days RED (#3)

1 1

ovirt and AMD virtualization bug
by Hetz Ben Hamo 09 Mar '19

09 Mar '19

Hi, I've done some research on a bug which I found on the AMD Zen/Zen+ based CPU's. Apparently, when using nested virtualization, the CPU doesn't expose the "monitor" CPU flag. While other virtualization solutions (Xen, ESXi, Hyper-V) don't care about this issue and let you run with nested without any problem, oVirt doesn't let you launch or create any VM and it complains that this "monitor" CPU flag is missing. Since QEMU/KVM cannot do anything about it, I was wondering if the ovirt dev team could ignore this flag please. Thanks

2 1

[VDSM] build-artifacts on s390x broken again (bad mock/dnf cache?)
by Nir Soffer 09 Mar '19

09 Mar '19

build-artifacts on s390x fail now with: [Errno 2] No such file or directory: '/var/cache/dnf/updates-07b7057ee4fded96/packages/systemd-238-12.git07f8cd5.fc28.s390x.rpm' Here are few example failures: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… This is going to block the change queue when we merge these patches, and we have 4.3 build soon. We had this issue few week ago, and Barak fixed this by deleting caches on the slaves. I hope this can be fixed quickly this time. Nir

1 0

ovirt-optimizer: subproject health check
by Sandro Bonazzola 08 Mar '19

08 Mar '19

Hi, oVirt Optimizer has been introduced in oVirt 3.5.2 back in 2014. Last update on master branch is dated November 2017 with version 0.15 The package has been shipped till oVirt 4.2.8 and it has not been shipped in oVirt 4.3 despite no official deprecation has been announced. According to Gerrit, current maintainers are Doron Fediuck and Martin Sivak. According to Bugzilla there is one bug open since February 2018: Bug 1545077 <https://bugzilla.redhat.com/show_bug.cgi?id=1545077> - cancel optimization button is enabled all the time which is currently untargeted, unassigned and without activity since March 2018. The project has feature / development documentation at https://ovirt.org/develop/release-management/features/sla/optaplanner.html but no official documentation. Can we declare this subproject retired from oVirt project and drop CI automation for it? Thanks, -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

1 0

ovirt-scheduler-proxy: subproject health check
by Sandro Bonazzola 08 Mar '19

08 Mar '19

Hi, oVirt Scheduler Proxy has been introduced in oVirt 3.3.1 back in 2013. Last update on master branch is dated November 2017 with 0.1.8 released. The package has been shipped till oVirt 4.2.8 and it has not been shipped in oVirt 4.3 despite no official deprecation has been announced. According to Gerrit, current maintainers are Doron Fediuck and Martin Sivak. According to Bugzilla there are no open bugs or RFEs for this project. The project has no documentation available on oVirt website. Can we declare this subproject retired from oVirt project and drop CI automation for it? Thanks, -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

2 2

[ci test tool] how to start build-artifacts job using jenkins API?
by Nir Soffer 08 Mar '19

08 Mar '19

I'm trying to a write a tool that given a gerrit patch number, will: 1. Start build artifacts job (instead of adding "ci please build" comment)\ 2. Wait until the job is complete 3. Start OST with CUSTOM_REPOS using the build artifacts job from step 1 4. Wait until OST complets 5. Post a comment with the OST link to gerrit So testing patches will easy as: $ ci test 12345 Building rpms... Starting OST... OST succeeded, congratulations! I can start OST and watch it, but I could not find a way to start vdsm build-artifacts job. This starts a job: curl -i \ --user USERNAME:API_TOKEN \ -X POST \ https://jenkins.ovirt.org/job/vdsm_standard-check-patch/build But the fail quickly in the first "detecting" stage. https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… Clicking "Build Now" button on the job page fails in the same way. And this: curl -i \ --user USERNAME:API_TOKEN \ -X POST \ https://jenkins.ovirt.org/job/vdsm_standard-check-patch/buildWithParameters Fails with 500 Internal server error, with a huge traceback, hiding this expected message: java.lang.IllegalStateException: This build is not parameterized! Looks like the job need to be parametrized: - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/configure - https://jenkins.ovirt.org/job/ovirt-system-tests_manual/configure I guess we can workaround this by posting "ci please build" comment on gerrit using gerrit API, and then waiting waiting an grabbing the build link from the comment added by jenkins, but I really don't want to go in this direction. I hope we can have a proper solution in the job configuration. Nir

1 1

[ OST Failure Report ] [ oVirt Master ] [ 07-03-2019 ] [ 008_basic_ui_sanity.initialize_firefox]
by Galit Rosenthal 07 Mar '19

07 Mar '19

Hi, we are failing basic suite master (vdsm) Can you please have a look at the issue? *Error Message:* Message: Error forwarding the new session new session request for webdriver should contain a location header or an 'application/json;charset=UTF-8' response body with the session ID. *from the [2]:* *08:22:04* + xmllint --format /dev/shm/ost/deployment-basic-suite-master/008_basic_ui_sanity.py.junit.xml*08:22:04* <?xml version="1.0" encoding="UTF-8"?>*08:22:04* <testsuite name="nosetests" tests="7" errors="1" failures="0" skip="0">*08:22:04* <testcase classname="008_basic_ui_sanity" name="init" time="0.001"/>*08:22:04* <testcase classname="008_basic_ui_sanity" name="start_grid" time="144.413">*08:22:04* <system-out><![CDATA[executing shell: docker ps*08:22:04* CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES*08:22:04* *08:22:04* *08:22:04* executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub*08:22:04* *08:22:04* Error response from daemon: No such container: grid_node_chrome*08:22:04* Error response from daemon: No such container: grid_node_firefox*08:22:04* Error response from daemon: No such container: selenium-hub*08:22:04* *08:22:04* executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub*08:22:04* *08:22:04* Error response from daemon: Cannot kill container grid_node_chrome: No such container: grid_node_chrome*08:22:04* Error response from daemon: Cannot kill container grid_node_firefox: No such container: grid_node_firefox*08:22:04* Error response from daemon: Cannot kill container selenium-hub: No such container: selenium-hub*08:22:04* *08:22:04* executing shell: docker ps*08:22:04* CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES*08:22:04* *08:22:04* *08:22:04* executing shell: docker network ls*08:22:04* NETWORK ID NAME DRIVER SCOPE*08:22:04* f710c07f46da bridge bridge local*08:22:04* 7d05935c9513 host host local*08:22:04* 415e4fd8022f none null local*08:22:04* *08:22:04* *08:22:04* executing shell: docker network rm grid*08:22:04* *08:22:04* Error response from daemon: network grid not found*08:22:04* *08:22:04* executing shell: docker network ls*08:22:04* NETWORK ID NAME DRIVER SCOPE*08:22:04* f710c07f46da bridge bridge local*08:22:04* 7d05935c9513 host host local*08:22:04* 415e4fd8022f none null local*08:22:04* *08:22:04* *08:22:04* executing shell: docker network ls*08:22:04* NETWORK ID NAME DRIVER SCOPE*08:22:04* f710c07f46da bridge bridge local*08:22:04* 7d05935c9513 host host local*08:22:04* 415e4fd8022f none null local*08:22:04* *08:22:04* *08:22:04* creating docker network for grid*08:22:04* executing shell: docker network create grid*08:22:04* 7406c15c141eccac968e5fba9f091d83831a29c92fc9f61b95bc4fba47f5e73f*08:22:04* *Global link to the CQ fail[3]* [1] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_chang… [2] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_chang… [3] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_chang… -- GALIT ROSENTHAL SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit(a)gmail.com T: 972-9-7692230 <https://red.ht/sig>

4 3

[ OST Failure Report ] [ oVirt Master ] [ 07-03-2019 ] [ / 004_basic_sanity.verify_glance_import]
by Galit Rosenthal 07 Mar '19

07 Mar '19

Hi, we are failing basic suite master ( ovirt-engine) It looks a problem in ovirt-engine Can you please have a look at the issue? Thanks, Galit Error [1]: 'NoneType' object has no attribute 'status' -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f5e9feb3ed8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 1153, in <lambda> lambda: api.disks.get(disk_name).status.state == 'ok', AttributeError: 'NoneType' object has no attribute 'status' --------------------- >> end captured logging << ------------- US CQ of the ovirt-engine results can be found [2] We see the same error DS as well [3] [1] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_chang… [2] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_chang… [3] https://rhv-devops-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/blue/rest/organi… -- GALIT ROSENTHAL SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit(a)gmail.com T: 972-9-7692230 <https://red.ht/sig>

5 9

Please don't remove instance type
by Baptiste Agasse 07 Mar '19

07 Mar '19

Hi all, We are happy oVirt users for some years now (we started with 3.6, now on 4.2) and we manage most of our virtualization stacks with it. To provision and manage our machines, we use the foreman (for bare metal and virtual machines) on top of it. I made some little contributions to the foreman and other underlying stuff to have a deeper integration with oVirt, like to be able to select instance type directly from foreman interface/api and we rely on it. We use instance types to standardize our vms by defining system resources (memory, cpu and cpu topology) console type, boot options. On top of that we plan to use templates to apply OS (CentOS 7 and CentOS 6 actually). Having resources definitions separated from OS installation help us to keep instance types and templates lists small and don't bother users about some technical underlying stuff. As we are interested in automating oVirt maintenance tasks and configuration with ansible, I asked at FOSDEM oVirt booth if there is any ansible module to manage instance types in Ovirt as I didn't find it in ovirt ansible infra repo. The person to whom I asked the question said that you are planning to remove instance types from ovirt, and this make me sad :(. So here I am to ask why do you plan to remove instance types from oVirt. As far as I know, it's fairly common to have "instance types" / "flavors" / "sizes" on one side and then templates (bare OS, preinstalled appliances...) on other side and pick one of each to make an instance. If this first part is missing in future version of ovirt, it will be a pain point for us. So, my question is, do you really plan to remove instances type definitely ? Cheers. -- Baptiste

3 4

Taking down the engine without setting global maintenance
by Yedidyah Bar David 07 Mar '19

07 Mar '19

Hi all, How about making this change: Right before the engine goes down cleanly, it marks the shared storage saying it did not crash but exited cleanly, and then HE-HA will not try to restart it on another host. Perhaps make this optional, so that users can do clean shutdowns and still test HA cleanly (or some other use cases, where users might not want this). This should help a lot cases where people restarted their engine for some reason, e.g. upgrade, and forgot to set maintenance. Makes sense? -- Didi

3 5