May 2020 - Infra - oVirt List Archives

oVirt 4.4.0 Release is now generally available
by Sandro Bonazzola 20 May '20

20 May '20

oVirt 4.4.0 Release is now generally available The oVirt Project is excited to announce the general availability of the oVirt 4.4.0 Release, as of May 20th, 2020 This release unleashes an altogether more powerful and flexible open source virtualization solution that encompasses hundreds of individual changes and a wide range of enhancements across the engine, storage, network, user interface, and analytics, as compared to oVirt 4.3. Important notes before you install / upgrade Some of the features included in the oVirt 4.4.0 release require content that will be available in CentOS Linux 8.2 but cannot be tested on RHEL 8.2 yet due to some incompatibility in the openvswitch package that is shipped in CentOS Virt SIG, which requires rebuilding openvswitch on top of CentOS 8.2. The cluster switch type OVS is not implemented for CentOS 8 hosts. Please note that oVirt 4.4 only supports clusters and datacenters with compatibility version 4.2 and above. If clusters or datacenters are running with an older compatibility version, you need to upgrade them to at least 4.2 (4.3 is recommended). Please note that in RHEL 8 / CentOS 8 several devices that worked on EL7 are no longer supported. For example, megaraid_sas driver is removed. If you use Enterprise Linux 8 hosts you can try to provide the necessary drivers for the deprecated hardware using the DUD method (See users mailing list thread on this at https://lists.ovirt.org/archives/list/users@ovirt.org/thread/NDSVUZSESOXEFJ… ) Installation instructions For the engine: either use the oVirt appliance or install CentOS Linux 8 minimal by following these steps: - Install the CentOS Linux 8 image from http://centos.mirror.garr.it/centos/8.1.1911/isos/x86_64/CentOS-8.1.1911-x8… - dnf install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm - dnf update (reboot if needed) - dnf module enable -y javapackages-tools pki-deps postgresql:12 - dnf install ovirt-engine - engine-setup For the nodes: Either use oVirt Node ISO or: - Install CentOS Linux 8 from http://centos.mirror.garr.it/centos/8.1.1911/isos/x86_64/CentOS-8.1.1911-x8…, selecting the minimal installation. - dnf install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm - dnf update (reboot if needed) - Attach the host to the engine and let it be deployed. Update instructionsUpdate from oVirt 4.4 Release Candidate On the engine side and on CentOS hosts, you’ll need to switch from ovirt44-pre to ovirt44 repositories. In order to do so, you need to: 1. dnf remove ovirt-release44-pre 2. rm -f /etc/yum.repos.d/ovirt-4.4-pre-dependencies.repo 3. rm -f /etc/yum.repos.d/ovirt-4.4-pre.repo 4. dnf install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm 5. dnf update On the engine side you’ll need to run engine-setup only if you were not already on the latest release candidate. On oVirt Node, you’ll need to upgrade with: 1. Move node to maintenance 2. dnf install https://resources.ovirt.org/pub/ovirt-4.4/rpm/el8/noarch/ovirt-node-ng-imag… 3. Reboot 4. Activate the host Update from oVirt 4.3 oVirt 4.4 is available only for CentOS 8. In-place upgrades from previous installations, based on CentOS 7, are not possible. For the engine, use backup, and restore that into a new engine. Nodes will need to be reinstalled. A 4.4 engine can still manage existing 4.3 hosts, but you can’t add new ones. For a standalone engine, please refer to upgrade procedure at https://ovirt.org/documentation/upgrade_guide/#Upgrading_from_4-3 If needed, run ovirt-engine-rename (see engine rename tool documentation at https://www.ovirt.org/documentation/admin-guide/chap-Utilities.html ) When upgrading hosts: You need to upgrade one host at a time. 1. Turn host to maintenance. Virtual machines on that host should migrate automatically to a different host. 2. Remove it from the engine 3. Re-install it with el8 or oVirt Node as per installation instructions 4. Re-add the host to the engine Please note that you may see some issues live migrating VMs from el7 to el8. If you hit such a case, please turn off the vm on el7 host and get it started on the new el8 host in order to be able to move the next el7 host to maintenance. What’s new in oVirt 4.4.0 Release? - Hypervisors based on CentOS Linux 8 (rebuilt from award winning RHEL8), for both oVirt Node and standalone CentOS Linux hosts. - Easier network management and configuration flexibility with NetworkManager. - VMs based on a more modern Q35 chipset with legacy SeaBIOS and UEFI firmware. - Support for direct passthrough of local host disks to VMs. - Live migration improvements for High Performance guests. - New Windows guest tools installer based on WiX framework now moved to VirtioWin project. - Dropped support for cluster level prior to 4.2. - Dropped API/SDK v3 support deprecated in past versions. - 4K block disk support only for file-based storage. iSCSI/FC storage do not support 4K disks yet. - You can export a VM to a data domain. - You can edit floating disks. - Ansible Runner (ansible-runner) is integrated within the engine, enabling more detailed monitoring of playbooks executed from the engine. - Adding and reinstalling hosts is now completely based on Ansible, replacing ovirt-host-deploy, which is not used anymore. - The OpenStack Neutron Agent cannot be configured by oVirt anymore, it should be configured by TripleO instead. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.1 * CentOS Linux (or similar) 8.1 This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.1 * CentOS Linux (or similar) 8.1 * oVirt Node 4.4 based on CentOS Linux 8.1 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. If you manage more than one oVirt instance, OKD or RDO we also recommend to try ManageIQ <http://manageiq.org/>. In such a case, please be sure to take the qc2 image and not the ova image. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 Additional Resources: * Read more about the oVirt 4.4.0 release highlights: http://www.ovirt.org/release/4.4.0/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.0/ [2] http://resources.ovirt.org/pub/ovirt-4.4/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> [image: |Our code is open_] <https://www.redhat.com/en/our-code-is-open> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Re: [ovirt-devel] oVirt and Fedora
by Michal Skrivanek 20 May '20

20 May '20

> On 19 May 2020, at 14:06, Neal Gompa <ngompa13(a)gmail.com> wrote: > > On Mon, May 11, 2020 at 11:45 AM Michal Skrivanek > <michal.skrivanek(a)redhat.com> wrote: >> >> >> >>> On 11 May 2020, at 14:49, Neal Gompa <ngompa13(a)gmail.com> wrote: >>> >>> On Mon, May 11, 2020 at 8:32 AM Nir Soffer <nsoffer(a)redhat.com> wrote: >>>> >>>> On Mon, May 11, 2020 at 2:24 PM Neal Gompa <ngompa13(a)gmail.com> wrote: >>>>> >>>>> As far as the oVirt software keeping up with Fedora, the main problem here has always been that people aren't integrating their software into the distribution itself. >> >> it was never a good fit for oVirt to be part of other distributions. We had individual packages part of Fedora in history, but there are things which are hard to accept (like automatically enabling of installed services, UIDs), and overall it’s just too complex, we’re rather a distribution than a simple app on top of base OS. >> > > None of those things are hard to do in Fedora. They're incredibly easy > to do. I know this because I've gone through this process already > before. > > But fine, let's assume I consider this argument valid. Then there's > still no reason not to be continually providing support for Fedora as > an add-on, as you have before. the reason is mentioned in the original email, the lack of resources to keep actively supporting 3 different platforms. If you want to provide a helping hand and maintain Fedora infrastructure I don’t think anyone would object > >>>>> That's how everything can get tested together. And this comes back to the old bug about fixing vdsm so that it doesn't use /rhev, but instead something FHS-compliant (RHBZ#1369102). Once that is resolved, pretty much the entire stack can go into Fedora. And then you benefit from the Fedora community being able to use, test, and contribute to the oVirt project. As it stands, why would anyone do this for you when you don't even run on the cutting edge platform that feeds into Red Hat Enterprise Linux? >>>> >>>> This was actually fixed a long time ago. With this commit: >>>> https://github.com/oVirt/vdsm/commit/67ba9c4bc860840d6e103fe604b16f494f60a0… >>>> >>>> You can configure a compatible vdsm that does not use /rhev. >>>> >>>> Of course it is not backward compatible, for this we need much more >>>> work to support live migration >>>> between old and new vdsm using different data-center configurations. >>>> >>> >>> It'd probably be simpler to just *change* it to an FHS-compatible path >>> going forward with EL8 and Fedora and set up a migration path there, >>> but it's a bit late for that... :( >> >> It wouldn’t. We always support live migration across several versions (now it’s 4.2-4.4) and it needs to stay the same or youo have to go with arcane code to mangle it back and forth which gets a bit ugly when you consider suspend/resume, snapshots, etc >> > > Erk. At some point you need to bite the bullet though... it’s about capacity as well, it’s just a matter of someone writing a code which can handle the (long) transition period Thanks, michal

1 0

Build failed in Jenkins: system-mk_mirrors_index-yum #85291
by jenkins＠jenkins.phx.ovirt.org 20 May '20

20 May '20

See <https://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/85291/display/red…> Changes: ------------------------------------------ Started by upstream project "system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64" build number 630 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:909) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1131) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1856) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Could not resolve host: gerrit.ovirt.org; Unknown error at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2430) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2044) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:81) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:569) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to mirrors.phx.ovirt.org at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:998) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy90.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:907) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1131) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1856) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) ERROR: Error fetching remote repo 'origin'

1 1

oVirt infra daily report - unstable production jobs - 1196
by jenkins＠jenkins.phx.ovirt.org 20 May '20

20 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1196//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2943) Very slow 'git pull' for gerrit.ovirt.org:ovirt-engine
by Artur Socha (oVirt JIRA) 19 May '20

19 May '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2943?page=com.atlassian.jira.… ] Artur Socha commented on OVIRT-2943: ------------------------------------ It seems the problem is gone. Since yesterday evening I have not noticed lags. > Very slow 'git pull' for gerrit.ovirt.org:ovirt-engine > --------------------------------------------------------- > > Key: OVIRT-2943 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2943 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: oVirt Infra > Reporter: Artur Socha > Assignee: infra > > I have noticed that pulling updates for ovirt-engine takes very long now. In my case simple git pull took ~4minutes on master. > With GIT_TRACE=1 I observed low transfer speeds but even time to got to this point was unusually big. > Here is the git trace: https://pastebin.pl/view/ec457057 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

oVirt infra daily report - unstable production jobs - 1195
by jenkins＠jenkins.phx.ovirt.org 19 May '20

19 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1195//artifact/exported… Cheers, Jenkins

1 0

[Feedback needed] oVirt 4.4.0 Test week
by Sandro Bonazzola 18 May '20

18 May '20

Hi, oVirt team is planning to release oVirt 4.4.0 Ga in the next couple of weeks. oVirt 4.4.0 release candidate was released yesterday and we'd like to gather as much feedback as possible. Please join us testing this release candidate next week, starting Sunday May 10th 2020 till Friday May 15th 2020! We are going to coordinate the testing effort with a public Trello board at https://trello.com/b/5ZNJgPC3/ovirt-440-test-day You'll find instructions on how to use the board there. For joining the board you can use this link: https://trello.com/invite/b/5ZNJgPC3/f1b1826ee4902f348c44607765a15099/ovirt… If you have not an environment dedicated to testing, remember you can set up a few VMs and test the deployment with nested virtualization using your production environment creating a virtual test environment. In this case please be careful avoiding touching the production environment from the testing one. The oVirt team will monitor the Trello board, the #ovirt IRC channel on irc.oftc.net server and the users(a)ovirt.org mailing list to assist with the testing and debugging issues. Basic instructions for setting up a minimal system are available in release candidate announce at: https://lists.ovirt.org/archives/list/announce@ovirt.org/message/3QORBKVKTA… Release notes for this release candidate are available here: https://ovirt.org/release/4.4.0/ Thanks -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> [image: |Our code is open_] <https://www.redhat.com/en/our-code-is-open> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 1

[JIRA] (OVIRT-2943) Very slow 'git pull' for gerrit.ovirt.org:ovirt-engine
by Artur Socha (oVirt JIRA) 18 May '20

18 May '20

Artur Socha created OVIRT-2943: ---------------------------------- Summary: Very slow 'git pull' for gerrit.ovirt.org:ovirt-engine Key: OVIRT-2943 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2943 Project: oVirt - virtualization made easy Issue Type: Task Components: oVirt Infra Reporter: Artur Socha Assignee: infra I have noticed that pulling updates for ovirt-engine takes very long now. In my case simple git pull took ~4minutes on master. With GIT_TRACE=1 I observed low transfer speeds but even time to got to this point was unusually big. Here is the git trace: https://pastebin.pl/view/ec457057 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

[JIRA] (OVIRT-2942) [FIRING:1] MirrorIsOutOfSync (raw.githubusercontent.com:80 oVirt-mirrors-health http://ftp.snt.utwente.nl/pub/software/ovirt)
by Shlomi Zidmi (oVirt JIRA) 18 May '20

18 May '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2942?page=com.atlassian.jira.… ] Shlomi Zidmi reassigned OVIRT-2942: ----------------------------------- Assignee: Shlomi Zidmi (was: infra) > [FIRING:1] MirrorIsOutOfSync (raw.githubusercontent.com:80 oVirt-mirrors-health http://ftp.snt.utwente.nl/pub/software/ovirt) > ----------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-2942 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2942 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: Shlomi Zidmi > Labels: ALERT{alertname="MirrorIsOutOfSync"} > > Labels: > - alertname = MirrorIsOutOfSync > - instance = raw.githubusercontent.com:80 > - job = oVirt-mirrors-health > - url = http://ftp.snt.utwente.nl/pub/software/ovirt > Annotations: > - description = http://ftp.snt.utwente.nl/pub/software/ovirt (measured by oVirt-mirrors-health) has not been synced in the past two days. > Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

[JIRA] (OVIRT-2942) [FIRING:1] MirrorIsOutOfSync (raw.githubusercontent.com:80 oVirt-mirrors-health http://ftp.snt.utwente.nl/pub/software/ovirt)
by Alertmanager_Bot (oVirt JIRA) 18 May '20

18 May '20

Alertmanager_Bot created OVIRT-2942: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync (raw.githubusercontent.com:80 oVirt-mirrors-health http://ftp.snt.utwente.nl/pub/software/ovirt) Key: OVIRT-2942 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2942 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = raw.githubusercontent.com:80 - job = oVirt-mirrors-health - url = http://ftp.snt.utwente.nl/pub/software/ovirt Annotations: - description = http://ftp.snt.utwente.nl/pub/software/ovirt (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

[CQ]: 109011, 2 (vdsm) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 18 May '20

18 May '20

A system test invoked by the "ovirt-4.3" change queue including change 109011,2 (vdsm) failed. However, this change seems not to be the root cause for this failure. Change 108810,1 (vdsm) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 108810,1 (vdsm) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/109011/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/108810/1 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3445/

1 0

[JIRA] (OVIRT-2940) Stuck job - running 8 days
by Evgheni Dereveanchin (oVirt JIRA) 18 May '20

18 May '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2940?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-2940: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > Stuck job - running 8 days > -------------------------- > > Key: OVIRT-2940 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2940 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: Evgheni Dereveanchin > > I found this stuck job in jenkins: > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/5660/ > Don't we have a timeout for killing run away jobs? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

[CQ]: 108164, 7 (vdsm) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 18 May '20

18 May '20

A system test invoked by the "ovirt-4.3" change queue including change 108164,7 (vdsm) failed. However, this change seems not to be the root cause for this failure. Change 108810,1 (vdsm) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 108810,1 (vdsm) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/108164/7 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/108810/1 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3441/

1 0

[JIRA] (OVIRT-2941) resources.ovirt.org doesn't reply on https
by Evgheni Dereveanchin (oVirt JIRA) 18 May '20

18 May '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2941?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-2941: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > resources.ovirt.org doesn't reply on https > ------------------------------------------ > > Key: OVIRT-2941 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2941 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Sandro Bonazzola > Assignee: Evgheni Dereveanchin > > resources.ovirt.org doesn't reply on https; still reachable over ssh -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

[JIRA] (OVIRT-2941) resources.ovirt.org doesn't reply on https
by Sandro Bonazzola (oVirt JIRA) 18 May '20

18 May '20

Sandro Bonazzola created OVIRT-2941: --------------------------------------- Summary: resources.ovirt.org doesn't reply on https Key: OVIRT-2941 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2941 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Sandro Bonazzola Assignee: infra resources.ovirt.org doesn't reply on https; still reachable over ssh -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100126)

1 0

oVirt infra daily report - unstable production jobs - 1194
by jenkins＠jenkins.phx.ovirt.org 18 May '20

18 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1194//artifact/exported… Cheers, Jenkins

1 0

otopi and change-queue
by Yedidyah Bar David 17 May '20

17 May '20

Hi all, I recently merged a patch to otopi: https://gerrit.ovirt.org/#/c/108590/ It failed in CQ: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/23512/ Now commented there "ci re-merge", and it passed: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/23736/ But didn't run basic-suite-master, and I do not understand if it will be published. Was anything changed in how change-queue works? Should I do anything else to make sure it's published? Thanks, -- Didi

2 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 11 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 May '20

17 May '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/11/ Build Number: 11 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #11 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 005_network_by_label.assign_labeled_network Error Message: False != True after 180 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/005_network_by_label.py", line 149, in assign_labeled_network host_service, LABELED_NET_NAME)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds

1 1

oVirt infra daily report - unstable production jobs - 1193
by jenkins＠jenkins.phx.ovirt.org 17 May '20

17 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1193//artifact/exported… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1192
by jenkins＠jenkins.phx.ovirt.org 16 May '20

16 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1192//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1 - Failure!
by jenkins＠jenkins.phx.ovirt.org 15 May '20

15 May '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff91385dcd0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], ) > assert result.code == 0, \ 'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../basic-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff91367fc08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 3

oVirt infra daily report - unstable production jobs - 1191
by jenkins＠jenkins.phx.ovirt.org 15 May '20

15 May '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1191//artifact/exported… Cheers, Jenkins

1 0

[JENKINS] Failed to setup proejct vdsm_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 14 May '20

14 May '20

Failed to run project_setup.sh for: #21173 vdsm [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on vm0139.workers-phx.ovirt.org.

1 0

[CQ]: 109021, 2 (ovirt-engine) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 14 May '20

14 May '20

A system test invoked by the "ovirt-4.3" change queue including change 109021,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 109003,1 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 109003,1 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/109021/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/109003/1 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3431/

1 0

[CQ]: 109020, 2 (ovirt-engine) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 14 May '20

14 May '20

A system test invoked by the "ovirt-4.3" change queue including change 109020,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 109003,1 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 109003,1 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/109020/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/109003/1 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3430/

1 0