On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: > > Hi > > Seems that in the last few days the el8 CI for vdsm cannot run, were there any el8 repo changes? > > [2020-06-21T09:20:03.003Z] Mock Version: 2.2 > [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 > [2020-06-21T09:20:03.987Z] Start: dnf install > [2020-06-21T09:20:12.326Z] ERROR: Command failed: > [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot > /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ > --releasever 8 --setopt=deltarpm=False --allowerasing > --disableplugin=local --disableplugin=spacewalk > --disableplugin=local --disableplugin=spacewalk install dnf tar > gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc coreutils > unzip shadow-utils diffutils cpio bash gawk rpm-build info patch > util-linux findutils grep python36 autoconf automake createrepo dnf > dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils > libguestfs-tools-c lshw make openvswitch ovirt-imageio-common > python3-augeas python3-blivet python3-coverage python3-dateutil > python3-dbus python3-decorator python3-devel python3-dmidecode > python3-inotify python3-ioprocess-1.4.1 python3-libselinux > python3-libvirt python3-magic python3-netaddr python3-nose > python3-pip python3-policycoreutils python3-pyyaml python3-requests > python3-sanlock python3-six python3-yaml rpm-build rpmlint sanlock > sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts > [2020-06-21T09:20:12.326Z] No matches found for the following > disable plugin patterns: local, spacewalk > [2020-06-21T09:20:12.326Z] Last metadata expiration check: 0:00:02 ago on Sun Jun 21 09:20:07 2020. > [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch > > Taken from https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText I think this is a result of trying to move to CentOS 8.2 before all the packages are ready.

Please use Travis until this issue is resolved. You should use it anyway for storage patches. Nir

On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: > Nir Soffer <nsoffer(a)redhat.com&gt; writes: > > > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: > >> > >> Hi > > > >> > >> Seems that in the last few days the el8 CI for vdsm cannot run, were > there any el8 repo changes? > >> > >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 > >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 > >> [2020-06-21T09:20:03.987Z] Start: dnf install > >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: > >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot > >> /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ > >> --releasever 8 --setopt=deltarpm=False --allowerasing > >> --disableplugin=local --disableplugin=spacewalk > >> --disableplugin=local --disableplugin=spacewalk install dnf tar > >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc coreutils > >> unzip shadow-utils diffutils cpio bash gawk rpm-build info patch > >> util-linux findutils grep python36 autoconf automake createrepo dnf > >> dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils > >> libguestfs-tools-c lshw make openvswitch ovirt-imageio-common > >> python3-augeas python3-blivet python3-coverage python3-dateutil > >> python3-dbus python3-decorator python3-devel python3-dmidecode > >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux > >> python3-libvirt python3-magic python3-netaddr python3-nose > >> python3-pip python3-policycoreutils python3-pyyaml python3-requests > >> python3-sanlock python3-six python3-yaml rpm-build rpmlint sanlock > >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts > >> [2020-06-21T09:20:12.326Z] No matches found for the following > >> disable plugin patterns: local, spacewalk > >> [2020-06-21T09:20:12.326Z] Last metadata expiration check: 0:00:02 ago > on Sun Jun 21 09:20:07 2020. > >> [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch > >> > >> Taken from > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText > > > > I think this is a result of trying to move to CentOS 8.2 before all > > the packages are > > ready. > > There are problems with openvswitch package since the CentOS 8.2 update. > It also blocks Vdsm builds. Sandro and Dominik work on solving it. > > This seems to be something unrelated in the build system. CI should work since last Tuesday again.

Is the list of allowed packages for the CI limited, or is every package from ovirt-repo allowed? > > Please use Travis until this issue is resolved. You should use it > > anyway for storage > > patches. > > > > Nir > >

Dominik Holler <dholler(a)redhat.com&gt; writes: > On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: > >> Nir Soffer <nsoffer(a)redhat.com&gt; writes: >> >> > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: >> >> >> >> Hi >> > >> >> >> >> Seems that in the last few days the el8 CI for vdsm cannot run, were >> there any el8 repo changes? >> >> >> >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 >> >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 >> >> [2020-06-21T09:20:03.987Z] Start: dnf install >> >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: >> >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot >> >> /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ >> >> --releasever 8 --setopt=deltarpm=False --allowerasing >> >> --disableplugin=local --disableplugin=spacewalk >> >> --disableplugin=local --disableplugin=spacewalk install dnf tar >> >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc coreutils >> >> unzip shadow-utils diffutils cpio bash gawk rpm-build info patch >> >> util-linux findutils grep python36 autoconf automake createrepo dnf >> >> dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils >> >> libguestfs-tools-c lshw make openvswitch ovirt-imageio-common >> >> python3-augeas python3-blivet python3-coverage python3-dateutil >> >> python3-dbus python3-decorator python3-devel python3-dmidecode >> >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux >> >> python3-libvirt python3-magic python3-netaddr python3-nose >> >> python3-pip python3-policycoreutils python3-pyyaml python3-requests >> >> python3-sanlock python3-six python3-yaml rpm-build rpmlint sanlock >> >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts >> >> [2020-06-21T09:20:12.326Z] No matches found for the following >> >> disable plugin patterns: local, spacewalk >> >> [2020-06-21T09:20:12.326Z] Last metadata expiration check: 0:00:02 ago >> on Sun Jun 21 09:20:07 2020. >> >> [2020-06-21T09:20:12.326Z] No match for argument: openvswitch >> >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch >> >>

>> >> Taken from >> https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText >> > >> > I think this is a result of trying to move to CentOS 8.2 before all >> > the packages are >> > ready. >> >> There are problems with openvswitch package since the CentOS 8.2 update. >> It also blocks Vdsm builds. Sandro and Dominik work on solving it. >> >> > This seems to be something unrelated in the build system. > CI should work since last Tuesday again. I'm not sure which CI you mean, but Vdsm CI on master is still broken, even with Sandro's patch adding the oVirt 4.4 virt repo (https://gerrit.ovirt.org/109714): Running transaction check [2020-06-22T08:54:51.868Z] warning: Generating 18 missing index(es), please wait... [2020-06-22T08:54:51.868Z] Error: transaction check vs depsolve: [2020-06-22T08:54:51.868Z] /bin/sh is needed by ovirt-openvswitch-2.11-0.2020061801.el8.noarch [2020-06-22T08:54:51.868Z] bash is needed by ovirt-openvswitch-2.11-0.2020061801.el8.noarch [2020-06-22T08:54:51.868Z] systemd is needed by ovirt-openvswitch-2.11-0.2020061801.el8.noarch [2020-06-22T08:54:51.868Z] To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'. [2020-06-22T08:54:51.868Z] You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the issue. [2020-06-22T08:54:51.868Z] The downloaded packages were saved in cache until the next successful transaction. [2020-06-22T08:54:51.868Z] You can remove cached packages by executing 'dnf clean packages'. Besides it puts -1 on each patch, we can't have any Vdsm builds until it is fixed. Without the Sandro's patch, we get the error posted by Amit above. Any ideas how to proceed? Why is RPMDB reported as corrupted, were caches on the CI flushed?

> Is the list of allowed packages for the CI limited, or is every package > from ovirt-repo allowed? > > >> > Please use Travis until this issue is resolved. You should use it >> > anyway for storage >> > patches. >> > >> > Nir >> >>

On Tue, 23 Jun 2020 12:49:47 +0200 Dominik Holler <dholler(a)redhat.com&gt; wrote: > On Tue, Jun 23, 2020 at 12:07 PM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: > > > Dominik Holler <dholler(a)redhat.com&gt; writes: > > > > > On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal <mzamazal(a)redhat.com&gt; > > wrote: > > > > > >> Nir Soffer <nsoffer(a)redhat.com&gt; writes: > > >> > > >> > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: > > >> >> > > >> >> Hi > > >> > > > >> >> > > >> >> Seems that in the last few days the el8 CI for vdsm cannot run, were > > >> there any el8 repo changes? > > >> >> > > >> >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 > > >> >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 > > >> >> [2020-06-21T09:20:03.987Z] Start: dnf install > > >> >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: > > >> >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot > > >> >> > > /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ > > >> >> --releasever 8 --setopt=deltarpm=False --allowerasing > > >> >> --disableplugin=local --disableplugin=spacewalk > > >> >> --disableplugin=local --disableplugin=spacewalk install dnf tar > > >> >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc coreutils > > >> >> unzip shadow-utils diffutils cpio bash gawk rpm-build info patch > > >> >> util-linux findutils grep python36 autoconf automake createrepo dnf > > >> >> dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils > > >> >> libguestfs-tools-c lshw make openvswitch ovirt-imageio-common > > >> >> python3-augeas python3-blivet python3-coverage python3-dateutil > > >> >> python3-dbus python3-decorator python3-devel python3-dmidecode > > >> >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux > > >> >> python3-libvirt python3-magic python3-netaddr python3-nose > > >> >> python3-pip python3-policycoreutils python3-pyyaml python3-requests > > >> >> python3-sanlock python3-six python3-yaml rpm-build rpmlint sanlock > > >> >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts > > >> >> [2020-06-21T09:20:12.326Z] No matches found for the following > > >> >> disable plugin patterns: local, spacewalk > > >> >> [2020-06-21T09:20:12.326Z] Last metadata expiration check: 0:00:02 > > ago > > >> on Sun Jun 21 09:20:07 2020. > > >> >> [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > > >> >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch > > >> >> > > > > That > https://gerrit.ovirt.org/#/c/109714/ > was required to fix this raises the question from which repo openvswitch > was consumed before. > But never mind, it is fixed now. What exactly is fixed? It is still failing for me for the same reason.

Tomas > > > > >> >> Taken from > > >> > > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText > > >> > > > >> > I think this is a result of trying to move to CentOS 8.2 before all > > >> > the packages are > > >> > ready. > > >> > > >> There are problems with openvswitch package since the CentOS 8.2 update. > > >> It also blocks Vdsm builds. Sandro and Dominik work on solving it. > > >> > > >> > > > This seems to be something unrelated in the build system. > > > CI should work since last Tuesday again. > > > > I'm not sure which CI you mean, but Vdsm CI on master is still broken, > > even with Sandro's patch adding the oVirt 4.4 virt repo > > (https://gerrit.ovirt.org/109714): > > > > Running transaction check > > [2020-06-22T08:54:51.868Z] warning: Generating 18 missing index(es), > > please wait... > > [2020-06-22T08:54:51.868Z] Error: transaction check vs depsolve: > > [2020-06-22T08:54:51.868Z] /bin/sh is needed by > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > [2020-06-22T08:54:51.868Z] bash is needed by > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > [2020-06-22T08:54:51.868Z] systemd is needed by > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > [2020-06-22T08:54:51.868Z] To diagnose the problem, try running: 'rpm > > -Va --nofiles --nodigest'. > > [2020-06-22T08:54:51.868Z] You probably have corrupted RPMDB, running > > 'rpm --rebuilddb' might fix the issue. > > [2020-06-22T08:54:51.868Z] The downloaded packages were saved in cache > > until the next successful transaction. > > [2020-06-22T08:54:51.868Z] You can remove cached packages by executing > > 'dnf clean packages'. > > > > Besides it puts -1 on each patch, we can't have any Vdsm builds until it > > is fixed. Without the Sandro's patch, we get the error posted by Amit > > above. > > > > Any ideas how to proceed? Why is RPMDB reported as corrupted, were > > caches on the CI flushed? > > > > > Anton and Evgheni, can you please help us? >

> > > > > Is the list of allowed packages for the CI limited, or is every package > > > from ovirt-repo allowed? > > > > > > > > >> > Please use Travis until this issue is resolved. You should use it > > >> > anyway for storage > > >> > patches. > > >> > > > >> > Nir > > >> > > >> > > > > -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt;

On Tue, 23 Jun 2020 17:36:25 +0200 Dominik Holler <dholler(a)redhat.com&gt; wrote: > On Tue, Jun 23, 2020 at 5:22 PM Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > wrote: > > > On Tue, 23 Jun 2020 12:49:47 +0200 > > Dominik Holler <dholler(a)redhat.com&gt; wrote: > > > > > On Tue, Jun 23, 2020 at 12:07 PM Milan Zamazal <mzamazal(a)redhat.com&gt; > > wrote: > > > > > > > Dominik Holler <dholler(a)redhat.com&gt; writes: > > > > > > > > > On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal < mzamazal(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > >> Nir Soffer <nsoffer(a)redhat.com&gt; writes: > > > > >> > > > > >> > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; > > wrote: > > > > >> >> > > > > >> >> Hi > > > > >> > > > > > >> >> > > > > >> >> Seems that in the last few days the el8 CI for vdsm cannot run, > > were > > > > >> there any el8 repo changes? > > > > >> >> > > > > >> >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 > > > > >> >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 > > > > >> >> [2020-06-21T09:20:03.987Z] Start: dnf install > > > > >> >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: > > > > >> >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot > > > > >> >> > > > > > > /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ > > > > >> >> --releasever 8 --setopt=deltarpm=False --allowerasing > > > > >> >> --disableplugin=local --disableplugin=spacewalk > > > > >> >> --disableplugin=local --disableplugin=spacewalk install dnf tar > > > > >> >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc > > coreutils > > > > >> >> unzip shadow-utils diffutils cpio bash gawk rpm-build info patch > > > > >> >> util-linux findutils grep python36 autoconf automake createrepo > > dnf > > > > >> >> dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils > > > > >> >> libguestfs-tools-c lshw make openvswitch ovirt-imageio-common > > > > >> >> python3-augeas python3-blivet python3-coverage python3-dateutil > > > > >> >> python3-dbus python3-decorator python3-devel python3-dmidecode > > > > >> >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux > > > > >> >> python3-libvirt python3-magic python3-netaddr python3-nose > > > > >> >> python3-pip python3-policycoreutils python3-pyyaml > > python3-requests > > > > >> >> python3-sanlock python3-six python3-yaml rpm-build rpmlint > > sanlock > > > > >> >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts > > > > >> >> [2020-06-21T09:20:12.326Z] No matches found for the following > > > > >> >> disable plugin patterns: local, spacewalk > > > > >> >> [2020-06-21T09:20:12.326Z] Last metadata expiration check: > > 0:00:02 > > > > ago > > > > >> on Sun Jun 21 09:20:07 2020. > > > > >> >> [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > > > > >> >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: > > openvswitch > > > > >> >> > > > > > > > > > > That > > > https://gerrit.ovirt.org/#/c/109714/ > > > was required to fix this raises the question from which repo openvswitch > > > was consumed before. > > > But never mind, it is fixed now. > > > > What exactly is fixed? It is still failing for me for the same reason. > > > > > The error > [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch > is fixed by adding the ovirt44testing repo. Well the last CI state on that patch is -1, that does not look like a fix too much. But if it is, could we rebease & merge it?

Tomas > > > > > Tomas > > > > > > > > > > > > >> >> Taken from > > > > >> > > > > > > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText > > > > > > >> > > > > > >> > I think this is a result of trying to move to CentOS 8.2 before > > all > > > > >> > the packages are > > > > >> > ready. > > > > >> > > > > >> There are problems with openvswitch package since the CentOS 8.2 > > update. > > > > >> It also blocks Vdsm builds. Sandro and Dominik work on solving it. > > > > >> > > > > >> > > > > > This seems to be something unrelated in the build system. > > > > > CI should work since last Tuesday again. > > > > > > > > I'm not sure which CI you mean, but Vdsm CI on master is still broken, > > > > even with Sandro's patch adding the oVirt 4.4 virt repo > > > > (https://gerrit.ovirt.org/109714): > > > > > > > > Running transaction check > > > > [2020-06-22T08:54:51.868Z] warning: Generating 18 missing index(es), > > > > please wait... > > > > [2020-06-22T08:54:51.868Z] Error: transaction check vs depsolve: > > > > [2020-06-22T08:54:51.868Z] /bin/sh is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] bash is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] systemd is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] To diagnose the problem, try running: 'rpm > > > > -Va --nofiles --nodigest'. > > > > [2020-06-22T08:54:51.868Z] You probably have corrupted RPMDB, running > > > > 'rpm --rebuilddb' might fix the issue. > > > > [2020-06-22T08:54:51.868Z] The downloaded packages were saved in > > cache > > > > until the next successful transaction. > > > > [2020-06-22T08:54:51.868Z] You can remove cached packages by > > executing > > > > 'dnf clean packages'. > > > > > > > > Besides it puts -1 on each patch, we can't have any Vdsm builds until > > it > > > > is fixed. Without the Sandro's patch, we get the error posted by Amit > > > > above. > > > > > > > > Any ideas how to proceed? Why is RPMDB reported as corrupted, were > > > > caches on the CI flushed? > > > > > > > > > > > Anton and Evgheni, can you please help us? > > > > > > > This error is still open. > Can this be reproduced locally, or does this only happen on CI environment? > > > > > > > > > > > > > Is the list of allowed packages for the CI limited, or is every > > package > > > > > from ovirt-repo allowed? > > > > > > > > > > > > > > >> > Please use Travis until this issue is resolved. You should use it > > > > >> > anyway for storage > > > > >> > patches. > > > > >> > > > > > >> > Nir > > > > >> > > > > >> > > > > > > > > > > > > > > -- > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > > > > -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt;

On Wed, Jun 24, 2020 at 12:12 PM Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > > > Il giorno mar 23 giu 2020 alle ore 19:14 Tomáš Golembiovský < > tgolembi(a)redhat.com&gt; ha scritto: > >> On Tue, 23 Jun 2020 17:36:25 +0200 >> Dominik Holler <dholler(a)redhat.com&gt; wrote: >> >> > On Tue, Jun 23, 2020 at 5:22 PM Tomáš Golembiovský < >> tgolembi(a)redhat.com&gt; >> > wrote: >> > >> > > On Tue, 23 Jun 2020 12:49:47 +0200 >> > > Dominik Holler <dholler(a)redhat.com&gt; wrote: >> > > >> > > > On Tue, Jun 23, 2020 at 12:07 PM Milan Zamazal < >> mzamazal(a)redhat.com&gt; >> > > wrote: >> > > > >> > > > > Dominik Holler <dholler(a)redhat.com&gt; writes: >> > > > > >> > > > > > On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal < >> mzamazal(a)redhat.com&gt; >> > > >> > > > > wrote: >> > > > > > >> > > > > >> Nir Soffer <nsoffer(a)redhat.com&gt; writes: >> > > > > >> >> > > > > >> > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer < >> abawer(a)redhat.com&gt; >> > > wrote: >> > > > > >> >> >> > > > > >> >> Hi >> > > > > >> > >> > > > > >> >> >> > > > > >> >> Seems that in the last few days the el8 CI for vdsm cannot >> run, >> > > were >> > > > > >> there any el8 repo changes? >> > > > > >> >> >> > > > > >> >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 >> > > > > >> >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 >> > > > > >> >> [2020-06-21T09:20:03.987Z] Start: dnf install >> > > > > >> >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: >> > > > > >> >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot >> > > > > >> >> >> > > > > >> > > >> /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ >> > > > > >> >> --releasever 8 --setopt=deltarpm=False --allowerasing >> > > > > >> >> --disableplugin=local --disableplugin=spacewalk >> > > > > >> >> --disableplugin=local --disableplugin=spacewalk install >> dnf tar >> > > > > >> >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip >> gcc >> > > coreutils >> > > > > >> >> unzip shadow-utils diffutils cpio bash gawk rpm-build info >> patch >> > > > > >> >> util-linux findutils grep python36 autoconf automake >> createrepo >> > > dnf >> > > > > >> >> dnf-utils e2fsprogs gcc gdb git iproute-tc >> iscsi-initiator-utils >> > > > > >> >> libguestfs-tools-c lshw make openvswitch >> ovirt-imageio-common >> > > > > >> >> python3-augeas python3-blivet python3-coverage >> python3-dateutil >> > > > > >> >> python3-dbus python3-decorator python3-devel >> python3-dmidecode >> > > > > >> >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux >> > > > > >> >> python3-libvirt python3-magic python3-netaddr python3-nose >> > > > > >> >> python3-pip python3-policycoreutils python3-pyyaml >> > > python3-requests >> > > > > >> >> python3-sanlock python3-six python3-yaml rpm-build >> rpmlint >> > > sanlock >> > > > > >> >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts >> > > > > >> >> [2020-06-21T09:20:12.326Z] No matches found for the >> following >> > > > > >> >> disable plugin patterns: local, spacewalk >> > > > > >> >> [2020-06-21T09:20:12.326Z] Last metadata expiration >> check: >> > > 0:00:02 >> > > > > ago >> > > > > >> on Sun Jun 21 09:20:07 2020. >> > > > > >> >> [2020-06-21T09:20:12.326Z] No match for argument: >> openvswitch >> > > > > >> >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: >> > > openvswitch >> > > > > >> >> >> > > > > >> > > > >> > > > That >> > > > https://gerrit.ovirt.org/#/c/109714/ >> > > > was required to fix this raises the question from which repo >> openvswitch >> > > > was consumed before. >> > > > But never mind, it is fixed now. >> > > >> > > What exactly is fixed? It is still failing for me for the same >> reason. >> > > >> > > >> > The error >> > [2020-06-21T09:20:12.326Z] No match for argument: openvswitch >> > [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch >> > is fixed by adding the ovirt44testing repo. >> >> Well the last CI state on that patch is -1, that does not look like a >> fix too much. But if it is, could we rebease & merge it? >> > > Just pushed an update there. > Issue is that ovirt-openvswitch is requiring stuff at pre-trans stage and > it's being installed at buildroot creation. > This obviously fail because the root being created misses all the > pre-trans requirements. > Trying to remove the openvswitch installation in build root creation and > move its installation into tests requiring it. > > > The problematic code in the spec file is only required to update from CentOS 8.1 to 8.2. I could create an rpm, which requires manual re-enabling of ovn and ovs services on the hosts on update of CentOs 8.1 to 8.2, without the problematic code which should help to get CI working again. We can decide later if we want this behavior in a released oVirt version or not. Do you think this would be a good idea?

> > > >> >> Tomas >> >> > >> > >> > >> > > Tomas >> > > >> > > > >> > > > >> > > > > >> >> Taken from >> > > > > >> >> > > > > >> > > >> https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText >> > > >> > > > > >> > >> > > > > >> > I think this is a result of trying to move to CentOS 8.2 >> before >> > > all >> > > > > >> > the packages are >> > > > > >> > ready. >> > > > > >> >> > > > > >> There are problems with openvswitch package since the CentOS >> 8.2 >> > > update. >> > > > > >> It also blocks Vdsm builds. Sandro and Dominik work on >> solving it. >> > > > > >> >> > > > > >> >> > > > > > This seems to be something unrelated in the build system. >> > > > > > CI should work since last Tuesday again. >> > > > > >> > > > > I'm not sure which CI you mean, but Vdsm CI on master is still >> broken, >> > > > > even with Sandro's patch adding the oVirt 4.4 virt repo >> > > > > (https://gerrit.ovirt.org/109714): >> > > > > >> > > > > Running transaction check >> > > > > [2020-06-22T08:54:51.868Z] warning: Generating 18 missing >> index(es), >> > > > > please wait... >> > > > > [2020-06-22T08:54:51.868Z] Error: transaction check vs >> depsolve: >> > > > > [2020-06-22T08:54:51.868Z] /bin/sh is needed by >> > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch >> > > > > [2020-06-22T08:54:51.868Z] bash is needed by >> > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch >> > > > > [2020-06-22T08:54:51.868Z] systemd is needed by >> > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch >> > > > > [2020-06-22T08:54:51.868Z] To diagnose the problem, try >> running: 'rpm >> > > > > -Va --nofiles --nodigest'. >> > > > > [2020-06-22T08:54:51.868Z] You probably have corrupted RPMDB, >> running >> > > > > 'rpm --rebuilddb' might fix the issue. >> > > > > [2020-06-22T08:54:51.868Z] The downloaded packages were saved >> in >> > > cache >> > > > > until the next successful transaction. >> > > > > [2020-06-22T08:54:51.868Z] You can remove cached packages by >> > > executing >> > > > > 'dnf clean packages'. >> > > > > >> > > > > Besides it puts -1 on each patch, we can't have any Vdsm builds >> until >> > > it >> > > > > is fixed. Without the Sandro's patch, we get the error posted >> by Amit >> > > > > above. >> > > > > >> > > > > Any ideas how to proceed? Why is RPMDB reported as corrupted, >> were >> > > > > caches on the CI flushed? >> > > > > >> > > > > >> > > > Anton and Evgheni, can you please help us? >> > > > >> > > >> > >> > This error is still open. >> > Can this be reproduced locally, or does this only happen on CI >> environment? >> > >> > >> > > > >> > > > >> > > > > > Is the list of allowed packages for the CI limited, or is >> every >> > > package >> > > > > > from ovirt-repo allowed? >> > > > > > >> > > > > > >> > > > > >> > Please use Travis until this issue is resolved. You should >> use it >> > > > > >> > anyway for storage >> > > > > >> > patches. >> > > > > >> > >> > > > > >> > Nir >> > > > > >> >> > > > > >> >> > > > > >> > > > > >> > > >> > > >> > > -- >> > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; >> > > >> > > >> >> >> -- >> Tomáš Golembiovský <tgolembi(a)redhat.com&gt; >> >> > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA <https://www.redhat.com/> > > sbonazzo(a)redhat.com > <https://www.redhat.com/> > > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours. > <https://mojo.redhat.com/docs/DOC-1199578>* >

On Tue, 23 Jun 2020 17:36:25 +0200 Dominik Holler <dholler(a)redhat.com&gt; wrote: > On Tue, Jun 23, 2020 at 5:22 PM Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > wrote: > > > On Tue, 23 Jun 2020 12:49:47 +0200 > > Dominik Holler <dholler(a)redhat.com&gt; wrote: > > > > > On Tue, Jun 23, 2020 at 12:07 PM Milan Zamazal <mzamazal(a)redhat.com&gt; > > wrote: > > > > > > > Dominik Holler <dholler(a)redhat.com&gt; writes: > > > > > > > > > On Mon, Jun 22, 2020 at 9:34 AM Milan Zamazal <mzamazal(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > >> Nir Soffer <nsoffer(a)redhat.com&gt; writes: > > > > >> > > > > >> > On Sun, Jun 21, 2020 at 1:18 PM Amit Bawer <abawer(a)redhat.com&gt; > > wrote: > > > > >> >> > > > > >> >> Hi > > > > >> > > > > > >> >> > > > > >> >> Seems that in the last few days the el8 CI for vdsm cannot run, > > were > > > > >> there any el8 repo changes? > > > > >> >> > > > > >> >> [2020-06-21T09:20:03.003Z] Mock Version: 2.2 > > > > >> >> [2020-06-21T09:20:03.003Z] INFO: Mock Version: 2.2 > > > > >> >> [2020-06-21T09:20:03.987Z] Start: dnf install > > > > >> >> [2020-06-21T09:20:12.326Z] ERROR: Command failed: > > > > >> >> [2020-06-21T09:20:12.326Z] # /usr/bin/dnf --installroot > > > > >> >> > > > > > > /var/lib/mock/epel-8-x86_64-8e9eeb575ab4da7bf0fbfdc80a25b9c0-30232/root/ > > > > >> >> --releasever 8 --setopt=deltarpm=False --allowerasing > > > > >> >> --disableplugin=local --disableplugin=spacewalk > > > > >> >> --disableplugin=local --disableplugin=spacewalk install dnf tar > > > > >> >> gcc-c++ redhat-rpm-config which xz sed make bzip2 gzip gcc > > coreutils > > > > >> >> unzip shadow-utils diffutils cpio bash gawk rpm-build info patch > > > > >> >> util-linux findutils grep python36 autoconf automake createrepo > > dnf > > > > >> >> dnf-utils e2fsprogs gcc gdb git iproute-tc iscsi-initiator-utils > > > > >> >> libguestfs-tools-c lshw make openvswitch ovirt-imageio-common > > > > >> >> python3-augeas python3-blivet python3-coverage python3-dateutil > > > > >> >> python3-dbus python3-decorator python3-devel python3-dmidecode > > > > >> >> python3-inotify python3-ioprocess-1.4.1 python3-libselinux > > > > >> >> python3-libvirt python3-magic python3-netaddr python3-nose > > > > >> >> python3-pip python3-policycoreutils python3-pyyaml > > python3-requests > > > > >> >> python3-sanlock python3-six python3-yaml rpm-build rpmlint > > sanlock > > > > >> >> sudo systemd-udev xfsprogs --setopt=tsflags=nocontexts > > > > >> >> [2020-06-21T09:20:12.326Z] No matches found for the following > > > > >> >> disable plugin patterns: local, spacewalk > > > > >> >> [2020-06-21T09:20:12.326Z] Last metadata expiration check: > > 0:00:02 > > > > ago > > > > >> on Sun Jun 21 09:20:07 2020. > > > > >> >> [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > > > > >> >> [2020-06-21T09:20:12.326Z] Error: Unable to find a match: > > openvswitch > > > > >> >> > > > > > > > > > > That > > > https://gerrit.ovirt.org/#/c/109714/ > > > was required to fix this raises the question from which repo openvswitch > > > was consumed before. > > > But never mind, it is fixed now. > > > > What exactly is fixed? It is still failing for me for the same reason. > > > > > The error > [2020-06-21T09:20:12.326Z] No match for argument: openvswitch > [2020-06-21T09:20:12.326Z] Error: Unable to find a match: openvswitch > is fixed by adding the ovirt44testing repo. Well the last CI state on that patch is -1, that does not look like a fix too much. But if it is, could we rebease & merge it?

> > > > >> >> Taken from > > > > >> > > > > > > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/21966/consoleText > > > > > > >> > > > > > >> > I think this is a result of trying to move to CentOS 8.2 before > > all > > > > >> > the packages are > > > > >> > ready. > > > > >> > > > > >> There are problems with openvswitch package since the CentOS 8.2 > > update. > > > > >> It also blocks Vdsm builds. Sandro and Dominik work on solving it. > > > > >> > > > > >> > > > > > This seems to be something unrelated in the build system. > > > > > CI should work since last Tuesday again. > > > > > > > > I'm not sure which CI you mean, but Vdsm CI on master is still broken, > > > > even with Sandro's patch adding the oVirt 4.4 virt repo > > > > (https://gerrit.ovirt.org/109714): > > > > > > > > Running transaction check > > > > [2020-06-22T08:54:51.868Z] warning: Generating 18 missing index(es), > > > > please wait... > > > > [2020-06-22T08:54:51.868Z] Error: transaction check vs depsolve: > > > > [2020-06-22T08:54:51.868Z] /bin/sh is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] bash is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] systemd is needed by > > > > ovirt-openvswitch-2.11-0.2020061801.el8.noarch > > > > [2020-06-22T08:54:51.868Z] To diagnose the problem, try running: 'rpm > > > > -Va --nofiles --nodigest'. > > > > [2020-06-22T08:54:51.868Z] You probably have corrupted RPMDB, running > > > > 'rpm --rebuilddb' might fix the issue. > > > > [2020-06-22T08:54:51.868Z] The downloaded packages were saved in > > cache > > > > until the next successful transaction. > > > > [2020-06-22T08:54:51.868Z] You can remove cached packages by > > executing > > > > 'dnf clean packages'. > > > > > > > > Besides it puts -1 on each patch, we can't have any Vdsm builds until > > it > > > > is fixed. Without the Sandro's patch, we get the error posted by Amit > > > > above. > > > > > > > > Any ideas how to proceed? Why is RPMDB reported as corrupted, were > > > > caches on the CI flushed? > > > > > > > > > > > Anton and Evgheni, can you please help us? > > > > > > > This error is still open. > Can this be reproduced locally, or does this only happen on CI environment? > > > > > > > > > > > > > Is the list of allowed packages for the CI limited, or is every > > package > > > > > from ovirt-repo allowed? > > > > > > > > > > > > > > >> > Please use Travis until this issue is resolved. You should use it > > > > >> > anyway for storage > > > > >> > patches. > > > > >> > > > > > >> > Nir > > > > >> > > > > >> > > > > > > > > > > > > > > -- > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > > > >