On 18. 11. 2021, at 10:10, Anton Marchukov <amarchuk(a)redhat.com&gt; wrote: https://resources02.phx.ovirt.org

On 18. 11. 2021, at 11:18, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: Anton Marchukov <amarchuk(a)redhat.com&gt; writes: > Hello Milan. > > That’s unfortunate and sorry for that. Please check if you can use > https://resources02.phx.ovirt.org for now to be unblocked. This is the > old server. Hi Anton, thank you for the suggestion. I tried but it looks to me that packages are missing there too. Am I mistaken or is it so really? > During migration some files were not migrated as there is really tons > of old stuff, plus all those CI files are being distributed to mirrors > that is not instantaneous and unneeded. We tried to check which ones > are used and which are not. I think the plan is to have a separate > file server for CI files as a permanent solution. > > CCing Denis also. > > Anton. > >> On 18. 11. 2021, at 9:44, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >> >> Hi, >> >> CI is not working for me (at least) in ovirt-4.4.z Vdsm branch. As >> Sandro has found out, one part of that is the following: >> >> something or someone destroyed the tested repo, it's missing a >> huge amount of packages there >> this caused the nighlty reaper to also cleanup the >> ovirt-master-snapshot repo as no corresponding package was in >> tested repo >> we had a massive data loss there, can you please report it to >> devops / amarchuk 's team? >> I hope they have backups >> if not... we'll have to rebuild the missing packages. >> >> Could you please take a look and tell us what can be done about it? >> >> It is currently one of the issues blocking today's 4.4.10 Vdsm build. >> >> Thanks, >> Milan >>

On 18. 11. 2021, at 11:24, Anton Marchukov <amarchuk(a)redhat.com&gt; wrote: E.g. the “tested” I am aware off is here: https://resources02.phx.ovirt.org/repos/ovirt/tested/ It is also on the new server: https://resources.ovirt.org/repos/ovirt/tested/ Anton. > On 18. 11. 2021, at 11:21, Anton Marchukov <amarchuk(a)redhat.com&gt; wrote: > > I am not aware about any on purpose changes on the old server. When was the last time it was working and which url is exactly missing? > > Anton. > >> On 18. 11. 2021, at 11:18, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >> >> Anton Marchukov <amarchuk(a)redhat.com&gt; writes: >> >>> Hello Milan. >>> >>> That’s unfortunate and sorry for that. Please check if you can use >>> https://resources02.phx.ovirt.org for now to be unblocked. This is the >>> old server. >> >> Hi Anton, >> >> thank you for the suggestion. I tried but it looks to me that packages >> are missing there too. Am I mistaken or is it so really? >> >>> During migration some files were not migrated as there is really tons >>> of old stuff, plus all those CI files are being distributed to mirrors >>> that is not instantaneous and unneeded. We tried to check which ones >>> are used and which are not. I think the plan is to have a separate >>> file server for CI files as a permanent solution. >>> >>> CCing Denis also. >>> >>> Anton. >>> >>>> On 18. 11. 2021, at 9:44, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >>>> >>>> Hi, >>>> >>>> CI is not working for me (at least) in ovirt-4.4.z Vdsm branch. As >>>> Sandro has found out, one part of that is the following: >>>> >>>> something or someone destroyed the tested repo, it's missing a >>>> huge amount of packages there >>>> this caused the nighlty reaper to also cleanup the >>>> ovirt-master-snapshot repo as no corresponding package was in >>>> tested repo >>>> we had a massive data loss there, can you please report it to >>>> devops / amarchuk 's team? >>>> I hope they have backups >>>> if not... we'll have to rebuild the missing packages. >>>> >>>> Could you please take a look and tell us what can be done about it? >>>> >>>> It is currently one of the issues blocking today's 4.4.10 Vdsm build. >>>> >>>> Thanks, >>>> Milan >>>> >> > > -- > Anton Marchukov > Associate Manager - RHV DevOps - Red Hat > > > > > > -- Anton Marchukov Associate Manager - RHV DevOps - Red Hat

On 18. 11. 2021, at 11:29, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: Anton Marchukov <amarchuk(a)redhat.com&gt; writes: > I am not aware about any on purpose changes on the old server. When > was the last time it was working and which url is exactly missing? Correct me if I'm wrong, but is it for example right that https://resources02.phx.ovirt.org/pub/ovirt-master-snapshot/rpm/el8/noarch/ is empty? I don't know when it was last working but we've experienced similar issues at least several days ago in some patches (don't know if related). > Anton. > >> On 18. 11. 2021, at 11:18, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >> >> Anton Marchukov <amarchuk(a)redhat.com&gt; writes: >> >>> Hello Milan. >>> >>> That’s unfortunate and sorry for that. Please check if you can use >>> https://resources02.phx.ovirt.org for now to be unblocked. This is the >>> old server. >> >> Hi Anton, >> >> thank you for the suggestion. I tried but it looks to me that packages >> are missing there too. Am I mistaken or is it so really? >> >>> During migration some files were not migrated as there is really tons >>> of old stuff, plus all those CI files are being distributed to mirrors >>> that is not instantaneous and unneeded. We tried to check which ones >>> are used and which are not. I think the plan is to have a separate >>> file server for CI files as a permanent solution. >>> >>> CCing Denis also. >>> >>> Anton. >>> >>>> On 18. 11. 2021, at 9:44, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >>>> >>>> Hi, >>>> >>>> CI is not working for me (at least) in ovirt-4.4.z Vdsm branch. As >>>> Sandro has found out, one part of that is the following: >>>> >>>> something or someone destroyed the tested repo, it's missing a >>>> huge amount of packages there >>>> this caused the nighlty reaper to also cleanup the >>>> ovirt-master-snapshot repo as no corresponding package was in >>>> tested repo >>>> we had a massive data loss there, can you please report it to >>>> devops / amarchuk 's team? >>>> I hope they have backups >>>> if not... we'll have to rebuild the missing packages. >>>> >>>> Could you please take a look and tell us what can be done about it? >>>> >>>> It is currently one of the issues blocking today's 4.4.10 Vdsm build. >>>> >>>> Thanks, >>>> Milan >>>> >>

The CI was switched to the new server: https://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el8/noarch/ is not empty. I guess the old one might be cleaned up by cron.

> host resources.ovirt.org resources.ovirt.org has address 169.62.7.164 Can you please check your system uses the correct DNS name for resources and resolve to the correct IP?

If it is then we need to know which exact packages are missing now on the new server (regular resources.ovirt.org) as this is where CI publishers are pushing stuff.