[JIRA] (OVIRT-901) bad mirror caused check-merged jobs to fail

[Barak Korren (oVirt JIRA)]

    [ https://ovirt-jira.atlassian.net/browse/OVIRT-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=23802#comment-23802 ] 

Barak Korren commented on OVIRT-901:
------------------------------------

One approach to setting up a mirror that never fails is to make it "transactional" by actually seyting up a whole new mirror every time you sync it and then create some kind of a meta-data file that point to where it is.
That way when a job starts running it just takes the latest meta-data  file in it is guaranteed that the thing this file points to will never change while the job is running.

This doesn't mean that syncing the mirror will be slow, there are quite a few things we can do to make it fast:
* Use hardlinks - before running the mirror sync tool (reposync, repoman, rsync, etc.) copy the older mirror (by creating hardlinks so the copy is fast) to the new location.
* Use LVM snapshots - Always sync the mirror to the same place, but after the sync is done, create an LVM snapshot of that place, mount it and put the location it is mounted into as the path in the metadata file.
* Use a tool the has this kind of functionality built-in (Katallo/Pulp with content views)



> bad mirror caused check-merged jobs to fail
> -------------------------------------------
>
>                 Key: OVIRT-901
>                 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-901
>             Project: oVirt - virtualization made easy
>          Issue Type: Bug
>            Reporter: eyal edri [Administrator]
>            Assignee: infra
>
> check-merged jobs fails on bad mirrors:
> If we can't defend ourselves from repomd.xml file errors, lets setup a mirror when we control the mirror sync and we can decide on the update times ( maybe even disable jenkins jobs when the update is running to avoid errors )
> http://jenkins.ovirt.org/job/repos_3.6_check-closure_el7_merged/103/console
> 07:35:30 Can't download or revert repomd.xml for check-epel-el7
> 07:35:30 Some dependencies may not be complete for this repository
> 07:35:30 Run as root to get all dependencies or use -t to enable a user temp cache
> 07:35:30 Checking Dependencies
> 07:35:30 failure: repodata/151a911cc4b434fcd135e243eec4a4f1d8c1943595a92e2bb838cda5f03577ff-primary.sqlite.xz from check-epel-el7: [Errno 256] No more mirrors to try.
> 07:35:30 http://mirror.switch.ch/ftp/mirror/epel/7/x86_64/repodata/151a911cc4b434fcd135e243eec4a4f1d8c1943595a92e2bb838cda5f03577ff-primary.sqlite.xz: [Errno 14] HTTP Error 404 - Not Found



--
This message was sent by Atlassian JIRA
(v1000.610.2#100023)