Looking at squid logs there have been no requests to theproxy for patternfly-2.59.2.tgz and the URL seems to work fine for the moment. This looks like an instability on the
yarnpkg.com CDN and using the PHX proxy may indeed help stabilize things here.
Seems like other people are facing similar issues as well:
I've integrated some of the suggestions in that issue's thread with [0].
Advice ranges from increasing yarn timeouts to decreasing the size of packages uploaded to their registry so caching stuff on the proxy and not relying on upstream seems like the way to go.
Caching on the proxy won't work since everything is https, unless squid is setup to intercept https traffic...
One thought - is there any difference between [1] and [2] on jenkins? Runners, disk size, memory size? I ask since I had a "ci build" run successfully and then within a few minutes have the merge job fail over and over.
Adding Evgheni.
> On 6 Mar 2020, at 13:41, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
>
> Any other suggestion?
> It’s still mostly failing, just see builds at https://jenkins.ovirt.org/job/ovirt-engine-nodejs-modules_standard-on-merge/
>
>
>> On 3 Mar 2020, at 21:56, Scott Dickerson <sdickers@redhat.com> wrote:
>>
>>
>>
>> On Tue, Mar 3, 2020 at 3:51 PM Scott Dickerson <sdickers@redhat.com> wrote:
>> Note: I updated ovirt-engine-nodejs-modules with a new patch that did not require any new yarn downloads [1]. This gets CI working for ovirt-engine-ui-extensions and ovirt-web-ui changes that do not require a pre-seed update. I created a new patch [2] to contain my pre-seed requirements (and therefore new yarn downloads). CI needs to do a successful 'ci build' on el7, el8, fc29 and fc30 before a handful of patches can be unblocked.
>>
>> I looked into the http proxy and there is no way to turn them off from being set when calling the automation scripts. I can adjust the scripts, but since wget seems to work ok with the proxy environment vars, the problem is with yarn. I'll try a few options, starting with telling yarn to explicitly turn off the proxy and see what happens.
>>
>> Can someone look at the squid logs to see if anything is getting through to the yarn repos? E.g. "https://registry.yarnpkg.com/@patternfly/react-core/-/react-core-3.134.2.tgz"
>>
>> Regards,
>> Scott
>>
>> [1] - https://gerrit.ovirt.org/107367
>> [2] - https://gerrit.ovirt.org/107381
>>
>> Correction, the pre-seed patch is actually: https://gerrit.ovirt.org/107382
>>
>> On Tue, Mar 3, 2020 at 10:20 AM Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
>>
>>
>>> On 3 Mar 2020, at 15:18, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
>>>
>>>
>>>
>>> Il giorno mar 3 mar 2020 alle ore 15:09 Scott Dickerson <sdickers@redhat.com> ha scritto:
>>>
>>>
>>> On Tue, Mar 3, 2020 at 8:35 AM Sharon Gratch <sgratch@redhat.com> wrote:
>>>
>>>
>>> On Tue, Mar 3, 2020 at 3:17 PM Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
>>>
>>>
>>>> On 3 Mar 2020, at 09:14, Barak Korren <bkorren@redhat.com> wrote:
>>>>
>>>> It matters - because it would block sending the relevant builds to the CQ.
>>>
>>> yes
>>>
>>>>
>>>> One thing that could make YARN fail in CI but not in local mock is the fact the we have HTTP_PROXY defined in the CI environment and pointing to a Squid server.
>>>>
>>>> A typical issue we see people having is when connecting to `localhost` and ending up being blocked by the proxy. Please make sure the NO_PROXY env var is set appropriately if that is the case?
>>>
>>> good point, we can try that I guess. Weird thing is it succeeds some time.
>>> Scott, can you try that?
>>>
>>>
>>>
>>> @Sandro Bonazzola tries that by using a specific local proxy solution: https://gerrit.ovirt.org/107358
>>>
>>>
>>>
>>> I was just watching the jenkins build on his patch and it failed with exactly the same error as before.
>>>
>>> FWIW, the pre-seed patch in question (https://gerrit.ovirt.org/107309) only needs to do a total of 8 npm/yarnpkg repo downloads per build-artifacts run. So it isn't exactly asking for a huge effort. ;-)
>>>
>>> Yes, still failing on
>>> error An unexpected error occurred: "https://registry.yarnpkg.com/@patternfly/react-core/-/react-core-3.134.2.tgz: unexpected end of file".
>>> so even using a proxy is not helping at all.
>>>
>>> It appears to affect everyone using yarn and common solution I found is just retry on failure till it works :-(
>>
>> how to use NO_PROXY? We don’t really need nor want any proxy
>>
>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> Why is this not failing in check-patch as well BTW?
>>>
>>> I think it doesn’t do any downloads, just checks the presence of package.json.
>>>
>>> Thanks,
>>> michal
>>>
>>>>
>>>> On Tue, 3 Mar 2020 at 10:07, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
>>>> is this a better list? or no one cares?
>>>>
>>>>> On 2 Mar 2020, at 09:40, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On 1 Mar 2020, at 17:54, Scott Dickerson <sdickers@redhat.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On the merge phase on patch [1], both the el7 and fc30 build have been failing. Examples are [2] and [3]. I'm guessing there are environmental issues that I can't fix from the project's perspective. Typical example of the error from the log:
>>>>>
>>>>> it seems to fail randomly, in run 93 el8 and fc30 fails while el7 and fc29 succeeds, in run 94 it’s fc30 that’s failing and el8 succeeded.
>>>>>
>>>>>>
>>>>>> [2020-03-01T15:55:19.475Z] + yarn install --pure-lockfile --har
>>>>>> [2020-03-01T15:55:19.475Z] + /home/jenkins/workspace/ovirt-engine-nodejs-modules_standard-on-merge/ovirt-engine-nodejs-modules/yarn-1.17.3.js install --pure-lockfile --har
>>>>>> [2020-03-01T15:55:19.475Z] yarn install v1.17.3
>>>>>> [2020-03-01T15:55:19.475Z] [1/5] Resolving packages...
>>>>>> [2020-03-01T15:55:19.475Z] [2/5] Fetching packages...
>>>>>> [2020-03-01T15:55:19.475Z] error An unexpected error occurred: "https://registry.yarnpkg.com/@patternfly/react-core/-/react-core-3.134.2.tgz: unexpected end of file".
>>>>>>
>>>>>> Running the build in mock_runner locally targeted for el7, el8, fc29 and fc30 work just fine.
>>>>>>
>>>>>> Help please!
>>>>>>
>>>>>> [1] - https://gerrit.ovirt.org/#/c/107309/
>>>>>> [2] - https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-engine-nodejs-modules_standard-on-merge/detail/ovirt-engine-nodejs-modules_standard-on-merge/88/pipeline
>>>>>> [3] - https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-engine-nodejs-modules_standard-on-merge/detail/ovirt-engine-nodejs-modules_standard-on-merge/92/pipeline
>>>>>>
>>>>>> --
>>>>>> Scott Dickerson
>>>>>> Senior Software Engineer
>>>>>> RHV-M Engineering - UX Team
>>>>>> Red Hat, Inc
>>>>>
>>>>
>>>> _______________________________________________
>>>> Infra mailing list -- infra@ovirt.org
>>>> To unsubscribe send an email to infra-leave@ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/MJAIPCG6VJJAXKD4JKSEHEQ6UUX6HXMD/
>>>>
>>>>
>>>> --
>>>> Barak Korren
>>>> RHV DevOps team , RHCE, RHCi
>>>> Red Hat EMEA
>>>> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
>>>
>>>
>>>
>>> --
>>> Scott Dickerson
>>> Senior Software Engineer
>>> RHV-M Engineering - UX Team
>>> Red Hat, Inc
>>>
>>>
>>> --
>>> Sandro Bonazzola
>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>>> Red Hat EMEA
>>> sbonazzo@redhat.com
>>>
>>> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.
>>
>>
>>
>> --
>> Scott Dickerson
>> Senior Software Engineer
>> RHV-M Engineering - UX Team
>> Red Hat, Inc
>>
>>
>> --
>> Scott Dickerson
>> Senior Software Engineer
>> RHV-M Engineering - UX Team
>> Red Hat, Inc
>
--
Anton Marchukov
Associate Manager - RHV DevOps - Red Hat
--
Regards,
Evgheni Dereveanchin