I believe that we always deployed the mom rpms from patches but we just didn't know about it until now when we have a real patch breaking VDSM. I've sent [1] to split created & merged jobs for mom so we'll deploy only the merged ones, this should deploy only mom rpms created from merged commits. I've deleted the bad mom from the repo and running tests again to see if it works. [1] https://gerrit.ovirt.org/#/c/63152/1 On Thu, Sep 1, 2016 at 5:53 PM, Eyal Edri <eedri@redhat.com> wrote:
I think I know what the problem is (not 100% sure). I see the experimental job [1] is taking mom rpms from an unmerged patch with -1 [2], so if this patch has a bug then it deployed broken mom to the experimental repo and failing tests. I'm trying to exclude the created jobs for mom out of this workflow.
[1] http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/1283/ [2] https://gerrit.ovirt.org/#/c/63130/
On Thu, Sep 1, 2016 at 3:32 PM, Barak Korren <bkorren@redhat.com> wrote:
Hi there,
The oVirt "experimental" system tests form "master and 4.0 started failing today at approximately 9:20 ([1], [2]) at the AddHost stage.
I suspect the failure is due to a mom build that was performed today at 9:05 ([3], [4]). This is because the following can be found in the VDSM logs ([5], [6]):
MainThread::ERROR::2016-09-01 06:18:41,076::vdsm::151::vds::(run) Exception raised Traceback (most recent call last): File "/usr/share/vdsm/vdsm", line 149, in run serve_clients(log) File "/usr/share/vdsm/vdsm", line 104, in serve_clients cif = clientIF.getInstance(irs, log, scheduler) File "/usr/share/vdsm/clientIF.py", line 204, in getInstance cls._instance = clientIF(irs, log, scheduler) File "/usr/share/vdsm/clientIF.py", line 105, in __init__ self._prepareMOM() File "/usr/share/vdsm/clientIF.py", line 273, in _prepareMOM self.mom = MomClient(momconf) File "/usr/lib/python2.7/site-packages/vdsm/momIF.py", line 46, in __init__ raise MomNotAvailableError() MomNotAvailableError MainThread::INFO::2016-09-01 06:18:41,077::vdsm::153::vds::(run) Stopping threads MainThread::INFO::2016-09-01 06:18:41,077::vdsm::164::vds::(run) Exiting
Please have a look. Thanks, Barak.
[1]: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma ster/1283/ [2]: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/1077/ [3]: http://jenkins.ovirt.org/job/mom_master_create-rpms-el7-x86_ 64_created/15/ [4]: http://jenkins.ovirt.org/job/mom_4.0_create-rpms-el7-x86_64_ created/11/ [5]: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma ster/1283/artifact/exported-artifacts/basic_suite_master.sh- fc24/exported-artifacts/test_logs/basic_suite_master/post- 002_bootstrap.py/lago_basic_suite_master_host0/_var_log_vdsm/vdsm.log [6]: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4. 0/1077/artifact/exported-artifacts/basic_suite_4.0.sh-fc23/ exported-artifacts/test_logs/basic_suite_4.0/post-002_boots trap.py/lago_basic_suite_4_0_host0/_var_log_vdsm/vdsm.log
-- Barak Korren bkorren@redhat.com RHEV-CI Team
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)