- Infra - oVirt List Archives

[JIRA] (OVIRT-1490) Simplify oVirt sotrage configuration
by Barak Korren (oVirt JIRA) 29 Jun '17

29 Jun '17

Barak Korren created OVIRT-1490: ----------------------------------- Summary: Simplify oVirt sotrage configuration Key: OVIRT-1490 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1490 Project: oVirt - virtualization made easy Issue Type: Improvement Components: storage Reporter: Barak Korren Assignee: infra Priority: Highest Today's outage was a clear reminder that our current storage configuration does not serve us well. We hardly know how to debug it, it seems to not be resistant to the very issues it was supposed to protect against and introduce potential failure scenarios of its own. I suggest we implement a new storage layout that meets the following criteria: # Ultimate simplicity at the lower level of the stack. More specifically: ## The storage severs should be simple NFS or iSCSI servers. No DRBD and no exotic file-systems. ## Only simple storage will be presented to oVirt for use as storage domains # Separation of resources between critical services - The 'Jenkins" master for e.g. should not share resources with the "resources" server or anything else.The separation should hold true down to the physical spindle level. # Duplication of services and use of local storage where possible - this is a longer term effort - but we have some low hanging fruits here like artifactory, where simple DNS/LB-based fail-over between two identical hosts would probably suffice. # Complexity only where needed and up the stack. For example we can just have the storage for Jenkins be mirrored at the VM level with fail-over to a backup VM. -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0

[JIRA] (OVIRT-1490) Simplify oVirt sotrage configuration
by Barak Korren (oVirt JIRA) 29 Jun '17

29 Jun '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1490?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1490: -------------------------------- Epic Link: OVIRT-403 > Simplify oVirt sotrage configuration > ------------------------------------ > > Key: OVIRT-1490 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1490 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: storage > Reporter: Barak Korren > Assignee: infra > Priority: Highest > Labels: infra, storage > > Today's outage was a clear reminder that our current storage configuration does not serve us well. We hardly know how to debug it, it seems to not be resistant to the very issues it was supposed to protect against and introduce potential failure scenarios of its own. > I suggest we implement a new storage layout that meets the following criteria: > # Ultimate simplicity at the lower level of the stack. More specifically: > ## The storage severs should be simple NFS or iSCSI servers. No DRBD and no exotic file-systems. > ## Only simple storage will be presented to oVirt for use as storage domains > # Separation of resources between critical services - The 'Jenkins" master for e.g. should not share resources with the "resources" server or anything else.The separation should hold true down to the physical spindle level. > # Duplication of services and use of local storage where possible - this is a longer term effort - but we have some low hanging fruits here like artifactory, where simple DNS/LB-based fail-over between two identical hosts would probably suffice. > # Complexity only where needed and up the stack. For example we can just have the storage for Jenkins be mirrored at the VM level with fail-over to a backup VM. -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0

[JIRA] (OVIRT-1489) refactor git push map file to be regex instead of url
by Gil Shinar (oVirt JIRA) 29 Jun '17

29 Jun '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1489?page=com.atlassian.jira.… ] Gil Shinar reassigned OVIRT-1489: --------------------------------- Assignee: Gil Shinar (was: infra) > refactor git push map file to be regex instead of url > ----------------------------------------------------- > > Key: OVIRT-1489 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1489 > Project: oVirt - virtualization made easy > Issue Type: Task > Reporter: Gil Shinar > Assignee: Gil Shinar > > changing the way the poll script reads stuff from the git push map file. > Instead of using implicit URL, use regex to be able to support in whatever comes in the future -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0

[JIRA] (OVIRT-1489) refactor git push map file to be regex instead of url
by Gil Shinar (oVirt JIRA) 29 Jun '17

29 Jun '17

Gil Shinar created OVIRT-1489: --------------------------------- Summary: refactor git push map file to be regex instead of url Key: OVIRT-1489 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1489 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Gil Shinar Assignee: infra changing the way the poll script reads stuff from the git push map file. Instead of using implicit URL, use regex to be able to support in whatever comes in the future -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0

[JIRA] (OVIRT-1231) Security: do we need HSTS for oVirt services?
by Marc Dequènes (Duck) (oVirt JIRA) 28 Jun '17

28 Jun '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1231?page=com.atlassian.jira.… ] Marc Dequènes (Duck) commented on OVIRT-1231: --------------------------------------------- So, the only place using it is the new ML3 server, which is on production only for redirects. We're currently using the 'httpd' Ansible role to deploy the configuration, which activates it. The role also activates 'includeSubDomains'; this is a desired setting but only when all the vhosts on the domain are able to do HTTPS. This is not the case on all oVirt infra yet so it was deactivated manually at some point IIRC. So, this solution is not perfect but avoiding protocol downgrade is already a very important protection and we should use it. We should also use 'includeSubDomains' too when all our vhosts are ready. And we must not create new vhosts without HTTPS support even for testing. Here are my recommendations. > Security: do we need HSTS for oVirt services? > --------------------------------------------- > > Key: OVIRT-1231 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1231 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Reporter: eedri > Assignee: infra > > https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security > Most of the browsers already supports it and some websites started to enforce it. > cc [~dfediuck] -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0

Outage in network affecting oVirt services
by Eyal Edri 28 Jun '17

28 Jun '17

FYI, It looks like we have an outage in the DC we're running our oVirt services, it is currently affecting multiple services like Jenkins and Resources but not Gerrit. We are escalating it to the relevant teams to check what is causing it and will report once we have an update. -- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[oVirt Jenkins] ovirt-node-ng_master_build-artifacts-el7-x86_64 - Build # 457 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Jun '17

28 Jun '17

------=_Part_125_2130288522.1498681300825 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build Number: 457 Build Status: Failure Triggered By: Started by user Yuval Turgeman ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #457 [Yuval Turgeman] automation: adding timestamp to squashfs ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_125_2130288522.1498681300825--

1 1

[oVirt Jenkins] ovirt_master_hc-system-tests - Build # 154 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Jun '17

28 Jun '17

------=_Part_84_325694316.1498620535803 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/154/ Build Number: 154 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #154 [Daniel Belenky] Exclude packages from ovirt-master.repo [Barak Korren] Filter builds sent to change queues by version [Barak Korren] Make change queue invoke OST basic suit [Barak Korren] Various UX improvements in change-queue jobs [Barak Korren] Add a job for direct deployments to 'tested' [Barak Korren] Use new job to update 'tested' from experimental [Barak Korren] Use new tested deploy job in timed builders [Eyal Edri] update build retention policy for big artifacts ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in add_hosts testlib.assert_true_within(_host_is_up, timeout=15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << --------------------- ------=_Part_84_325694316.1498620535803--

1 1

planned Gerrit maintenance
by Evgheni Dereveanchin 28 Jun '17

28 Jun '17

Hi everyone, I will be performing updates on gerrit.ovirt.org during the next two hours. Within this period the Gerrit UI and Git repositories may be unavailable. I will follow up as soon as maintenance activities are over. -- Regards, Evgheni Dereveanchin

2 3

[JIRA] (OVIRT-1488) gerrit is down
by Evgheni Dereveanchin (oVirt JIRA) 28 Jun '17

28 Jun '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1488?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-1488: --------------------------------------------- This was a planned maintenance window and alerts have been sent to devel and infra lists. Gerrit should be back up and running now, sorry for the inconvenience. > gerrit is down > -------------- > > Key: OVIRT-1488 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1488 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: gshereme(a)redhat.com > Assignee: Evgheni Dereveanchin > Priority: Highest > > Hi, > https://gerrit.ovirt.org/ > Service Temporarily Unavailable > The server is temporarily unable to service your request due to maintenance > downtime or capacity problems. Please try again later. > Apache/2.2.15 (Red Hat) Server at gerrit.ovirt.org Port 443 > -- > Greg Sheremeta, MBA > Sr. Software Engineer > Red Hat, Inc. > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.1092.0#100053)

1 0