[ANN] introducing ovirt-openshift-extensions
by Roy Golan
Hi all,
Running Openshift on oVirt seems more and more attractive and is starting
to get attention.
It is very easy to do so today without needing any special configuration,
however, to tighten the integration and to take advantage of the underlying
infra
provider(ovirt) we can do better. For example, oVirt can connect various
storage
providers, and serve disk space to containers[1][2]. Also, Openshift can
ask oVirt
for VMs and deploy them as master or application nodes for its usage,
without
making the administrator doing all that manually.
This project[1] is the home for the ovirt-flexvolume-driver and
ovirt-provisioner[3]
and merging ovirt-cloudprovider[4] is a work in progress.
The code under this repository is work-in-progress and moving quickly,
however,
it has automation (stdci v2 :)) and is working and does at least what you
observe in
the demo videos. Yet I highly appreciate if any of you that will try will
provide feedback
be that #ovirt channel/ mailing list or to report bugs directly in the Github
project page <https://github.com/oVirt/ovirt-openshift-extensions>
[1] https://github.com/oVirt/ovirt-openshift-extensions
[2] https://ovirt.org/blog/2018/02/your-container-volumes-served-by-ovirt/
[3] https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/
[4] https://github.com/rgolangh/ovirt-k8s-cloudprovider
Thanks,
Roy
5 years, 11 months
Uploaded ISO images
by Sandro Bonazzola
Hi,
Just noticed that when I upload an ISO into a data domain from the browser,
the content is detected as ISO but once upload finishes it's listed as
Type:Image. Shouldn't it be Type:ISO?
Software Version:4.2.3.5-1.el7.centos
Thanks,
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://red.ht/sig>
<https://redhat.com/summit>
5 years, 11 months
Propose Dominik Holler as a Network-Backend Maintainer
by Alona Kaplan
Hi all,
Dominik Holler has been working on the oVirt project for more than 1.5
years.
To share some of Dominik's great stats -
~ 120 patches related to the network backend/ui
~ 95 patches for ovirt-provider-ovn
~ 44 vdsm patches
~ 80 bug fixes
He was the feature owner of 'auto sync network provider', 'lldp-reporting'
and 'network-filter-parameters'.
For the last few months Dominik is helping review network-backend related
patches and is doing a great and thorough work.
Dominik showed a deep understanding of all the parts of code that he
touched or reviewed.
He learns fast, thorough and uncompromising.
I've reviewed most of Dominik's engine related work (code and reviews).
I trust his opinion and think he will be a good addition to the maintainers
team.
I would like to propose Dominik as a Network backend maintainer.
Thanks,
Alona.
5 years, 11 months
[ANNOUNCE] Introducing STDCI V2
by Barak Korren
The CI team is thrilled to announce the general availability of the second
version of the oVirt CI standard. Work on this version included almost a
complete rewrite of the CI backend. The major user-visible features are:
- Project maintainers no longer need to maintain YAML in the ‘jenkins’
repository. Details that were specified there, including targeted
distributions, architectures and oVirt versions should now be specified in a
YAML file under the project’s own repository (In a different syntax).
- We now support “sub stages” which provide the ability to run multiple
different scripts in parallel within the same STDCI stage. There is also a
conditional syntax which allows controlling which scripts get executed
according to which files were changed in the patch being tested.
- The STDCI script file names and locations can now be customized via the above
mentioned YAML file. This means that for e.g. using the same script for
different stages can now be done by assigning it to the stages in the YAML
file instead of by using symlinks.
Inspecting job results in STDCI V2
----------------------------------
As already mentioned, the work on STDCI V2 consisted of a major rewrite of the
CI backend, one of the changes made is switching from using multiple “FreeStyle”
type jobs per project to just two (pre-merge and post-merge) pipeline jobs. This
has implications for the way job results are to be inspected.
Since all the different parallel tasks now happen within the same job, looking
at the job output can be rather confusing as it includes the merged output of
all the tasks. Instead, the “Blue Ocean” view should be used. The “Blue Ocean”
view displays a graphical layout of the job execution allowing one to quickly
learn which parts of the job failed. It also allows drilling down and viewing
the logs of individual parts of the job.
Apart from using the “Blue Ocean” view, job logs are also stored as artifact
files. The ‘exported-artifacts’ directory seen in the job results will now
include different subdirectories for the different parts of the job. Assuming we
have a ‘check-patch’ stage script running on ‘el7/x86_64’, we can find its
output under ‘exported-artifacts’ in:
check-patch.el7.x86_64/mock_logs/script/stdout_stderr.log
Any additional artifacts generated by the script would be present in the
‘check-patch.el7.x86-64’ directory as well.
I have a CI YAML file in my project already, is this really new?
----------------------------------------------------------------
We’ve been working on this for a while, and occasionally introduced V2 features
into individual projects as needed. In particular, our GitHub support was always
based on STDCI V2 code, so all GitHub projects (except Lago, which is
‘special’…) are already using STDCI V2.
A few Gerrit-based projects have already been converted to V2 as well as part of
our efforts to test and debug the V2 code. Most notably, the “OST” and “Jenkins”
projects have been switched, although they are running the STDCI V1 jobs as well
for the time being.
What is the process for switching my project to STDCI V2?
---------------------------------------------------------
The CI team is going to proactively work with project maintainers to switch them
to V2. The process for switching is as follows:
- Send a one-line patch to the ‘jenkins’ repo to enable the V2 jobs for the
project - at this point the V2 jobs will run side-by-side with the V1 jobs,
and will execute the STDCI scripts on el7/x86_64.
- Create an STDCI YAML file to define the target distributions, architectures
and oVirt versions for the project. (See below for a sample file that would be
equivalent to what many projects have defined in V1 currently). As soon as a
patch with the new YAML file is submitted to the project, the V2 job will
parse it and follow the instructions in it. This allows for an easy
verification of the file functionality in CI.
- Remove the STDCI V1 job configuration from the ‘jenkins’ repo. This should be
the last patch project maintainers have to send to the ‘jenkins’ repo.
What does the new YAML file look like?
--------------------------------------
We defined multiple optional names for the file, so that each project owner can
choose which name seems most adequate. The following names can be used:
- stdci.yaml
- automation.yaml
- ovirtci.yaml
A dot (.) can also optionally be added at the beginning of the file name to make
the file hidden, the file extension could also be “yml”. If multiple matching
files exist in the project repo, the first matching file according to the order
listed above will be used.
The file conforms to the YAML syntax. The key names in the file are
case-agnostic, and hyphens (-) underscores (_) and spaces ( ) in key names are
ignored. Additionally we support multiple forms of the same word so you don’t
need to remember if the key should be ‘distro’, ‘distros’, ‘distributions’,
‘operating-systems’ or ‘OperatingSystems’ as all these forms (and others) will
work and mean the same thing.
To create complex test/build matrices, ‘stage’, ‘distribution’, ‘architecture’
and ‘sub-stage’ definitions can be nested within one another. We find this to be
more intuitive than having to maintain tedious ‘exclude’ lists as was needed in
V1.
Here is an example of an STDCI V2 YAML file that is compatible with the current
master branch V1 configuration of many oVirt projects:
---
Architectures:
- x86_64:
Distributions: [ "el7", "fc27" ]
- ppc64le:
Distribution: el7
- s390x:
Distribution: fc27
Release Branches:
master: ovirt-master
Note: since the file is committed into the project’s own repo, having different
configuration for different branches can be done by simply having different
files in the different branches, so there is no need for a big convoluted file
to configure all branches.
Since the above file does not mention stages, any STDCI scripts that exists in
the project repo and belong to a particular stage will be run on all specified
distribution and architecture combinations. Since it is sometimes desired to run
‘check-patch.sh’ on less platforms then build-artifacts for example, a slightly
different file would be needed:
---
Architectures:
- x86_64:
Distributions: [ “el7”, “fc27” ]
- ppc64le:
Distribution: el7
- s390x:
Distribution: fc27
Stages:
- check-patch:
Architecture: x86_64
Distribution: el7
- build-artifacts
Release Branches:
master: ovirt-master
The above file makes ‘check-patch’ run only on el7/x86_64, while build-artifacts
runs on all platforms specified and check-merged would not run at all because it
is not listed in the file.
Great efforts have been made to make the file format very flexible but intuitive
to use. Additionally there are many defaults in place to allow specifying
complex behaviours with very brief YAML code. For further details about the file
format, please see the documentation linked below.
About the relation between STDCI V2 and the change-queue
--------------------------------------------------------
In STDCI V1 the change queue that would run the OST tests and release a given
patch was determined by looking at the “version” part of the name of the
project’s build-artifacts jobs that got invoked for the patch.
This was confusing for people as most people understood “version” to mean the
internal version for their own project rather then the oVirt version.
In V2 we decided to be more explicit and simply include a map from branches to
change queues in the YAML configuration under the “release-branches” option, as
can be seen in the examples above.
We also chose to no longer allow specifying the oVirt version as a shorthand for
the equivalent queue name (E.g specifying ‘4.2’ instead of ‘ovirt-4.2’), this
should reduce the chance for confusing between project versions and queue names,
and also allows us to create and use change queues for projects that are not
oVirt.
A project can choose not to include a “release-branches” option, in which case
its patches will not get submitted to any queues.
Further information
-------------------
The documentation for STDCI can be found at [1].
The documentation update for V2 are still in progress and expected to be merged
soon. In the meatine, the GitHub-specific documentation [2] already provides a
great deal of information which is relevant for V2.
[1]: http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standards
[2]: http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub
---
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
5 years, 11 months
[ OST Failure Report ] [ oVirt Master (otopi+imgbased) ] [ 11-05-2018 ] [ 001_initialize_engine.test_initialize_engine ]
by Dafna Ron
Hi,
We are failing in 001_initialize_engine.test_initialize_engine in the
upgrade suite.
the issue seems to be related to ovn configuration.
The changes reported by CQ are not the cause of this failure and I may be
mistaken but I suspect it may be related to one of the below changes.
*Link and headline of suspected patches: *
*https://gerrit.ovirt.org/#/c/90784/ <https://gerrit.ovirt.org/#/c/90784/>
- network: default ovn provider client is returned by
fixturehttps://gerrit.ovirt.org/#/c/90327/
<https://gerrit.ovirt.org/#/c/90327/> - backend, packing: Add default MTU
for tunnelled networksLink to Job:*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/>*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7488/
*Link to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/artifa...>(Relevant)
error snippet from the log: <error>*
2018-05-11 04:14:34,940-0400 DEBUG otopi.context
context._executeMethod:143 method exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133,
in _executeMethod
method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py",
line 779, in _customization
self._query_install_ovn()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py",
line 399, in _query_install_ovn
default=True
File "/usr/lib/python2.7/site-packages/ovirt_setup_lib/dialog.py",
line 47, in queryBoolean
default=true if default else false,
File "/usr/share/otopi/plugins/otopi/dialog/human.py", line 211, in
queryString
value = self._readline(hidden=hidden)
File "/usr/lib/python2.7/site-packages/otopi/dialog.py", line 248,
in _readline
raise IOError(_('End of file'))
IOError: End of file
2018-05-11 04:14:34,942-0400 ERROR otopi.context
context._executeMethod:152 Failed to execute stage 'Environment
customization': End of file
2018-05-11 04:14:34,972-0400 DEBUG
otopi.plugins.otopi.debug.debug_failure.debug_failure
debug_failure._notification:100 tcp connections:
id uid local foreign state pid exe
0: 0 0.0.0.0:111 0.0.0.0:0 LISTEN 1829 /usr/sbin/rpcbind
1: 29 0.0.0.0:662 0.0.0.0:0 LISTEN 1868 /usr/sbin/rpc.statd
2: 0 0.0.0.0:22 0.0.0.0:0 LISTEN 970 /usr/sbin/sshd
3: 0 192.168.201.2:3260 0.0.0.0:0 LISTEN UnknownPID UnknownEXE
4: 0 192.168.200.2:3260 0.0.0.0:0 LISTEN UnknownPID UnknownEXE
5: 0 0.0.0.0:892 0.0.0.0:0 LISTEN 1874 /usr/sbin/rpc.mountd
6: 0 0.0.0.0:2049 0.0.0.0:0 LISTEN UnknownPID UnknownEXE
7: 0 0.0.0.0:32803 0.0.0.0:0 LISTEN UnknownPID UnknownEXE
8: 0 192.168.201.2:22 192.168.201.1:33338 ESTABLISHED 5544 /usr/sbin/sshd
2018-05-11 04:14:34,973-0400 DEBUG otopi.context
context.dumpEnvironment:859 ENVIRONMENT DUMP - BEGIN
2018-05-11 04:14:34,973-0400 DEBUG otopi.context
context.dumpEnvironment:869 ENV BASE/error=bool:'True'
2018-05-11 04:14:34,973-0400 DEBUG otopi.context
context.dumpEnvironment:869 ENV BASE/exceptionInfo=list:'[(<type
'exceptions.IOError'>, IOError('End of file',), <traceback object at
0x239ab90>)]'
2018-05-11 04:14:34,974-0400 DEBUG otopi.context
context.dumpEnvironment:873 ENVIRONMENT DUMP - END
2018-05-11 04:14:34,975-0400 INFO otopi.context
context.runSequence:741 Stage: Clean up
2018-05-11 04:14:34,975-0400 DEBUG otopi.context
context.runSequence:745 STAGE cleanup
2018-05-11 04:14:34,976-0400 DEBUG otopi.context
context._executeMethod:128 Stage cleanup METHOD
otopi.plugins.otopi.dialog.answer_file.Plugin._generate_answer_file
2018-05-11 04:14:34,977-0400 DEBUG otopi.context
context.dumpEnvironment:859 ENVIRONMENT DUMP - BEGIN
2018-05-11 04:14:34,977-0400 DEBUG otopi.context
context.dumpEnvironment:869 ENV DIALOG/answerFileContent=str:'# OTOPI
answer file, generated by human dialog
[environment:default]
'
*</error>Thanks, Dafna*
5 years, 11 months
[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 10-05-2018 ] [ 001_upgrade_engine.test_initialize_engine ]
by Dafna Ron
Hi,
We are failing on deploying engine (test:
001_upgrade_engine.test_initialize_engine) duet to version bump for package
vdsm-jsonrpc.
Can you please revert the change or add the missing package?
Please note that all ovirt-engine changes are failing and there are a few.
*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/91084/ <https://gerrit.ovirt.org/#/c/91084/> -
jsonrpc: version bump Link to Job:*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7281/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7281/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7281/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7281/artifact>(Relevant)
error snippet from the log: <error>*
[ INFO ] Checking for an update for Setup...
[ ERROR ] Yum [u'ovirt-engine-backend-4.3.0-0.0.master.20180509105818.gitf6b5283.el7.centos.noarch
requires vdsm-jsonrpc-java >= 1.4.13']
[ INFO ] Yum Performing yum transaction rollback
[ ERROR ] Failed to execute stage 'Environment customization':
[u'ovirt-engine-backend-4.3.0-0.0.master.20180509105818.gitf6b5283.el7.centos.noarch
requires vdsm-jsonrpc-java >= 1.4.13']
[ INFO ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20180509074146-er3dzr.log
[ INFO ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20180509074148-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
("FATAL Internal error (main):
[u'ovirt-engine-backend-4.3.0-0.0.master.20180509105818.gitf6b5283.el7.centos.noarch
requires vdsm-jsonrpc-java >= 1.4.13']",)
lago.ssh: DEBUG: Command f3aaeaac on
lago-upgrade-from-release-suite-master-engine errors:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line 88, in main
installer.execute()
File "/usr/lib/python2.7/site-packages/otopi/main.py", line 157, in execute
self.context.runSequence()
File "/usr/lib/python2.7/site-packages/otopi/context.py", line 771,
in runSequence
util.raiseExceptionInformation(infos[0])
File "/usr/lib/python2.7/site-packages/otopi/util.py", line 81, in
raiseExceptionInformation
exec('raise info[1], None, info[2]')
File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133,
in _executeMethod
method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-common/distro-rpm/packages.py",
line 417, in _customization
) = self._checkForProductUpdate()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-common/distro-rpm/packages.py",
line 248, in _checkForProductUpdate
if mpm.buildTransaction():
File "/usr/lib/python2.7/site-packages/otopi/miniyum.py", line 920,
in buildTransaction
raise yum.Errors.YumBaseError(msg)
YumBaseError: [u'ovirt-engine-backend-4.3.0-0.0.master.20180509105818.gitf6b5283.el7.centos.noarch
requires vdsm-jsonrpc-java >= 1.4.13']
*</error>*
5 years, 11 months
Jupiter isn't just a planet - it's how we test things in the Engine
by Allon Mureinik
Hi all,
Over the past couple of days I've merged a series of patches that upgrade the Engine's usage of JUnit to JUnit Jupiter 5.2 in order to make it easier to write new tests.
Despite the patches being quite straight-forward, since Jupiter is **not** backwards compatible to JUnit 4, or even JUnit 5 Vintage, here's the short version of what you need to know:
1. You no longer use annotations from the org.junit package, but the junit.org.jupiter.api packages. @Before and @After have been replaced with @BeforeEach and AfterEach respectively, and @BeforeClass and @AfterClass have been replaced with @BeforeAll and @AfterAll.
2. Assertions now live in the org.junit.jupiter.api.Assertions class. Note that the optional message parameter is now the **last** parameter in all the assertions, not the first. Also worth noting that assertThat has been removed. More often than not, assertTrue can be used with a Supplier<Boolean>, but if you absolutely need an assertThat method, just use the one that Hamcrest provides.
3. Similarly, assumptions live in org.junit.jupiter.api.Assumptions.
4. No more runners - extending tests in JUnit Jupiter is done with Extensions. There's no longer a need for the Parameterized and Theories runners, as you can just use the built-in @ParameterizedTest annotation. Instead of the MockitoJUnitRunner we used to heavily rely on, you should now use MockitoExtension.
5. No more Rules - these too were replaced with extensions. Our custom rules were replaced with extensions, which are pretty well documented both in their javadoc and in wiki under [1]
For additional information, you could take a look at the official documentation [2]
Happy testing!
Allon
[1] https://ovirt.org/develop/dev-process/unit-testing-utilities/
[2] https://junit.org/junit5/docs/current/user-guide/
5 years, 11 months
Jupiter isn't just a planet - it's how we test things in the Engine
by Allon Mureinik
Hi all,
Over the past couple of days I've merged a series of patches that upgrade the Engine's usage of JUnit to JUnit Jupiter 5.2 in order to make it easier to write new tests.
Despite the patches being quite straight-forward, since Jupiter is **not** backwards compatible to JUnit 4, or even JUnit 5 Vintage, here's the short version of what you need to know:
1. You no longer use annotations from the org.junit package, but the junit.org.jupiter.api packages. @Before and @After have been replaced with @BeforeEach and AfterEach respectively, and @BeforeClass and @AfterClass have been replaced with @BeforeAll and @AfterAll.
2. Assertions now live in the org.junit.jupiter.api.Assertions class. Note that the optional message parameter is now the **last** parameter in all the assertions, not the first. Also worth noting that assertThat has been removed. More often than not, assertTrue can be used with a Supplier<Boolean>, but if you absolutely need an assertThat method, just use the one that Hamcrest provides.
3. Similarly, assumptions live in org.junit.jupiter.api.Assumptions.
4. No more runners - extending tests in JUnit Jupiter is done with Extensions. There's no longer a need for the Parameterized and Theories runners, as you can just use the built-in @ParameterizedTest annotation. Instead of the MockitoJUnitRunner we used to heavily rely on, you should now use MockitoExtension.
5. No more Rules - these too were replaced with extensions. Our custom rules were replaced with extensions, which are pretty well documented both in their javadoc and in wiki under [1]
For additional information, you could take a look at the official documentation [2]
Happy testing!
Allon
[1] https://ovirt.org/develop/dev-process/unit-testing-utilities/
[2] https://junit.org/junit5/docs/current/user-guide/
5 years, 11 months
