- Devel - oVirt List Archives

Libvirt secrets management - take 2
by Nir Soffer 16 Jun '15

16 Jun '15

Hi all, Recently support for Ceph network disk landed in master. It its possible now to start a vm using Ceph network disk or hot-plug/unplug such disk using Cephx authentication. However, to make it work, you must add the relevant Ceph secret to libvirt manually, in the same way it is done in OpenStack deployment. Our goal is to manage secrets automatically and use ephemeral (safer) secrets. The next patches in the Ceph topic [1], implement secret management in the same way we manage storage domains or server connections: The concept is - all hosts can use all secrets, so you can migrate a vm using Ceph disk to any host in the cluster. 1. When host becomes up, we register the secrets associated with all the current active domains with libvirt 2. When activating a domain, we register the secrets associated with the new domain with libvirt 3. When deactivating a domain, we unregister the secrets associated with the domain from libvirt 4. When moving host to maintenance, we clear all secrets 5. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep stale or unneeded secrets on a host This system seems to work, but Federico pointed few issues and suggested a new (simpler?) approach. In future libvirt version, libvirt will support the concept of transient secrets so you can start a transient vm using secret without registering the secret with libvirt before starting the vm. The secret will be specified in the vm XML (for starting a vm) or disk XML (for hot-plug). This will make our secret management system and APIs useless. Managing state on multiple hosts is hard; we will probably have to deal with nasty edge cases (e.g. lost messages, network errors), which may lead to host with missing secret, which cannot run some vms. We probably do this right for storage domains (after 8 years?), and we should not assume that we are smarter and secret management will work in the first try. The new approach is to *not* manage state or multiple hosts. Instead, send the required secrets only to the host that starting a vm or hot-plugging a disk that need a libvirt secret: 1. When starting a vm, add the required secrets to the vm description. On the host, vdsm will register these secrets with libvirt before starting the vm. 2. When migrating a vm, add the required secrets to the vm description. On the host, vdsm will send these secrets to the destination host, and on the destination host, vdsm will register the secrets with libvirt before starting the vm. 3. When hot-plugging a disk, send the secret if needed in the disk description. On the host, vdsm will register the secrets with libvirt. 4. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep stale or unneeded secrets on a host 5. We never unregister secrets, since they are ephemeral anyway. 6. Alternatively, we can implement secrets reference counting so when a vm stops or disk is hot-unplugged we decrease the reference count on the secrets associated with this vm/disk, and if no other vms need the secret, we can unregister the secret from libvirt. The new approach is simpler, if we avoid the fancy secret reference counting. I believe we can get it merged in couple of weeks with help from the virt team. Please share your thoughts on these alternative solutions. Thanks, Nir [1] https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:c…

4 14

master broken
by Piotr Kliczewski 16 Jun '15

16 Jun '15

Guys, I attempted to add a host using latest master code and got following exception: 2015-06-16 10:01:28,854 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (http--0_0_0_0_0_0_0_0-8080-3) [14cc5839] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server 192.168.1.7,VDS_SECURITY_CONNECTION_ERROR,$ErrorMessage Couldn't store fingerprint to db for host 192.168.1.7: javax.persistence.PersistenceException: org.hibernate.id.IdentifierGenerationException: ids for this class must be manually assigned before calling save(): org.ovirt.engine.core.common.businessentities.VdsStatic,VDS_CANNOT_AUTHENTICATE_TO_SERVER 2015-06-16 10:01:41,724 ERROR [org.ovirt.engine.core.utils.transaction.TransactionalInterceptor] (http--0_0_0_0_0_0_0_0-8080-3) [740e96b9] Failed to run operation in a new transaction: javax.persistence.PersistenceException: org.hibernate.id.IdentifierGenerationException: ids for this class must be manually assigned before calling save(): org.ovirt.engine.core.common.businessentities.VdsStatic at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1361) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1289) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1295) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.merge(AbstractEntityManagerImpl.java:876) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.jboss.as.jpa.container.AbstractEntityManager.merge(AbstractEntityManager.java:548) [jboss-as-jpa-7.1.1.Final.jar:7.1.1.Final] at org.ovirt.engine.core.dao.jpa.AbstractJpaDao.update(AbstractJpaDao.java:60) [dal.jar:] at org.ovirt.engine.core.dao.VdsStaticDAODbFacadeImpl$Proxy$_$$_WeldSubclass.update(VdsStaticDAODbFacadeImpl$Proxy$_$$_WeldSubclass.java) Thanks, Piotr

3 5

vdsm_master_unit-tests_created and vdsm_master_unit-tests_merged have been disabled
by Sandro Bonazzola 16 Jun '15

16 Jun '15

Hi, I disabled vdsm_master_unit-tests_created and vdsm_master_unit-tests_merged since they get stuck testing jsonrpc and cause jenkins job to grow over 500 jobs taking busy for days the slaves. Please either fix vdsm or the unit tests before enabling these jobs again. Thanks. -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

3 5

Repository closure is currently failing for 3.5
by Sandro Bonazzola 16 Jun '15

16 Jun '15

Hi, repository closure is currently failing for 3.5 on: 01:18:15 package: ovirt-optimizer-jboss-0.9-1.el6.noarch from check-custom-el6 01:18:15 unresolved deps: 01:18:15 ovirt-engine-wildfly >= 0:8.2 Since Wildfly support is targeted to 3.6 please provide a 3.5 branch for ovirt-optimizer or give ack for removing ovirt-optimizer from published rpms. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

2 1

VDSM from 3.5 branch is currently failing on Fedora 20
by Sandro Bonazzola 16 Jun '15

16 Jun '15

VDSM from 3.5 branch is currently failing on Fedora 20: make[1]: Entering directory `/builddir/build/BUILD/vdsm-4.16.20' make check-local make[2]: Entering directory `/builddir/build/BUILD/vdsm-4.16.20' python -c 'import pyflakes; print("pyflakes-%s" % pyflakes.__version__)' pyflakes-0.7.3 find . -path './.git' -prune -type f -o \ -name '*.py' -o -name '*.py.in' | xargs /usr/bin/pyflakes | \ grep -w -v "\./vdsm/storage/lvm\.py.*: list comprehension redefines 'lv' from line .*" | \ while read LINE; do echo "$LINE"; false; done ./tests/functional/networkTests.py:80: redefinition of unused 'vdsm' from line 43 ./tests/functional/networkTests.py:88: redefinition of unused 'vdsm' from line 43 make[2]: Leaving directory `/builddir/build/BUILD/vdsm-4.16.20' make[2]: *** [check-local] Error 1 make[1]: *** [check-am] Error 2 make[1]: Leaving directory `/builddir/build/BUILD/vdsm-4.16.20' make: *** [check-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.fmuLh5 (%check) Please fix it. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

2 1

Automated builds and releases
by David Caro 16 Jun '15

16 Jun '15

--kkRamCq5m5VQq0L6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi everyone! In an effort to improve the project workflow and ease the maintenance and improve the quality of the project releases I want to propose start working towards automated builds and releases, the main ideas are the following: * Stop building differently for release and non-release: - Building only once, testing what you build and release what you test - Don't use two different version strings, one for testing and one for release * Automate the build process, and the release process, directly getting the code from the repos (no manual build tarballs) * Adopt semantic versioning, it's a lot more meaningful than the current sc= heme and fits very well with the above points This will ease and lower the maintenance and the extra work required by maintainers, release engineers (sandro) and infra itself by making releases= as easy as hitting a button at any time. That will allow us to lower the time features and fixes get to the users, and deliver packages and builds that h= ave passed through all the tests we have, instead of rebuilding on another env,= at another time, by someone else, and passing only manual testing. wdyt? --=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com Web: www.redhat.com RHT Global #: 82-62605 --kkRamCq5m5VQq0L6 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVcZDBAAoJEEBxx+HSYmnDT08IAJcqZK4lgNV1DqOTvfXrB7Tl 7f7ceRfBezUVVnwQigvDuUz7LqT9SPO9jaNonFzHHuEiBanaug7W+YUdsFPyIEXF khJriJRocs2brZKFlL4IGfHC1yQGh6FomILAJkIUBWyzo+YJWhtobkfz1346q2dl 8B8k7ReZxTCnrB3Rb+dU0i4XY75IGKX0GFMOHQx9Ws0cFbhQBrqd7vW8svTmuFcY 6mtdzvQBmARqYM/wwkC9lLVUAZ0zlfsEoPAU2JSaSCXNEQkd0cy5XLua1oofUVuv t4epBTRu4RXTwblGwcCqrxC5XUoSerHWM2O7Yv5yaXYa6LP4OXSOy4cvEfju+Zk= =8dph -----END PGP SIGNATURE----- --kkRamCq5m5VQq0L6--

5 6

[ANN] oVirt 3.5.3 Second Release Candidate is now available for testing
by Sandro Bonazzola 16 Jun '15

16 Jun '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The oVirt team is pleased to announce that the 3.5.3 Second Release Candidate is now available for testing as of May 28th 2015. The release candidate is available now for Fedora 20, Red Hat Enterprise Linux 6.6, CentOS 6.6 (or similar) and Red Hat Enterprise Linux 7.1, CentOS 7.1 (or similar). This release of oVirt includes numerous bug fixes and one CVE fix for the VENOM Vulnerability. See the release notes [1] for a list of the new features and bugs fixed. Please refer to release notes [1] for Installation / Upgrade instructions. A new oVirt Live ISO is already available as well[2]. Please note that mirrors[3] may need usually one day before being synchronized. Please refer to the release notes for known issues in this release. [1] http://www.ovirt.org/OVirt_3.5.3_Release_Notes [2] http://resources.ovirt.org/pub/ovirt-3.5-pre/iso/ [3] http://www.ovirt.org/Repository_mirrors#Current_mirrors - -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJVZrrzAAoJEHw3u57E4QAOnX4QAMFi+d4Vq5jnZ0e+LaD1vB9R GmT+k21q6vybjIvovPh7q6StCtPS0sg15BoH5JTmTfl5IWL01qMsFvw/G32PIjZI u3mm2Z3evbxPg9qsqJPyJm/jAlRHv/L6ntLJaJBDPDSO6Y0iqiHTVUQG16easW3L ZnUC2IM5TpHuU3PcgVp8zY9gqFqQWnnsA8deJXBvif9J8RdSf7DnkFha9/Ei3v0E 1btkzt0XVS3Cate8pEaTT8zSoE/tOBBS/D65HsnnZdD13RAxYkolzuPvT+7BlmUO QMhyIE7msWex048HrcDVTEDLrkYF2SpAmYix3sA2O9+AIujenpICHJ8JyxAc+BmJ scsvv99iDLU3wkpMwKiIABj7nE8gIiUbyZBNaxsxQJ0jurLZGHGIhQGLAJ2q/rd7 MTkcDXOBOZHEadqs87mSEzHt1pQCmZ40KSJCvyWbqa7vFv6kyOgOs3PldoizsOyV Z1DlDmfMM8hs/6LOP/Mr/4p/l3JgS3gbOy34L2q9J/B+BX/XU7H2bOk6GTEMVPOn av7m2kDs8DlQJzSqZAURZNt6XTnGGw05THOBWD8NmHuvlVZsSF1mxKb0BC+LpRcs QmLGAtT12Y2Wnq1xwzcRav9+wiIeEYTVJCbyohMLyqTHqPgRstpeF3JulpqILo5I rHhGapreYDlzy06Ki7X7 =pUqF -----END PGP SIGNATURE-----

2 1

latest master
by Alexander Wels 15 Jun '15

15 Jun '15

Hi, I just rebased and re-ran the engine setup to make sure I have the latest schema. I am getting the following exception when starting the engine: Caused by: org.postgresql.util.PSQLException: The column name ksm_merge_across_nodes was not found in this ResultSet. at org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2542) at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBoolean(AbstractJdbc2ResultSet.java:2390) at org.jboss.jca.adapters.jdbc.WrappedResultSet.getBoolean(WrappedResultSet.java:615) at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl$VdsGroupRowMapper.mapRow(VdsGroupDAODbFacadeImpl.java:319) [dal.jar:] at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl$VdsGroupRowMapper.mapRow(VdsGroupDAODbFacadeImpl.java:267) [dal.jar:] at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] ... 68 more It appears the code is looking for a column that wasn't added to the schema? ksm_merge_across_nodes Can someone merge the column into master so I am not stuck? thanks.

3 2

Fwd: New architecture in Copr: PPC64LE
by Sandro Bonazzola 15 Jun '15

15 Jun '15

FYI -------- Messaggio Inoltrato -------- Oggetto: New architecture in Copr: PPC64LE Data: Mon, 15 Jun 2015 15:38:59 +0200 Mittente: Miroslav Suchy <msuchy(a)redhat.com> Rispondi-a: Development discussions related to Fedora <devel(a)lists.fedoraproject.org> A: Development discussions related to Fedora <devel(a)lists.fedoraproject.org> Hi, I just enabled new architecture in Copr - PowerPC 64 LE (little endian). There are those chroots available: * fedora-21-ppc64le * fedora-22-ppc64le * fedora-rawhide-ppc64le This happened as co-operation of Red Hat, Brno University of Technology and IBM. I would like to thanks D, Horak and J. Capik from Secondary Architectures for cooperation and making this happen. To enable this architecture on existing project, you need to navigate to your project -> click on Edit tab -> choose PPC64LE chroots -> Save and then resubmit your SRPMs. Enjoy and happy hacking Mirek Suchy -- devel mailing list devel(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

3 2

Select storage domain when creating VM from template with python-SDK
by Philip Strömbäck 15 Jun '15

15 Jun '15

Hi, I have a question about python-SDK for Ovirt Engine API. When creating a new VM from a template in Clone/Independent-mode, I'm not unable to select storage domain for the new disk. It seems like it always get created on the same disk at the template. I can move the disk after creation but it's a little bit time consuming. Example: vm_params = params.VM ( name = vm_name, cluster = api.clusters.get ( "Cluster-1" ), template = api.templates.get ( "RHEL7" ), storage_domain = api.storagedomains.get ( "iscsi1" ), disks = params.Disks ( clone = True ) ) try: api.vms.add ( vm_params ) except Exception, e: raise SystemExit ( "Could not create VM from template: %s" % ( e ) ) The VM is added correctly but the disk is located on iscsi0 ( the same as the template ). Have anybody an idea how to get this sorted out? Best regards, Philip

2 1