- Devel - oVirt List Archives

Re: [ovirt-devel] [VDSM] Live snapshot with ceph disks
by Christopher Pereira 20 Jun '15

20 Jun '15

Hi Nir, Regarding "3. Engine creates snapshot *via cinder*"... What are the benefits of creating snapshots via cinder vs via libvirt? Libvirt and qemu are offering core VM-aware storage and memory snapshot features. Besides, snapshot-create-as has no VM downtime. It would be a mistake to implement snapshoting on the ceph layer. At some point, you would need VM-aware code (eg: the VM memory state) and organically go back to the libvirt + qemu way. There seems to be qemu + libvirt support for ceph snapshots (via rbd commands) which probably offers some (?) VM-awareness, but what are the benefits of not using the good old core libvirt + qemu snapshot features? I must be missing something... 2) Not related: It seems like oVirt shifted focus towards Ceph recently... I would like to drop Gluster for Ceph if the latter supports SEEK HOLE reading and optimal sparse files operations. Can someone please confirm if Ceph is supporting SEEK_HOLE? I saw some related code, but would like to ask for comments before setting up and benchmarking Ceph sparse image file operations.

2 1

vdsm master is failing unit tests
by Sandro Bonazzola 18 Jun '15

18 Jun '15

Hi, just an heads up that vdsm on master is failing unit testing: 14:00:25 Ran 1090 tests in 92.403s 14:00:25 14:00:25 FAILED (SKIP=40, errors=7, failures=2) http://jenkins.ovirt.org/job/vdsm_master_unit-tests_merged/297/consoleFull Please fix it. -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 1

Re: [ovirt-devel] Libvirt secrets management - take 2
by Oved Ourfali 18 Jun '15

18 Jun '15

On Jun 12, 2015 14:21, Nir Soffer <nsoffer(a)redhat.com> wrote: > > Hi all, > > Recently support for Ceph network disk landed in master. It its possible > now to start a vm using Ceph network disk or hot-plug/unplug such disk > using Cephx authentication. > > However, to make it work, you must add the relevant Ceph secret to > libvirt manually, in the same way it is done in OpenStack deployment. > Our goal is to manage secrets automatically and use ephemeral (safer) > secrets. > > The next patches in the Ceph topic [1], implement secret management in > the same way we manage storage domains or server connections: > > The concept is - all hosts can use all secrets, so you can migrate a vm > using Ceph disk to any host in the cluster. > > 1. When host becomes up, we register the secrets associated with all the > current active domains with libvirt > > 2. When activating a domain, we register the secrets associated with the > new domain with libvirt > > 3. When deactivating a domain, we unregister the secrets associated with > the domain from libvirt > > 4. When moving host to maintenance, we clear all secrets > > 5. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep > stale or unneeded secrets on a host > > This system seems to work, but Federico pointed few issues and suggested > a new (simpler?) approach. > > In future libvirt version, libvirt will support the concept of transient > secrets so you can start a transient vm using secret without registering > the secret with libvirt before starting the vm. The secret will be > specified in the vm XML (for starting a vm) or disk XML (for hot-plug). > This will make our secret management system and APIs useless. When is this planned to be added to libvirt? iiuc, once added, you will no longer need to register the secrets with libvirt, right? > > Managing state on multiple hosts is hard; we will probably have to deal > with nasty edge cases (e.g. lost messages, network errors), which may > lead to host with missing secret, which cannot run some vms. We probably > do this right for storage domains (after 8 years?), and we should not > assume that we are smarter and secret management will work in the first > try. > > The new approach is to *not* manage state or multiple hosts. Instead, > send the required secrets only to the host that starting a vm or > hot-plugging a disk that need a libvirt secret: > > 1. When starting a vm, add the required secrets to the vm description. > On the host, vdsm will register these secrets with libvirt before > starting the vm. > > 2. When migrating a vm, add the required secrets to the vm description. > On the host, vdsm will send these secrets to the destination host, > and on the destination host, vdsm will register the secrets with libvirt > before starting the vm. > Will these secrets be part of the domain xml? If so, no need for special treatment in case of VM migration. > 3. When hot-plugging a disk, send the secret if needed in the disk > description. On the host, vdsm will register the secrets with libvirt. > > 4. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep > stale or unneeded secrets on a host > > 5. We never unregister secrets, since they are ephemeral anyway. > > 6. Alternatively, we can implement secrets reference counting so when a vm > stops or disk is hot-unplugged we decrease the reference count on the > secrets associated with this vm/disk, and if no other vms need the > secret, we can unregister the secret from libvirt. Doesn't seem necessary to me. > > The new approach is simpler, if we avoid the fancy secret reference > counting. I believe we can get it merged in couple of weeks with help > from the virt team. > > Please share your thoughts on these alternative solutions. > Keep in mind also hosted engine. Sounds to me like the new approach will be a better fit for it, as he won't need to deal with storage domain secrets, but just pass it in the VM description... Although I'm not a hosted engine expert, so not sure it makes a difference, but it sounds simpler in that aspect as well. > Thanks, > Nir > > [1] https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:c… > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > >

3 2

help with karma for mom
by Eyal Edri 17 Jun '15

17 Jun '15

Hi, can you please help with karma for new mom pkgs? https://admin.fedoraproject.org/updates/FEDORA-2015-9899/mom-0.4.5-2.fc22 https://admin.fedoraproject.org/updates/FEDORA-2015-9898/mom-0.4.5-2.fc21 https://admin.fedoraproject.org/updates/FEDORA-EPEL-2015-6686/mom-0.4.5-2.e… this will reduce custom hacks so that ci jobs will work with nightly builds (rather than take it from epel/fedora) -- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

3 2

oVirt UI log records persisted in browser's local storage
by Vojtech Szocs 17 Jun '15

17 Jun '15

Hi, with patches [1,2] merged recently into master branch, oVirt UI (WebAdmin and UserPortal) now persists client-side log records in browser's local storage [3] in addition to logging them via console.log API [4]. This means that client-side logs are kept in browser's storage between browser restarts, unless the user manually clears the storage. To inspect "localStorage" object in Firefox: * Menu / Developer / Toggle Tools * on Console tab, type "inspect(localStorage)" Client-side logs now use application-specific key prefix, e.g. "ENGINE_WebAdmin_MyKey" or "ENGINE_UserPortal_MyKey" to avoid clashes. For backwards compatibility (any existing items with un-prefixed keys), UI has a fallback to read the un-prefixed key when the prefixed one is missing. (Newly persisted items always use the key prefix.) In addition to changes above, any uncaught errors that originate from UI code are now properly caught and logged. This means that you shouldn't see uncaught JavaScript errors in browser's error console anymore. Regards, Vojtech [1] https://gerrit.ovirt.org/#/c/25444/ [2] https://gerrit.ovirt.org/#/c/41404/ [3] http://www.w3.org/TR/webstorage/#the-localstorage-attribute [4] https://developer.mozilla.org/en-US/docs/Web/API/Console/log

1 0

CI flag enabled to all projects
by David Caro 17 Jun '15

17 Jun '15

--47eKBCiAZYFK5l32 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi everyone, to ease our current issues, the CI flag has been enabled to all the projects. It will be the only flag that jenkins will set/unset and it c= an also be set by a maintainer. A +1 is required to enable the submit but a -1 will not block (so if jenkins -1 it and a maintainer +1 the +1 prevails). The permissions to the maintainers is added manually, that means that I mig= ht have messed some of them, if you have any issues ping me and I'll check. Any project that does not have any jenkins jobs, will have to get it set manually by a maintainer (or create a jenkins job, that would be best...). Thanks!! --=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com Web: www.redhat.com RHT Global #: 82-62605 --47eKBCiAZYFK5l32 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVgRszAAoJEEBxx+HSYmnDiQEIAIB+mB1/JWgaSpEyOnqwumQ/ oKKNUgtGK43dt29dxMSoXT5Z1nUxrvnwDNbx9fDqTY4/x/E2vNvIkR7CrxdI+B/R OGA01EB8x2ZxDxUNiZtRMvTwfwh2IFCgFd1tOujPGgcOxJRUgywWQVaiIuH5SfHv no7WyuEZjzF5JhBHEOUfGGbF/kS4YoDB93rDrwABLQMbk1sKnSe6M2DTpl3hTyBO or+Vbw1n2PmOk+YNQcsJVZr7msz1H4Cmwi1+4M9XdElR3Vj7tTuML7/PMlymiT7F FpetYzm1sHeKBNjtn3iqmfVbgV2DaK+LOH6XbL9B2DBK6g3vq0mqkh9Nt9SiSAI= =Onxt -----END PGP SIGNATURE----- --47eKBCiAZYFK5l32--

1 0

Postponing 3.6.0 second alpha to next week
by Sandro Bonazzola 17 Jun '15

17 Jun '15

Hi, due to instability of the current code, we need to postpone second alpha to next week. Maintainers: - please ensure that all packages you maintain builds in jenkins - please ensure that all the dependencies required by your packages are available either in ovirt repository or in an external repository included in ovirt-release repo files. - please ensure that engine is at least able to add a couple of hosts, create a vm on one host and migrate it to the other one. Infra: - please help keeping jenkins monitored and stabilizing it. Community: - please help testing nightly build while stabilizing it for the second alpha release. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 0

Libvirt secrets management - take 2
by Nir Soffer 16 Jun '15

16 Jun '15

Hi all, Recently support for Ceph network disk landed in master. It its possible now to start a vm using Ceph network disk or hot-plug/unplug such disk using Cephx authentication. However, to make it work, you must add the relevant Ceph secret to libvirt manually, in the same way it is done in OpenStack deployment. Our goal is to manage secrets automatically and use ephemeral (safer) secrets. The next patches in the Ceph topic [1], implement secret management in the same way we manage storage domains or server connections: The concept is - all hosts can use all secrets, so you can migrate a vm using Ceph disk to any host in the cluster. 1. When host becomes up, we register the secrets associated with all the current active domains with libvirt 2. When activating a domain, we register the secrets associated with the new domain with libvirt 3. When deactivating a domain, we unregister the secrets associated with the domain from libvirt 4. When moving host to maintenance, we clear all secrets 5. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep stale or unneeded secrets on a host This system seems to work, but Federico pointed few issues and suggested a new (simpler?) approach. In future libvirt version, libvirt will support the concept of transient secrets so you can start a transient vm using secret without registering the secret with libvirt before starting the vm. The secret will be specified in the vm XML (for starting a vm) or disk XML (for hot-plug). This will make our secret management system and APIs useless. Managing state on multiple hosts is hard; we will probably have to deal with nasty edge cases (e.g. lost messages, network errors), which may lead to host with missing secret, which cannot run some vms. We probably do this right for storage domains (after 8 years?), and we should not assume that we are smarter and secret management will work in the first try. The new approach is to *not* manage state or multiple hosts. Instead, send the required secrets only to the host that starting a vm or hot-plugging a disk that need a libvirt secret: 1. When starting a vm, add the required secrets to the vm description. On the host, vdsm will register these secrets with libvirt before starting the vm. 2. When migrating a vm, add the required secrets to the vm description. On the host, vdsm will send these secrets to the destination host, and on the destination host, vdsm will register the secrets with libvirt before starting the vm. 3. When hot-plugging a disk, send the secret if needed in the disk description. On the host, vdsm will register the secrets with libvirt. 4. When vdsm shutdown or starts, clear all secrets to ensure that we don't keep stale or unneeded secrets on a host 5. We never unregister secrets, since they are ephemeral anyway. 6. Alternatively, we can implement secrets reference counting so when a vm stops or disk is hot-unplugged we decrease the reference count on the secrets associated with this vm/disk, and if no other vms need the secret, we can unregister the secret from libvirt. The new approach is simpler, if we avoid the fancy secret reference counting. I believe we can get it merged in couple of weeks with help from the virt team. Please share your thoughts on these alternative solutions. Thanks, Nir [1] https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:c…

4 14

master broken
by Piotr Kliczewski 16 Jun '15

16 Jun '15

Guys, I attempted to add a host using latest master code and got following exception: 2015-06-16 10:01:28,854 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (http--0_0_0_0_0_0_0_0-8080-3) [14cc5839] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server 192.168.1.7,VDS_SECURITY_CONNECTION_ERROR,$ErrorMessage Couldn't store fingerprint to db for host 192.168.1.7: javax.persistence.PersistenceException: org.hibernate.id.IdentifierGenerationException: ids for this class must be manually assigned before calling save(): org.ovirt.engine.core.common.businessentities.VdsStatic,VDS_CANNOT_AUTHENTICATE_TO_SERVER 2015-06-16 10:01:41,724 ERROR [org.ovirt.engine.core.utils.transaction.TransactionalInterceptor] (http--0_0_0_0_0_0_0_0-8080-3) [740e96b9] Failed to run operation in a new transaction: javax.persistence.PersistenceException: org.hibernate.id.IdentifierGenerationException: ids for this class must be manually assigned before calling save(): org.ovirt.engine.core.common.businessentities.VdsStatic at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1361) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1289) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1295) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.hibernate.ejb.AbstractEntityManagerImpl.merge(AbstractEntityManagerImpl.java:876) [hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final] at org.jboss.as.jpa.container.AbstractEntityManager.merge(AbstractEntityManager.java:548) [jboss-as-jpa-7.1.1.Final.jar:7.1.1.Final] at org.ovirt.engine.core.dao.jpa.AbstractJpaDao.update(AbstractJpaDao.java:60) [dal.jar:] at org.ovirt.engine.core.dao.VdsStaticDAODbFacadeImpl$Proxy$_$$_WeldSubclass.update(VdsStaticDAODbFacadeImpl$Proxy$_$$_WeldSubclass.java) Thanks, Piotr

3 5

vdsm_master_unit-tests_created and vdsm_master_unit-tests_merged have been disabled
by Sandro Bonazzola 16 Jun '15

16 Jun '15

Hi, I disabled vdsm_master_unit-tests_created and vdsm_master_unit-tests_merged since they get stuck testing jsonrpc and cause jenkins job to grow over 500 jobs taking busy for days the slaves. Please either fix vdsm or the unit tests before enabling these jobs again. Thanks. -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

3 5