April 2013 - Devel - oVirt List Archives

[Engine-devel] ovirt-engine-sdk-java 1.0.0.6-1 released
by Michael Pasternak 25 Dec '13

25 Dec '13

- added new collection ClusterGlusterVolumeGlusterBrickStatistics - added new properties to the GlusterBrick - to vm added cpu.mode - to host install action added "image" parameter - ignore case in factory method lookup More details can be found at [1]. [1] http://www.ovirt.org/Java-sdk-changelog -- Michael Pasternak RedHat, ENG-Virtualization R&D

1 10

[Engine-devel] ovirt-sdk-python 3.3.0.1 released
by Michael Pasternak 25 Dec '13

25 Dec '13

- added new collection ClusterGlusterVolumeBrickStatistics - added NetUp 'cloned disk' importing capabilities - to ClusterGlusterVolumeBricks.list() added new argument "all_content" - to ClusterGlusterVolumeBricks.get() added new argument "all_content" - to host install method added new parameter "image" - to host add method added new parameter "display.address" - to vm add method added new parameter "vm.cpu.mode" - to vm add method added new overload for creating vm from snapshot - to cluster added tunnel_migration - to display added keyboard_layout - to display added smartcard_enabled - to DataCenter added Clusters sub-collection - to StorageDomain added Disks sub-collection - to Disks.list() added 'unregistered' argument - to Host added display.address - to Snapshot added commit/preview/undo actions - to vms.add() added overload for creating vm from snapshot - to hosts.add() added [@param host.display.address: string] - to vm.ticket() added [@param action.ticket.value: string] - return error if amount of fetched objects in X.get() is > 1 #890327 - session based authentication for /localhost is broken #916285 - explicitly ask for reply in xml - Add constructor parameter validate_cert_chain=True #915225 - defend against malformed server reply For more details see [1]. [1] http://wiki.ovirt.org/Python-sdk-changelog -- Michael Pasternak RedHat, ENG-Virtualization R&D

1 5

[Engine-devel] ovirt-cli 3.3.0.1 released
by Michael Pasternak 14 Nov '13

14 Nov '13

- at vm.start() --vm-os-boot doesn't send the order of devices #921464 - rephrase status command help - add option to retrieve system summary #854369 - accept IP address as FQ argument rather than string #886067 - fix broken pipe - Bad error message when trying to create a new Role #908284 - add flag --dont-validate-cert-chain #915231 - collection-based-options could be passed in 2 ways #859684 - make NO_SUCH_ACTION error a bit more clear - ovirt-cli DistributionNotFound exception on f18 #881011 - ovirt-shell misleading help for command "connect" #907943 - show event -id accept strings instead of numeric values #886786 - Use vncviewer passwordFile instead of passwdInput More details can be found at [1]. [1] http://wiki.ovirt.org/Cli-changelog -- Michael Pasternak RedHat, ENG-Virtualization R&D

1 5

[Engine-devel] Cloud-Init integration
by Greg Padgett 17 Jun '13

17 Jun '13

Hi Everyone, I'd like to propose a feature we've been doing some investigation into, which is to integrate cloud-init support into oVirt. Cloud-init is used to help provision new Linux systems by setting the hostname, ip, ssh keys, timezone, injecting files, and more. It's used by OpenStack (amongst others) now, and has a lot of features that may be helpful to our users. Details are still evolving, but for more info please see the wiki page: http://www.ovirt.org/Features/Cloud-Init_Integration All feedback is welcome! Thanks, Greg

8 11

[Engine-devel] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
by Vinzenz Feenstra 30 May '13

30 May '13

This is a multi-part message in MIME format. --------------020305070807050907000005 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Please find the prettier version on the wiki: http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval Proposal VDSM - Engine Data Statistics Retrieval VDSM <=> Engine data retrieval optimization Motivation: Currently the RHEVM engine is polling the a lot of data from VDSM every 15 seconds. This should be optimized and the amount of data requested should be more specific. For each VM the data currently contains much more information than actually needed which blows up the size of the XML content quite big. We could optimize this by splitting the reply on the getVmStats based on the request of the engine into sections. For this reason Omer Frenkel and me have split up the data into parts based on their usage. This data can and usually does change during the lifetime of the VM. Rarely Changed: This data is change not very frequent and it should be enough to update this only once in a while. Most commonly this data changes after changes made in the UI or after a migration of the VM to another Host. *Status* = Running *acpiEnable* = true *vmType* = kvm *guestName* = W864GUESTAGENTT *displayType* = qxl *guestOs* = Win 8 *kvmEnable* = true #/*this should be constant and never changed*/ *pauseCode* = NOERR *monitorResponse* = 0 *session* = Locked # unused *netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}] *appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] *pid* = 11314 *guestIPs* = 10.34.60.148 # duplicated info *displayIp* = 0 *displayPort* = 5902 *displaySecurePort* = 5903 *username* = user@W864GUESTAGENTT *clientIp* = *lastLogin* = 1361976900.67 Often Changed: This data is changed quite often however it is not necessary to update this data every 15 seconds. As this is cumulative data and reflects the current status, and it does not need to be snapshotted every 15 seconds to retrieve statistics. The data can be retrieved in much more generous time slices. (e.g. Every 5 minutes) *network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}} *disksUsage* = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 'used': '3490912256'}] *timeOffset* = 14422 *elapsedTime* = 68591 *hash* = 2335461227228498964 *statsAge* = 0.09 # unused Often Changed but unused This data does not seem to be used in the engine at all. It is *not* even used in the data warehouse. *memoryStats* = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': '1466884'} *balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152} *disks* = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549', 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}} Very frequent uppdates needed by webadmin portal: This data is mostly needed for the webadmin portal and might be required to be updated quite often. An exception here is the statsAge field, which seems to be unused by the Engine. This data could be requested every 15 seconds to keep things as they are now. *cpuSys* = 2.32 *cpuUser* = 1.34 *memUsage* = 30 Proposed Solution for VDSM & Engine: We will introduce new optional parameters to getVmStats, getAllVmStats and list to allow a finer grained specification of data which should be included. *Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats only) *Allowed values:* * full (default to keep backwards compatibility) * app-list (Just send the application list) * rare (include everything from rarely changed to very frequent) * often (include everything from often changed to very frequent) * frequent (only send the very frequently changed items) *Parameter:* *clientId*=*<string>* The client id is specified by the client and should be unique however constantly used. *Parameter:* *diff*=*<boolean>* In combination with the clientId VDSM will send only differences to the previous request from the named clientId. (if diff=true) Additional Change: Besides the introduction of the new parameters for list, getVmStats and getAllVmStats it might make sense to include a hash for the appList into the rarely changed section of the response which would allow to identify changes and avoid having to sent the complete appList every so often and only if the hash known to the client is outdated. *Note:* The appList (Application List) reported by the guest agent could be fully implemented on request only, as long as the guest agent installed supports this. As there seems to be a request to have the complete list of installed applications on all guests this data could be quite extensive and a huge list. On the other hand this data is only rarely visible and therefore it should not be requested all the time and only on demand. Improvement of the Guest Agent: As part of the proposed solution it is necessary to improve the guest agent as well. For the full application list there should be implemented a caching system which will be fully reactive and should not poll the application list for example all the time. The guest can create a prepared data file containing all data in the JSON format (as used for the communication with VDSM via VIO) and just have to read that file from disk and directly sends it to VDSM. However it is quite possible that this list is to big and it might have to be chunked into pieces. (Multiple messages, which would have to be supported by VDSM then as well) The solution for this is to make VDSM request this data and it will retrieve the data necessary on request only. -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com --------------020305070807050907000005 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> </head> <body bgcolor="#FFFFFF" text="#000000"> Please find the prettier version on the wiki: <a class="moz-txt-link-freetext" href="http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval">http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval</a> <h1 id="firstHeading" class="firstHeading page-header">Proposal VDSM - Engine Data Statistics Retrieval</h1> <h2>VDSM <=> Engine data retrieval optimization </h2> <h3> Motivation: </h3> Currently the RHEVM engine is polling the a lot of data from VDSM every 15 seconds. This should be optimized and the amount of data requested should be more specific. For each VM the data currently contains much more information than actually needed which blows up the size of the XML content quite big. We could optimize this by splitting the reply on the getVmStats based on the request of the engine into sections. For this reason Omer Frenkel and me have split up the data into parts based on their usage. This data can and usually does change during the lifetime of the VM. <h4> Rarely Changed: </h4> This data is change not very frequent and it should be enough to update this only once in a while. Most commonly this data changes after changes made in the UI or after a migration of the VM to another Host. <pre> Status = Running acpiEnable = true vmType = kvm guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8 kvmEnable = true # this should be constant and never changed pauseCode = NOERR monitorResponse = 0 session = Locked # unused netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid = 11314 guestIPs = 10.34.60.148 # duplicated info displayIp = 0 displayPort = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT clientIp = lastLogin = 1361976900.67 </pre> <h4> Often Changed: </h4> This data is changed quite often however it is not necessary to update this data every 15 seconds. As this is cumulative data and reflects the current status, and it does not need to be snapshotted every 15 seconds to retrieve statistics. The data can be retrieved in much more generous time slices. (e.g. Every 5 minutes) <pre> network = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}} disksUsage = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 'used': '3490912256'}] timeOffset = 14422 elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09 # unused </pre> <h4> Often Changed but unused </h4> This data does not seem to be used in the engine at all. It is not even used in the data warehouse. <pre> memoryStats = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': '1466884'} balloonInfo = {'balloon_max': 2097152, 'balloon_cur': 2097152} disks = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549', 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}} </pre> <h4> Very frequent uppdates needed by webadmin portal: </h4> This data is mostly needed for the webadmin portal and might be required to be updated quite often. An exception here is the statsAge field, which seems to be unused by the Engine. This data could be requested every 15 seconds to keep things as they are now. <pre> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 </pre> <h2> Proposed Solution for VDSM & Engine: </h2> We will introduce new optional parameters to getVmStats, getAllVmStats and list to allow a finer grained specification of data which should be included. Parameter: statsType=<string> (getVmStats, getAllVmStats only) Allowed values: <ul> <li> full (default to keep backwards compatibility) </li> <li> app-list (Just send the application list) </li> <li> rare (include everything from rarely changed to very frequent) </li> <li> often (include everything from often changed to very frequent) </li> <li> frequent (only send the very frequently changed items) </li> </ul> Parameter: clientId=<string> The client id is specified by the client and should be unique however constantly used. Parameter: diff=<boolean> In combination with the clientId VDSM will send only differences to the previous request from the named clientId. (if diff=true) <h3> Additional Change: </h3> Besides the introduction of the new parameters for list, getVmStats and getAllVmStats it might make sense to include a hash for the appList into the rarely changed section of the response which would allow to identify changes and avoid having to sent the complete appList every so often and only if the hash known to the client is outdated. Note: The appList (Application List) reported by the guest agent could be fully implemented on request only, as long as the guest agent installed supports this. As there seems to be a request to have the complete list of installed applications on all guests this data could be quite extensive and a huge list. On the other hand this data is only rarely visible and therefore it should not be requested all the time and only on demand. <h3> Improvement of the Guest Agent: </h3> As part of the proposed solution it is necessary to improve the guest agent as well. For the full application list there should be implemented a caching system which will be fully reactive and should not poll the application list for example all the time. The guest can create a prepared data file containing all data in the JSON format (as used for the communication with VDSM via VIO) and just have to read that file from disk and directly sends it to VDSM. However it is quite possible that this list is to big and it might have to be chunked into pieces. (Multiple messages, which would have to be supported by VDSM then as well) The solution for this is to make VDSM request this data and it will retrieve the data necessary on request only. <pre class="moz-signature" cols="72">-- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com</pre> </body> </html> --------------020305070807050907000005--

8 23

[Engine-devel] ovirt user portal and attach a cd
by Andrej Bagon 17 May '13

17 May '13

Hi all, we would like to limit a user as much as possible and give him only actions that he needs with his VM. If we disallow a user the right in system|configure system|manipulate permissions), we give the user, in user portal, only basic mode. But then three is no option to attach a cd in the basic user portal. In basic mode user interface there is no chance in attaching a cd (as this is a basic operation a solution to this would be a button attaching a cd when starting a vm). If we allow the user to manipulate permissions, we get the extended menu back and the user is limited, but we found unacceptable when he is adding permission to the VM, he sees all the users (name, surname, email!) in the ovirt authentication mechanism? A solution to this would be authenticating to the directory service as a user an not as ovirt admin user - meaning that the user will see only users that we allowed he sees in the ldap directory. Thank you. Best Regards, Andrej

4 8

[Engine-devel] [ANN] Development environment and Gentoo packaging (preview)
by Alon Bar-Lev 16 May '13

16 May '13

Hello All, We[1] are working to rewrite the current installer to something more portable and flexible. Currently two milestones had been reached: 1. Ability to install almost fully functioning ovirt-engine at $HOME for development environment. 2. Porting ovirt-engine to run on different distribution, Gentoo. We will be glad to receive feedback on either. Code is located at github[2] for now, at otopi branch. Instructions for setting up development environment are available[3]. Gentoo overlay for live ebuilds is available[4], the following packages are valid: - app-emulation/otopi - app-emulation/ovirt-host-deploy - app-emulation/ovirt-jboss-bin - app-emulation/ovirt-engine Why Gentoo first? because source based distributions demands the highest level of customization, solving the complex issue ease to continue porting to binary based distributions. Please remember that this is work in progress, and not guarantee to be stable or even work... The installer was re-written from scratch so expect issues at this point. Any feedback is welcomed, we are focusing first in providing the functionality of the existing installer to be able to replace it entirely before going into new adventures. Regards, Alon Bar-Lev. [1] Alex Lourie, Sandro Bonazzola, Alon Bar-Lev [2] https://github.com/alonbl/ovirt-engine [3] https://github.com/alonbl/ovirt-engine/blob/otopi/README.developer [4] https://github.com/alonbl/ovirt-overlay

2 1

[Engine-devel] Snapshots with RAM feature
by Arik Hadas 16 May '13

16 May '13

Hi All, The proposed feature will make it possible to run a VM which was reverted to live snapshot or created from live snapshot with the same memory state as it was at the moment the live snapshot was taken. http://www.ovirt.org/Features/RAM_Snapshots All feedback is welcome! Regards, Arik

5 8

[Engine-devel] spice-xpi(firefox) plugin for Windows
by Christophe Fergeau 13 May '13

13 May '13

--tsOsTdHNUZQcU9Ye Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi everyone, Lately I've been hacking on getting the spice-xpi plugin to work on Windows. This is the Firefox plugin that is used on Linux to implement the portal SPICE console. This work has now been merged upstream ( http://cgit.freedesktop.org/spice/spice-xpi/ ), and I've pushed a scratch build using mingw available from http://koji.fedoraproject.org/koji/taskinfo?taskID=5265107 (this will go away in several days). Copying this dll + its dependencies to the C:\Program Files\Mozilla Firefox\plugins directory are enough to get a spice-xpi plugin to show up in about:plugins, and to then be able to start a SPICE client from the spice-xpi test page ( http://teuf.fedorapeople.org/plugins/test.html ). The dependencies I had to copy are: gspawn-win32-helper-console.exe gspawn-win32-helper.exe iconv.dll libffi-6.dll libgcc_s_sjlj-1.dll libgio-2.0-0.dll libglib-2.0-0.dll libgmodule-2.0-0.dll libgobject-2.0-0.dll libgthread-2.0-0.dll libintl-8.dll libstdc++-6.dll pthreadGC2.dll zlib1.dll (I picked them from the various mingw-* fedora packages). The reason for this email is that to be usable from oVirt, some portal changes are needed as it currently only tries to use the spice-xpi plugin on Linux platforms. Another fuzzy bit is how to get this plugin to the users' machines... And that's about all I had to say in that email ;) Let me know if that's useful to oVirt, how you want to integrate this, ... Cheers, Christophe --tsOsTdHNUZQcU9Ye Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) iEYEARECAAYFAlFumXkACgkQJKRp+3pW945+vwCglgqxiHD3g6EOC12U9Ydg8pVS kmMAoIRXM5MepmgnvK+UJMF1rjiVJ4r/ =QcGM -----END PGP SIGNATURE----- --tsOsTdHNUZQcU9Ye--

5 6

[Engine-devel] VDSM Fake
by Libor Spevak 12 May '13

12 May '13

Hi, let me introduce an experimental project called VDSM Fake. This is a small Java web application simulating selected VDSM features, but only from the Engine point of view. The aim was to get oVirt-Engine top performance characteristics in core features simulating hundreds of fake hosts and VMs, and in ecologic way. More info: http://www.ovirt.org/VDSM_Fake Sources: git clone git://github.com/lspevak/ovirt-vdsmfake.git git clone git://github.com/lspevak/ovirt-restapiconf.git Regards, Libor

5 6