- Devel - oVirt List Archives

Re: [Engine-devel] Odd Host Activation with mgmt-if
by Dead Horse 03 Jun '13

03 Jun '13

Still puzzling on this all I get for an error when I see it is: WARN [org.ovirt.engine.core.bll. network.host.SetupNetworksCommand] (pool-5-thread-42) [21543ac] CanDoAction of action SetupNetworks failed. Reasons:VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORKS_ALREADY_ATTACHED_TO_IFACES,$NETWORKS_ALREADY_ATTACHED_TO_IFACES_LIST ovirtmgmt Nothing is logged VDSM wise other then the above standard host activation probe. - DHC On Mon, May 13, 2013 at 1:19 PM, Dead Horse <deadhorseconsulting(a)gmail.com>wrote: > Seeing this as of late when activating hosts. The odd this is that it > reports a failure to activate the host (EL6.4) but still does it anyways. > > Engine side: > > 2013-05-13 12:53:38,547 INFO > [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-5-thread-42) > [21543ac] Running command: HandleVdsVersionCommand internal: true. Entities > affected : ID: 15f11023-9746-49de-b33f-e3cc7dca6f65 Type: VDS > 2013-05-13 12:53:38,549 INFO > [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-5-thread-42) > [21543ac] FINISH, ActivateVdsVDSCommand, return: Host[durotar], log id: > 3796a7bd > 2013-05-13 12:53:38,625 WARN > [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] > (pool-5-thread-42) [21543ac] CanDoAction of action SetupNetworks failed. > Reasons:VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORKS_ALREADY_ATTACHED_TO_IFACES,$NETWORKS_ALREADY_ATTACHED_TO_IFACES_LIST > ovirtmgmt > 2013-05-13 12:53:39,368 INFO > [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-42) [21543ac] > Activate finished. Lock released. Monitoring can run now for host durotar > from data-center Azeroth > > VDSM Side: > > Thread-13::DEBUG::2013-05-13 > 12:53:37,841::BindingXMLRPC::933::vds::(wrapper) return getCapabilities > with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': > {'iSCSI': [{'InitiatorName': 'iqn.2012-09.net.azeroth:durotar'}], 'FC': > []}, 'packages2': {'kernel': {'release': '358.6.1.el6.x86_64', 'buildtime': > 1366751713.0, 'version': '2.6.32'}, 'glusterfs-rdma': {'release': > '0.3.alpha3.el6', 'buildtime': 1367604893L, 'version': '3.4.0'}, > 'glusterfs-fuse': {'release': '0.3.alpha3.el6', 'buildtime': 1367604893L, > 'version': '3.4.0'}, 'spice-server': {'release': '12.el6', 'buildtime': > 1361573005L, 'version': '0.12.0'}, 'vdsm': {'release': '17.el6', > 'buildtime': 1368196305L, 'version': '4.10.3'}, 'qemu-kvm': {'release': > '2.355.el6_4.2', 'buildtime': 1362691270L, 'version': '0.12.1.2'}, > 'qemu-img': {'release': '2.355.el6_4.2', 'buildtime': 1362691270L, > 'version': '0.12.1.2'}, 'libvirt': {'release': '18.el6_4.4', 'buildtime': > 1366301801L, 'version': '0.10.2'}, 'glusterfs': {'release': > '0.3.alpha3.el6', 'buildtime': 1367604893L, 'version': '3.4.0'}, 'mom': > {'release': '1.el6', 'buildtime': 1349470062L, 'version': '0.3.0'}, > 'glusterfs-server': {'release': '0.3.alpha3.el6', 'buildtime': 1367604893L, > 'version': '3.4.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU X5570 @ > 2.93GHz', 'hooks': {'before_vm_start': {'50_sriov': {'md5': > '3ebc60cd2e4eb089820102285fad7c45'}, '50_pincpu': {'md5': > '0b5fb99ff0e7acb9ad534b87c02c59e3'}, '50_qos': {'md5': > '18b596a6b4e4bad80357f240ba122a5e'}, '50_vmfex': {'md5': > '9f5abb892ddb6b3daa779985d38d9f55'}, '50_scratchpad': {'md5': > '7db25a4b8cb04f6e7132cb7c2300c111'}, '50_numa': {'md5': > '5008c2826714ac5b63748780aabd2f25'}, '50_floppy': {'md5': > '202fe18705a7d4c50c40c126e8f8dbe8'}, '50_qemucmdline': {'md5': > 'a884929ad6f5eb039887157288867409'}, '50_macspoof': {'md5': '25deea559772719 > > - DHC >

5 9

[Engine-devel] OS info wiki
by Roy Golan 03 Jun '13

03 Jun '13

Hi, A new feature page for OS info is out: http://www.ovirt.org/OS_info Please review. Roy

2 4

[Engine-devel] Disk BE very small refactoring
by Vered Volansky 02 Jun '13

02 Jun '13

Hi All, Please express your opinion if you have any - Currently Disk BE has a plugged property, which should be a property of the relationship between vm(or template) and a disk. I plan to remove this property from the Disk entity, and add new entity, called DeviceDisk. This should inherit from Disk and contain the vm/template guid and the plugged property at first round. At second round it'll also contain the readOnly property, for RO disks, TBD right after. Appreciate any input, Vered

7 11

[Engine-devel] cannot execute command in the class of InitVdsOnUpCommand
by Chen, Wei D 02 Jun '13

02 Jun '13

Hi, We suppose when one VDS is added into cluster, executeCommand method in InitVdsOnUpCommand.java will be invoked, so, some logic will be checked there. But per our experiment, this is depended on what we actually did. Here are some cases: 1. when reboot VDS, the logic in " executeCommand" will be invoked. 2. when re-deploy engine, the logic in " executeCommand" will be invoked. 3. when we add a new VDS (vdsm is running) into cluster, the logic in " executeCommand" will not be invoked. What we expect is the logic will also be invoked in the third case, but why it's not invoked? What's behind all of these? Best Regards, Dave Chen

2 1

[Engine-devel] [oVirt jenkins] Weekly report on open tasks for ovirt-engine
by Jenkins ci oVirt Server 01 Jun '13

01 Jun '13

------=_Part_4421_1536138156.1370041330863 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Files scanned: '**/*.java, **/*.py'. Strings searched: FIXME | TODO | @deprecated Report: http://jenkins.ovirt.org/job/ovirt_engine_scan_open_tasks/8/tasksResult/? ------=_Part_4421_1536138156.1370041330863--

1 0

[Engine-devel] ovirt fails to install on f19 from fedora repos
by Mike Burns 31 May '13

31 May '13

A couple problems came up today on IRC. * Current version of oVirt in Fedora is 3.1. * installing oVirt on F19 beta (from fedora repos, not ovirt.org) fails because classpathx-mail was removed from fedora and is a dependency for ovirt-engine-notification-service AFAICT, the notification-service was obsoleted in 3.2 at some point. Can we either get ovirt-engine rpms updated in F19 or removed completely? Mike

1 0

[Engine-devel] [JENKINS][GERRIT][ANN] new jenkins jobs added to run per patch on ovirt-engine
by Eyal Edri 31 May '13

31 May '13

fyi, starting from now a new jenkins job [1] will run on every patch in ovirt-engine that has java files in it. the job will verify: 1. compilation 2. findbugs (HIGH + NORMAL) if your patch will fail in one of the above, jenkins will give -1 to the patch and you'll have to send a new patchset to fix it. if the failure was not justified (infra error/network/etc...) - contact infra(a)ovirt.org and you can re-trigger your job manually [2], by searching for your commit sha or changeid. Eyal Edri ovirt infra team. [1] http://jenkins.ovirt.org/job/ovirt_engine_find_bugs_gerrit/ [2] http://jenkins.ovirt.org/gerrit_manual_trigger/?

1 0

[Engine-devel] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
by Vinzenz Feenstra 30 May '13

30 May '13

This is a multi-part message in MIME format. --------------020305070807050907000005 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Please find the prettier version on the wiki: http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval Proposal VDSM - Engine Data Statistics Retrieval VDSM <=> Engine data retrieval optimization Motivation: Currently the RHEVM engine is polling the a lot of data from VDSM every 15 seconds. This should be optimized and the amount of data requested should be more specific. For each VM the data currently contains much more information than actually needed which blows up the size of the XML content quite big. We could optimize this by splitting the reply on the getVmStats based on the request of the engine into sections. For this reason Omer Frenkel and me have split up the data into parts based on their usage. This data can and usually does change during the lifetime of the VM. Rarely Changed: This data is change not very frequent and it should be enough to update this only once in a while. Most commonly this data changes after changes made in the UI or after a migration of the VM to another Host. *Status* = Running *acpiEnable* = true *vmType* = kvm *guestName* = W864GUESTAGENTT *displayType* = qxl *guestOs* = Win 8 *kvmEnable* = true #/*this should be constant and never changed*/ *pauseCode* = NOERR *monitorResponse* = 0 *session* = Locked # unused *netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}] *appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] *pid* = 11314 *guestIPs* = 10.34.60.148 # duplicated info *displayIp* = 0 *displayPort* = 5902 *displaySecurePort* = 5903 *username* = user@W864GUESTAGENTT *clientIp* = *lastLogin* = 1361976900.67 Often Changed: This data is changed quite often however it is not necessary to update this data every 15 seconds. As this is cumulative data and reflects the current status, and it does not need to be snapshotted every 15 seconds to retrieve statistics. The data can be retrieved in much more generous time slices. (e.g. Every 5 minutes) *network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}} *disksUsage* = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 'used': '3490912256'}] *timeOffset* = 14422 *elapsedTime* = 68591 *hash* = 2335461227228498964 *statsAge* = 0.09 # unused Often Changed but unused This data does not seem to be used in the engine at all. It is *not* even used in the data warehouse. *memoryStats* = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': '1466884'} *balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152} *disks* = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549', 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}} Very frequent uppdates needed by webadmin portal: This data is mostly needed for the webadmin portal and might be required to be updated quite often. An exception here is the statsAge field, which seems to be unused by the Engine. This data could be requested every 15 seconds to keep things as they are now. *cpuSys* = 2.32 *cpuUser* = 1.34 *memUsage* = 30 Proposed Solution for VDSM & Engine: We will introduce new optional parameters to getVmStats, getAllVmStats and list to allow a finer grained specification of data which should be included. *Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats only) *Allowed values:* * full (default to keep backwards compatibility) * app-list (Just send the application list) * rare (include everything from rarely changed to very frequent) * often (include everything from often changed to very frequent) * frequent (only send the very frequently changed items) *Parameter:* *clientId*=*<string>* The client id is specified by the client and should be unique however constantly used. *Parameter:* *diff*=*<boolean>* In combination with the clientId VDSM will send only differences to the previous request from the named clientId. (if diff=true) Additional Change: Besides the introduction of the new parameters for list, getVmStats and getAllVmStats it might make sense to include a hash for the appList into the rarely changed section of the response which would allow to identify changes and avoid having to sent the complete appList every so often and only if the hash known to the client is outdated. *Note:* The appList (Application List) reported by the guest agent could be fully implemented on request only, as long as the guest agent installed supports this. As there seems to be a request to have the complete list of installed applications on all guests this data could be quite extensive and a huge list. On the other hand this data is only rarely visible and therefore it should not be requested all the time and only on demand. Improvement of the Guest Agent: As part of the proposed solution it is necessary to improve the guest agent as well. For the full application list there should be implemented a caching system which will be fully reactive and should not poll the application list for example all the time. The guest can create a prepared data file containing all data in the JSON format (as used for the communication with VDSM via VIO) and just have to read that file from disk and directly sends it to VDSM. However it is quite possible that this list is to big and it might have to be chunked into pieces. (Multiple messages, which would have to be supported by VDSM then as well) The solution for this is to make VDSM request this data and it will retrieve the data necessary on request only. -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com --------------020305070807050907000005 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> </head> <body bgcolor="#FFFFFF" text="#000000"> Please find the prettier version on the wiki: <a class="moz-txt-link-freetext" href="http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval">http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval</a> <h1 id="firstHeading" class="firstHeading page-header">Proposal VDSM - Engine Data Statistics Retrieval</h1> <h2>VDSM <=> Engine data retrieval optimization </h2> <h3> Motivation: </h3> Currently the RHEVM engine is polling the a lot of data from VDSM every 15 seconds. This should be optimized and the amount of data requested should be more specific. For each VM the data currently contains much more information than actually needed which blows up the size of the XML content quite big. We could optimize this by splitting the reply on the getVmStats based on the request of the engine into sections. For this reason Omer Frenkel and me have split up the data into parts based on their usage. This data can and usually does change during the lifetime of the VM. <h4> Rarely Changed: </h4> This data is change not very frequent and it should be enough to update this only once in a while. Most commonly this data changes after changes made in the UI or after a migration of the VM to another Host. <pre> Status = Running acpiEnable = true vmType = kvm guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8 kvmEnable = true # this should be constant and never changed pauseCode = NOERR monitorResponse = 0 session = Locked # unused netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid = 11314 guestIPs = 10.34.60.148 # duplicated info displayIp = 0 displayPort = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT clientIp = lastLogin = 1361976900.67 </pre> <h4> Often Changed: </h4> This data is changed quite often however it is not necessary to update this data every 15 seconds. As this is cumulative data and reflects the current status, and it does not need to be snapshotted every 15 seconds to retrieve statistics. The data can be retrieved in much more generous time slices. (e.g. Every 5 minutes) <pre> network = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}} disksUsage = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 'used': '3490912256'}] timeOffset = 14422 elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09 # unused </pre> <h4> Often Changed but unused </h4> This data does not seem to be used in the engine at all. It is not even used in the data warehouse. <pre> memoryStats = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': '1466884'} balloonInfo = {'balloon_max': 2097152, 'balloon_cur': 2097152} disks = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549', 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}} </pre> <h4> Very frequent uppdates needed by webadmin portal: </h4> This data is mostly needed for the webadmin portal and might be required to be updated quite often. An exception here is the statsAge field, which seems to be unused by the Engine. This data could be requested every 15 seconds to keep things as they are now. <pre> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 </pre> <h2> Proposed Solution for VDSM & Engine: </h2> We will introduce new optional parameters to getVmStats, getAllVmStats and list to allow a finer grained specification of data which should be included. Parameter: statsType=<string> (getVmStats, getAllVmStats only) Allowed values: <ul> <li> full (default to keep backwards compatibility) </li> <li> app-list (Just send the application list) </li> <li> rare (include everything from rarely changed to very frequent) </li> <li> often (include everything from often changed to very frequent) </li> <li> frequent (only send the very frequently changed items) </li> </ul> Parameter: clientId=<string> The client id is specified by the client and should be unique however constantly used. Parameter: diff=<boolean> In combination with the clientId VDSM will send only differences to the previous request from the named clientId. (if diff=true) <h3> Additional Change: </h3> Besides the introduction of the new parameters for list, getVmStats and getAllVmStats it might make sense to include a hash for the appList into the rarely changed section of the response which would allow to identify changes and avoid having to sent the complete appList every so often and only if the hash known to the client is outdated. Note: The appList (Application List) reported by the guest agent could be fully implemented on request only, as long as the guest agent installed supports this. As there seems to be a request to have the complete list of installed applications on all guests this data could be quite extensive and a huge list. On the other hand this data is only rarely visible and therefore it should not be requested all the time and only on demand. <h3> Improvement of the Guest Agent: </h3> As part of the proposed solution it is necessary to improve the guest agent as well. For the full application list there should be implemented a caching system which will be fully reactive and should not poll the application list for example all the time. The guest can create a prepared data file containing all data in the JSON format (as used for the communication with VDSM via VIO) and just have to read that file from disk and directly sends it to VDSM. However it is quite possible that this list is to big and it might have to be chunked into pieces. (Multiple messages, which would have to be supported by VDSM then as well) The solution for this is to make VDSM request this data and it will retrieve the data necessary on request only. <pre class="moz-signature" cols="72">-- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com</pre> </body> </html> --------------020305070807050907000005--

8 23

[Engine-devel] oVirt Scheduler API Design
by Gilad Chaplik 30 May '13

30 May '13

Hi all, Attached links for oVirt Scheduler API design wiki page. High level overview: http://www.ovirt.org/Features/oVirtScheduler Detailed design: http://www.ovirt.org/Features/oVirtSchedulerAPI You are more than welcome to share you thoughts. Thanks, Gilad.

4 4

[Engine-devel] findbugs [normal priority] - *please fix ASAP*
by Einav Cohen 29 May '13

29 May '13

lvernia: find bugs error: http://jenkins.ovirt.org/job/ovirt_engine_find_bugs/4323/findbugsResult/NOR… in file: frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/Uri.java caused by patch: http://gerrit.ovirt.org/#/c/14695/ [webadmin: Added validation to ProviderModel, merged: May 27] ---- mkublin: find bugs error: http://jenkins.ovirt.org/job/ovirt_engine_find_bugs/4323/findbugsResult/NOR… in file: backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/RunVmCommandBase.java caused by patch: http://gerrit.ovirt.org/#/c/13740/ [engine: Fixing perfomance issue between VdsRunTimeInfo and RunVmCommandBase, merged: Apr 10] ---- Thanks, Einav

3 3