- Devel - oVirt List Archives

Re: [Engine-devel] [vdsm] RFD: NEW API getAllTasks
by Doron Fediuck 08 May '12

08 May '12

On 07/05/12 21:33, Adam Litke wrote: > The current APIs for retrieving all task information do not actually return all > task information. I would like to introduce a new API that corrects this and > other issues with the current API while preserving backwards compatibility with > ovirt-engine for as long as is necessary. > > The current APIs: > > getAllTasksInfo(spUUID=None, options = None): > - Returns a dictionary that maps a task UUID to a task verb. > - Despite having 'all' in the name, this API only returns tasks that have an > 'spm' tag. > - This call returns only one piece of information for each task. > - The spUUID parameter is deprecated and ignored. > > getAllTasksStatuses(spUUID=None, options = None): > - Returns a dictionary of task status information. > - Despite having 'all' in the name, this API only returns tasks that have an > 'spm' tag. > - The spUUID parameter is deprecated and ignored. > > > I propose the following new API: > > getAllTasks(tag=None, options=None): > - Returns a dictionary of task information. The info from both of the above > functions would be merged into a single result set. > - If tag is None, all tasks are returned. otherwise, only tasks matching the > tag are returned. > - The spUUID parameter is dropped. options is for future extension and is > currently not used. > > This new API includes all functionality that is available in the old calls. In > the future, ovirt-engine could switch to this API and preserve the current > semantics by passing tag='spm' to getAllTasks. Meanwhile, API users that really > want all tasks (gluster and the REST API) can get what they need. > > Thoughts on this idea? > (Adding engine-devel, as this relates to the engine API). AFAIR, in the original design (when a-sync tasks where introduced into vdsm), most (if not all) of the tasks were SPM tasks, and this is the reason for this behavior. Improving the API is welcomed. The suggested design should work. I'd like to verify: - Backwards compatibility works; so running engine's shouldn't be replaced. Dan: any news on this? - Going forward with potential changes in SPM concepts should be supported as well. Dan/Ayal/Livnat: do you think it works? ie- anything else needed than alternate 'spm' tag? -- /d "All computers wait at the same speed."

2 1

Re: [Engine-devel] Supporting native USB in oVirt
by Eli Mesika 08 May '12

08 May '12

----- Original Message ----- > From: "Dan Kenigsberg" <danken(a)redhat.com> > To: "Oved Ourfalli" <ovedo(a)redhat.com>, "Dave Allan" <dallan(a)redhat.com> > Cc: "Dave Allan" <dallan(a)redhat.com>, "Jiri Denemark" <jdenemar(a)redhat.com>, "Michal Privoznik" > <mprivozn(a)redhat.com>, "Itamar Heim" <iheim(a)redhat.com>, "Igor Lvovsky" <ilvovsky(a)redhat.com>, "Eli Mesika" > <emesika(a)redhat.com>, "Hans de Goede" <hdegoede(a)redhat.com>, "Andrew Cathrow" <acathrow(a)redhat.com> > Sent: Tuesday, May 8, 2012 11:41:25 AM > Subject: Re: Supporting native USB in oVirt > > On Tue, May 08, 2012 at 03:19:32AM -0400, Oved Ourfalli wrote: > > Hi, > > > > We are now working on adding the support for native USB devices on > > oVirt. > > As far as we understand, in order to use it we need to pass the > > following devices with the following restrictions (according to > > the EHCI spec): > > 1. EHCI USB controller - with the highest function number, #7. > > > > devices='{type:controller,device:usb,model:ich9-ehci1,address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x7}}' > > > > 2. 3 UHCI USB controllers, on the same bus and PCI slot as the EHCI > > controller. Set the multifunction bit to on, on the controller > > with function #0. > > > > devices='{type:controller,device:usb,model:ich9-uhci1,master:{startport:0},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x0,multifunction:on}}' > > devices='{type:controller,device:usb,model:ich9-uhci2,master:{startport:2},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x1}}' > > devices='{type:controller,device:usb,model:ich9-uhci3,master:{startport:4},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x2}}' > > I'd like to make it clear that the suggested API request Engine to > pass > a "master" attribute for the controller device, with a nested > dictionary > within. This makes sense, as I am certain that libvirt's modeling of > the > device has merits. However, AFAIK Engine does not do stuff like that > at > the momemnt. And should not do in future IMHO, this should be transparent to the engine core that should know only the USB device. > > > > > 3. USB redirect devices (according to the needed number of USB > > slots, maximum 6) on the same bus, each one having a different > > port. > > > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:1}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:2}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:3}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:4}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:5}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:6}}' > > > > 4. If we want more than 6 USB slots, we need to have 2 EHCI > > controllers, and 6 UHCI controllers, that are consistent with the > > restrictions above, on different bus. > > (we need them to be on different bus, since the connection between > > the redirect devices and the controllers is the bus). > > > > According to Hans (cc-ed), if we let libvirt pick its own > > addresses, it will put each controller of the composite USB > > controller device on its own pci slot, which is a violation of the > > EHCI spec. > > Is there an open libvirt bug for this? > > > > > The problem is that the oVirt engine is not aware of addresses. > > They aren't managed by it. > > They are chosen automatically by libvirt, returned to the engine, > > and they saved in the engine database in order to provide the > > ability to retain the same PCI addresses after VM restart. > > Yes, we should be able to trust libvirt's assignement, certainly for > the > masses of VMs. Few VMs of certain customer may want to tweak the > location of their devices. Maybe Engine does not need to expose this > feature at the moment (I think it is better that each device address > is > considered a "blob" by Engine), but the Engine/Vdsm API should > support > that. I know that in future we may need to tweak the device location. But still, the general principle of address allocation should be kept simple: If you send an address, this address is honored by libvirt If you don't, you will get a valid one from libvirt. I think that engine should not handle any addresses apart from persisting the address in DB. > > > > > So, in order to support the EHCI spec, oVirt will have to be aware > > of addresses, manage them (allocate them, check for collisions, > > update on every libvirt change regarding that, etc...). Obviously, > > it doesn't feel right, and in fact it is also not feasible, to > > manage these addresses in the oVirt domain. > > That's not obvious to me. Engine could have a hard-coded addresses > for > the usb device gang. Libvirt could arrange the rest of the devices > around it, so Engine should not worry about collisions. Libvirt's > stable > API relieves Engine from worrying about "every libvirt change > regarding > that" - it is libvirt responsibility to make sure that every future > libvirt version would handle the USB addresses properly. A big hole for future bugs that will fall between libvirt & engine responsibility. I don't like that idea of spreading responsibilities between separate components and prefer that addresses will have one manager that is obviously libvirt. > > > > > Is all the above correct, or are we missing something? > > If so, can you address the issues above, and provide the ability to > > manage these devices in libvirt? > > > > Regards, > > Oved >

1 0

Re: [Engine-devel] Supporting native USB in oVirt
by Oved Ourfalli 08 May '12

08 May '12

cc-ing engine-devel. Oved ----- Original Message ----- > From: "Hans de Goede" <hdegoede(a)redhat.com> > To: "Oved Ourfalli" <ovedo(a)redhat.com> > Cc: "Dave Allan" <dallan(a)redhat.com>, "Jiri Denemark" <jdenemar(a)redhat.com>, "Michal Privoznik" > <mprivozn(a)redhat.com>, "Itamar Heim" <iheim(a)redhat.com>, "Igor Lvovsky" <ilvovsky(a)redhat.com>, "Eli Mesika" > <emesika(a)redhat.com>, "Dan Kenigsberg" <danken(a)redhat.com>, "Andrew Cathrow" <acathrow(a)redhat.com> > Sent: Tuesday, May 8, 2012 10:48:32 AM > Subject: Re: Supporting native USB in oVirt > > Hi, > > On 05/08/2012 09:19 AM, Oved Ourfalli wrote: > > Hi, > > > > We are now working on adding the support for native USB devices on > > oVirt. > > As far as we understand, in order to use it we need to pass the > > following devices with the following restrictions (according to > > the EHCI spec): > > 1. EHCI USB controller - with the highest function number, #7. > > > > devices='{type:controller,device:usb,model:ich9-ehci1,address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x7}}' > > > > 2. 3 UHCI USB controllers, on the same bus and PCI slot as the EHCI > > controller. Set the multifunction bit to on, on the controller > > with function #0. > > > > devices='{type:controller,device:usb,model:ich9-uhci1,master:{startport:0},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x0,multifunction:on}}' > > devices='{type:controller,device:usb,model:ich9-uhci2,master:{startport:2},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x1}}' > > devices='{type:controller,device:usb,model:ich9-uhci3,master:{startport:4},address:{type:pci,domain:0x0000,bus:0x00,slot:0x04,function:0x2}}' > > > > 3. USB redirect devices (according to the needed number of USB > > slots, maximum 6) on the same bus, each one having a different > > port. > > > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:1}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:2}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:3}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:4}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:5}}' > > devices='{type:redir,device:spicevmc,bus:usb,address:{type:usb,bus:0,port:6}}' > > > > To the best of my knowledge, the above is all correct. > > > 4. If we want more than 6 USB slots, we need to have 2 EHCI > > controllers, and 6 UHCI controllers, that are consistent with the > > restrictions above, on different bus. > > (we need them to be on different bus, since the connection between > > the redirect devices and the controllers is the bus). > > Correct, note that this may change in the future. Specifically I'm > thinking about > adding support for USB-2 hubs, and then have libvirt automatically > add new > redir-devices when all but one are in use. This is all just an idea > at the moment, > so don't count on it, I just wanted you to know that we are thinking > about making > it easier to make the number of devices dynamic in the future. So for > now you > could consider simply limiting redirection to max 6 devices. > > > According to Hans (cc-ed), if we let libvirt pick its own > > addresses, it will put each controller of the composite USB > > controller device on its own pci slot, which is a violation of the > > EHCI spec. > > Correct. > > > The problem is that the oVirt engine is not aware of addresses. They aren't managed by it. > > They are chosen automatically by libvirt, returned to the engine, and they saved in the engine database in order to provide the ability to retain the same PCI addresses after VM restart. > > > >So, in order to support the EHCI spec, oVirt will have to be aware of addresses, manage them (allocate them, check for collisions, update on every libvirt change regarding that, etc...). Obviously, it doesn't feel right, and in fact it is also not feasible, to manage these addresses in the oVirt domain. > > > > Is all the above correct, or are we missing something? > > If so, can you address the issues above, and provide the ability to manage these devices in libvirt? > Regards, > > Hans >

1 0

Re: [Engine-devel] Problem with detachment of host interface using ovirt-sdk
by Itzik Brown 07 May '12

07 May '12

Juan, I used the repository git://gerrit.ovirt.org/ovirt-engine-sdk Commit 3c721e60ab3af3ad07e1a20bc6fdbcdbc1344df0 The test script : #!/usr/bin/python import sys from ovirtsdk.api import API from ovirtsdk.xml import params OVIRT_ENGINE_SERVER = 'server1' OVIRT_ENGINE_PORT = '8080' OVIRT_ENGINE_USER = 'admin@internal' OVIRT_ENGINE_PASSWORD = 'password' url = 'http://%s:%s' % (OVIRT_ENGINE_SERVER, OVIRT_ENGINE_PORT) api = API(url = url, username = OVIRT_ENGINE_USER , password = OVIRT_ENGINE_PASSWORD) host = 'host1' if_name = 'eth4' net_name = 'net1' nic = api.hosts.get(name=host).nics.get(name=if_name) net = params.Network(name=net_name) act = params.Action(network=net) nic.detach(act) api.hosts.get(name=host).commitnetconfig() Thanks, Itzik

2 1

[Engine-devel] [Jenkins] new High priority bug in ovirt-engine
by Eyal Edri 06 May '12

06 May '12

fyi, a new HIGH priority bug was pushed to ovirt-engine, please check. bug details: http://jenkins.ovirt.org/job/ovirt_engine_find_bugs/765/findbugsResult/HIGH… Found in file: VmPropertiesUtils.java:362, GC_UNRELATED_TYPES Introduced after these commits: engine: Create Gluster Volume enhancements (detail / gitweb) core: removing async tasks and compensation data on upgrade (detail / gitweb) engine: support versions for vm properties (detail / gitweb) webadmin: support versions for vm properties (detail / gitweb) core: Fixing bll tests (detail / gitweb) engine: New query to fetch Gluster Volume by ID (detail / gitweb) Eyal.

2 1

[Engine-devel] vds_bootstrap.py 's residency
by Shu Ming 03 May '12

03 May '12

Hi, I am checking the VDSM and ovirt-engine workspace for "vds_bootstrap.py" file. It was found that vds_bootstrap.py file was in VDSM workspace and was packaged into vdsm-bootstrap rpm package. Also, it was found that in the host installation process, host node will try to get the "vds_bootstrap.py" from the engine server. But "vds_bootstrap.py" is not included in any engine packages. Does that mean we should install vdsm-bootstrap package into engine server? Why not package this file into engine packages instead of vdsm packages? -- Shu Ming<shuming(a)linux.vnet.ibm.com> IBM China Systems and Technology Laboratory

3 2

Re: [Engine-devel] Logical network Usages collection problematic approach
by Eoghan Glynn 03 May '12

03 May '12

> The new design of logical network 'usages' collection came out a bit > problematic or shall i say annoying. > The idea is to send the entire collection elements every time a user > wants to update a single usage otherwise the missing elements will be > automatically removed from the collection. > > Example: > having <usages><usage>VM</usage><usages> > 1. in order to add 'display' usage to the collection i must send 'vm' > as well. > 2. to remove an element from the collection, i must send the entire > collection without the desired element > (note: in this case it is only one extra element for every update but > in other cases it could be much more) Yep, this seems to conflict with the general idiom around the interpretation of missing properties in a PUT representation - i.e. any properties omitted from the representation are ignored and not changed. > The solution should be: > <usages><vm>true</vm><display>false</display></usages> > That way we can send a single usage having different boolean text > without including the entire collection elements. I wonder would a POST/DELETE idiom be more natural, if the individual usages would be pressed into service as a psuedo-OID, e.g: POST .../networks/network_id/usages HTTP/1.1 <usage id="VM"/> 201 Created Location: .../networks/network_id/usages/VM <usage id="VM" href=".../networks/network_id/usages/VM"/> -- GET .../networks/network_id/usages HTTP/1.1 200 OK Location: .../networks/network_id/usages/VM <usages> <usage id="VM" href=".../networks/network_id/usages/VM"/> </usages> -- POST .../networks/network_id/usages HTTP/1.1 <usage id="display"/> 201 Created Location: .../networks/network_id/usages/display <usage id="display" href=".../networks/network_id/usages/display"/> -- GET .../networks/network_id/usages HTTP/1.1 200 OK <usages> <usage id="VM" href=".../networks/network_id/usages/VM"/> <usage id="display" href=".../networks/network_id/usages/display"/> <usages> -- DELETE .../networks/network_id/usages/VM 204 No Content -- GET .../networks/network_id/usages HTTP/1.1 200 OK <usages> <usage id="display" href=".../networks/network_id/usages/display"/> <usages> So from each usage would appear like a first class resource from the perspective of the creation/deletion idiom. If the client providing the ID is considered inconsistent, this could be a "name" child element instead, e.g.: POST .../networks/network_id/usages HTTP/1.1 <usage> <name>display</name> </usage> 201 Created Location: .../networks/network_id/usages/display <usage id="display" href=".../networks/network_id/usages/display"> <name>display</name> </usage> Cheers, Eoghan

2 2

[Engine-devel] Naming convention in db upgrade script temporary functions
by Eli Mesika 03 May '12

03 May '12

Hi >From now on, please use the prefix __temp_<function name> for temporary functions defined in a db upgrade scripts The reasons are 1) Each time that you have to rename your upgrade script after sync with master you have to rename the function as well 2) As a result, may function are pushed with a wrong suffix (for example upgrade 03_01_0950 had a function name suffix of 08200) Since the reason for that numbering was in the first place for avoiding dropping real installed functions, now you will have just to put in a temporary function created in an upgrade script just the prefix "__temp_" Hope that this will make handling upgrade scripts more accurate and less painful. This was documented as well at the oVirt db upgrade wiki page: http://www.ovirt.org/wiki/OVirt-database-upgrade-procedure (see section 1.5 Temporary Functions in Upgrade scripts)

1 0

[Engine-devel] Fwd: Checkstyle- Webadmin & userportal localization
by Alona Kaplan 02 May '12

02 May '12

Hi, Yesterday some changes were made in oVirt's checkstyle (http://gerrit.ovirt.org/#change,3760) 1. Renaming built-tools to build-tools-root and separating it into two packages: * checksyles- for checkstyle xml files. * ovirt-checkstyle-extension- for adding new (not built-in) checkstyle checks. 2. Adding to the checkstyle.xml a new check- NlsCheck, which fails the compilation in case non-externalized strings appear in the code. [To understand more about the need of this check, see http://gerrit.ovirt.org/#change,3612] >From now on, all strings in the web-admin/user-portal java code should be one of the following: - Externalized (in case the string should be localized) - Have a "//$NON-NLS-N$" comment next to them (in case the string shouldn't be externalized, e.g. HashName of a component) If there is a non externalized string in a web-admin/user-portal java file and there is no "//$NON-NLS-N$" comment next to it, mvn compilation will fail. Note: NlsCheck is currently configured to run only on web-admin/user-portal projects. If you want NlsCheck check to run on another project, please add the following to the configuration of the checkstyle in the project's pom file: <propertyExpansion>runNlsCheck=true</propertyExpansion> Alona.

3 2

[Engine-devel] Updating ovirt engine from 3.0 to the 3.1.0 version
by Shu Ming 02 May '12

02 May '12

Hi, I built a set of ovirt engine RPM packages from near the latest ovirt engine source code. However, when I try to access the engine server with URL "http://ovirt-engine-112:80", the browser displayed a blank page without http error returned. Any clue about what is going on here? Why didn't the engine administrator home page be shown? And the packages were also installed successfully in my target system, checked this by "rpm -q -a|grep engine". I) upgraded the packages to the 3.1 [root@ovirt-engine-112 ~]# rpm -q -a |grep engine ovirt-engine-restapi-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-dbscripts-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-log-collector-3.0.0_0001-1.6.fc16.x86_64 ovirt-engine-genericapi-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-backend-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-setup-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-jbossas-1.2-2.fc16.x86_64 ovirt-engine-jboss-deps-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-config-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-webadmin-portal-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-userportal-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-tools-common-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-image-uploader-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-notification-service-3.1.0_0001-1.8.fc16.x86_64 ovirt-engine-iso-uploader-3.1.0_0001-1.8.fc16.x86_64 gtk2-engines-2.20.2-2.fc15.x86_64 Step II) Then I used "engine-setup" to setup the engine. It seems that everything is ok, see the log below. ------------- [root@ovirt-engine-112 ~]# engine-setup Welcome to oVirt Engine setup utility HTTP Port [80] : HTTPS Port [443] : Host fully qualified domain name, note that this name should be fully resolvable [ovirt-engine-112] : ERROR: domain is not a valid domain name User input failed validation, do you still wish to use it? (yes|no): yes Password for Administrator (admin@internal) : Warning: Weak Password. Confirm password : Organization Name for the Certificate: cstl The default storage type you will be using ['NFS'| 'FC'| 'ISCSI'] [NFS] : Enter DB type for installation ['remote'| 'local'] [local] : Local database password : Warning: Weak Password. Confirm password : Should the installer configure NFS share on this server to be used as an ISO Domain? ['yes'| 'no'] [yes] : Mount point path: /iso-dom ERROR: mount point already exists in /etc/exports Mount point path: /iso-dom Display name for the ISO Domain: iso-domains Firewall ports need to be opened. You can let the installer configure iptables automatically overriding the current configuration. The old configuration will be backed up. Alternately you can configure the firewall later using an example iptables file found under /usr/share/ovirt-engine/conf/iptables.example Configure iptables ? ['yes'| 'no']: yes oVirt Engine will be installed using the following configuration: ================================================================= http-port: 80 https-port: 443 host-fqdn: ovirt-engine-112 auth-pass: ******** org-name: cstl default-dc-type: NFS db-remote-install: local db-local-pass: ******** nfs-mp: /iso-dom iso-domain-name: iso-domains config-nfs: yes override-iptables: yes Proceed with the configuration listed above? (yes|no): yes Installing: Configuring oVirt-engine... [ DONE ] Creating CA... [ DONE ] Editing JBoss Configuration... [ DONE ] Setting Database Configuration... [ DONE ] Setting Database Security... [ DONE ] Creating Database... [ DONE ] Updating the Default Data Center Storage Type... [ DONE ] Editing oVirt Engine Configuration... [ DONE ] Editing Postgresql Configuration... [ DONE ] Configuring the Default ISO Domain... [ DONE ] Configuring Firewall (iptables)... [ DONE ] Starting JBoss Service... [ DONE ] Handling HTTPD... [ DONE ] **** Installation completed successfully ****** (Please allow oVirt Engine a few moments to start up.....) Additional information: * SSL Certificate fingerprint: C6:01:83:93:4B:2C:2A:38:65:C8:49:C9:17:34:FE:4B:1C:10:D5:FF * SSH Public key fingerprint: 69:8c:bd:05:43:17:0a:43:a3:cc:62:7e:f7:be:0c:42 * A default ISO share has been created on this host. If IP based access restrictions are required, please edit /iso-dom entry in /etc/exports * The firewall has been updated, the old iptables configuration file was saved to /usr/share/ovirt-engine/conf/iptables.backup.011513-04282012_2225 * The installation log file is available at: /var/log/ovirt-engine/engine-setup_2012_04_28_01_13_01.log * Please use the user "admin" and password specified in order to login into oVirt Engine * To configure additional users, first configure authentication domains using the 'engine-manage-domains' utility * To access oVirt Engine please go to the following URL: http://ovirt-engine-112:80 [root@ovirt-engine-112 ~]# -- Shu Ming<shuming(a)linux.vnet.ibm.com> IBM China Systems and Technology Laboratory

2 8