[ovirt-devel] Re: device compatibility interface for live migration with assigned devices

On Wed, Aug 19, 2020 at 02:57:34PM +0800, Jason Wang wrote: > On 2020/8/19 上午11:30, Yan Zhao wrote: >> hi All, >> could we decide that sysfs is the interface that every VFIO vendor driver >> needs to provide in order to support vfio live migration, otherwise the >> userspace management tool would not list the device into the compatible >> list? >> >> if that's true, let's move to the standardizing of the sysfs interface. >> (1) content >> common part: (must) >> - software_version: (in major.minor.bugfix scheme) > > This can not work for devices whose features can be negotiated/advertised > independently. (E.g virtio devices) > sorry, I don't understand here, why virtio devices need to use vfio interface?

I think this thread is discussing about vfio related devices. >> - device_api: vfio-pci or vfio-ccw ... >> - type: mdev type for mdev device or >> a signature for physical device which is a counterpart for >> mdev type. >> >> device api specific part: (must) >> - pci id: pci id of mdev parent device or pci id of physical pci >> device (device_api is vfio-pci)API here. > > So this assumes a PCI device which is probably not true. > for device_api of vfio-pci, why it's not true? for vfio-ccw, it's subchannel_type.

>> - subchannel_type (device_api is vfio-ccw) >> vendor driver specific part: (optional) >> - aggregator >> - chpid_type >> - remote_url > > For "remote_url", just wonder if it's better to integrate or reuse the > existing NVME management interface instead of duplicating it here. Otherwise > it could be a burden for mgmt to learn. E.g vendor A may use "remote_url" > but vendor B may use a different attribute. > it's vendor driver specific. vendor specific attributes are inevitable, and that's why we are discussing here of a way to standardizing of it.

our goal is that mgmt can use it without understanding the meaning of vendor specific attributes.

>> NOTE: vendors are free to add attributes in this part with a >> restriction that this attribute is able to be configured with the same >> name in sysfs too. e.g. > > Sysfs works well for common attributes belongs to a class, but I'm not sure > it can work well for device/vendor specific attributes. Does this mean mgmt > need to iterate all the attributes in both src and dst? > no. just attributes under migration directory. >> for aggregator, there must be a sysfs attribute in device node >> /sys/devices/pci0000:00/0000:00:02.0/882cc4da-dede-11e7-9180-078a62063ab1/intel_vgpu/aggregator, >> so that the userspace tool is able to configure the target device >> according to source device's aggregator attribute. >> >> >> (2) where and structure >> proposal 1: >> |- [path to device] >> |--- migration >> | |--- self >> | | |-software_version >> | | |-device_api >> | | |-type >> | | |-[pci_id or subchannel_type] >> | | |-<aggregator or chpid_type> >> | |--- compatible >> | | |-software_version >> | | |-device_api >> | | |-type >> | | |-[pci_id or subchannel_type] >> | | |-<aggregator or chpid_type> >> multiple compatible is allowed. >> attributes should be ASCII text files, preferably with only one value >> per file. >> >> >> proposal 2: use bin_attribute. >> |- [path to device] >> |--- migration >> | |--- self >> | |--- compatible >> >> so we can continue use multiline format. e.g. >> cat compatible >> software_version=0.1.0 >> device_api=vfio_pci >> type=i915-GVTg_V5_{val1:int:1,2,4,8} >> pci_id=80865963 >> aggregator={val1}/2 > > So basically two questions: > > - how hard to standardize sysfs API for dealing with compatibility check (to > make it work for most types of devices) sorry, I just know we are in the process of standardizing of it :)

> - how hard for the mgmt to learn with a vendor specific attributes (vs > existing management API) what is existing management API?

Thanks