I want to run containers and VMs side by side and not necessarily nested. The main reason for that is GPUs, Voltas mostly, used for CUDA machine learning not for VDI, which is what most of the VM orchestrators like oVirt or vSphere seem to focus on. And CUDA drivers are notorious for refusing to work under KVM unless you pay $esla. oVirt is more of a side show in my environment, used to run some smaller functional VMs alongside bigger containers, but also in order to consolidate and re-distribute the local compute node storage as a Gluster storage pool: Kibbutz storage and compute, if you want, very much how I understand the HCI philosophy behind oVirt. The full integration of containers and VMs is still very much on the roadmap I believe, but I was surprised to see that even co-existence seems to be a problem currently. So I set-up a 3-node HCI on CentOS7 (GPU-less and older) hosts and then added additional (beefier GPGPU) CentOS7 hosts, that have been running CUDA workloads on the latest Docker-CE v19 something. The installation works fine, I can migrate VMs to these extra hosts etc., but to my dismay Docker containers on these hosts lose access to the local network, that is the entire subnet the host is in. For some strange reason I can still ping Internet hosts, perhaps even everything behind the host's gateway, but local connections are blocked. It would seem that the ovritmgmt network that the oVirt installation puts in breaks the docker0 bridge that Docker put there first. I'd consider that a bug, but I'd like to gather some feedback first, if anyone else has run into this problem. I've repeated this several times in completely distinct environments with the same results: Simply add a host with a working Docker-CE as an oVirt host to an existing DC/cluster and then try if you can still ping anyone on that net, including the Docker host from a busybox container afterwards (should try that ping just before you actually add it). No, I didn't try this with podman yet, because that's separate challenge with CUDA: Would love to know if that is part of QA for oVirt already. _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WKLB3IAN7FJ...

Hi Strahil, color me surprised, too, especially considering where things are supposed to go in terms of roadmap. Yet again, both oVIrt and Docker could be excused to think that they "own the hardware" they are running on, because it's a rather natural assumption, even if there are good reasons to run VMs and containers side-by-side as well as nested. Yes, I believe, I have also done the reverse, put Docker on a system that was already running as oVirt compute host and it wasn't with better results, either. The biggest challenge is to repairing the node, without having to re-install the whole thing and I have gone through quite a few wobbles there, with colleagues who weren't too appreciative of having their ML jobs fail (those tend to be rather lenghty...) I've managed to get CUDA work on KVM VMs twiddling the XML config files in these ways documented on the Web. But with oVirt those KVM XML config files get generated on the fly in Python and I'd have to fiddle with the code which does that. Actually I *did* try doing that at one point in time about a year or two ago, but I never found the right place, where that code actually was taken from. You see, there are copies of that code on every node, but also on the management engine. And you know how Ansible squirts code from machine to machine to do its magic, so that's were I stopped at one point, because running ML workloads containerized was more natural anyway and I was happy to have CPU-only VMs at their side. Besides GPU access also disables live-migration and the abillity to move these long-running functional VMs around to manage resources for ML jobs is exactly what attracts me to oVirt. Currently I am mostly probing around here, to see if what I try would be considered totally esoteric or irresponsible or if it's worth reporting as a bug. _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6S7VBHOMI76...

i think it wouldn't work out of box ovirt will overwrite all your routes and network. you might try to tell ovirt do jot maintain the network of a interface where you got a docker and also add custom rules in the firewall ports template on the engine. <thomas(a)hoberg.net&gt; schrieb am So., 12. Apr. 2020, 15:51: > I want to run containers and VMs side by side and not necessarily nested. > The main reason for that is GPUs, Voltas mostly, used for CUDA machine > learning not for VDI, which is what most of the VM orchestrators like oVirt > or vSphere seem to focus on. And CUDA drivers are notorious for refusing to > work under KVM unless you pay $esla. > > oVirt is more of a side show in my environment, used to run some smaller > functional VMs alongside bigger containers, but also in order to > consolidate and re-distribute the local compute node storage as a Gluster > storage pool: Kibbutz storage and compute, if you want, very much how I > understand the HCI philosophy behind oVirt. > > The full integration of containers and VMs is still very much on the > roadmap I believe, but I was surprised to see that even co-existence seems > to be a problem currently. > > So I set-up a 3-node HCI on CentOS7 (GPU-less and older) hosts and then > added additional (beefier GPGPU) CentOS7 hosts, that have been running CUDA > workloads on the latest Docker-CE v19 something. > > The installation works fine, I can migrate VMs to these extra hosts etc., > but to my dismay Docker containers on these hosts lose access to the local > network, that is the entire subnet the host is in. For some strange reason > I can still ping Internet hosts, perhaps even everything behind the host's > gateway, but local connections are blocked. > > It would seem that the ovritmgmt network that the oVirt installation puts > in breaks the docker0 bridge that Docker put there first. > > I'd consider that a bug, but I'd like to gather some feedback first, if > anyone else has run into this problem. > > I've repeated this several times in completely distinct environments with > the same results: > > Simply add a host with a working Docker-CE as an oVirt host to an existing > DC/cluster and then try if you can still ping anyone on that net, including > the Docker host from a busybox container afterwards (should try that ping > just before you actually add it). > > No, I didn't try this with podman yet, because that's separate challenge > with CUDA: Would love to know if that is part of QA for oVirt already. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/WKLB3IAN7FJ... >