Hi Dan, How are you. As you know we've been using `lfedora1.lf-dev.marist.edu` to generate s390x build of oVirt. We've recently seen some failures that have to do with running our of space on the node. Some of this seems to be our fault, as clearing up stale mock chroots we created freed up about 14G, but after doing that I still see there are 48G used there (Is the OS image that big?). Can some more space be cleared up on the node? Could we perhaps have the disk space increased there?

I was also wondering, could we setup and use Docker on that node? We've been switching to using containers on our regular CI nodes, and it'd be a shame to leave s390x behind...

On Mon, 3 Dec 2018 at 10:37, Dan Horák <dhorak(a)redhat.com&gt; wrote: > Hi Barak, > > On Sun, 2 Dec 2018 09:50:34 +0200 > Barak Korren <bkorren(a)redhat.com&gt; wrote: > > > Hi Dan, > > > > How are you. > > > > As you know we've been using `lfedora1.lf-dev.marist.edu` to > > generate s390x build of oVirt. > > > > We've recently seen some failures that have to do with running > > our of space on the node. Some of this seems to be our fault, as > > clearing up stale mock chroots we created freed up about 14G, but > > after doing that I still see there are 48G used there (Is the OS > > image that big?). Can some more space be cleared up on the node? > > Could we perhaps have the disk space increased there? > > thanks for info, I'm looking into it. There are multiple users > sharing the machine, so someone else might have used the all free > space :-) How easily you could migrate your setup to our second > guest (same specs)? We could try the containers there. > I'd rather keep the current setup as it is, and have it keep working as we try out the containers. We can remove it once the containers are working well...

> I was also wondering, could we setup and use Docker on that node? > > We've been switching to using containers on our regular CI nodes, > > and it'd be a shame to leave s390x behind... > > I have been thinking about containers already as another level of > interaction. I would prefer podman (and co) for the runtime, it's RH > preferred technology, doesn't require a daemon and allows > non-privileged use. > I'm all for using podman down the line, but there are a few reasons why we need docker currently: 1. All our existing code had been developed and tested on Docker, we will switch to podman eventually, but we're not gonna be ready for that in the near future. 2. The main thing we want to do is use the jenkins-docker plugin to spin up and remove the containers for us - there is AFAIK is no plugin for podman ATM.

WRT non privileged use - we're currently still running mock inside the container, so we need it to be privileged...

בתאריך יום ב׳, 3 בדצמ׳ 2018, 15:22, מאת Dan Horák <dhorak(a)redhat.com&gt;: > On Mon, 3 Dec 2018 14:24:00 +0200 > Barak Korren <bkorren(a)redhat.com&gt; wrote: > > > On Mon, 3 Dec 2018 at 10:37, Dan Horák <dhorak(a)redhat.com&gt; wrote: > > > > > Hi Barak, > > > > > > On Sun, 2 Dec 2018 09:50:34 +0200 > > > Barak Korren <bkorren(a)redhat.com&gt; wrote: > > > > > > > Hi Dan, > > > > > > > > How are you. > > > > > > > > As you know we've been using `lfedora1.lf-dev.marist.edu` to > > > > generate s390x build of oVirt. > > > > > > > > We've recently seen some failures that have to do with running > > > > our of space on the node. Some of this seems to be our fault, > > > > as clearing up stale mock chroots we created freed up about > > > > 14G, but after doing that I still see there are 48G used > > > > there (Is the OS image that big?). Can some more space be > > > > cleared up on the node? Could we perhaps have the disk space > > > > increased there? > > > > > > thanks for info, I'm looking into it. There are multiple users > > > sharing the machine, so someone else might have used the all > > > free space :-) How easily you could migrate your setup to our > > > second guest (same specs)? We could try the containers there. > > > > > > > I'd rather keep the current setup as it is, and have it keep > > working as we try out the containers. We can remove it once the > > containers are working well... > > ok, makes sense > > I've already removed some old cached data, so jobs on the guest > should work again. I'm going update and reboot the guest, sometimes > there are removed, but not closed, files reducing the free disk > space. > > > > I was also wondering, could we setup and use Docker on that > > > node? > > > > We've been switching to using containers on our regular CI > > > > nodes, and it'd be a shame to leave s390x behind... > > > > > > I have been thinking about containers already as another level > > > of interaction. I would prefer podman (and co) for the runtime, > > > it's RH preferred technology, doesn't require a daemon and > > > allows non-privileged use. > > > > > > > I'm all for using podman down the line, but there are a few > > reasons why we need docker currently: > > > > 1. All our existing code had been developed and tested on > > Docker, we will switch to podman eventually, but we're not gonna > > be ready for that in the near future. > > 2. The main thing we want to do is use the jenkins-docker > > plugin to spin up and remove the containers for us - there is > > AFAIK is no plugin for podman ATM. > > might be worth to let the podman team know about > I think I heard this idea been floated around, but no actual work going on... I'd rather not hold my breath...

> > WRT non privileged use - we're currently still running mock > > inside the container, so we need it to be privileged... > > AFAIK podman gives you a root user in the container even when you > start the container as a regular user, which is why I like it for > shared machines like this. > I understand perfectly... But given that it still has some maturing to do, and I'd rather be able to use containers sooner then later, would you mind trusting us not to break the machine for a while? (We've been nice citizens so far no?) (Moving to the containers across the board is the 1st step in deprecating a lot of old code with have around mock and I'd hate to have to keep it just for s390x now that I know Docker is available...)