- Infra - oVirt List Archives

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1421: ------------------------------------- https://bugzilla.redhat.com/show_bug.cgi?id=1330714 > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1421: ------------------------------------- https://github.com/projectatomic/container-storage-setup/issues/53 > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1421: ------------------------------------- Looks like the DM loopback devices exist on both EL7 and FC24: {code} [root@vm0024 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 253:0 0 40G 0 disk ├─vda1 253:1 0 512M 0 part /boot ├─vda2 253:2 0 4G 0 part [SWAP] └─vda3 253:3 0 35.5G 0 part / loop0 7:0 0 100G 0 loop └─docker-253:3-101073748-pool 252:0 0 100G 0 dm loop1 7:1 0 2G 0 loop └─docker-253:3-101073748-pool 252:0 0 100G 0 dm {code} {code} [root@vm0064 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop1 7:1 0 2G 0 loop └─docker-252:3-33877273-pool 253:0 0 100G 0 dm sr0 11:0 1 1024M 0 rom loop0 7:0 0 100G 0 loop └─docker-252:3-33877273-pool 253:0 0 100G 0 dm vda 252:0 0 40G 0 disk ├─vda2 252:2 0 4G 0 part [SWAP] ├─vda3 252:3 0 35.8G 0 part / └─vda1 252:1 0 256M 0 part /boot {code} This looks more and more like a recent regression with "{{docker-storage-setup}}" that has more sever repercussions on FC24 because of a more strict systemd setup. > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1421: ------------------------------------- More information: The original issue was found on "{{vm0064.workers-phx.ovirt.org}}" which is an FC24 VM. This is what you get when you try to run "{{docker-storage-setup}}" manually on it: {code} # docker-storage-setup INFO: Volume group backing root filesystem could not be determined ERROR: No valid volume group found. Exiting. {code} We seem to be seeing a very similar issue on EL7. the following is from "{{vm0024.workers-phx.ovirt.org}}". {code} [root@vm0024 ~]# systemctl status docker-storage-setup ● docker-storage-setup.service - Docker Storage Setup Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2017-05-31 03:00:42 UTC; 3h 53min ago Process: 16223 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) Main PID: 16223 (code=exited, status=1/FAILURE) May 31 03:00:42 vm0024.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... May 31 03:00:42 vm0024.workers-phx.ovirt.org docker-storage-setup[16223]: INFO: Volume group backing root filesystem could not be determined May 31 03:00:42 vm0024.workers-phx.ovirt.org docker-storage-setup[16223]: ERROR: No valid volume group found. Exiting. May 31 03:00:42 vm0024.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: main process exited, code=exited, status=1/FAILURE May 31 03:00:42 vm0024.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. May 31 03:00:42 vm0024.workers-phx.ovirt.org systemd[1]: Unit docker-storage-setup.service entered failed state. May 31 03:00:42 vm0024.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service failed. {code} This is the output when running the command manually: {code} [root@vm0024 ~]# docker-storage-setup INFO: Volume group backing root filesystem could not be determined ERROR: No valid volume group found. Exiting. {code} This is, strangely enough, not causing processes to get stuck on EL7 though: {code} [root@vm0024 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES {code} In fact, things seem to be working fine despite the failure: {code} [root@vm0024 ~]# docker run --rm -it centos hostname 62d53a5ab5a7 {code} I suspect we're seeing a recent regression in docker, and previously the setup script knew how to handle nodes without LVM. I'm not finding evidence of an upgrade in yum/dnf logs though. > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

Job stuck in cleanup for 13 hours
by Nir Soffer 31 May '17

31 May '17

See http://jenkins.ovirt.org/job/ovirt-release_4.1_build-artifacts-el7-x86_64/2… Build stuck after 16 minutes (using elapsed time): *00:16:49.203* + sudo systemctl restart docker Failure detected after 6 hours:*06:00:07.301* Build timed out (after 360 minutes). Marking the build as failed. But the job is still running: *06:00:08.490* + xargs -r sudo docker rm -f Why build-artifacts job needs 6 hours timeout? Nir

2 2

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1421: ------------------------------------- Need to also look into why we didn't see this so far. Did no jobs run on FC24 slaves so far? Are slaves without LVM very rare? [~ederevea] can you provide some insights here? > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC24 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1421: -------------------------------- Summary: Docker storage issues on FC24 nodes (was: Docker storage issues on FC25 nodes) > Docker storage issues on FC24 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC25 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1421: -------------------------------- Labels: containers (was: ) > Docker storage issues on FC25 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: containers > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC25 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

Barak Korren created OVIRT-1421: ----------------------------------- Summary: Docker storage issues on FC25 nodes Key: OVIRT-1421 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 Project: oVirt - virtualization made easy Issue Type: Bug Components: oVirt CI Reporter: Barak Korren Assignee: infra Priority: High docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. The core reason behind the failure seems to be that the Docker storage setup failed: {code} [root@vm0064 ~]# systemctl status docker-storage-setup ● docker-storage-setup.service - Docker Storage Setup Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) Main PID: 1489 (code=exited, status=1/FAILURE) May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. {code} This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1421) Docker storage issues on FC25 nodes
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1421?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1421: -------------------------------- Epic Link: OVIRT-400 > Docker storage issues on FC25 nodes > ----------------------------------- > > Key: OVIRT-1421 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1421 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: High > > docekr startup seems to be failing on FC24 nodes and making other dokcer commands get stuck forever. > The core reason behind the failure seems to be that the Docker storage setup failed: > {code} > [root@vm0064 ~]# systemctl status docker-storage-setup > ● docker-storage-setup.service - Docker Storage Setup > Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) > Active: failed (Result: exit-code) since Tue 2017-05-30 03:14:49 UTC; 1 day 2h ago > Process: 1489 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE) > Main PID: 1489 (code=exited, status=1/FAILURE) > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Starting Docker Storage Setup... > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: INFO: Volume group backing root filesystem could not be determined > May 30 03:14:49 vm0064.workers-phx.ovirt.org docker-storage-setup[1489]: ERROR: No valid volume group found. Exiting. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: Failed to start Docker Storage Setup. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Unit entered failed state. > May 30 03:14:49 vm0064.workers-phx.ovirt.org systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. > {code} > This seem to be does to the fact that the system's root filesystem is on a plain disk rather then LVM. This is although the docker default DM storage driver is supposed to be able to use a plaing file on the filesystem. Need to check if this is an issue specific to this slave of something that has to do with all FC24 slaves. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1420) Add timeout to Docker setup and cleanup operations
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

Barak Korren created OVIRT-1420: ----------------------------------- Summary: Add timeout to Docker setup and cleanup operations Key: OVIRT-1420 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1420 Project: oVirt - virtualization made easy Issue Type: Improvement Components: oVirt CI Reporter: Barak Korren Assignee: infra Priority: Highest It seems that some docker operation cat get stuck forever if some docker configuration stages fail. We need to add time-outs for those operations and generate proper error messages. for example failure see: http://jenkins.ovirt.org/job/ovirt-release_4.1_build-artifacts-el7-x86_64/2… -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1420) Add timeout to Docker setup and cleanup operations
by Barak Korren (oVirt JIRA) 31 May '17

31 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1420?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1420: -------------------------------- Epic Link: OVIRT-400 > Add timeout to Docker setup and cleanup operations > -------------------------------------------------- > > Key: OVIRT-1420 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1420 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: Highest > > It seems that some docker operation cat get stuck forever if some docker configuration stages fail. > We need to add time-outs for those operations and generate proper error messages. > for example failure see: > http://jenkins.ovirt.org/job/ovirt-release_4.1_build-artifacts-el7-x86_64/2… -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

oVirt infra daily report - unstable production jobs - 338
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_188_627005002.1496185206254 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/338//artifact/exported-a… Cheers, Jenkins ------=_Part_188_627005002.1496185206254 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 30/05/2017 </th></tr><tr><th colspan=2 class=sub> <a href="http://jenkins.ovirt.org/">00 Unstable Critical</a> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/">ovirt_4.1_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/">ovirt_master_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/">test-repo_ovirt_experimental_4.1</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_188_627005002.1496185206254--

1 0

[oVirt Jenkins] deploy-to-ovirt_experimental - Build #1798 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_176_941409922.1496140818278 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental/1798/ Build Name: 1798 ovirt-engine (master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/76705 - title: engine: Reduce AuditLogDirector visibily to private in Commands - project: ovirt-engine - branch: master - author: Moti Asayag <masayag(a)redhat.com> ------=_Part_176_941409922.1496140818278--

1 0

[oVirt Jenkins] deploy-to-ovirt_experimental - Build #1802 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_174_1643814379.1496140741952 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental/1802/ Build Name: 1802 ovirt-engine (master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/77219 - title: engine: Use AuditLogableImpl for audit logging - project: ovirt-engine - branch: master - author: Moti Asayag <masayag(a)redhat.com> ------=_Part_174_1643814379.1496140741952--

1 0

[oVirt Jenkins] deploy-to-ovirt_experimental - Build #1799 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_172_916848938.1496140712984 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental/1799/ Build Name: 1799 ovirt-engine (master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/77218 - title: engine: Use AuditLogableImpl when possible - project: ovirt-engine - branch: master - author: Moti Asayag <masayag(a)redhat.com> ------=_Part_172_916848938.1496140712984--

1 0

[oVirt Jenkins] deploy-to-ovirt_experimental - Build #1792 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_170_818907884.1496140697567 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental/1792/ Build Name: 1792 ovirt-engine (master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/77215 - title: engine: Use AuditLogableImpl for logging events - project: ovirt-engine - branch: master - author: Moti Asayag <masayag(a)redhat.com> ------=_Part_170_818907884.1496140697567--

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #6899 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_106_1090521024.1496065552549 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6899/, Build Number: 6899, Build Status: FAILURE ------=_Part_106_1090521024.1496065552549--

2 17

[JIRA] (OVIRT-1419) Create nightly builds for ovirt-containers
by Barak Korren (oVirt JIRA) 30 May '17

30 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1419?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-1419: ----------------------------------- Assignee: Daniel Belenky (was: infra) > Create nightly builds for ovirt-containers > ------------------------------------------ > > Key: OVIRT-1419 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1419 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: Daniel Belenky > Labels: containers, standard-ci > > Before we make dependency-triggered builds to ovirt-containers as described in the design document (OVIRT-1371), we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. > For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. > This will be similar to the currently existing nightly builds for node and the appliance. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1419) Create nightly builds for ovirt-containers
by Barak Korren (oVirt JIRA) 30 May '17

30 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1419?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1419: -------------------------------- Description: Before we make dependency-triggered builds to ovirt-containers as described in the design document (OVIRT-1371), we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. This will be similar to the currently existing nightly builds for node and the appliance. was: Before we make dependency-triggered builds to ovirt-containers as described in the design document, we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. This will be similar to the currently existing nightly builds for node and the appliance. > Create nightly builds for ovirt-containers > ------------------------------------------ > > Key: OVIRT-1419 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1419 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: containers, standard-ci > > Before we make dependency-triggered builds to ovirt-containers as described in the design document (OVIRT-1371), we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. > For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. > This will be similar to the currently existing nightly builds for node and the appliance. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1419) Create nightly builds for ovirt-containers
by Barak Korren (oVirt JIRA) 30 May '17

30 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1419?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1419: -------------------------------- Labels: containers standard-ci (was: ) > Create nightly builds for ovirt-containers > ------------------------------------------ > > Key: OVIRT-1419 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1419 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: containers, standard-ci > > Before we make dependency-triggered builds to ovirt-containers as described in the design document, we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. > For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. > This will be similar to the currently existing nightly builds for node and the appliance. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1419) Create nightly builds for ovirt-containers
by Barak Korren (oVirt JIRA) 30 May '17

30 May '17

Barak Korren created OVIRT-1419: ----------------------------------- Summary: Create nightly builds for ovirt-containers Key: OVIRT-1419 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1419 Project: oVirt - virtualization made easy Issue Type: Improvement Components: oVirt CI Reporter: Barak Korren Assignee: infra Before we make dependency-triggered builds to ovirt-containers as described in the design document, we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. This will be similar to the currently existing nightly builds for node and the appliance. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[JIRA] (OVIRT-1419) Create nightly builds for ovirt-containers
by Barak Korren (oVirt JIRA) 30 May '17

30 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1419?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1419: -------------------------------- Epic Link: OVIRT-400 > Create nightly builds for ovirt-containers > ------------------------------------------ > > Key: OVIRT-1419 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1419 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > > Before we make dependency-triggered builds to ovirt-containers as described in the design document, we need to start running the build and deploy code more frequently to find issues and measure needed resources and other aspects. > For that we should setup jobs to product nightly builds of the containers from the oVirt nightly snapshot repos. > This will be similar to the currently existing nightly builds for node and the appliance. -- This message was sent by Atlassian JIRA (v1000.1010.2#100044)

1 0

[oVirt Jenkins] ovirt-node-ng_master_build-artifacts-el7-x86_64 - Build # 422 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 May '17

30 May '17

------=_Part_154_1461408565.1496117574306 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build Number: 422 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #422 [Sandro Bonazzola] ovirt-engine-nodejs: add fedora 26 [Sandro Bonazzola] ovirt-engine-nodejs-modules: add fedora 26 [Sandro Bonazzola] ovirt-engine-yarn: add fedora 26 [Eyal Edri] remove ovirt-web-ui project [Barak Korren] Add PyYAML package to all slaves [Barak Korren] Add retries to mirror_client on JSON errors [Barak Korren] Enabled loading mirrors data from local file [Barak Korren] Added GerritMergedChange class [Yuval Turgeman] packaging: merged image and image-update packages ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_154_1461408565.1496117574306--

2 1

oVirt infra daily report - unstable production jobs - 337
by jenkins＠jenkins.phx.ovirt.org 29 May '17

29 May '17

------=_Part_143_407749465.1496098807878 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/337//artifact/exported-a… Cheers, Jenkins ------=_Part_143_407749465.1496098807878 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 29/05/2017 </th></tr><tr><th colspan=2 class=sub> <a href="http://jenkins.ovirt.org/">00 Unstable Critical</a> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/">ovirt_4.1_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/">ovirt_master_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/">test-repo_ovirt_experimental_master</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_143_407749465.1496098807878--

1 0

[ OST Failure Report ] [ oVirt master ] [ 28-05-2017 ] [ 004_basic_sanity.hotunplug_disk ]
by Dafna Ron 29 May '17

29 May '17

This is a multi-part message in MIME format. --------------992DE8C067CE91FDF29339AC Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ** *Hi All, * * ** *We are seeing the same failure**sporadically**in *three*different places (hotunplug disk). * *We think this is a race condition which was introduced to OST with the the live storage migration test (lock on the disk is preventing the hotunplug). * Test failed: [ 004_basic_sanity.hotunplug_disk] Link to suspected patches: https://gerrit.ovirt.org/#/c/77083 Link to Job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881 ** Link to all logs: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif… Error snippet from the log: <error> * Error Message status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. Stacktrace Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 68, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 498, in hotunplug_disk disk.deactivate() File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31839, in deactivate headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. from engine log: 2017-05-28 09:05:32,891-04 INFO [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISKS_LOCKED$diskAliases vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_VM_IS_LOCKED>]'}' 2017-05-28 09:05:32,891-04 WARN [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Validation of action 'HotUnPlugDiskFromVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__HOT_UNPLUG,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED,$DiskName vm0_disk0 2017-05-28 09:05:32,895-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-10) [] Operation Failed: [Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.] 2017-05-28 09:05:33,323-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Ending command 'org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand' successfully. 2017-05-28 09:05:33,324-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Lock freed to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED$DiskName vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>]'}' ** *</error>* * * --------------992DE8C067CE91FDF29339AC Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Hi All, <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">We are seeing the same failure<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> sporadically<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> in <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">three<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> different places (hotunplug disk). <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">We think this is a race condition which was introduced to OST with the the live storage migration test (lock on the disk is preventing the hotunplug). <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Test failed: [ 004_basic_sanity.hotunplug_disk] <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/77083">https://gerrit.ovirt.org/#/c/77083</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif…">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif…</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Error snippet from the log: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error> <h3 style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Helvetica, Arial, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Error Message</h3> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.</pre> <h3 style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Helvetica, Arial, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Stacktrace</h3> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 68, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 498, in hotunplug_disk disk.deactivate() File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31839, in deactivate headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. from engine log: 2017-05-28 09:05:32,891-04 INFO [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISKS_LOCKED$diskAliases vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_VM_IS_LOCKED>]'}' 2017-05-28 09:05:32,891-04 WARN [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Validation of action 'HotUnPlugDiskFromVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__HOT_UNPLUG,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED,$DiskName vm0_disk0 2017-05-28 09:05:32,895-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-10) [] Operation Failed: [Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.] 2017-05-28 09:05:33,323-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Ending command 'org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand' successfully. 2017-05-28 09:05:33,324-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Lock freed to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED$DiskName vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>]'}' </pre> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error> </body> </html> --------------992DE8C067CE91FDF29339AC--

3 3

[JIRA] (OVIRT-1418) Lets remove the 'workflow' flag
by Daniel Belenky (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1418?page=com.atlassian.jira.… ] Daniel Belenky commented on OVIRT-1418: --------------------------------------- As far as I've been told, and as [~eedri] mentioned already, it's used to let others know that a patch is ready for review. But as we mark patches that are still under development with [WIP] I think that it's unnecessary. > Lets remove the 'workflow' flag > ------------------------------- > > Key: OVIRT-1418 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1418 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Gerrit/git > Reporter: Barak Korren > Assignee: infra > > I don't remember what was David's idea behind adding it, I'm pretty sure nobody else does, and for now its just annoying. People just forget to set it, and you need to have it set to +2 to merge. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1418) Lets remove the 'workflow' flag
by eedri (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1418?page=com.atlassian.jira.… ] eedri commented on OVIRT-1418: ------------------------------ +1, I also don't think it has real usage for now. We intended to use it to mark patches that are ready to review, not sure anyone uses it. On Mon, May 29, 2017 at 3:21 PM, Barak Korren (oVirt JIRA) < -- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) > Lets remove the 'workflow' flag > ------------------------------- > > Key: OVIRT-1418 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1418 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Gerrit/git > Reporter: Barak Korren > Assignee: infra > > I don't remember what was David's idea behind adding it, I'm pretty sure nobody else does, and for now its just annoying. People just forget to set it, and you need to have it set to +2 to merge. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1418) Lets remove the 'workflow' flag
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1418?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1418: -------------------------------- Epic Link: OVIRT-403 > Lets remove the 'workflow' flag > ------------------------------- > > Key: OVIRT-1418 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1418 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Gerrit/git > Reporter: Barak Korren > Assignee: infra > > I don't remember what was David's idea behind adding it, I'm pretty sure nobody else does, and for now its just annoying. People just forget to set it, and you need to have it set to +2 to merge. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

2 1

[JIRA] (OVIRT-1418) Lets remove the 'workflow' flag
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

Barak Korren created OVIRT-1418: ----------------------------------- Summary: Lets remove the 'workflow' flag Key: OVIRT-1418 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1418 Project: oVirt - virtualization made easy Issue Type: Improvement Components: Gerrit/git Reporter: Barak Korren Assignee: infra I don't remember what was David's idea behind adding it, I'm pretty sure nobody else does, and for now its just annoying. People just forget to set it, and you need to have it set to +2 to merge. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1417) mock_cleanup.sh script does not erash shut-down VMs well
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1417?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-1417: ----------------------------------- Assignee: Barak Korren (was: infra) > mock_cleanup.sh script does not erash shut-down VMs well > -------------------------------------------------------- > > Key: OVIRT-1417 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1417 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: Barak Korren > Priority: Highest > > '{{mock_cleanup.sh}}' includes the following code to cleanup VMs: > {code} > for UUID in $(virsh list --all --uuid); do > virsh destroy $UUID || : > sleep 2 > virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || : > done > {code} > This code probably worked at some point, but doesn't any more, instead if there are shut down VMs left on the machine, we get this error message: > {code} > error: Specified both --storage and --remove-all-storage > {code} > Need to fix this, the following was verified to work: > {code} > virsh undefine --remove-all-storage --snapshots-metadata $UUID > {code} -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1417) mock_cleanup.sh script does not erash shut-down VMs well
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

Barak Korren created OVIRT-1417: ----------------------------------- Summary: mock_cleanup.sh script does not erash shut-down VMs well Key: OVIRT-1417 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1417 Project: oVirt - virtualization made easy Issue Type: Improvement Components: oVirt CI Reporter: Barak Korren Assignee: infra Priority: Highest '{{mock_cleanup.sh}}' includes the following code to cleanup VMs: {code} for UUID in $(virsh list --all --uuid); do virsh destroy $UUID || : sleep 2 virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || : done {code} This code probably worked at some point, but doesn't any more, instead if there are shut down VMs left on the machine, we get this error message: {code} error: Specified both --storage and --remove-all-storage {code} Need to fix this, the following was verified to work: {code} virsh undefine --remove-all-storage --snapshots-metadata $UUID {code} -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1417) mock_cleanup.sh script does not erash shut-down VMs well
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1417?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1417: -------------------------------- Epic Link: OVIRT-400 > mock_cleanup.sh script does not erash shut-down VMs well > -------------------------------------------------------- > > Key: OVIRT-1417 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1417 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Priority: Highest > > '{{mock_cleanup.sh}}' includes the following code to cleanup VMs: > {code} > for UUID in $(virsh list --all --uuid); do > virsh destroy $UUID || : > sleep 2 > virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || : > done > {code} > This code probably worked at some point, but doesn't any more, instead if there are shut down VMs left on the machine, we get this error message: > {code} > error: Specified both --storage and --remove-all-storage > {code} > Need to fix this, the following was verified to work: > {code} > virsh undefine --remove-all-storage --snapshots-metadata $UUID > {code} -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1416) Preserve Gerrit CR flags on rebase.
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1416?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1416: -------------------------------- Epic Link: OVIRT-403 > Preserve Gerrit CR flags on rebase. > ------------------------------------ > > Key: OVIRT-1416 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1416 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Gerrit/git > Reporter: Barak Korren > Assignee: infra > > Sometimes a patch finishes review and then needs to be rebased for some reason. This causes all flags to be lost. > We currently have settings to keep the +v flag on trivial rebase, we need the same for at least the CR flag. > Doing this for the WF flag might not be necessary though, b/c it is controlled by the patch author. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1416) Preserve Gerrit CR flags on rebase.
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

Barak Korren created OVIRT-1416: ----------------------------------- Summary: Preserve Gerrit CR flags on rebase. Key: OVIRT-1416 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1416 Project: oVirt - virtualization made easy Issue Type: Improvement Components: Gerrit/git Reporter: Barak Korren Assignee: infra Sometimes a patch finishes review and then needs to be rebased for some reason. This causes all flags to be lost. We currently have settings to keep the +v flag on trivial rebase, we need the same for at least the CR flag. Doing this for the WF flag might not be necessary though, b/c it is controlled by the patch author. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #6895 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 29 May '17

29 May '17

------=_Part_95_1960724250.1496054851851 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6895/, Build Number: 6895, Build Status: FAILURE ------=_Part_95_1960724250.1496054851851--

1 1

[JIRA] (OVIRT-1415) Add more FC26 slaves
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

Barak Korren created OVIRT-1415: ----------------------------------- Summary: Add more FC26 slaves Key: OVIRT-1415 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1415 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Barak Korren Assignee: infra We only have one ATM, this means that many jobs need to wait in queue for it. Please add more. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1415) Add more FC26 slaves
by Barak Korren (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1415?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1415: -------------------------------- Epic Link: OVIRT-403 > Add more FC26 slaves > -------------------- > > Key: OVIRT-1415 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1415 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Barak Korren > Assignee: infra > > We only have one ATM, this means that many jobs need to wait in queue for it. > Please add more. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #382
by jenkins＠jenkins.phx.ovirt.org 29 May '17

29 May '17

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/382/displa…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/changes/13/75913/5:patch --prune > git rev-parse origin/patch^{commit} # timeout=10 > git rev-parse patch^{commit} # timeout=10 Checking out Revision 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f (patch) > git config core.sparsecheckout # timeout=10 > git checkout -f 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f > git rev-list 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/hudson4863573035083842664.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1434, in _latestRepoXML oxml = self._saveOldRepoXML(local) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1292, in _saveOldRepoXML shutil.copy2(local, old_local) File "/usr/lib64/python2.7/shutil.py", line 131, in copy2 copystat(src, dst) File "/usr/lib64/python2.7/shutil.py", line 98, in copystat os.utime(dst, (st.st_atime, st.st_mtime)) OSError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/fedora-base-fc25/repomd.xml.old.tmp' Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-1379) add support for fedora rawhide in mock_runner
by sbonazzo (oVirt JIRA) 29 May '17

29 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1379?page=com.atlassian.jira.… ] sbonazzo commented on OVIRT-1379: --------------------------------- I'm fine enabling Ci only for rawhide as default. For the releases we'll need the stable versions anyway, doesn't make sense to issue a stable release for rawhide. If single specific package requires to be tested on a stable fedora for some very good reason I would suggest to treat it as an exception. > add support for fedora rawhide in mock_runner > --------------------------------------------- > > Key: OVIRT-1379 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1379 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Barak Korren > Assignee: infra > > Moving this to Jira. > On 15 May 2017 at 10:31, Barak Korren <bkorren(a)redhat.com> wrote: > > On 15 May 2017 at 10:13, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > >> > >> > >> > >> On Sat, May 13, 2017 at 11:12 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: > >>> > >>> Hi Sandro, what is needed to build and public ovirt-imageio for Fedora rawhide? > >> > >> > >> We miss support for rawhide in mock runner, within jenkins git repo. > >> Adding Barak > > > > Get we get a mock configuration for it from somewhere? > > They are building it in Koji so I guess they have rawhide buildroots already no? > > > > Is there any other special behaviour needed besides the proper mock > > configuration? > > > > -- > > Barak Korren > > RHV DevOps team , RHCE, RHCi > > Red Hat EMEA > > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted > -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

Build failed in Jenkins: system-mk_mirrors_index-yum #5997
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

See <http://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/5997/display/redir…> ------------------------------------------ Started by upstream project "system-sync_mirrors-centos-base-el6-x86_64" build number 400 originally caused by: Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:799) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1055) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:496) at hudson.model.AbstractProject.checkout(AbstractProject.java:1281) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed connect to gerrit.ovirt.org:80; Connection timed out at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1596) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:336) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) at ......remote call to mirrors.phx.ovirt.org(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:830) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy70.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797) ... 11 more ERROR: null

1 1

Build failed in Jenkins: system-mk_mirrors_index-yum #5990
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

See <http://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/5990/display/redir…> ------------------------------------------ Started by upstream project "system-sync_mirrors-fedora-base-fc26-x86_64" build number 110 originally caused by: Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:799) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1055) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:496) at hudson.model.AbstractProject.checkout(AbstractProject.java:1281) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed connect to gerrit.ovirt.org:80; Connection timed out at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1596) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:336) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) at ......remote call to mirrors.phx.ovirt.org(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:830) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy70.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797) ... 11 more ERROR: null

1 1

oVirt infra daily report - unstable production jobs - 336
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

------=_Part_82_1015679253.1496012402178 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/336//artifact/exported-a… Cheers, Jenkins ------=_Part_82_1015679253.1496012402178 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 28/05/2017 </th></tr><tr><th colspan=2 class=sub> <a href="http://jenkins.ovirt.org/">00 Unstable Critical</a> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/">ovirt_4.1_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/">ovirt_master_hc-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/">system-mk_mirrors_index-yum</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_82_1015679253.1496012402178--

1 0

[oVirt Jenkins] deploy-to-ovirt_experimental - Build #1717 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

------=_Part_76_856465687.1495995213596 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental/1717/ Build Name: 1717 ovirt-engine-api-model (master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/76802 - title: Introduce Event.index - project: ovirt-engine-api-model - branch: master - author: Juan Hernandez <juan.hernandez(a)redhat.com> ------=_Part_76_856465687.1495995213596--

1 1

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #6883 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

------=_Part_71_1663417794.1495982654711 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6883/, Build Number: 6883, Build Status: FAILURE ------=_Part_71_1663417794.1495982654711--

1 1

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #6881 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 28 May '17

28 May '17

------=_Part_64_2012585112.1495976776807 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/, Build Number: 6881, Build Status: FAILURE ------=_Part_64_2012585112.1495976776807--

1 1

[JIRA] (OVIRT-1414) Enable mirro injection process to also inject oVirt repos
by Barak Korren (oVirt JIRA) 28 May '17

28 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1414?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-1414: ----------------------------------- Assignee: Barak Korren (was: infra) > Enable mirro injection process to also inject oVirt repos > --------------------------------------------------------- > > Key: OVIRT-1414 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1414 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: oVirt CI > Reporter: Barak Korren > Assignee: Barak Korren > > It may be desirable to allow injecting oVirt repos in a similar way to the way other mirrors are injected. This may be useful, for example to force OST to run with the 'tested' repo configured as the fallback repo without having to add it to extra_sources. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1414) Enable mirro injection process to also inject oVirt repos
by Barak Korren (oVirt JIRA) 28 May '17

28 May '17

Barak Korren created OVIRT-1414: ----------------------------------- Summary: Enable mirro injection process to also inject oVirt repos Key: OVIRT-1414 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1414 Project: oVirt - virtualization made easy Issue Type: New Feature Components: oVirt CI Reporter: Barak Korren Assignee: infra It may be desirable to allow injecting oVirt repos in a similar way to the way other mirrors are injected. This may be useful, for example to force OST to run with the 'tested' repo configured as the fallback repo without having to add it to extra_sources. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1414) Enable mirro injection process to also inject oVirt repos
by Barak Korren (oVirt JIRA) 28 May '17

28 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1414?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1414: -------------------------------- Epic Link: OVIRT-403 > Enable mirro injection process to also inject oVirt repos > --------------------------------------------------------- > > Key: OVIRT-1414 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1414 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > > It may be desirable to allow injecting oVirt repos in a similar way to the way other mirrors are injected. This may be useful, for example to force OST to run with the 'tested' repo configured as the fallback repo without having to add it to extra_sources. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0

[JIRA] (OVIRT-1413) Enable configuring mirrors from a file
by Barak Korren (oVirt JIRA) 28 May '17

28 May '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1413?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-1413: ----------------------------------- Assignee: Barak Korren (was: infra) > Enable configuring mirrors from a file > -------------------------------------- > > Key: OVIRT-1413 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1413 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: oVirt CI > Reporter: Barak Korren > Assignee: Barak Korren > Labels: mirrors > > Right now mirror configuration can only come from a JSON file stored on the mirrors server. > We want to enable taking the configuration from a local file also, this will enable jobs to manipulate mirror information before running the mirror injection process. > This would be especially useful for change queue test jobs in order to enable fine-grained control over the mirrors being tested. -- This message was sent by Atlassian JIRA (v1000.1010.1#100044)

1 0