On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David <didi@redhat.com> wrote:
basic suite failed for me too.

/var/log/messages has[1]:

Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job openvswitch.service/start failed with result 'dependency'.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch Forwarding Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job ovs-vswitchd.service/start failed with result 'dependency'.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 14.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 2 (TCP-like)
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 3 (TCP-Friendly Rate Control)
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables configured (bind 256/256)
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 17.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1

[1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/messages/*view*/


Talked with danken, he asked to check if it's an selinux issue. It is. audit lot has:

type=AVC msg=audit(1521967325.146:675): avc:  denied  { create } for  pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket
type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null)
type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61
type=AVC msg=audit(1521967325.150:676): avc:  denied  { create } for  pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket
type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null)
type=PROCTITLE msg=audit(1521967325.150:676): 

http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log
And it's 2.9:

Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch
 

On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
+ Network team.
I'm not sure if we've moved to ovs 2.9 already?
Y.

On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Hi,

Is there an ongoing engine master OST failure blocking?

[ INFO  ] Stage: Misc configuration
[ INFO  ]  Stage: Package installation
[ INFO  ]  Stage: Misc configuration
[ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start service \'openvswitch\'
[ INFO  ] Yum Performing yum transaction rollback


These are unrelated code changes:


and

But they both die in 001, with exactly 1.24MB in the log and 'Failed to start service openvswitch'
On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron <dron@redhat.com> wrote:
Hello,

I would like to update on this week's failures and OST current status.

On 19-03-2018 - the CI team reported 3 different failures.

On Master branch the failed changes reported were:

core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/
core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/

On 4.2 branch, the reported change was:

core: Call endAction() of all child commands in ImportVmCommand - https://gerrit.ovirt.org/#/c/89165/

The fix's for the regressions were merged the following day (20-03-2018)

https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() logic in ImportVmCommand
https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an instance type

On 20-03-2018 - the CI team discovered an issue on the job's cleanup which caused random failures on changes testing due to failure in docker cleanup. There is an open Jira on the issue: https://ovirt-jira.atlassian.net/browse/OVIRT-1939

Below you can see the chart for this week's resolved issues but cause of failure:

Code = regression of working components/functionalities
Configurations = package related issues
Other = failed build artifacts
Infra = infrastructure/OST/Lago related issues








Below is a chart of resolved failures based on ovirt version

 





Below is a chart showing failures by suite type:






Thank you,
Dafna


_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra




--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme


_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra




--
Didi



--
Didi