On Mon, Mar 26, 2018 at 3:29 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Eyal, danken told me that ovn cannot require selinux, so the images must be updated.

Yea, I got that, and Gal indeed went a head and updated the image, just wondered why it wasn't fixed also on OVN side.
 

Looks like Gal is updating the images here: https://gerrit.ovirt.org/#/c/89430/  (ng: Update the CentOS image, just merged)

Gal, will this fix it?

I think Gal verified it, so should be working now, try rebasing your patch.
 


On Sun, Mar 25, 2018 at 1:15 PM, Eyal Edri <eedri@redhat.com> wrote:
Just wondering, if the bug requires updated selinux, shouldn't the ovs PKG enforce it I'm the spec file?



On Mar 25, 2018 20:12, "Greg Sheremeta" <gshereme@redhat.com> wrote:
Indeed, updating selinux-policy on both engine and hosts passes.

Change If671d938: [test do not merge] test selinux-policy update on engine and hosts | gerrit.ovirt Code Review

So I guess the images need updates.

On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta <gshereme@redhat.com> wrote:
currently selinux-policy-3.13.1-166.el7_4.4.noarch

updating selinux-policy on engine gets me past 001, and then 002 dies:

15:05:45   # initialize_engine: 
rpm -qa:
15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', err='')
15:06:13 <snip> Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will be an update
<snip> Updated:  selinux-policy.noarch 0:3.13.1-166.el7_4.9
rpm -qa:
15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', err='')

But later in 002
15:08:47 RuntimeError: 1 hosts failed installation:
15:08:47 lago-basic-suite-master-host-0: install_failed

Perhaps selinux-policy needs to be updated on the hosts too? Not my area of expertise :)

Greg

On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg <danken@redhat.com> wrote:
Which version of selinux-policy do we have on the Engine image?

Bug 1538936 - Open vSwitch selinux policy needs updating [rhel-7.4.z]

was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm


On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David <didi@redhat.com> wrote:
On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David <didi@redhat.com> wrote:
basic suite failed for me too.

/var/log/messages has[1]:

Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job openvswitch.service/start failed with result 'dependency'.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch Forwarding Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job ovs-vswitchd.service/start failed with result 'dependency'.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 14.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session 17 of user root.
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 2 (TCP-like)
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 3 (TCP-Friendly Rate Control)
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit...
Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables configured (bind 256/256)
Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 17.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root.
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED]
Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1

[1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/messages/*view*/


Talked with danken, he asked to check if it's an selinux issue. It is. audit lot has:

type=AVC msg=audit(1521967325.146:675): avc:  denied  { create } for  pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket
type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null)
type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61
type=AVC msg=audit(1521967325.150:676): avc:  denied  { create } for  pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket
type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null)
type=PROCTITLE msg=audit(1521967325.150:676): 

http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log
And it's 2.9:

Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch
 

On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
+ Network team.
I'm not sure if we've moved to ovs 2.9 already?
Y.

On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Hi,

Is there an ongoing engine master OST failure blocking?

[ INFO  ] Stage: Misc configuration
[ INFO  ]  Stage: Package installation
[ INFO  ]  Stage: Misc configuration
[ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start service \'openvswitch\'
[ INFO  ] Yum Performing yum transaction rollback


These are unrelated code changes:


and

But they both die in 001, with exactly 1.24MB in the log and 'Failed to start service openvswitch'
On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron <dron@redhat.com> wrote:
Hello,

I would like to update on this week's failures and OST current status.

On 19-03-2018 - the CI team reported 3 different failures.

On Master branch the failed changes reported were:

core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/
core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/

On 4.2 branch, the reported change was:

core: Call endAction() of all child commands in ImportVmCommand - https://gerrit.ovirt.org/#/c/89165/

The fix's for the regressions were merged the following day (20-03-2018)

https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() logic in ImportVmCommand
https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an instance type

On 20-03-2018 - the CI team discovered an issue on the job's cleanup which caused random failures on changes testing due to failure in docker cleanup. There is an open Jira on the issue: https://ovirt-jira.atlassian.net/browse/OVIRT-1939

Below you can see the chart for this week's resolved issues but cause of failure:

Code = regression of working components/functionalities
Configurations = package related issues
Other = failed build artifacts
Infra = infrastructure/OST/Lago related issues








Below is a chart of resolved failures based on ovirt version

 





Below is a chart showing failures by suite type:






Thank you,
Dafna


_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra




--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme


_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra




--
Didi



--
Didi




--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme




--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme


_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra




--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme




--

Eyal edri


MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D


Red Hat EMEA

TRIED. TESTED. TRUSTED.
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)