Is there any way you can run "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" like it suggests? The logs below don't give any indication why it failed to start, afaict. On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal <grosenth(a)redhat.com&gt; wrote: > Hi > > I receive this error message both in CQ and check_patch: > > 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql > 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None > 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 > 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: > > > 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: > Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. > > 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' > 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod > method['method']() > File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc > self._provisioning.provision() > File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision > self.restartPG() > File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG > state=state, > File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state > service=name, > RuntimeError: Failed to start service 'rh-postgresql95-postgresql' > 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' > 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: > id uid local foreign state pid exe > > > What can cause it? > > > Thanks > > Galit > > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... > > > https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... > > > > Regards, > > Galit > > > -- > > GALIT ROSENTHAL > > SOFTWARE ENGINEER > > Red Hat > > <https://www.redhat.com/> > > galit(a)gmail.com T: 972-9-7692230 > <https://red.ht/sig> > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme(a)redhat.com IRC: gshereme <https://red.ht/sig>

I think this is a regression causing rh-postgress to fail to start on selinux conf. the issue is probably with the selinux packages I ran lago locally to debug and ssh-ed to the vms and this is the output from the processes start: Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine postfix/postdrop[9028]: warning: unable to look up public/pickup: No such file or directory Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine postfix/postdrop[9029]: warning: unable to look up public/pickup: No such file or directory Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine systemd[1]: Starting PostgreSQL database server... -- Subject: Unit rh-postgresql95-postgresql.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit rh-postgresql95-postgresql.service has begun starting up. Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: pg_ctl: could not start server Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: Examine the log output. Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: rh-postgresql95-postgresql.service: control process exited, code=exited status=1 Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: Failed to start PostgreSQL database server. -- Subject: Unit rh-postgresql95-postgresql.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit rh-postgresql95-postgresql.service has failed. -- -- The result is failed. Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state. Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: rh-postgresql95-postgresql.service failed. Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Unregistered Authentication Agent for unix-process:9033:93610 (system bus name :1.160, object path /org/freedesktop/PolicyKit1/Authent Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine systemd[1]: Started Session 51 of user root. -- Subject: Unit session-51.scope has finished start-up -- Defined-By: systemd Secure log: Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Unregistered Authentication Agent for unix-process:9033:93610 (system bus name :1.160, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus) after setenforce: root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0 [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl start rh-postgresql95-postgresql.service [root@lago-upgrade-from-release-suite-master-engine ~]# [root@lago-upgrade-from-release-suite-master-engine ~]# [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl status rh-postgresql95-postgresql.service ● rh-postgresql95-postgresql.service - PostgreSQL database server Loaded: loaded (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; vendor preset: disabled) Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s ago Process: 9137 ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS) Process: 9134 ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir %N (code=exited, status=0/SUCCESS) Main PID: 9143 (postgres) CGroup: /system.slice/rh-postgresql95-postgresql.service ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D /var/opt/rh/rh-postgresql95/lib/pgsql/data ├─9144 postgres: logger process ├─9146 postgres: checkpointer process ├─9147 postgres: writer process ├─9148 postgres: wal writer process ├─9149 postgres: autovacuum launcher process └─9150 postgres: stats collector process Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine systemd[1]: Starting PostgreSQL database server... Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9137]: LOG: redirecting log output to logging collector process Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9137]: HINT: Future log output will appear in directory "pg_log". Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine systemd[1]: Started PostgreSQL database server. [root@lago-upgrade-from-release-suite-master-engine ~]# Not sure who deals with this configuration but this is a blocker as upgrade from release is failing for both ovirt-engine and vdsm. Thanks, Dafna On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal <grosenth(a)redhat.com&gt; wrote: > Thanks Greg > > I will check this > > > On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta <gshereme(a)redhat.com&gt; > wrote: > >> Is there any way you can run >> "systemctl status rh-postgresql95-postgresql.service" and "journalctl >> -xe" >> like it suggests? >> The logs below don't give any indication why it failed to start, afaict. >> >> On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >> wrote: >> >>> Hi >>> >>> I receive this error message both in CQ and check_patch: >>> >>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql >>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None >>> 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 >>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: >>> >>> >>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: >>> Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. >>> >>> 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' >>> 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>> method['method']() >>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc >>> self._provisioning.provision() >>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision >>> self.restartPG() >>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG >>> state=state, >>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state >>> service=name, >>> RuntimeError: Failed to start service 'rh-postgresql95-postgresql' >>> 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' >>> 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: >>> id uid local foreign state pid exe >>> >>> >>> What can cause it? >>> >>> >>> Thanks >>> >>> Galit >>> >>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... >>> >>> >>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... >>> >>> >>> >>> Regards, >>> >>> Galit >>> >>> >>> -- >>> >>> GALIT ROSENTHAL >>> >>> SOFTWARE ENGINEER >>> >>> Red Hat >>> >>> <https://www.redhat.com/> >>> >>> galit(a)gmail.com T: 972-9-7692230 >>> <https://red.ht/sig> >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... >>> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> <https://www.redhat.com/> >> >> gshereme(a)redhat.com IRC: gshereme >> <https://red.ht/sig> >> > > > -- > > GALIT ROSENTHAL > > SOFTWARE ENGINEER > > Red Hat > > <https://www.redhat.com/> > > galit(a)gmail.com T: 972-9-7692230 > <https://red.ht/sig> > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTO... > _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CSNQENF4J6Z...

Hi, I'm unable to reproduce that issue outside OST, following scenarios worked without any issues: Scenario 1 1. Make sure that selinux-policy-*3.13.1-229.el7_6.9 is not installed 2. Install and configure ovirt-engine 4.2.8 3. Login to webadmin - everything works fine 4. Update to selinux-policy-*3.13.1-229.el7_6.9 5. Login to webadmin - everything works fine 6. Try to restart ovirt-engine and rh-postgresql95-postgresql services 7. Login to webadmin - everything works fine 8. Upgrade all other available packages 9. Login to webadmin - everything works fine 10. Reboot the machine 11. Login to webadmin - everything works fine Senario 2 1. Update CentOS to latest version and make sure that selinux-policy-*3.13.1-229.el7_6.9 is installed 2. Install and configure ovirt-engine 4.2.8 3. Login to webadmin - everything works fine So continuing the investigation, but so far it seems to me related only to OST Martin On Mon, Feb 18, 2019 at 7:39 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: > Just to add some coal to the fire, here are my findings for failures of > the 4.2 OST network suite: > > Following the selinux update [0], engine setup fails because what looks > like failure of engine to communicate with postgresql. > In [1]: > > Feb 16 19:26:55 lago-network-suite-4-2-engine systemd: Starting PostgreSQL database server... > Feb 16 19:26:55 lago-network-suite-4-2-engine postgresql-ctl: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission denied > Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: pg_ctl: could not start server > Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: Examine the log output. > Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service: control process exited, code=exited status=1 > Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Failed to start PostgreSQL database server. > Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Unit rh-postgresql95-postgresql.service entered failed state. > Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service failed. > > and in [2] there are selinux access denials for pg_ctl to read the postgres.conf file: > > type=AVC msg=audit(1550363215.978:1067): avc: denied { read } for pid=8648 comm="pg_ctl" name="postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 > type=SYSCALL msg=audit(1550363215.978:1067): arch=c000003e syscall=2 success=no exit=-13 a0=7ffe611ff730 a1=0 a2=1b6 a3=24 items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) > type=PROCTITLE msg=audit(1550363215.978:1067): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 > type=AVC msg=audit(1550363215.978:1068): avc: denied { getattr } for pid=8648 comm="pg_ctl" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/PG_VERSION" dev="vda4" ino=888709 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 > type=SYSCALL msg=audit(1550363215.978:1068): arch=c000003e syscall=4 success=no exit=-13 a0=60a640 a1=7ffe611ffa50 a2=7ffe611ffa50 a3=2f62696c2f35396c items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) > type=PROCTITLE msg=audit(1550363215.978:1068): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 > type=AVC msg=audit(1550363215.994:1069): avc: denied { getattr } for pid=8654 comm="postgres" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 > type=SYSCALL msg=audit(1550363215.994:1069): arch=c000003e syscall=4 success=no exit=-13 a0=1d862b0 a1=7fff91968710 a2=7fff91968710 a3=2f62696c2f35396c items=0 ppid=8648 pid=8654 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/opt/rh/rh-postgresql95/root/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null) > > whereas in [3] - the build just before the selinux package update, these errors did not occur. > > Looks like alongside enabling selinux a policy update is required. > > thanks > > > [0] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/900/ > [1] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... > [2] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... > [3] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/899/ar... > > > On Sun, Feb 17, 2019 at 11:16 PM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> I think this is a regression causing rh-postgress to fail to start on >> selinux conf. >> the issue is probably with the selinux packages >> >> I ran lago locally to debug and ssh-ed to the vms and this is the output >> from the processes start: >> >> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >> postfix/postdrop[9028]: warning: unable to look up public/pickup: No such >> file or directory >> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >> postfix/postdrop[9029]: warning: unable to look up public/pickup: No such >> file or directory >> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob >> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Starting PostgreSQL database server... >> -- Subject: Unit rh-postgresql95-postgresql.service has begun start-up >> -- Defined-By: systemd >> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel >> -- >> -- Unit rh-postgresql95-postgresql.service has begun starting up. >> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >> postgresql-ctl[9041]: postgres cannot access the server configuration file >> "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> postgresql-ctl[9041]: pg_ctl: could not start server >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> postgresql-ctl[9041]: Examine the log output. >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> systemd[1]: rh-postgresql95-postgresql.service: control process exited, >> code=exited status=1 >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Failed to start PostgreSQL database server. >> -- Subject: Unit rh-postgresql95-postgresql.service has failed >> -- Defined-By: systemd >> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel >> -- >> -- Unit rh-postgresql95-postgresql.service has failed. >> -- >> -- The result is failed. >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state. >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> systemd[1]: rh-postgresql95-postgresql.service failed. >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> polkitd[2720]: Unregistered Authentication Agent for >> unix-process:9033:93610 (system bus name :1.160, object path >> /org/freedesktop/PolicyKit1/Authent >> Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Started Session 51 of user root. >> -- Subject: Unit session-51.scope has finished start-up >> -- Defined-By: systemd >> >> >> >> Secure log: >> >> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], >> object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale >> en_US.UTF-8) >> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >> polkitd[2720]: Unregistered Authentication Agent for >> unix-process:9033:93610 (system bus name :1.160, object path >> /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) >> (disconnected from bus) >> >> after setenforce: >> >> root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0 >> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl start >> rh-postgresql95-postgresql.service >> [root@lago-upgrade-from-release-suite-master-engine ~]# >> [root@lago-upgrade-from-release-suite-master-engine ~]# >> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >> status rh-postgresql95-postgresql.service >> ● rh-postgresql95-postgresql.service - PostgreSQL database server >> Loaded: loaded >> (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; >> vendor preset: disabled) >> Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s ago >> Process: 9137 >> ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D >> ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS) >> Process: 9134 >> ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir >> %N (code=exited, status=0/SUCCESS) >> Main PID: 9143 (postgres) >> CGroup: /system.slice/rh-postgresql95-postgresql.service >> ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D >> /var/opt/rh/rh-postgresql95/lib/pgsql/data >> ├─9144 postgres: logger process >> ├─9146 postgres: checkpointer process >> ├─9147 postgres: writer process >> ├─9148 postgres: wal writer process >> ├─9149 postgres: autovacuum launcher process >> └─9150 postgres: stats collector process >> >> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Starting PostgreSQL database server... >> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >> postgresql-ctl[9137]: LOG: redirecting log output to logging collector >> process >> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >> postgresql-ctl[9137]: HINT: Future log output will appear in directory >> "pg_log". >> Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine >> systemd[1]: Started PostgreSQL database server. >> [root@lago-upgrade-from-release-suite-master-engine ~]# >> >> Not sure who deals with this configuration but this is a blocker as >> upgrade from release is failing for both ovirt-engine and vdsm. >> >> Thanks, >> Dafna >> >> >> On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >> wrote: >> >>> Thanks Greg >>> >>> I will check this >>> >>> >>> On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta <gshereme(a)redhat.com&gt; >>> wrote: >>> >>>> Is there any way you can run >>>> "systemctl status rh-postgresql95-postgresql.service" and "journalctl >>>> -xe" >>>> like it suggests? >>>> The logs below don't give any indication why it failed to start, >>>> afaict. >>>> >>>> On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> I receive this error message both in CQ and check_patch: >>>>> >>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql >>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None >>>>> 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 >>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: >>>>> >>>>> >>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: >>>>> Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. >>>>> >>>>> 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' >>>>> 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception >>>>> Traceback (most recent call last): >>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>> method['method']() >>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc >>>>> self._provisioning.provision() >>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision >>>>> self.restartPG() >>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG >>>>> state=state, >>>>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state >>>>> service=name, >>>>> RuntimeError: Failed to start service 'rh-postgresql95-postgresql' >>>>> 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' >>>>> 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: >>>>> id uid local foreign state pid exe >>>>> >>>>> >>>>> What can cause it? >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Galit >>>>> >>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... >>>>> >>>>> >>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... >>>>> >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Galit >>>>> >>>>> >>>>> -- >>>>> >>>>> GALIT ROSENTHAL >>>>> >>>>> SOFTWARE ENGINEER >>>>> >>>>> Red Hat >>>>> >>>>> <https://www.redhat.com/> >>>>> >>>>> galit(a)gmail.com T: 972-9-7692230 >>>>> <https://red.ht/sig> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... >>>>> >>>> >>>> >>>> -- >>>> >>>> GREG SHEREMETA >>>> >>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>> >>>> Red Hat NA >>>> >>>> <https://www.redhat.com/> >>>> >>>> gshereme(a)redhat.com IRC: gshereme >>>> <https://red.ht/sig> >>>> >>> >>> >>> -- >>> >>> GALIT ROSENTHAL >>> >>> SOFTWARE ENGINEER >>> >>> Red Hat >>> >>> <https://www.redhat.com/> >>> >>> galit(a)gmail.com T: 972-9-7692230 >>> <https://red.ht/sig> >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTO... >>> >> _______________________________________________ >> Devel mailing list -- devel(a)ovirt.org >> To unsubscribe send an email to devel-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CSNQENF4J6Z... >> > -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.

On Mon, Feb 18, 2019 at 9:35 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: > Martin, > Is following package also installed in you env? > > selinux-policy-targeted-3.13.1-229.el7_6.9.noarch > > It is installed in the OST failing environment. > > Yes, both selinux-policy and selinux-policy-targeted are installed and they didn't cause any issues > > > Eitan > > > On Mon, Feb 18, 2019 at 9:56 AM Martin Perina <mperina(a)redhat.com&gt; wrote: > >> Hi, >> >> I'm unable to reproduce that issue outside OST, following scenarios >> worked without any issues: >> >> Scenario 1 >> 1. Make sure that selinux-policy-*3.13.1-229.el7_6.9 is not installed >> 2. Install and configure ovirt-engine 4.2.8 >> 3. Login to webadmin - everything works fine >> 4. Update to selinux-policy-*3.13.1-229.el7_6.9 >> 5. Login to webadmin - everything works fine >> 6. Try to restart ovirt-engine and rh-postgresql95-postgresql services >> 7. Login to webadmin - everything works fine >> 8. Upgrade all other available packages >> 9. Login to webadmin - everything works fine >> 10. Reboot the machine >> 11. Login to webadmin - everything works fine >> >> Senario 2 >> 1. Update CentOS to latest version and make sure that >> selinux-policy-*3.13.1-229.el7_6.9 is installed >> 2. Install and configure ovirt-engine 4.2.8 >> 3. Login to webadmin - everything works fine >> >> So continuing the investigation, but so far it seems to me related only >> to OST >> >> Martin >> >> >> On Mon, Feb 18, 2019 at 7:39 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: >> >>> Just to add some coal to the fire, here are my findings for failures of >>> the 4.2 OST network suite: >>> >>> Following the selinux update [0], engine setup fails because what looks >>> like failure of engine to communicate with postgresql. >>> In [1]: >>> >>> Feb 16 19:26:55 lago-network-suite-4-2-engine systemd: Starting PostgreSQL database server... >>> Feb 16 19:26:55 lago-network-suite-4-2-engine postgresql-ctl: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission denied >>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: pg_ctl: could not start server >>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: Examine the log output. >>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service: control process exited, code=exited status=1 >>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Failed to start PostgreSQL database server. >>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Unit rh-postgresql95-postgresql.service entered failed state. >>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service failed. >>> >>> and in [2] there are selinux access denials for pg_ctl to read the postgres.conf file: >>> >>> type=AVC msg=audit(1550363215.978:1067): avc: denied { read } for pid=8648 comm="pg_ctl" name="postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>> type=SYSCALL msg=audit(1550363215.978:1067): arch=c000003e syscall=2 success=no exit=-13 a0=7ffe611ff730 a1=0 a2=1b6 a3=24 items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>> type=PROCTITLE msg=audit(1550363215.978:1067): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>> type=AVC msg=audit(1550363215.978:1068): avc: denied { getattr } for pid=8648 comm="pg_ctl" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/PG_VERSION" dev="vda4" ino=888709 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>> type=SYSCALL msg=audit(1550363215.978:1068): arch=c000003e syscall=4 success=no exit=-13 a0=60a640 a1=7ffe611ffa50 a2=7ffe611ffa50 a3=2f62696c2f35396c items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>> type=PROCTITLE msg=audit(1550363215.978:1068): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>> type=AVC msg=audit(1550363215.994:1069): avc: denied { getattr } for pid=8654 comm="postgres" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>> type=SYSCALL msg=audit(1550363215.994:1069): arch=c000003e syscall=4 success=no exit=-13 a0=1d862b0 a1=7fff91968710 a2=7fff91968710 a3=2f62696c2f35396c items=0 ppid=8648 pid=8654 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/opt/rh/rh-postgresql95/root/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null) >>> >>> whereas in [3] - the build just before the selinux package update, these errors did not occur. >>> >>> Looks like alongside enabling selinux a policy update is required. >>> >>> thanks >>> >>> >>> [0] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/900/ >>> [1] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>> [2] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>> [3] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/899/ar... >>> >>> >>> On Sun, Feb 17, 2019 at 11:16 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> I think this is a regression causing rh-postgress to fail to start on >>>> selinux conf. >>>> the issue is probably with the selinux packages >>>> >>>> I ran lago locally to debug and ssh-ed to the vms and this is the >>>> output from the processes start: >>>> >>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>> postfix/postdrop[9028]: warning: unable to look up public/pickup: No such >>>> file or directory >>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>> postfix/postdrop[9029]: warning: unable to look up public/pickup: No such >>>> file or directory >>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob >>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Starting PostgreSQL database server... >>>> -- Subject: Unit rh-postgresql95-postgresql.service has begun start-up >>>> -- Defined-By: systemd >>>> -- Support: >>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>> -- >>>> -- Unit rh-postgresql95-postgresql.service has begun starting up. >>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>> postgresql-ctl[9041]: postgres cannot access the server configuration file >>>> "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> postgresql-ctl[9041]: pg_ctl: could not start server >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> postgresql-ctl[9041]: Examine the log output. >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: rh-postgresql95-postgresql.service: control process exited, >>>> code=exited status=1 >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Failed to start PostgreSQL database server. >>>> -- Subject: Unit rh-postgresql95-postgresql.service has failed >>>> -- Defined-By: systemd >>>> -- Support: >>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>> -- >>>> -- Unit rh-postgresql95-postgresql.service has failed. >>>> -- >>>> -- The result is failed. >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state. >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: rh-postgresql95-postgresql.service failed. >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> polkitd[2720]: Unregistered Authentication Agent for >>>> unix-process:9033:93610 (system bus name :1.160, object path >>>> /org/freedesktop/PolicyKit1/Authent >>>> Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Started Session 51 of user root. >>>> -- Subject: Unit session-51.scope has finished start-up >>>> -- Defined-By: systemd >>>> >>>> >>>> >>>> Secure log: >>>> >>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], >>>> object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale >>>> en_US.UTF-8) >>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>> polkitd[2720]: Unregistered Authentication Agent for >>>> unix-process:9033:93610 (system bus name :1.160, object path >>>> /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) >>>> (disconnected from bus) >>>> >>>> after setenforce: >>>> >>>> root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0 >>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>> start rh-postgresql95-postgresql.service >>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>> status rh-postgresql95-postgresql.service >>>> ● rh-postgresql95-postgresql.service - PostgreSQL database server >>>> Loaded: loaded >>>> (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; >>>> vendor preset: disabled) >>>> Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s ago >>>> Process: 9137 >>>> ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D >>>> ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS) >>>> Process: 9134 >>>> ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir >>>> %N (code=exited, status=0/SUCCESS) >>>> Main PID: 9143 (postgres) >>>> CGroup: /system.slice/rh-postgresql95-postgresql.service >>>> ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D >>>> /var/opt/rh/rh-postgresql95/lib/pgsql/data >>>> ├─9144 postgres: logger process >>>> ├─9146 postgres: checkpointer process >>>> ├─9147 postgres: writer process >>>> ├─9148 postgres: wal writer process >>>> ├─9149 postgres: autovacuum launcher process >>>> └─9150 postgres: stats collector process >>>> >>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Starting PostgreSQL database server... >>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>> postgresql-ctl[9137]: LOG: redirecting log output to logging collector >>>> process >>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>> postgresql-ctl[9137]: HINT: Future log output will appear in directory >>>> "pg_log". >>>> Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine >>>> systemd[1]: Started PostgreSQL database server. >>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>> >>>> Not sure who deals with this configuration but this is a blocker as >>>> upgrade from release is failing for both ovirt-engine and vdsm. >>>> >>>> Thanks, >>>> Dafna >>>> >>>> >>>> On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Thanks Greg >>>>> >>>>> I will check this >>>>> >>>>> >>>>> On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta <gshereme(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Is there any way you can run >>>>>> "systemctl status rh-postgresql95-postgresql.service" and >>>>>> "journalctl -xe" >>>>>> like it suggests? >>>>>> The logs below don't give any indication why it failed to start, >>>>>> afaict. >>>>>> >>>>>> On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> I receive this error message both in CQ and check_patch: >>>>>>> >>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql >>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None >>>>>>> 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 >>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: >>>>>>> >>>>>>> >>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: >>>>>>> Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. >>>>>>> >>>>>>> 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' >>>>>>> 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>> Traceback (most recent call last): >>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>> method['method']() >>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc >>>>>>> self._provisioning.provision() >>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision >>>>>>> self.restartPG() >>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG >>>>>>> state=state, >>>>>>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state >>>>>>> service=name, >>>>>>> RuntimeError: Failed to start service 'rh-postgresql95-postgresql' >>>>>>> 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' >>>>>>> 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: >>>>>>> id uid local foreign state pid exe >>>>>>> >>>>>>> >>>>>>> What can cause it? >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Galit >>>>>>> >>>>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... >>>>>>> >>>>>>> >>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... >>>>>>> >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Galit >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> GALIT ROSENTHAL >>>>>>> >>>>>>> SOFTWARE ENGINEER >>>>>>> >>>>>>> Red Hat >>>>>>> >>>>>>> <https://www.redhat.com/> >>>>>>> >>>>>>> galit(a)gmail.com T: 972-9-7692230 >>>>>>> <https://red.ht/sig> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> GREG SHEREMETA >>>>>> >>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>> >>>>>> Red Hat NA >>>>>> >>>>>> <https://www.redhat.com/> >>>>>> >>>>>> gshereme(a)redhat.com IRC: gshereme >>>>>> <https://red.ht/sig> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> GALIT ROSENTHAL >>>>> >>>>> SOFTWARE ENGINEER >>>>> >>>>> Red Hat >>>>> >>>>> <https://www.redhat.com/> >>>>> >>>>> galit(a)gmail.com T: 972-9-7692230 >>>>> <https://red.ht/sig> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTO... >>>>> >>>> _______________________________________________ >>>> Devel mailing list -- devel(a)ovirt.org >>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CSNQENF4J6Z... >>>> >>> >> >> -- >> Martin Perina >> Associate Manager, Software Engineering >> Red Hat Czech s.r.o. >> > -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.

are you running upgrade from release or basic? On Mon, Feb 18, 2019 at 8:46 AM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > On Mon, Feb 18, 2019 at 9:35 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: > >> Martin, >> Is following package also installed in you env? >> >> selinux-policy-targeted-3.13.1-229.el7_6.9.noarch >> >> It is installed in the OST failing environment. >> >> > Yes, both selinux-policy and selinux-policy-targeted are installed and > they didn't cause any issues > >> >> >> Eitan >> >> >> On Mon, Feb 18, 2019 at 9:56 AM Martin Perina <mperina(a)redhat.com&gt; >> wrote: >> >>> Hi, >>> >>> I'm unable to reproduce that issue outside OST, following scenarios >>> worked without any issues: >>> >>> Scenario 1 >>> 1. Make sure that selinux-policy-*3.13.1-229.el7_6.9 is not installed >>> 2. Install and configure ovirt-engine 4.2.8 >>> 3. Login to webadmin - everything works fine >>> 4. Update to selinux-policy-*3.13.1-229.el7_6.9 >>> 5. Login to webadmin - everything works fine >>> 6. Try to restart ovirt-engine and rh-postgresql95-postgresql services >>> 7. Login to webadmin - everything works fine >>> 8. Upgrade all other available packages >>> 9. Login to webadmin - everything works fine >>> 10. Reboot the machine >>> 11. Login to webadmin - everything works fine >>> >>> Senario 2 >>> 1. Update CentOS to latest version and make sure that >>> selinux-policy-*3.13.1-229.el7_6.9 is installed >>> 2. Install and configure ovirt-engine 4.2.8 >>> 3. Login to webadmin - everything works fine >>> >>> So continuing the investigation, but so far it seems to me related only >>> to OST >>> >>> Martin >>> >>> >>> On Mon, Feb 18, 2019 at 7:39 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: >>> >>>> Just to add some coal to the fire, here are my findings for failures >>>> of the 4.2 OST network suite: >>>> >>>> Following the selinux update [0], engine setup fails because what >>>> looks like failure of engine to communicate with postgresql. >>>> In [1]: >>>> >>>> Feb 16 19:26:55 lago-network-suite-4-2-engine systemd: Starting PostgreSQL database server... >>>> Feb 16 19:26:55 lago-network-suite-4-2-engine postgresql-ctl: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission denied >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: pg_ctl: could not start server >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: Examine the log output. >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service: control process exited, code=exited status=1 >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Failed to start PostgreSQL database server. >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Unit rh-postgresql95-postgresql.service entered failed state. >>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service failed. >>>> >>>> and in [2] there are selinux access denials for pg_ctl to read the postgres.conf file: >>>> >>>> type=AVC msg=audit(1550363215.978:1067): avc: denied { read } for pid=8648 comm="pg_ctl" name="postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>> type=SYSCALL msg=audit(1550363215.978:1067): arch=c000003e syscall=2 success=no exit=-13 a0=7ffe611ff730 a1=0 a2=1b6 a3=24 items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>> type=PROCTITLE msg=audit(1550363215.978:1067): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>>> type=AVC msg=audit(1550363215.978:1068): avc: denied { getattr } for pid=8648 comm="pg_ctl" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/PG_VERSION" dev="vda4" ino=888709 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>> type=SYSCALL msg=audit(1550363215.978:1068): arch=c000003e syscall=4 success=no exit=-13 a0=60a640 a1=7ffe611ffa50 a2=7ffe611ffa50 a3=2f62696c2f35396c items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>> type=PROCTITLE msg=audit(1550363215.978:1068): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>>> type=AVC msg=audit(1550363215.994:1069): avc: denied { getattr } for pid=8654 comm="postgres" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>> type=SYSCALL msg=audit(1550363215.994:1069): arch=c000003e syscall=4 success=no exit=-13 a0=1d862b0 a1=7fff91968710 a2=7fff91968710 a3=2f62696c2f35396c items=0 ppid=8648 pid=8654 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/opt/rh/rh-postgresql95/root/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>> >>>> whereas in [3] - the build just before the selinux package update, these errors did not occur. >>>> >>>> Looks like alongside enabling selinux a policy update is required. >>>> >>>> thanks >>>> >>>> >>>> [0] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/900/ >>>> [1] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>>> [2] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>>> [3] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/899/ar... >>>> >>>> >>>> On Sun, Feb 17, 2019 at 11:16 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>>> I think this is a regression causing rh-postgress to fail to start on >>>>> selinux conf. >>>>> the issue is probably with the selinux packages >>>>> >>>>> I ran lago locally to debug and ssh-ed to the vms and this is the >>>>> output from the processes start: >>>>> >>>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>>> postfix/postdrop[9028]: warning: unable to look up public/pickup: No such >>>>> file or directory >>>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>>> postfix/postdrop[9029]: warning: unable to look up public/pickup: No such >>>>> file or directory >>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob >>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Starting PostgreSQL database server... >>>>> -- Subject: Unit rh-postgresql95-postgresql.service has begun start-up >>>>> -- Defined-By: systemd >>>>> -- Support: >>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>>> -- >>>>> -- Unit rh-postgresql95-postgresql.service has begun starting up. >>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>> postgresql-ctl[9041]: postgres cannot access the server configuration file >>>>> "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> postgresql-ctl[9041]: pg_ctl: could not start server >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> postgresql-ctl[9041]: Examine the log output. >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: rh-postgresql95-postgresql.service: control process exited, >>>>> code=exited status=1 >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Failed to start PostgreSQL database server. >>>>> -- Subject: Unit rh-postgresql95-postgresql.service has failed >>>>> -- Defined-By: systemd >>>>> -- Support: >>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>>> -- >>>>> -- Unit rh-postgresql95-postgresql.service has failed. >>>>> -- >>>>> -- The result is failed. >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state. >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: rh-postgresql95-postgresql.service failed. >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> polkitd[2720]: Unregistered Authentication Agent for >>>>> unix-process:9033:93610 (system bus name :1.160, object path >>>>> /org/freedesktop/PolicyKit1/Authent >>>>> Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Started Session 51 of user root. >>>>> -- Subject: Unit session-51.scope has finished start-up >>>>> -- Defined-By: systemd >>>>> >>>>> >>>>> >>>>> Secure log: >>>>> >>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], >>>>> object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale >>>>> en_US.UTF-8) >>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>> polkitd[2720]: Unregistered Authentication Agent for >>>>> unix-process:9033:93610 (system bus name :1.160, object path >>>>> /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) >>>>> (disconnected from bus) >>>>> >>>>> after setenforce: >>>>> >>>>> root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0 >>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>>> start rh-postgresql95-postgresql.service >>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>>> status rh-postgresql95-postgresql.service >>>>> ● rh-postgresql95-postgresql.service - PostgreSQL database server >>>>> Loaded: loaded >>>>> (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; >>>>> vendor preset: disabled) >>>>> Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s ago >>>>> Process: 9137 >>>>> ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D >>>>> ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS) >>>>> Process: 9134 >>>>> ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir >>>>> %N (code=exited, status=0/SUCCESS) >>>>> Main PID: 9143 (postgres) >>>>> CGroup: /system.slice/rh-postgresql95-postgresql.service >>>>> ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D >>>>> /var/opt/rh/rh-postgresql95/lib/pgsql/data >>>>> ├─9144 postgres: logger process >>>>> ├─9146 postgres: checkpointer process >>>>> ├─9147 postgres: writer process >>>>> ├─9148 postgres: wal writer process >>>>> ├─9149 postgres: autovacuum launcher process >>>>> └─9150 postgres: stats collector process >>>>> >>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Starting PostgreSQL database server... >>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>> postgresql-ctl[9137]: LOG: redirecting log output to logging collector >>>>> process >>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>> postgresql-ctl[9137]: HINT: Future log output will appear in directory >>>>> "pg_log". >>>>> Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine >>>>> systemd[1]: Started PostgreSQL database server. >>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>> >>>>> Not sure who deals with this configuration but this is a blocker as >>>>> upgrade from release is failing for both ovirt-engine and vdsm. >>>>> >>>>> Thanks, >>>>> Dafna >>>>> >>>>> >>>>> On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Thanks Greg >>>>>> >>>>>> I will check this >>>>>> >>>>>> >>>>>> On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta <gshereme(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Is there any way you can run >>>>>>> "systemctl status rh-postgresql95-postgresql.service" and >>>>>>> "journalctl -xe" >>>>>>> like it suggests? >>>>>>> The logs below don't give any indication why it failed to start, >>>>>>> afaict. >>>>>>> >>>>>>> On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal < >>>>>>> grosenth(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> I receive this error message both in CQ and check_patch: >>>>>>>> >>>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql >>>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None >>>>>>>> 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 >>>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: >>>>>>>> >>>>>>>> >>>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: >>>>>>>> Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. >>>>>>>> >>>>>>>> 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' >>>>>>>> 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>>> Traceback (most recent call last): >>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>>> method['method']() >>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc >>>>>>>> self._provisioning.provision() >>>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision >>>>>>>> self.restartPG() >>>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG >>>>>>>> state=state, >>>>>>>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state >>>>>>>> service=name, >>>>>>>> RuntimeError: Failed to start service 'rh-postgresql95-postgresql' >>>>>>>> 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' >>>>>>>> 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: >>>>>>>> id uid local foreign state pid exe >>>>>>>> >>>>>>>> >>>>>>>> What can cause it? >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Galit >>>>>>>> >>>>>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... >>>>>>>> >>>>>>>> >>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Galit >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> GALIT ROSENTHAL >>>>>>>> >>>>>>>> SOFTWARE ENGINEER >>>>>>>> >>>>>>>> Red Hat >>>>>>>> >>>>>>>> <https://www.redhat.com/> >>>>>>>> >>>>>>>> galit(a)gmail.com T: 972-9-7692230 >>>>>>>> <https://red.ht/sig> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> GREG SHEREMETA >>>>>>> >>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>> >>>>>>> Red Hat NA >>>>>>> >>>>>>> <https://www.redhat.com/> >>>>>>> >>>>>>> gshereme(a)redhat.com IRC: gshereme >>>>>>> <https://red.ht/sig> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> GALIT ROSENTHAL >>>>>> >>>>>> SOFTWARE ENGINEER >>>>>> >>>>>> Red Hat >>>>>> >>>>>> <https://www.redhat.com/> >>>>>> >>>>>> galit(a)gmail.com T: 972-9-7692230 >>>>>> <https://red.ht/sig> >>>>>> _______________________________________________ >>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTO... >>>>>> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CSNQENF4J6Z... >>>>> >>>> >>> >>> -- >>> Martin Perina >>> Associate Manager, Software Engineering >>> Red Hat Czech s.r.o. >>> >> > > -- > Martin Perina > Associate Manager, Software Engineering > Red Hat Czech s.r.o. >

On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > Also, try to run the same in mock which would isolate your environment > My environment is minimal CentOS 7 VM installed from scratch, why would mock help me? Unfortunately I have FC29 on laptop and I haven't been able to run OST on it so far :-( I have been discussing with Galit over IRC and so far we haven't been able to identify any different package version on my CentOS (where everything works as expected) and her OST environment (where PG 9.5 failed to start).

> On Mon, Feb 18, 2019 at 9:28 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> are you running upgrade from release or basic? >> >> >> On Mon, Feb 18, 2019 at 8:46 AM Martin Perina <mperina(a)redhat.com&gt; >> wrote: >> >>> >>> >>> On Mon, Feb 18, 2019 at 9:35 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: >>> >>>> Martin, >>>> Is following package also installed in you env? >>>> >>>> selinux-policy-targeted-3.13.1-229.el7_6.9.noarch >>>> >>>> It is installed in the OST failing environment. >>>> >>>> >>> Yes, both selinux-policy and selinux-policy-targeted are installed and >>> they didn't cause any issues >>> >>>> >>>> >>>> Eitan >>>> >>>> >>>> On Mon, Feb 18, 2019 at 9:56 AM Martin Perina <mperina(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm unable to reproduce that issue outside OST, following scenarios >>>>> worked without any issues: >>>>> >>>>> Scenario 1 >>>>> 1. Make sure that selinux-policy-*3.13.1-229.el7_6.9 is not >>>>> installed >>>>> 2. Install and configure ovirt-engine 4.2.8 >>>>> 3. Login to webadmin - everything works fine >>>>> 4. Update to selinux-policy-*3.13.1-229.el7_6.9 >>>>> 5. Login to webadmin - everything works fine >>>>> 6. Try to restart ovirt-engine and rh-postgresql95-postgresql >>>>> services >>>>> 7. Login to webadmin - everything works fine >>>>> 8. Upgrade all other available packages >>>>> 9. Login to webadmin - everything works fine >>>>> 10. Reboot the machine >>>>> 11. Login to webadmin - everything works fine >>>>> >>>>> Senario 2 >>>>> 1. Update CentOS to latest version and make sure that >>>>> selinux-policy-*3.13.1-229.el7_6.9 is installed >>>>> 2. Install and configure ovirt-engine 4.2.8 >>>>> 3. Login to webadmin - everything works fine >>>>> >>>>> So continuing the investigation, but so far it seems to me related >>>>> only to OST >>>>> >>>>> Martin >>>>> >>>>> >>>>> On Mon, Feb 18, 2019 at 7:39 AM Eitan Raviv <eraviv(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Just to add some coal to the fire, here are my findings for failures >>>>>> of the 4.2 OST network suite: >>>>>> >>>>>> Following the selinux update [0], engine setup fails because what >>>>>> looks like failure of engine to communicate with postgresql. >>>>>> In [1]: >>>>>> >>>>>> Feb 16 19:26:55 lago-network-suite-4-2-engine systemd: Starting PostgreSQL database server... >>>>>> Feb 16 19:26:55 lago-network-suite-4-2-engine postgresql-ctl: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission denied >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: pg_ctl: could not start server >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine postgresql-ctl: Examine the log output. >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service: control process exited, code=exited status=1 >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Failed to start PostgreSQL database server. >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: Unit rh-postgresql95-postgresql.service entered failed state. >>>>>> Feb 16 19:26:56 lago-network-suite-4-2-engine systemd: rh-postgresql95-postgresql.service failed. >>>>>> >>>>>> and in [2] there are selinux access denials for pg_ctl to read the postgres.conf file: >>>>>> >>>>>> type=AVC msg=audit(1550363215.978:1067): avc: denied { read } for pid=8648 comm="pg_ctl" name="postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>>>> type=SYSCALL msg=audit(1550363215.978:1067): arch=c000003e syscall=2 success=no exit=-13 a0=7ffe611ff730 a1=0 a2=1b6 a3=24 items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>>>> type=PROCTITLE msg=audit(1550363215.978:1067): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>>>>> type=AVC msg=audit(1550363215.978:1068): avc: denied { getattr } for pid=8648 comm="pg_ctl" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/PG_VERSION" dev="vda4" ino=888709 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>>>> type=SYSCALL msg=audit(1550363215.978:1068): arch=c000003e syscall=4 success=no exit=-13 a0=60a640 a1=7ffe611ffa50 a2=7ffe611ffa50 a3=2f62696c2f35396c items=0 ppid=1 pid=8648 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="pg_ctl" exe="/opt/rh/rh-postgresql95/root/usr/bin/pg_ctl" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>>>> type=PROCTITLE msg=audit(1550363215.978:1068): proctitle=2F6F70742F72682F72682D706F737467726573716C39352F726F6F742F7573722F62696E2F70675F63746C007374617274002D44002F7661722F6F70742F72682F72682D706F737467726573716C39352F6C69622F706773716C2F64617461002D73002D77002D7400323730 >>>>>> type=AVC msg=audit(1550363215.994:1069): avc: denied { getattr } for pid=8654 comm="postgres" path="/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf" dev="vda4" ino=888710 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0 >>>>>> type=SYSCALL msg=audit(1550363215.994:1069): arch=c000003e syscall=4 success=no exit=-13 a0=1d862b0 a1=7fff91968710 a2=7fff91968710 a3=2f62696c2f35396c items=0 ppid=8648 pid=8654 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/opt/rh/rh-postgresql95/root/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null) >>>>>> >>>>>> whereas in [3] - the build just before the selinux package update, these errors did not occur. >>>>>> >>>>>> Looks like alongside enabling selinux a policy update is required. >>>>>> >>>>>> thanks >>>>>> >>>>>> >>>>>> [0] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/900/ >>>>>> [1] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>>>>> [2] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/901/ar... >>>>>> [3] https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/899/ar... >>>>>> >>>>>> >>>>>> On Sun, Feb 17, 2019 at 11:16 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>>>> >>>>>>> I think this is a regression causing rh-postgress to fail to start >>>>>>> on selinux conf. >>>>>>> the issue is probably with the selinux packages >>>>>>> >>>>>>> I ran lago locally to debug and ssh-ed to the vms and this is the >>>>>>> output from the processes start: >>>>>>> >>>>>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>>>>> postfix/postdrop[9028]: warning: unable to look up public/pickup: No such >>>>>>> file or directory >>>>>>> Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine >>>>>>> postfix/postdrop[9029]: warning: unable to look up public/pickup: No such >>>>>>> file or directory >>>>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>>>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob >>>>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Starting PostgreSQL database server... >>>>>>> -- Subject: Unit rh-postgresql95-postgresql.service has begun >>>>>>> start-up >>>>>>> -- Defined-By: systemd >>>>>>> -- Support: >>>>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>>>>> -- >>>>>>> -- Unit rh-postgresql95-postgresql.service has begun starting up. >>>>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>>>> postgresql-ctl[9041]: postgres cannot access the server configuration file >>>>>>> "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> postgresql-ctl[9041]: pg_ctl: could not start server >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> postgresql-ctl[9041]: Examine the log output. >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: rh-postgresql95-postgresql.service: control process exited, >>>>>>> code=exited status=1 >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Failed to start PostgreSQL database server. >>>>>>> -- Subject: Unit rh-postgresql95-postgresql.service has failed >>>>>>> -- Defined-By: systemd >>>>>>> -- Support: >>>>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>>>>>> -- >>>>>>> -- Unit rh-postgresql95-postgresql.service has failed. >>>>>>> -- >>>>>>> -- The result is failed. >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state. >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: rh-postgresql95-postgresql.service failed. >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> polkitd[2720]: Unregistered Authentication Agent for >>>>>>> unix-process:9033:93610 (system bus name :1.160, object path >>>>>>> /org/freedesktop/PolicyKit1/Authent >>>>>>> Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Started Session 51 of user root. >>>>>>> -- Subject: Unit session-51.scope has finished start-up >>>>>>> -- Defined-By: systemd >>>>>>> >>>>>>> >>>>>>> >>>>>>> Secure log: >>>>>>> >>>>>>> Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine >>>>>>> polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 >>>>>>> (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], >>>>>>> object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale >>>>>>> en_US.UTF-8) >>>>>>> Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine >>>>>>> polkitd[2720]: Unregistered Authentication Agent for >>>>>>> unix-process:9033:93610 (system bus name :1.160, object path >>>>>>> /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) >>>>>>> (disconnected from bus) >>>>>>> >>>>>>> after setenforce: >>>>>>> >>>>>>> root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0 >>>>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>>>>> start rh-postgresql95-postgresql.service >>>>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# systemctl >>>>>>> status rh-postgresql95-postgresql.service >>>>>>> ● rh-postgresql95-postgresql.service - PostgreSQL database server >>>>>>> Loaded: loaded >>>>>>> (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; >>>>>>> vendor preset: disabled) >>>>>>> Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s >>>>>>> ago >>>>>>> Process: 9137 >>>>>>> ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D >>>>>>> ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS) >>>>>>> Process: 9134 >>>>>>> ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir >>>>>>> %N (code=exited, status=0/SUCCESS) >>>>>>> Main PID: 9143 (postgres) >>>>>>> CGroup: /system.slice/rh-postgresql95-postgresql.service >>>>>>> ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D >>>>>>> /var/opt/rh/rh-postgresql95/lib/pgsql/data >>>>>>> ├─9144 postgres: logger process >>>>>>> ├─9146 postgres: checkpointer process >>>>>>> ├─9147 postgres: writer process >>>>>>> ├─9148 postgres: wal writer process >>>>>>> ├─9149 postgres: autovacuum launcher process >>>>>>> └─9150 postgres: stats collector process >>>>>>> >>>>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Starting PostgreSQL database server... >>>>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>>>> postgresql-ctl[9137]: LOG: redirecting log output to logging collector >>>>>>> process >>>>>>> Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine >>>>>>> postgresql-ctl[9137]: HINT: Future log output will appear in directory >>>>>>> "pg_log". >>>>>>> Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine >>>>>>> systemd[1]: Started PostgreSQL database server. >>>>>>> [root@lago-upgrade-from-release-suite-master-engine ~]# >>>>>>> >>>>>>> Not sure who deals with this configuration but this is a blocker as >>>>>>> upgrade from release is failing for both ovirt-engine and vdsm. >>>>>>> >>>>>>> Thanks, >>>>>>> Dafna >>>>>>> >>>>>>> >>>>>>> On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal < >>>>>>> grosenth(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> Thanks Greg >>>>>>>> >>>>>>>> I will check this >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta < >>>>>>>> gshereme(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> Is there any way you can run >>>>>>>>> "systemctl status rh-postgresql95-postgresql.service" and >>>>>>>>> "journalctl -xe" >>>>>>>>> like it suggests? >>>>>>>>> The logs below don't give any indication why it failed to start, >>>>>>>>> afaict. >>>>>>>>> >>>>>>>>> On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal < >>>>>>>>> grosenth(a)redhat.com&gt; wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> I receive this error message both in CQ and check_patch: >>>>>>>>>> >>>>>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql >>>>>>>>>> 2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None >>>>>>>>>> 2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1 >>>>>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: >>>>>>>>>> Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. >>>>>>>>>> >>>>>>>>>> 2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' >>>>>>>>>> 2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>>>>> Traceback (most recent call last): >>>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>>>>> method['method']() >>>>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc >>>>>>>>>> self._provisioning.provision() >>>>>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision >>>>>>>>>> self.restartPG() >>>>>>>>>> File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG >>>>>>>>>> state=state, >>>>>>>>>> File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state >>>>>>>>>> service=name, >>>>>>>>>> RuntimeError: Failed to start service 'rh-postgresql95-postgresql' >>>>>>>>>> 2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' >>>>>>>>>> 2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: >>>>>>>>>> id uid local foreign state pid exe >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> What can cause it? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> Galit >>>>>>>>>> >>>>>>>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Galit >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> GALIT ROSENTHAL >>>>>>>>>> >>>>>>>>>> SOFTWARE ENGINEER >>>>>>>>>> >>>>>>>>>> Red Hat >>>>>>>>>> >>>>>>>>>> <https://www.redhat.com/> >>>>>>>>>> >>>>>>>>>> galit(a)gmail.com T: 972-9-7692230 >>>>>>>>>> <https://red.ht/sig> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>> List Archives: >>>>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPE... >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> GREG SHEREMETA >>>>>>>>> >>>>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>>>> >>>>>>>>> Red Hat NA >>>>>>>>> >>>>>>>>> <https://www.redhat.com/> >>>>>>>>> >>>>>>>>> gshereme(a)redhat.com IRC: gshereme >>>>>>>>> <https://red.ht/sig> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> GALIT ROSENTHAL >>>>>>>> >>>>>>>> SOFTWARE ENGINEER >>>>>>>> >>>>>>>> Red Hat >>>>>>>> >>>>>>>> <https://www.redhat.com/> >>>>>>>> >>>>>>>> galit(a)gmail.com T: 972-9-7692230 >>>>>>>> <https://red.ht/sig> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTO... >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CSNQENF4J6Z... >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Martin Perina >>>>> Associate Manager, Software Engineering >>>>> Red Hat Czech s.r.o. >>>>> >>>> >>> >>> -- >>> Martin Perina >>> Associate Manager, Software Engineering >>> Red Hat Czech s.r.o. >>> >> -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.

On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: >> >> Also, try to run the same in mock which would isolate your environment > > > My environment is minimal CentOS 7 VM installed from scratch, why would mock help me? > > Unfortunately I have FC29 on laptop and I haven't been able to run OST on it so far :-( > > I have been discussing with Galit over IRC and so far we haven't been able to identify any different package version on my CentOS (where everything works as expected) and her OST environment (where PG 9.5 failed to start). Have you tried running OST 4.2 locally? When it fails, it should keep everything available for you to ssh into. It has been the 4th night that the out 4.2 suite is failing, which is not a healthy condition to be in.

On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > > > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; > wrote: > > > > > > > > > > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > > >> > > >> Also, try to run the same in mock which would isolate your > environment > > > > > > > > > My environment is minimal CentOS 7 VM installed from scratch, why > would mock help me? > > > > > > Unfortunately I have FC29 on laptop and I haven't been able to run > OST on it so far :-( > > > > > > I have been discussing with Galit over IRC and so far we haven't been > able to identify any different package version on my CentOS (where > everything works as expected) and her OST environment (where PG 9.5 failed > to start). > > > > > > Have you tried running OST 4.2 locally? When it fails, it should keep > > everything available for you to ssh into. > > It has been the 4th night that the out 4.2 suite is failing, which is > > not a healthy condition to be in. > > Reproduced and added this comment: > > https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 > > If we do not get a solution soon, we can consider a workaround (as we > did in bug 1518253). > Galit sent a WA which seems to work, do we want to merge it? [1]

or maybe consider reverting the vmconsole patch? [2] which might have started this whole issue?

[1] https://gerrit.ovirt.org/#/c/97843/ [2] https://gerrit.ovirt.org/#/c/97704/ > > Best regards, > -- > Didi > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... > -- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; wrote: > >> >> >> On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; >> wrote: >> >>> On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> > >>> > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> > > >>> > > >>> > > >>> > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: >>> > >> >>> > >> Also, try to run the same in mock which would isolate your >>> environment >>> > > >>> > > >>> > > My environment is minimal CentOS 7 VM installed from scratch, why >>> would mock help me? >>> > > >>> > > Unfortunately I have FC29 on laptop and I haven't been able to run >>> OST on it so far :-( >>> > > >>> > > I have been discussing with Galit over IRC and so far we haven't >>> been able to identify any different package version on my CentOS (where >>> everything works as expected) and her OST environment (where PG 9.5 failed >>> to start). >>> > >>> > >>> > Have you tried running OST 4.2 locally? When it fails, it should keep >>> > everything available for you to ssh into. >>> > It has been the 4th night that the out 4.2 suite is failing, which is >>> > not a healthy condition to be in. >>> >>> Reproduced and added this comment: >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 >>> >>> If we do not get a solution soon, we can consider a workaround (as we >>> did in bug 1518253). >>> >> >> Galit sent a WA which seems to work, do we want to merge it? [1] >> > > Current version of this patch is only for the upgrade suites. I do not > object to merging, we might want to handle somewhere for all suites. > > I am working on a workaround in engine-setup. > Seems like it would not be so easy, so perhaps for now we can update our images in CI to make sure they already have latest selinux packages. I think this will fix too.

> > >> or maybe consider reverting the vmconsole patch? [2] which might have >> started this whole issue? >> > > I think it's not related, not sure. > > >> >> >> [1] https://gerrit.ovirt.org/#/c/97843/ >> [2] https://gerrit.ovirt.org/#/c/97704/ >> >> >>> >>> Best regards, >>> -- >>> Didi >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... >>> >> >> >> -- >> >> Eyal edri >> >> >> MANAGER >> >> RHV/CNV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA <https://www.redhat.com/> >> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >> <https://redhat.com/trusted> >> phone: +972-9-7692018 >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> > > > -- > Didi > -- Didi _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ...

On 19 Feb 2019, at 10:40, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: On Tue, Feb 19, 2019 at 10:34 AM Doron Fediuck <dfediuck(a)redhat.com&gt; wrote: On Tue, 19 Feb 2019 at 10:25, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; wrote: On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > > > > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> > >> Also, try to run the same in mock which would isolate your environment > > > > > > My environment is minimal CentOS 7 VM installed from scratch, why would mock help me? > > > > Unfortunately I have FC29 on laptop and I haven't been able to run OST on it so far :-( > > > > I have been discussing with Galit over IRC and so far we haven't been able to identify any different package version on my CentOS (where everything works as expected) and her OST environment (where PG 9.5 failed to start). > > > Have you tried running OST 4.2 locally? When it fails, it should keep > everything available for you to ssh into. > It has been the 4th night that the out 4.2 suite is failing, which is > not a healthy condition to be in. Reproduced and added this comment: https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 If we do not get a solution soon, we can consider a workaround (as we did in bug 1518253). Galit sent a WA which seems to work, do we want to merge it? [1] Current version of this patch is only for the upgrade suites. I do not object to merging, we might want to handle somewhere for all suites. I am working on a workaround in engine-setup. Seems like it would not be so easy, so perhaps for now we can update our images in CI to make sure they already have latest selinux packages. I think this will fix too. So is this an selinux policy bug? Or is it on our side? It's a bug in the selinux policy package, yes, not on our side. Now pushed a workaround: https://gerrit.ovirt.org/97870 or maybe consider reverting the vmconsole patch? [2] which might have started this whole issue? I think it's not related, not sure. [1] https://gerrit.ovirt.org/#/c/97843/ [2] https://gerrit.ovirt.org/#/c/97704/ Best regards, -- Didi _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... -- EYAL EDRI MANAGER RHV/CNV DEVOPS EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Didi -- Didi _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ... -- Didi _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7RTHHBALWRS...

Il giorno mar 19 feb 2019 alle ore 10:53 Anton Marchukov < amarchuk(a)redhat.com&gt; ha scritto: > Hello Tal, Sandro. > > Since engine from previous tag was failing and hence it is not yet build > we might consider including this fix into the new tag that we need to do > anyway (in case the bumps are not merged yet). > I'm trying to verify didi's patch, but it's kind of hard to reproduce on my laptop. > > Anton. > > > On 19 Feb 2019, at 10:40, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > > > > On Tue, Feb 19, 2019 at 10:34 AM Doron Fediuck <dfediuck(a)redhat.com&gt; > wrote: > > > > > > On Tue, 19 Feb 2019 at 10:25, Yedidyah Bar David <didi(a)redhat.com&gt; > wrote: > > On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David <didi(a)redhat.com&gt; > wrote: > > On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; wrote: > > > > > > On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; > wrote: > > On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; > wrote: > > > > > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; > wrote: > > > > > > > > > > > > > > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > > > >> > > > >> Also, try to run the same in mock which would isolate your > environment > > > > > > > > > > > > My environment is minimal CentOS 7 VM installed from scratch, why > would mock help me? > > > > > > > > Unfortunately I have FC29 on laptop and I haven't been able to run > OST on it so far :-( > > > > > > > > I have been discussing with Galit over IRC and so far we haven't > been able to identify any different package version on my CentOS (where > everything works as expected) and her OST environment (where PG 9.5 failed > to start). > > > > > > > > > Have you tried running OST 4.2 locally? When it fails, it should keep > > > everything available for you to ssh into. > > > It has been the 4th night that the out 4.2 suite is failing, which is > > > not a healthy condition to be in. > > > > Reproduced and added this comment: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 > > > > If we do not get a solution soon, we can consider a workaround (as we > > did in bug 1518253). > > > > Galit sent a WA which seems to work, do we want to merge it? [1] > > > > Current version of this patch is only for the upgrade suites. I do not > object to merging, we might want to handle somewhere for all suites. > > > > I am working on a workaround in engine-setup. > > > > Seems like it would not be so easy, so perhaps for now we can update > our images in CI to make sure they already have latest selinux packages. I > think this will fix too. > > > > So is this an selinux policy bug? Or is it on our side? > > > > It's a bug in the selinux policy package, yes, not on our side. > > > > Now pushed a workaround: > > > > https://gerrit.ovirt.org/97870 > > > > > > > > or maybe consider reverting the vmconsole patch? [2] which might have > started this whole issue? > > > > I think it's not related, not sure. > > > > > > > > [1] https://gerrit.ovirt.org/#/c/97843/ > > [2] https://gerrit.ovirt.org/#/c/97704/ > > > > > > Best regards, > > -- > > Didi > > _______________________________________________ > > Devel mailing list -- devel(a)ovirt.org > > To unsubscribe send an email to devel-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... > > > > > > -- > > EYAL EDRI > > > > MANAGER > > RHV/CNV DEVOPS > > EMEA VIRTUALIZATION R&D > > > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > phone: +972-9-7692018 > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > > > -- > > Didi > > > > > > -- > > Didi > > _______________________________________________ > > Devel mailing list -- devel(a)ovirt.org > > To unsubscribe send an email to devel-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ... > > > > > > -- > > Didi > > _______________________________________________ > > Devel mailing list -- devel(a)ovirt.org > > To unsubscribe send an email to devel-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7RTHHBALWRS... > > -- > Anton Marchukov > Team Lead - Release Management - RHV DevOps - Red Hat > > > > > > -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

On Tue, Feb 19, 2019, 19:45 Galit Rosenthal <grosenth(a)redhat.com wrote: > Hi > > U/S CQ master passed after the use of an updated image > +1, glad to hear that. are we going to apply also the other suggestion for workarounds for postgresql or abandon them?

> Regards > Galit > > > > בתאריך יום ג׳, 19 בפבר׳ 2019, 17:25, מאת Sandro Bonazzola < > sbonazzo(a)redhat.com&gt;: > >> >> >> Il giorno mar 19 feb 2019 alle ore 10:53 Anton Marchukov < >> amarchuk(a)redhat.com&gt; ha scritto: >> >>> Hello Tal, Sandro. >>> >>> Since engine from previous tag was failing and hence it is not yet >>> build we might consider including this fix into the new tag that we need to >>> do anyway (in case the bumps are not merged yet). >>> >> >> I'm trying to verify didi's patch, but it's kind of hard to reproduce on >> my laptop. >> >> >> >>> >>> Anton. >>> >>> > On 19 Feb 2019, at 10:40, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >>> > >>> > On Tue, Feb 19, 2019 at 10:34 AM Doron Fediuck <dfediuck(a)redhat.com&gt; >>> wrote: >>> > >>> > >>> > On Tue, 19 Feb 2019 at 10:25, Yedidyah Bar David <didi(a)redhat.com&gt; >>> wrote: >>> > On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>> wrote: >>> > On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; wrote: >>> > >>> > >>> > On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>> wrote: >>> > On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> > > >>> > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> > > > >>> > > > >>> > > > >>> > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; >>> wrote: >>> > > >> >>> > > >> Also, try to run the same in mock which would isolate your >>> environment >>> > > > >>> > > > >>> > > > My environment is minimal CentOS 7 VM installed from scratch, why >>> would mock help me? >>> > > > >>> > > > Unfortunately I have FC29 on laptop and I haven't been able to >>> run OST on it so far :-( >>> > > > >>> > > > I have been discussing with Galit over IRC and so far we haven't >>> been able to identify any different package version on my CentOS (where >>> everything works as expected) and her OST environment (where PG 9.5 failed >>> to start). >>> > > >>> > > >>> > > Have you tried running OST 4.2 locally? When it fails, it should >>> keep >>> > > everything available for you to ssh into. >>> > > It has been the 4th night that the out 4.2 suite is failing, which >>> is >>> > > not a healthy condition to be in. >>> > >>> > Reproduced and added this comment: >>> > >>> > https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 >>> > >>> > If we do not get a solution soon, we can consider a workaround (as we >>> > did in bug 1518253). >>> > >>> > Galit sent a WA which seems to work, do we want to merge it? [1] >>> > >>> > Current version of this patch is only for the upgrade suites. I do >>> not object to merging, we might want to handle somewhere for all suites. >>> > >>> > I am working on a workaround in engine-setup. >>> > >>> > Seems like it would not be so easy, so perhaps for now we can update >>> our images in CI to make sure they already have latest selinux packages. I >>> think this will fix too. >>> > >>> > So is this an selinux policy bug? Or is it on our side? >>> > >>> > It's a bug in the selinux policy package, yes, not on our side. >>> > >>> > Now pushed a workaround: >>> > >>> > https://gerrit.ovirt.org/97870 >>> > >>> > >>> > >>> > or maybe consider reverting the vmconsole patch? [2] which might >>> have started this whole issue? >>> > >>> > I think it's not related, not sure. >>> > >>> > >>> > >>> > [1] https://gerrit.ovirt.org/#/c/97843/ >>> > [2] https://gerrit.ovirt.org/#/c/97704/ >>> > >>> > >>> > Best regards, >>> > -- >>> > Didi >>> > _______________________________________________ >>> > Devel mailing list -- devel(a)ovirt.org >>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> > oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> > List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... >>> > >>> > >>> > -- >>> > EYAL EDRI >>> > >>> > MANAGER >>> > RHV/CNV DEVOPS >>> > EMEA VIRTUALIZATION R&D >>> > >>> > Red Hat EMEA >>> > TRIED. TESTED. TRUSTED. >>> > phone: +972-9-7692018 >>> > irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> > >>> > >>> > -- >>> > Didi >>> > >>> > >>> > -- >>> > Didi >>> > _______________________________________________ >>> > Devel mailing list -- devel(a)ovirt.org >>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> > oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> > List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ... >>> > >>> > >>> > -- >>> > Didi >>> > _______________________________________________ >>> > Devel mailing list -- devel(a)ovirt.org >>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> > oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> > List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7RTHHBALWRS... >>> >>> -- >>> Anton Marchukov >>> Team Lead - Release Management - RHV DevOps - Red Hat >>> >>> >>> >>> >>> >>> >> >> -- >> >> SANDRO BONAZZOLA >> >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> >> Red Hat EMEA <https://www.redhat.com/> >> >> sbonazzo(a)redhat.com >> <https://red.ht/sig> >> > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GZ7ZNVNPOH3... > _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J3QWUYU4TUN...

I think having the new base image working confirms the assumption that the problem was related to some old selinux package.

Then we just need to estimate the probability of users having those package installed and not upgraded?

Anton. On Tue, Feb 19, 2019 at 7:47 PM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > On Tue, 19 Feb 2019, 19:22 Eyal Edri <eedri(a)redhat.com wrote: > >> >> >> On Tue, Feb 19, 2019, 19:45 Galit Rosenthal <grosenth(a)redhat.com wrote: >> >>> Hi >>> >>> U/S CQ master passed after the use of an updated image >>> >> >> +1, glad to hear that. >> are we going to apply also the other suggestion for workarounds for >> postgresql or abandon them? >> > > I'd say let's use updated image to unblock OST, we discus further > improvements later... > > >> >>> Regards >>> Galit >>> >>> >>> >>> בתאריך יום ג׳, 19 בפבר׳ 2019, 17:25, מאת Sandro Bonazzola < >>> sbonazzo(a)redhat.com&gt;: >>> >>>> >>>> >>>> Il giorno mar 19 feb 2019 alle ore 10:53 Anton Marchukov < >>>> amarchuk(a)redhat.com&gt; ha scritto: >>>> >>>>> Hello Tal, Sandro. >>>>> >>>>> Since engine from previous tag was failing and hence it is not yet >>>>> build we might consider including this fix into the new tag that we need to >>>>> do anyway (in case the bumps are not merged yet). >>>>> >>>> >>>> I'm trying to verify didi's patch, but it's kind of hard to reproduce >>>> on my laptop. >>>> >>>> >>>> >>>>> >>>>> Anton. >>>>> >>>>> > On 19 Feb 2019, at 10:40, Yedidyah Bar David <didi(a)redhat.com&gt; >>>>> wrote: >>>>> > >>>>> > On Tue, Feb 19, 2019 at 10:34 AM Doron Fediuck <dfediuck(a)redhat.com&gt; >>>>> wrote: >>>>> > >>>>> > >>>>> > On Tue, 19 Feb 2019 at 10:25, Yedidyah Bar David <didi(a)redhat.com&gt; >>>>> wrote: >>>>> > On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>>>> wrote: >>>>> > On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>>> > >>>>> > >>>>> > On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; >>>>> wrote: >>>>> > On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; >>>>> wrote: >>>>> > > >>>>> > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina < >>>>> mperina(a)redhat.com&gt; wrote: >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; >>>>> wrote: >>>>> > > >> >>>>> > > >> Also, try to run the same in mock which would isolate your >>>>> environment >>>>> > > > >>>>> > > > >>>>> > > > My environment is minimal CentOS 7 VM installed from scratch, >>>>> why would mock help me? >>>>> > > > >>>>> > > > Unfortunately I have FC29 on laptop and I haven't been able to >>>>> run OST on it so far :-( >>>>> > > > >>>>> > > > I have been discussing with Galit over IRC and so far we >>>>> haven't been able to identify any different package version on my CentOS >>>>> (where everything works as expected) and her OST environment (where PG 9.5 >>>>> failed to start). >>>>> > > >>>>> > > >>>>> > > Have you tried running OST 4.2 locally? When it fails, it should >>>>> keep >>>>> > > everything available for you to ssh into. >>>>> > > It has been the 4th night that the out 4.2 suite is failing, >>>>> which is >>>>> > > not a healthy condition to be in. >>>>> > >>>>> > Reproduced and added this comment: >>>>> > >>>>> > https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 >>>>> > >>>>> > If we do not get a solution soon, we can consider a workaround (as >>>>> we >>>>> > did in bug 1518253). >>>>> > >>>>> > Galit sent a WA which seems to work, do we want to merge it? [1] >>>>> > >>>>> > Current version of this patch is only for the upgrade suites. I do >>>>> not object to merging, we might want to handle somewhere for all suites. >>>>> > >>>>> > I am working on a workaround in engine-setup. >>>>> > >>>>> > Seems like it would not be so easy, so perhaps for now we can >>>>> update our images in CI to make sure they already have latest selinux >>>>> packages. I think this will fix too. >>>>> > >>>>> > So is this an selinux policy bug? Or is it on our side? >>>>> > >>>>> > It's a bug in the selinux policy package, yes, not on our side. >>>>> > >>>>> > Now pushed a workaround: >>>>> > >>>>> > https://gerrit.ovirt.org/97870 >>>>> > >>>>> > >>>>> > >>>>> > or maybe consider reverting the vmconsole patch? [2] which might >>>>> have started this whole issue? >>>>> > >>>>> > I think it's not related, not sure. >>>>> > >>>>> > >>>>> > >>>>> > [1] https://gerrit.ovirt.org/#/c/97843/ >>>>> > [2] https://gerrit.ovirt.org/#/c/97704/ >>>>> > >>>>> > >>>>> > Best regards, >>>>> > -- >>>>> > Didi >>>>> > _______________________________________________ >>>>> > Devel mailing list -- devel(a)ovirt.org >>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> > oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> > List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... >>>>> > >>>>> > >>>>> > -- >>>>> > EYAL EDRI >>>>> > >>>>> > MANAGER >>>>> > RHV/CNV DEVOPS >>>>> > EMEA VIRTUALIZATION R&D >>>>> > >>>>> > Red Hat EMEA >>>>> > TRIED. TESTED. TRUSTED. >>>>> > phone: +972-9-7692018 >>>>> > irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> > >>>>> > >>>>> > -- >>>>> > Didi >>>>> > >>>>> > >>>>> > -- >>>>> > Didi >>>>> > _______________________________________________ >>>>> > Devel mailing list -- devel(a)ovirt.org >>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> > oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> > List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ... >>>>> > >>>>> > >>>>> > -- >>>>> > Didi >>>>> > _______________________________________________ >>>>> > Devel mailing list -- devel(a)ovirt.org >>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> > oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> > List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7RTHHBALWRS... >>>>> >>>>> -- >>>>> Anton Marchukov >>>>> Team Lead - Release Management - RHV DevOps - Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> SANDRO BONAZZOLA >>>> >>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>> >>>> Red Hat EMEA <https://www.redhat.com/> >>>> >>>> sbonazzo(a)redhat.com >>>> <https://red.ht/sig> >>>> >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GZ7ZNVNPOH3... >>> >> _______________________________________________ >> Devel mailing list -- devel(a)ovirt.org >> To unsubscribe send an email to devel-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J3QWUYU4TUN... >> > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YQUJCUP3PWC... > -- Anton Marchukov Team Lead - Release Management - RHV DevOps - Red Hat _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/INV4VMSF3QN...

For building the centos/rhel images I'm using the information from this document https://docs.google.com/document/d/1fxXBcgHnQr3r2qrSzXiSlpNMMLWnbq899s8rU...

On Wed, Feb 20, 2019 at 11:29 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Tue, Feb 19, 2019 at 8:51 PM Anton Marchukov <amarchuk(a)redhat.com&gt; > wrote: > >> I think having the new base image working confirms the assumption that >> the problem was related to some old selinux package. >> > > How are the images built? I searched a bit and found [1]. Do we run it > manually? In jenkins? Elsewhere? Do we keep logs? To investigate deeper, > I'd like to understand how exactly we built the broken images. > > [1] https://github.com/lago-project/lago-images > > >> Then we just need to estimate the probability of users having those >> package installed and not upgraded? >> > > Not very probable, AFAICT. I *think* the flow is something like this, but > this is not more than a (not-so-)wild guess: > > 1. Install el 7.2 (with selinux-policy 3.12) > 2. Upgrade to later > 3. Reboot the machine without a working /dev/console > 4. Install scl pg and initdb > > (1.) or (2.) might be about how we create the images, (3.) might be > specific to libvirt/lago. > > >> Anton. >> >> On Tue, Feb 19, 2019 at 7:47 PM Martin Perina <mperina(a)redhat.com&gt; >> wrote: >> >>> >>> >>> On Tue, 19 Feb 2019, 19:22 Eyal Edri <eedri(a)redhat.com wrote: >>> >>>> >>>> >>>> On Tue, Feb 19, 2019, 19:45 Galit Rosenthal <grosenth(a)redhat.com >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> U/S CQ master passed after the use of an updated image >>>>> >>>> >>>> +1, glad to hear that. >>>> are we going to apply also the other suggestion for workarounds for >>>> postgresql or abandon them? >>>> >>> >>> I'd say let's use updated image to unblock OST, we discus further >>> improvements later... >>> >>> >>>> >>>>> Regards >>>>> Galit >>>>> >>>>> >>>>> >>>>> בתאריך יום ג׳, 19 בפבר׳ 2019, 17:25, מאת Sandro Bonazzola < >>>>> sbonazzo(a)redhat.com&gt;: >>>>> >>>>>> >>>>>> >>>>>> Il giorno mar 19 feb 2019 alle ore 10:53 Anton Marchukov < >>>>>> amarchuk(a)redhat.com&gt; ha scritto: >>>>>> >>>>>>> Hello Tal, Sandro. >>>>>>> >>>>>>> Since engine from previous tag was failing and hence it is not yet >>>>>>> build we might consider including this fix into the new tag that we need to >>>>>>> do anyway (in case the bumps are not merged yet). >>>>>>> >>>>>> >>>>>> I'm trying to verify didi's patch, but it's kind of hard to >>>>>> reproduce on my laptop. >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Anton. >>>>>>> >>>>>>> > On 19 Feb 2019, at 10:40, Yedidyah Bar David <didi(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > >>>>>>> > On Tue, Feb 19, 2019 at 10:34 AM Doron Fediuck < >>>>>>> dfediuck(a)redhat.com&gt; wrote: >>>>>>> > >>>>>>> > >>>>>>> > On Tue, 19 Feb 2019 at 10:25, Yedidyah Bar David <didi(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > On Tue, Feb 19, 2019 at 9:52 AM Yedidyah Bar David < >>>>>>> didi(a)redhat.com&gt; wrote: >>>>>>> > On Tue, Feb 19, 2019 at 9:49 AM Eyal Edri <eedri(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > >>>>>>> > >>>>>>> > On Tue, Feb 19, 2019 at 9:29 AM Yedidyah Bar David < >>>>>>> didi(a)redhat.com&gt; wrote: >>>>>>> > On Tue, Feb 19, 2019 at 9:15 AM Dan Kenigsberg <danken(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > > >>>>>>> > > On Mon, Feb 18, 2019 at 11:39 AM Martin Perina < >>>>>>> mperina(a)redhat.com&gt; wrote: >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > On Mon, Feb 18, 2019 at 10:32 AM Dafna Ron <dron(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > > >> >>>>>>> > > >> Also, try to run the same in mock which would isolate your >>>>>>> environment >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > My environment is minimal CentOS 7 VM installed from scratch, >>>>>>> why would mock help me? >>>>>>> > > > >>>>>>> > > > Unfortunately I have FC29 on laptop and I haven't been able >>>>>>> to run OST on it so far :-( >>>>>>> > > > >>>>>>> > > > I have been discussing with Galit over IRC and so far we >>>>>>> haven't been able to identify any different package version on my CentOS >>>>>>> (where everything works as expected) and her OST environment (where PG 9.5 >>>>>>> failed to start). >>>>>>> > > >>>>>>> > > >>>>>>> > > Have you tried running OST 4.2 locally? When it fails, it >>>>>>> should keep >>>>>>> > > everything available for you to ssh into. >>>>>>> > > It has been the 4th night that the out 4.2 suite is failing, >>>>>>> which is >>>>>>> > > not a healthy condition to be in. >>>>>>> > >>>>>>> > Reproduced and added this comment: >>>>>>> > >>>>>>> > https://bugzilla.redhat.com/show_bug.cgi?id=1518599#c6 >>>>>>> > >>>>>>> > If we do not get a solution soon, we can consider a workaround >>>>>>> (as we >>>>>>> > did in bug 1518253). >>>>>>> > >>>>>>> > Galit sent a WA which seems to work, do we want to merge it? [1] >>>>>>> > >>>>>>> > Current version of this patch is only for the upgrade suites. I >>>>>>> do not object to merging, we might want to handle somewhere for all suites. >>>>>>> > >>>>>>> > I am working on a workaround in engine-setup. >>>>>>> > >>>>>>> > Seems like it would not be so easy, so perhaps for now we can >>>>>>> update our images in CI to make sure they already have latest selinux >>>>>>> packages. I think this will fix too. >>>>>>> > >>>>>>> > So is this an selinux policy bug? Or is it on our side? >>>>>>> > >>>>>>> > It's a bug in the selinux policy package, yes, not on our side. >>>>>>> > >>>>>>> > Now pushed a workaround: >>>>>>> > >>>>>>> > https://gerrit.ovirt.org/97870 >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > or maybe consider reverting the vmconsole patch? [2] which might >>>>>>> have started this whole issue? >>>>>>> > >>>>>>> > I think it's not related, not sure. >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > [1] https://gerrit.ovirt.org/#/c/97843/ >>>>>>> > [2] https://gerrit.ovirt.org/#/c/97704/ >>>>>>> > >>>>>>> > >>>>>>> > Best regards, >>>>>>> > -- >>>>>>> > Didi >>>>>>> > _______________________________________________ >>>>>>> > Devel mailing list -- devel(a)ovirt.org >>>>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XYD4CJQGHAU... >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > EYAL EDRI >>>>>>> > >>>>>>> > MANAGER >>>>>>> > RHV/CNV DEVOPS >>>>>>> > EMEA VIRTUALIZATION R&D >>>>>>> > >>>>>>> > Red Hat EMEA >>>>>>> > TRIED. TESTED. TRUSTED. >>>>>>> > phone: +972-9-7692018 >>>>>>> > irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > Didi >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > Didi >>>>>>> > _______________________________________________ >>>>>>> > Devel mailing list -- devel(a)ovirt.org >>>>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7QF4RSTMYHZ... >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > Didi >>>>>>> > _______________________________________________ >>>>>>> > Devel mailing list -- devel(a)ovirt.org >>>>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7RTHHBALWRS... >>>>>>> >>>>>>> -- >>>>>>> Anton Marchukov >>>>>>> Team Lead - Release Management - RHV DevOps - Red Hat >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> SANDRO BONAZZOLA >>>>>> >>>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>>> >>>>>> Red Hat EMEA <https://www.redhat.com/> >>>>>> >>>>>> sbonazzo(a)redhat.com >>>>>> <https://red.ht/sig> >>>>>> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GZ7ZNVNPOH3... >>>>> >>>> _______________________________________________ >>>> Devel mailing list -- devel(a)ovirt.org >>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J3QWUYU4TUN... >>>> >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YQUJCUP3PWC... >>> >> >> >> -- >> Anton Marchukov >> Team Lead - Release Management - RHV DevOps - Red Hat >> >> _______________________________________________ >> Devel mailing list -- devel(a)ovirt.org >> To unsubscribe send an email to devel-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/INV4VMSF3QN... >> > > > -- > Didi > -- GALIT ROSENTHAL SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit(a)gmail.com T: 972-9-7692230 <https://red.ht/sig>

On Tue, 19 Feb 2019, 19:22 Eyal Edri <eedri(a)redhat.com wrote: > > > On Tue, Feb 19, 2019, 19:45 Galit Rosenthal <grosenth(a)redhat.com wrote: > >> Hi >> >> U/S CQ master passed after the use of an updated image >> > > +1, glad to hear that. > are we going to apply also the other suggestion for workarounds for > postgresql or abandon them? > I'd say let's use updated image to unblock OST, we discus further improvements later...