I think this is a regression causing rh-postgress to fail to start on selinux conf.
the issue is probably with the selinux packages

I ran lago locally to debug and ssh-ed to the vms and this is the output from the processes start:

Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine postfix/postdrop[9028]: warning: unable to look up public/pickup: No such file or directory
Feb 17 16:02:01 lago-upgrade-from-release-suite-master-engine postfix/postdrop[9029]: warning: unable to look up public/pickup: No such file or directory
Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], ob
Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine systemd[1]: Starting PostgreSQL database server...
-- Subject: Unit rh-postgresql95-postgresql.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit rh-postgresql95-postgresql.service has begun starting up.
Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: postgres cannot access the server configuration file "/var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf": Permission d
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: pg_ctl: could not start server
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9041]: Examine the log output.
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: rh-postgresql95-postgresql.service: control process exited, code=exited status=1
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: Failed to start PostgreSQL database server.
-- Subject: Unit rh-postgresql95-postgresql.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit rh-postgresql95-postgresql.service has failed.
--
-- The result is failed.
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: Unit rh-postgresql95-postgresql.service entered failed state.
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine systemd[1]: rh-postgresql95-postgresql.service failed.
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Unregistered Authentication Agent for unix-process:9033:93610 (system bus name :1.160, object path /org/freedesktop/PolicyKit1/Authent
Feb 17 16:03:01 lago-upgrade-from-release-suite-master-engine systemd[1]: Started Session 51 of user root.
-- Subject: Unit session-51.scope has finished start-up
-- Defined-By: systemd



Secure log:

Feb 17 16:02:34 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Registered Authentication Agent for unix-process:9033:93610 (system bus name :1.160 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Feb 17 16:02:35 lago-upgrade-from-release-suite-master-engine polkitd[2720]: Unregistered Authentication Agent for unix-process:9033:93610 (system bus name :1.160, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)

after setenforce:

root@lago-upgrade-from-release-suite-master-engine ~]# setenforce 0
[root@lago-upgrade-from-release-suite-master-engine ~]# systemctl start rh-postgresql95-postgresql.service
[root@lago-upgrade-from-release-suite-master-engine ~]#
[root@lago-upgrade-from-release-suite-master-engine ~]#
[root@lago-upgrade-from-release-suite-master-engine ~]# systemctl status rh-postgresql95-postgresql.service
● rh-postgresql95-postgresql.service - PostgreSQL database server
   Loaded: loaded (/usr/lib/systemd/system/rh-postgresql95-postgresql.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-02-17 16:08:18 EST; 7s ago
  Process: 9137 ExecStart=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-ctl start -D ${PGDATA} -s -w -t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS)
  Process: 9134 ExecStartPre=/opt/rh/rh-postgresql95/root/usr/libexec/postgresql-check-db-dir %N (code=exited, status=0/SUCCESS)
 Main PID: 9143 (postgres)
   CGroup: /system.slice/rh-postgresql95-postgresql.service
           ├─9143 /opt/rh/rh-postgresql95/root/usr/bin/postgres -D /var/opt/rh/rh-postgresql95/lib/pgsql/data
           ├─9144 postgres: logger process  
           ├─9146 postgres: checkpointer process  
           ├─9147 postgres: writer process  
           ├─9148 postgres: wal writer process  
           ├─9149 postgres: autovacuum launcher process  
           └─9150 postgres: stats collector process  

Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine systemd[1]: Starting PostgreSQL database server...
Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9137]: LOG:  redirecting log output to logging collector process
Feb 17 16:08:17 lago-upgrade-from-release-suite-master-engine postgresql-ctl[9137]: HINT:  Future log output will appear in directory "pg_log".
Feb 17 16:08:18 lago-upgrade-from-release-suite-master-engine systemd[1]: Started PostgreSQL database server.
[root@lago-upgrade-from-release-suite-master-engine ~]#

Not sure who deals with this configuration but this is a blocker as upgrade from release is failing for both ovirt-engine and vdsm.

Thanks,
Dafna


On Sun, Feb 17, 2019 at 10:55 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Thanks Greg

I will check this 


On Sun, Feb 17, 2019 at 12:51 PM Greg Sheremeta <gshereme@redhat.com> wrote:
Is there any way you can run
"systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe"
like it suggests?
The logs below don't give any indication why it failed to start, afaict.

On Sun, Feb 17, 2019 at 4:59 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Hi 

I receive this error message both in CQ and check_patch:

2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd systemd.state:130 starting service rh-postgresql95-postgresql
2019-02-16 16:28:06,874-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:813 execute: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), executable='None', cwd='None', env=None
2019-02-16 16:28:07,913-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.executeRaw:863 execute-result: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service'), rc=1
2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:921 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stdout:


2019-02-16 16:28:07,914-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr:
Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details.

2019-02-16 16:28:07,915-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf''
2019-02-16 16:28:07,916-0500 DEBUG otopi.context context._executeMethod:143 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc
    self._provisioning.provision()
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision
    self.restartPG()
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG
    state=state,
  File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state
    service=name,
RuntimeError: Failed to start service 'rh-postgresql95-postgresql'
2019-02-16 16:28:07,918-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql'
2019-02-16 16:28:07,958-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
id uid local foreign state pid exe

What can cause it?

Thanks
Galit
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/12916/testReport/junit/(root)/001_initialize_engine/running_tests___upgrade_from_release_suite_el7_x86_64___test_initialize_engine/

https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/3207/pipeline/101


Regards,
Galit

--

GALIT ROSENTHAL

SOFTWARE ENGINEER

Red Hat 

galit@gmail.com    T: 972-9-7692230    

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QNDG65M6UPEXTCT3HXORRTZ67RVXH653/


--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme



--

GALIT ROSENTHAL

SOFTWARE ENGINEER

Red Hat 

galit@gmail.com    T: 972-9-7692230    

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YROV4PLNBTOWKYMT2EL25CN3C26HOU2R/