On Fri, Jul 28, 2017 at 8:37 PM, Richard Chan < richard(a)treeboxsolutions.com&gt; wrote: > After an upgrade to 4.0 I have a single host that cannot start supervdsmd > because of IOError on /dev/stdout. All other hosts upgraded correctly. > Upgrade from which version? 3.6? Did you stay on 4.0 or continued to 4.1? > > In the systemd unit I have to hack StandardOutput=null. > > Any thing I have overlooked? The hosts are all identical and it is just > this one > that has this weird behaviour. > Any logs that you can share? Y. > > -- > Richard Chan > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > >

Hi, Seems like I already bumped in such issue awhile ago - https://bugzilla.redhat.com/show_bug.cgi?id=1216880 but now I see what's wrong - it happens after failing to read /etc/vdsm/svdsm.logger.conf for some reason then we have a bug in our fallback which tries to run logging.basicConfig(filename='/dev/stdout', filemode='w+', level=logging.DEBUG) and this fails while running as systemd daemon sorry about that. check why it can't load /etc/vdsm/svdsm.logger.conf and update us after fixing the access issue check if you can run logging.config.fileConfig("/etc/vdsm/svdsm.logger.conf" , disable_existing_loggers=False) Then supervdsmd should start properly I'll reopen the bugzilla and fix the wrong fallback to syslog print Thanks! On Sun, Jul 30, 2017 at 3:23 PM Richard Chan <richard(a)treeboxsolutions.com&gt; wrote: > Hi Yaniv, > > On this one node, it happened from 3.6 -> 4.0. It persisted after > 4.0->4.1. > Current: vdsm-4.19.24-1.el7.centos.x86_64 > > > In systemd override I now have to have: > StandardOutput=null > > > > > Jul 29 01:19:21 xxxxxxxx systemd[1]: Starting Auxiliary vdsm service for > running helper functions as root... > Jul 29 01:19:21 xxxxxxxx python2[39124]: detected unhandled Python > exception in '/usr/share/vdsm/supervdsmServer' > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: Traceback (most recent > call last): > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File "/usr/share/vdsm/supervdsmServer", > line 45, in <module> > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: level=logging.DEBUG) > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File > "/usr/lib64/python2.7/logging/__init__.py", line 1529, in basicConfig > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: hdlr = > FileHandler(filename, mode) > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File > "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: > StreamHandler.__init__(self, self._open()) > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File > "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: stream = > open(self.baseFilename, self.mode) > Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: IOError: [Errno 6] No such > device or address: '/dev/stdout' > Jul 29 01:19:21 xxxxxxxx systemd[1]: supervdsmd.service: main process > exited, code=exited, status=1/FAILURE > > > > On Sun, Jul 30, 2017 at 3:47 PM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: > >> >> >> On Fri, Jul 28, 2017 at 8:37 PM, Richard Chan < >> richard(a)treeboxsolutions.com&gt; wrote: >> >>> After an upgrade to 4.0 I have a single host that cannot start >>> supervdsmd because of IOError on /dev/stdout. All other hosts upgraded >>> correctly. >>> >> >> Upgrade from which version? 3.6? Did you stay on 4.0 or continued to 4.1? >> >> >>> >>> In the systemd unit I have to hack StandardOutput=null. >>> >>> Any thing I have overlooked? The hosts are all identical and it is just >>> this one >>> that has this weird behaviour. >>> >> >> Any logs that you can share? >> Y. >> >> >>> >>> -- >>> Richard Chan >>> >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >> > > > -- > Richard Chan > Chief Architect > > TreeBox Solutions Pte Ltd > 1 Commonwealth Lane #03-01 > Singapore 149544 > Tel: 6570 3725 > http://www.treeboxsolutions.com > > Co.Reg.No. 201100585R > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > -- Yaniv Bronhaim.

Hi, Could this be the cause of the access issue (old system upgraded from 3.4 all the way to 4.1)? ## change in python class name?? [handler_logfile] -class=logUtils.UserGroupEnforcingHandler +class=vdsm.logUtils.UserGroupEnforcingHandler When I copied svdsm.logger.conf.rpmnew over svdsm.logger.conf, it could start up. Thanks! On Tue, Aug 1, 2017 at 5:02 PM, Yaniv Bronheim <ybronhei(a)redhat.com&gt; wrote: > Hi, > > Seems like I already bumped in such issue awhile ago - > https://bugzilla.redhat.com/show_bug.cgi?id=1216880 > but now I see what's wrong - it happens after failing to read > /etc/vdsm/svdsm.logger.conf for some reason > then we have a bug in our fallback which tries to run > logging.basicConfig(filename='/dev/stdout', filemode='w+', > level=logging.DEBUG) > > and this fails while running as systemd daemon > > sorry about that. check why it can't load /etc/vdsm/svdsm.logger.conf > and update us > after fixing the access issue check if you can run > logging.config.fileConfig("/etc/vdsm/svdsm.logger.conf" , > disable_existing_loggers=False) > Then supervdsmd should start properly > > I'll reopen the bugzilla and fix the wrong fallback to syslog print > > Thanks! > > > On Sun, Jul 30, 2017 at 3:23 PM Richard Chan < > richard(a)treeboxsolutions.com&gt; wrote: > >> Hi Yaniv, >> >> On this one node, it happened from 3.6 -> 4.0. It persisted after >> 4.0->4.1. >> Current: vdsm-4.19.24-1.el7.centos.x86_64 >> >> >> In systemd override I now have to have: >> StandardOutput=null >> >> >> >> >> Jul 29 01:19:21 xxxxxxxx systemd[1]: Starting Auxiliary vdsm service for >> running helper functions as root... >> Jul 29 01:19:21 xxxxxxxx python2[39124]: detected unhandled Python >> exception in '/usr/share/vdsm/supervdsmServer' >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: Traceback (most recent >> call last): >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File >> "/usr/share/vdsm/supervdsmServer", line 45, in <module> >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: level=logging.DEBUG) >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File >> "/usr/lib64/python2.7/logging/__init__.py", line 1529, in basicConfig >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: hdlr = >> FileHandler(filename, mode) >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File >> "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: >> StreamHandler.__init__(self, self._open()) >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: File >> "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: stream = >> open(self.baseFilename, self.mode) >> Jul 29 01:19:21 xxxxxxxx daemonAdapter[39124]: IOError: [Errno 6] No >> such device or address: '/dev/stdout' >> Jul 29 01:19:21 xxxxxxxx systemd[1]: supervdsmd.service: main process >> exited, code=exited, status=1/FAILURE >> >> >> >> On Sun, Jul 30, 2017 at 3:47 PM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: >> >>> >>> >>> On Fri, Jul 28, 2017 at 8:37 PM, Richard Chan < >>> richard(a)treeboxsolutions.com&gt; wrote: >>> >>>> After an upgrade to 4.0 I have a single host that cannot start >>>> supervdsmd because of IOError on /dev/stdout. All other hosts upgraded >>>> correctly. >>>> >>> >>> Upgrade from which version? 3.6? Did you stay on 4.0 or continued to >>> 4.1? >>> >>> >>>> >>>> In the systemd unit I have to hack StandardOutput=null. >>>> >>>> Any thing I have overlooked? The hosts are all identical and it is >>>> just this one >>>> that has this weird behaviour. >>>> >>> >>> Any logs that you can share? >>> Y. >>> >>> >>>> >>>> -- >>>> Richard Chan >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>>> >>>> >>> >> >> >> -- >> Richard Chan >> Chief Architect >> >> TreeBox Solutions Pte Ltd >> 1 Commonwealth Lane #03-01 >> Singapore 149544 >> Tel: 6570 3725 >> http://www.treeboxsolutions.com >> >> Co.Reg.No. 201100585R >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> > -- > Yaniv Bronhaim. > -- Richard Chan Chief Architect TreeBox Solutions Pte Ltd 1 Commonwealth Lane #03-01 Singapore 149544 Tel: 6570 3725 http://www.treeboxsolutions.com Co.Reg.No. 201100585R