Re: [ovirt-users] host-deploy on HE hosts failing: Job for vdsmd.service canceled.

From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; To: "Simone Tiraboschi" <stirabos(a)redhat.com&gt; Cc: Users(a)ovirt.org Sent: Wednesday, June 17, 2015 7:06:53 PM Subject: Re: [ovirt-users] host-deploy on HE hosts failing: Job for vdsmd.service canceled. On 17.06.2015 18:30, Simone Tiraboschi wrote: > > >> Il giorno 17/giu/2015, alle ore 18:21, Daniel Helgenberger >> <daniel.helgenberger(a)m-box.de&gt; ha scritto: >> >> >> >>> On 17.06.2015 17:01, Simone Tiraboschi wrote: >>> >>> >>> ----- Original Message ----- >>>> From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; >>>> To: "Simone Tiraboschi" <stirabos(a)redhat.com&gt; >>>> Cc: Users(a)ovirt.org, "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>> Sent: Wednesday, June 17, 2015 4:49:07 PM >>>> Subject: Re: [ovirt-users] host-deploy on HE hosts failing: Job for >>>> vdsmd.service canceled. >>>> >>>> >>>> >>>>> On 17.06.2015 16:47, Simone Tiraboschi wrote: >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; >>>>>> To: "Simone Tiraboschi" <stirabos(a)redhat.com&gt; >>>>>> Cc: Users(a)ovirt.org, "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>>> Sent: Wednesday, June 17, 2015 4:40:45 PM >>>>>> Subject: Re: [ovirt-users] host-deploy on HE hosts failing: Job for >>>>>> vdsmd.service canceled. >>>>>> >>>>>> >>>>>> >>>>>>> On 17.06.2015 16:35, Simone Tiraboschi wrote: >>>>>>> >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; >>>>>>>> To: Users(a)ovirt.org, "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>>>>> Sent: Wednesday, June 17, 2015 3:24:35 PM >>>>>>>> Subject: [ovirt-users] host-deploy on HE hosts failing: Job for >>>>>>>> vdsmd.service canceled. >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> I have the following problem with host-deploy on HE hosts: >>>>>>>> >>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.plugins.otopi.services.systemd >>>>>>>>> plugin.execute:937 execute-output: ('/bin/systemctl', 'stop', >>>>>>>>> 'vdsmd.service') stderr: >>>>>>>>> Job for vdsmd.service canceled. >>>>>>>>> >>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.context context._executeMethod:152 >>>>>>>>> method >>>>>>>>> exception >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "/tmp/ovirt-1WPaDxJOpX/pythonlib/otopi/context.py", line >>>>>>>>> 142, >>>>>>>>> in >>>>>>>>> _executeMethod >>>>>>>>> method['method']() >>>>>>>>> File >>>>>>>>> "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/ovirt-host-deploy/vdsm/packages.py", >>>>>>>>> line 106, in _packages >>>>>>>>> self.services.state('vdsmd', False) >>>>>>>>> File >>>>>>>>> "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/otopi/services/systemd.py", >>>>>>>>> line 138, in state >>>>>>>>> 'start' if state else 'stop' >>>>>>>>> File >>>>>>>>> "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/otopi/services/systemd.py", >>>>>>>>> line 77, in _executeServiceCommand >>>>>>>>> raiseOnError=raiseOnError >>>>>>>>> File "/tmp/ovirt-1WPaDxJOpX/pythonlib/otopi/plugin.py", line >>>>>>>>> 942, >>>>>>>>> in >>>>>>>>> execute >>>>>>>>> command=args[0], >>>>>>>>> RuntimeError: Command '/bin/systemctl' failed to execute >>>>>>>>> 2015-06-17 13:22:32 ERROR otopi.context context._executeMethod:161 >>>>>>>>> Failed >>>>>>>>> to execute stage 'Package installation': Command '/bin/systemctl' >>>>>>>>> failed >>>>>>>>> to execute >>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.transaction transaction.abort:131 >>>>>>>>> aborting >>>>>>>>> 'Yum Transaction' >>>>>>>>> 2015-06-17 13:22:32 INFO otopi.plugins.otopi.packagers.yumpackager >>>>>>>>> yumpackager.info:92 Yum Performing yum transaction rollback >>>>>>>>> Loaded plugins: auto-update-debuginfo, fastestmirror >>>>>>>> >>>>>>>> >>>>>>>> I do not have 'standard' (=non ha) hosts; so I cannot tell for >>>>>>>> the >>>>>>>> others but suspect it relates to HE. >>>>>>> >>>>>>> Yes, it's the hosted-engine HA agent which keeps VDSM up so >>>>>>> host-deploy >>>>>>> fails on that. >>>>>>> The capability to re-deploy an hosted-engine host from the Web UI is >>>>>>> not >>>>>>> available on oVirt 3.5, it will included in 3.6; please help us >>>>>>> testing >>>>>>> it >>>>>>> on the first beta in a few weeks. >>>>>>> >>>>>>> By the way, why are you trying to re-deploy an host-deploy host from >>>>>>> the >>>>>>> web UI? >>>>>> Ok, did not know about that. I use 'Reinstall' from the UI. >>>>>> >>>>>> Until now, I thought the best way to update the hosts after new engine >>>>>> relases was: >>>>>> 1- update engine >>>>>> 2. set host to maintane >>>>>> 4. yum update on host for latest vdsm >>>>>> 3- Use reinstall from UI. >>>>>> >>>>>> Please advise! >>>>> >>>>> Running on your HE host: >>>>> # hosted-engine --set-maintenance=local >>>>> waiting for everything to be migrated if needed >>>>> # yum update >>>>> # hosted-engine --set-maintenance=none >>>>> >>>>> should do the job >>>> Ok, I see; I think one should restart vdsmd also; IIRC it is not >>>> restarted in the process? >>> >>> If updated it will be restarted by its rpm postscript. >> >> Ok. However, I still need reinstall for configuration updates like >> gluster / firewall etc. > > RPM should perform intelligent upgrading of packages with configuration > files at least saving them on conf.rpmsave and alerting if not forward > compatible. If not, it's a packaging bug. > >> Also, hosted-enginge-deploy is using 'reinstall'. How do I work around >> that? > > Sorry, I didn't get it. I meant 'Reinstall' does a lot of other things apart from updating packages. For instance, what am I supposed to do if I switch on kdump integration or gluster on the cluster? AFIK this only works with transparently for engine 'Reinstall'.

As for the he-setup part; when deploying the first or any additional hosts the same routine as reinstall will be triggered - in the 'Installing host' section. > >> >>> >>>>>>>> Steps: >>>>>>>> 1. set the host to maintenance >>>>>>>> At this point, only the HE datacenter is still mounted: >>>>>>>> mount |grep nfs >>>>>>>> nexstor01.sec.int.m-box.de:/volumes/ovirt/engine on >>>>>>>> /rhev/data-center/mnt/nexstor01.sec.int.m-box.de:_volumes_ovirt_engine >>>>>>>> type nfs >>>>>>>> (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=10.11.0.30,mountvers=3,mountport=58144,mountproto=udp,local_lock=none,addr=10.11.0.30) >>>>>>>> >>>>>>>> 2. Reinstall host >>>>>>>> -> Error, host deactivated (Job for vdsmd.service canceled.) >>>>>>>> >>>>>>>> 3. A workaround is to shutdown sanlock: >>>>>>>> >>>>>>>> # systemctl stop vdsmd >>>>>>>> Job for vdsmd.service canceled. >>>>>>>> >>>>>>>> # sanlock shutdown >>>>>>>> # systemctl stop vdsmd >>>>>>>> >>>>>>>> 4. Now, host deploy works as expected. >>>>>>>> >>>>>>>> >>>>>>>> Versions: >>>>>>>> EL7 centos >>>>>>>> ovirt-3.5.2 -3.5.3 >>>>>>>> sanlock-3.2.2-2.el7.x86_64 >>>>>>>> vdsm 4.14.14 - 4.14.20 >>>>>>>> >>>>>>>> -- >>>>>>>> Daniel Helgenberger >>>>>>>> m box bewegtbild GmbH >>>>>>>> >>>>>>>> P: +49/30/2408781-22 >>>>>>>> F: +49/30/2408781-10 >>>>>>>> >>>>>>>> ACKERSTR. 19 >>>>>>>> D-10115 BERLIN >>>>>>>> >>>>>>>> >>>>>>>> www.m-box.de www.monkeymen.tv >>>>>>>> >>>>>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list >>>>>>>> Users(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>> >>>>>> -- >>>>>> Daniel Helgenberger >>>>>> m box bewegtbild GmbH >>>>>> >>>>>> P: +49/30/2408781-22 >>>>>> F: +49/30/2408781-10 >>>>>> >>>>>> ACKERSTR. 19 >>>>>> D-10115 BERLIN >>>>>> >>>>>> >>>>>> www.m-box.de www.monkeymen.tv >>>>>> >>>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>> >>>> -- >>>> Daniel Helgenberger >>>> m box bewegtbild GmbH >>>> >>>> P: +49/30/2408781-22 >>>> F: +49/30/2408781-10 >>>> >>>> ACKERSTR. 19 >>>> D-10115 BERLIN >>>> >>>> >>>> www.m-box.de www.monkeymen.tv >>>> >>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >> >> -- >> Daniel Helgenberger >> m box bewegtbild GmbH >> >> P: +49/30/2408781-22 >> F: +49/30/2408781-10 >> >> ACKERSTR. 19 >> D-10115 BERLIN >> >> >> www.m-box.de www.monkeymen.tv >> >> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 > -- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767