[ovirt-users] host-deploy on HE hosts failing: Job for vdsmd.service canceled.

Wed Jun 17 17:06:53 UTC 2015

On 17.06.2015 18:30, Simone Tiraboschi wrote:
>
>
>> Il giorno 17/giu/2015, alle ore 18:21, Daniel Helgenberger <daniel.helgenberger at m-box.de> ha scritto:
>>
>>
>>
>>> On 17.06.2015 17:01, Simone Tiraboschi wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Daniel Helgenberger" <daniel.helgenberger at m-box.de>
>>>> To: "Simone Tiraboschi" <stirabos at redhat.com>
>>>> Cc: Users at ovirt.org, "Alon Bar-Lev" <alonbl at redhat.com>
>>>> Sent: Wednesday, June 17, 2015 4:49:07 PM
>>>> Subject: Re: [ovirt-users] host-deploy on HE hosts failing: Job for    vdsmd.service canceled.
>>>>
>>>>
>>>>
>>>>> On 17.06.2015 16:47, Simone Tiraboschi wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Daniel Helgenberger" <daniel.helgenberger at m-box.de>
>>>>>> To: "Simone Tiraboschi" <stirabos at redhat.com>
>>>>>> Cc: Users at ovirt.org, "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>> Sent: Wednesday, June 17, 2015 4:40:45 PM
>>>>>> Subject: Re: [ovirt-users] host-deploy on HE hosts failing: Job for
>>>>>>     vdsmd.service canceled.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 17.06.2015 16:35, Simone Tiraboschi wrote:
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Daniel Helgenberger" <daniel.helgenberger at m-box.de>
>>>>>>>> To: Users at ovirt.org, "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>>>> Sent: Wednesday, June 17, 2015 3:24:35 PM
>>>>>>>> Subject: [ovirt-users] host-deploy on HE hosts failing: Job for
>>>>>>>>     vdsmd.service canceled.
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have the following problem with host-deploy on HE hosts:
>>>>>>>>
>>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.plugins.otopi.services.systemd
>>>>>>>>> plugin.execute:937 execute-output: ('/bin/systemctl', 'stop',
>>>>>>>>> 'vdsmd.service') stderr:
>>>>>>>>> Job for vdsmd.service canceled.
>>>>>>>>>
>>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.context context._executeMethod:152
>>>>>>>>> method
>>>>>>>>> exception
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>      File "/tmp/ovirt-1WPaDxJOpX/pythonlib/otopi/context.py", line 142,
>>>>>>>>>      in
>>>>>>>>>      _executeMethod
>>>>>>>>>        method['method']()
>>>>>>>>>      File
>>>>>>>>>      "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/ovirt-host-deploy/vdsm/packages.py",
>>>>>>>>>      line 106, in _packages
>>>>>>>>>        self.services.state('vdsmd', False)
>>>>>>>>>      File
>>>>>>>>>      "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/otopi/services/systemd.py",
>>>>>>>>>      line 138, in state
>>>>>>>>>        'start' if state else 'stop'
>>>>>>>>>      File
>>>>>>>>>      "/tmp/ovirt-1WPaDxJOpX/otopi-plugins/otopi/services/systemd.py",
>>>>>>>>>      line 77, in _executeServiceCommand
>>>>>>>>>        raiseOnError=raiseOnError
>>>>>>>>>      File "/tmp/ovirt-1WPaDxJOpX/pythonlib/otopi/plugin.py", line 942,
>>>>>>>>>      in
>>>>>>>>>      execute
>>>>>>>>>        command=args[0],
>>>>>>>>> RuntimeError: Command '/bin/systemctl' failed to execute
>>>>>>>>> 2015-06-17 13:22:32 ERROR otopi.context context._executeMethod:161
>>>>>>>>> Failed
>>>>>>>>> to execute stage 'Package installation': Command '/bin/systemctl'
>>>>>>>>> failed
>>>>>>>>> to execute
>>>>>>>>> 2015-06-17 13:22:32 DEBUG otopi.transaction transaction.abort:131
>>>>>>>>> aborting
>>>>>>>>> 'Yum Transaction'
>>>>>>>>> 2015-06-17 13:22:32 INFO otopi.plugins.otopi.packagers.yumpackager
>>>>>>>>> yumpackager.info:92 Yum Performing yum transaction rollback
>>>>>>>>> Loaded plugins: auto-update-debuginfo, fastestmirror
>>>>>>>>
>>>>>>>>
>>>>>>>>      I do not have 'standard' (=non ha) hosts; so I cannot tell for the
>>>>>>>> others but suspect it relates to HE.
>>>>>>>
>>>>>>> Yes, it's the hosted-engine HA agent which keeps VDSM up so host-deploy
>>>>>>> fails on that.
>>>>>>> The capability to re-deploy an hosted-engine host from the Web UI is not
>>>>>>> available on oVirt 3.5, it will included in 3.6; please help us testing
>>>>>>> it
>>>>>>> on the first beta in  a few weeks.
>>>>>>>
>>>>>>> By the way, why are you trying to re-deploy an host-deploy host from the
>>>>>>> web UI?
>>>>>> Ok, did not know about that. I use 'Reinstall' from the UI.
>>>>>>
>>>>>> Until now, I thought the best way to update the hosts after new engine
>>>>>> relases was:
>>>>>> 1- update engine
>>>>>> 2. set host to maintane
>>>>>> 4. yum update on host for latest vdsm
>>>>>> 3- Use reinstall from UI.
>>>>>>
>>>>>> Please advise!
>>>>>
>>>>> Running on your HE host:
>>>>> # hosted-engine --set-maintenance=local
>>>>> waiting for everything to be migrated if needed
>>>>> # yum update
>>>>> # hosted-engine --set-maintenance=none
>>>>>
>>>>> should do the job
>>>> Ok, I see; I think one should restart vdsmd also; IIRC it is not
>>>> restarted in the process?
>>>
>>> If updated it will be restarted by its rpm postscript.
>>
>> Ok. However, I still need reinstall for configuration updates like
>> gluster / firewall etc.
>
> RPM should perform intelligent upgrading of packages with configuration files at least saving them on conf.rpmsave and alerting if not forward compatible. If not, it's a packaging bug.
>
>> Also, hosted-enginge-deploy is using 'reinstall'. How do I work around that?
>
> Sorry, I didn't get it.

I meant 'Reinstall' does a lot of other things apart from updating 
packages. For instance, what am I supposed to do if I switch on kdump 
integration or gluster on the cluster?

AFIK this only works with transparently for engine 'Reinstall'.

As for the he-setup part; when deploying the first or any additional 
hosts the same routine as reinstall will be triggered - in the 
'Installing host' section.

>
>>
>>>
>>>>>>>> Steps:
>>>>>>>> 1. set the host to maintenance
>>>>>>>> At this point, only the HE datacenter is still mounted:
>>>>>>>> mount |grep nfs
>>>>>>>> nexstor01.sec.int.m-box.de:/volumes/ovirt/engine on
>>>>>>>> /rhev/data-center/mnt/nexstor01.sec.int.m-box.de:_volumes_ovirt_engine
>>>>>>>> type nfs
>>>>>>>> (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=10.11.0.30,mountvers=3,mountport=58144,mountproto=udp,local_lock=none,addr=10.11.0.30)
>>>>>>>>
>>>>>>>> 2. Reinstall host
>>>>>>>> -> Error, host deactivated (Job for vdsmd.service canceled.)
>>>>>>>>
>>>>>>>> 3. A workaround is to shutdown sanlock:
>>>>>>>>
>>>>>>>> # systemctl stop vdsmd
>>>>>>>> Job for vdsmd.service canceled.
>>>>>>>>
>>>>>>>> # sanlock shutdown
>>>>>>>> # systemctl stop vdsmd
>>>>>>>>
>>>>>>>> 4. Now, host deploy works as expected.
>>>>>>>>
>>>>>>>>
>>>>>>>> Versions:
>>>>>>>> EL7 centos
>>>>>>>> ovirt-3.5.2 -3.5.3
>>>>>>>> sanlock-3.2.2-2.el7.x86_64
>>>>>>>> vdsm 4.14.14 - 4.14.20
>>>>>>>>
>>>>>>>> --
>>>>>>>> Daniel Helgenberger
>>>>>>>> m box bewegtbild GmbH
>>>>>>>>
>>>>>>>> P: +49/30/2408781-22
>>>>>>>> F: +49/30/2408781-10
>>>>>>>>
>>>>>>>> ACKERSTR. 19
>>>>>>>> D-10115 BERLIN
>>>>>>>>
>>>>>>>>
>>>>>>>> www.m-box.de  www.monkeymen.tv
>>>>>>>>
>>>>>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
>>>>>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>> --
>>>>>> Daniel Helgenberger
>>>>>> m box bewegtbild GmbH
>>>>>>
>>>>>> P: +49/30/2408781-22
>>>>>> F: +49/30/2408781-10
>>>>>>
>>>>>> ACKERSTR. 19
>>>>>> D-10115 BERLIN
>>>>>>
>>>>>>
>>>>>> www.m-box.de  www.monkeymen.tv
>>>>>>
>>>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
>>>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
>>>>
>>>> --
>>>> Daniel Helgenberger
>>>> m box bewegtbild GmbH
>>>>
>>>> P: +49/30/2408781-22
>>>> F: +49/30/2408781-10
>>>>
>>>> ACKERSTR. 19
>>>> D-10115 BERLIN
>>>>
>>>>
>>>> www.m-box.de  www.monkeymen.tv
>>>>
>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
>>
>> --
>> Daniel Helgenberger
>> m box bewegtbild GmbH
>>
>> P: +49/30/2408781-22
>> F: +49/30/2408781-10
>>
>> ACKERSTR. 19
>> D-10115 BERLIN
>>
>>
>> www.m-box.de  www.monkeymen.tv
>>
>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
>

-- 
Daniel Helgenberger
m box bewegtbild GmbH

P: +49/30/2408781-22
F: +49/30/2408781-10

ACKERSTR. 19
D-10115 BERLIN

www.m-box.de  www.monkeymen.tv

Geschäftsführer: Martin Retschitzegger / Michaela Göllner
Handeslregister: Amtsgericht Charlottenburg / HRB 112767