Re: [ovirt-users] oVirt hosted-engine setup stuck in final phase registering host to engine
Hello, of course. See attached File. Bye Am 18.05.2016 um 11:13 schrieb Simone Tiraboschi:
It looks, correct.
Can you please share the whole log file?
On Wed, May 18, 2016 at 11:03 AM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Hello,
from the last setup Log:
2016-05-11 18:19:31 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:'hosted_engine_1'
Am 18.05.2016 um 10:59 schrieb Simone Tiraboschi:
On Wed, May 18, 2016 at 10:33 AM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Dear list,
i'm working for days on a hosted-engine setup on glusterfs and 'm stuck in the last phase of hosted-engine --deploy see Text output (for two days now :-( ). I've only ovirtmgmt Bridge up and running and all DNS-Names seen in the screenshot are resolvable. It shouldn't be related to your problem regarding multiple nic's in the engine-vm. As seen the hosted-engine setup is already succsessfull, engine-vm running, but the setup fails in registering the host (microcloud21...) in the engine.
[...]
|- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160517222119-ndcyxn.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160517223055-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=rxmgmt.databay.de <http://rxmgmt.databay.de>, CN=engine-mcIII.rxmgmt.databay.de.77792, Subject: C=US, O=rxmgmt.databay.de <http://rxmgmt.databay.de>, CN=engine-mcIII.rxmgmt.databay.de.77792, Fingerprint (SHA-1): D71A5EA7C5C2C2450F1473523C3A1E022945430 [ INFO ] Connecting to the Engine [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]: 1
Checking for oVirt-Engine status at engine-mcIII.rxmgmt.databay.de... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]:
Additionally I can see the error in the eninge-vm in /var/log/ovirt-engine/engine.log:
2016-05-17 14:08:15,296 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-13) [9277b0a] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server microcloud21.rxmgmt.databay.de <http://microcloud21.rxmgmt.databay.de>,VALIDATION_VDS_NAME_INVALID 2016-05-17 14:08:15,360 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Host name must be formed of alphanumeric characters, numbers or "-_."]
'microcloud21.rxmgmt.databay.de <http://microcloud21.rxmgmt.databay.de>' is the host address, in oVirt the host has also a name. Can you please look for OVEHOSTED_ENGINE/appHostName in your hosted-engine-setup logs?
The hostname that should be registered is the correct fqdn for the virtualization host that runs the engine-vm, points to the ip of the ovirtmgmt-Bridge and also the IP is reverse-resolvable to the hostname. I checked /etc/hostname of the host for non-printable characters and as you can see that is a valid hostname.
All is running on Centos 7 Versions: ovirt-engine-appliance.noarch 3.6-20160420.1.el7.centos @ovirt-3.6 ovirt-engine-sdk-python.noarch 3.6.5.0-1.el7.centos @ovirt-3.6 ovirt-host-deploy.noarch 1.4.1-1.el7.centos @ovirt-3.6 ovirt-hosted-engine-ha.noarch 1.3.5.3-1.1.el7 @centos-ovirt36 ovirt-hosted-engine-setup.noarch 1.3.5.0-1.1.el7 @centos-ovirt36 ovirt-setup-lib.noarch 1.0.1-1.el7.centos @ovirt-3.6 ovirt-vmconsole.noarch 1.0.0-1.el7.centos @ovirt-3.6 ovirt-vmconsole-host.noarch 1.0.0-1.el7.centos @ovirt-3.6
Since I already repeated the setup for many times now, I need your help. --
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <mailto:rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <mailto:rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
-- *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <mailto:rs@databay.de> *Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------------------------------------------------
The issue is here: 2016-05-18 00:16:17 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:''hosted_engine_1'' You are quoting it twice and so it will try to use the string with the inner quote which is not a valid name. Does it come from your custom '/root/engine_answers.conf'? if so please fix it there and try again. On Wed, May 18, 2016 at 11:22 AM, Ralf Schenk <rs@databay.de> wrote:
Hello,
of course. See attached File.
Bye
Am 18.05.2016 um 11:13 schrieb Simone Tiraboschi:
It looks, correct.
Can you please share the whole log file?
On Wed, May 18, 2016 at 11:03 AM, Ralf Schenk <rs@databay.de> wrote:
Hello,
from the last setup Log:
2016-05-11 18:19:31 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:'hosted_engine_1'
Am 18.05.2016 um 10:59 schrieb Simone Tiraboschi:
On Wed, May 18, 2016 at 10:33 AM, Ralf Schenk < <rs@databay.de> rs@databay.de> wrote:
Dear list,
i'm working for days on a hosted-engine setup on glusterfs and 'm stuck in the last phase of hosted-engine --deploy see Text output (for two days now :-( ). I've only ovirtmgmt Bridge up and running and all DNS-Names seen in the screenshot are resolvable. It shouldn't be related to your problem regarding multiple nic's in the engine-vm. As seen the hosted-engine setup is already succsessfull, engine-vm running, but the setup fails in registering the host (microcloud21...) in the engine.
[...]
|- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160517222119-ndcyxn.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160517223055-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=rxmgmt.databay.de, CN=engine-mcIII.rxmgmt.databay.de.77792, Subject: C=US, O= rxmgmt.databay.de, CN=engine-mcIII.rxmgmt.databay.de.77792, Fingerprint (SHA-1): D71A5EA7C5C2C2450F1473523C3A1E022945430 [ INFO ] Connecting to the Engine [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]: 1
Checking for oVirt-Engine status at engine-mcIII.rxmgmt.databay.de... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]:
Additionally I can see the error in the eninge-vm in /var/log/ovirt-engine/engine.log:
2016-05-17 14:08:15,296 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-13) [9277b0a] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server microcloud21.rxmgmt.databay.de,VALIDATION_VDS_NAME_INVALID 2016-05-17 14:08:15,360 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Host name must be formed of alphanumeric characters, numbers or "-_."]
'microcloud21.rxmgmt.databay.de' is the host address, in oVirt the host has also a name. Can you please look for OVEHOSTED_ENGINE/appHostName in your hosted-engine-setup logs?
The hostname that should be registered is the correct fqdn for the virtualization host that runs the engine-vm, points to the ip of the ovirtmgmt-Bridge and also the IP is reverse-resolvable to the hostname. I checked /etc/hostname of the host for non-printable characters and as you can see that is a valid hostname.
All is running on Centos 7 Versions: ovirt-engine-appliance.noarch 3.6-20160420.1.el7.centos @ovirt-3.6 ovirt-engine-sdk-python.noarch 3.6.5.0-1.el7.centos @ovirt-3.6 ovirt-host-deploy.noarch 1.4.1-1.el7.centos @ovirt-3.6 ovirt-hosted-engine-ha.noarch 1.3.5.3-1.1.el7 @centos-ovirt36 ovirt-hosted-engine-setup.noarch 1.3.5.0-1.1.el7 @centos-ovirt36 ovirt-setup-lib.noarch 1.0.1-1.el7.centos @ovirt-3.6 ovirt-vmconsole.noarch 1.0.0-1.el7.centos @ovirt-3.6 ovirt-vmconsole-host.noarch 1.0.0-1.el7.centos @ovirt-3.6
Since I already repeated the setup for many times now, I need your help. --
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail <rs@databay.de>*rs@databay.de <rs@databay.de>*
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
This is a multi-part message in MIME format. --------------9B2362D711CFA409D785EAEA Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hello, thanks a lot. Yesterday I stood at the edge off a cliff. Now I'm one step beyond... Setup finished sucsessfully, engine-vm was shutdown - and no HA Config, No running engine VM afterwards. :-( Restart of ovirt-ha-agent/ovirt-ha-broker doesn't work. Reboot of Host neither. No (mounted) /var/run/ovirt-hosted-engine-ha/vm.conf. I can do "hosted-engine --connect-storage" which mounts my replica 3 gluster under /rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de:_engine. I found the tar file with configs in my volume and I am able to extract all the files (fhanswers.conf, hosted-engine.conf, broker.conf, vm.conf) from the conf volume which is a tar file (/rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de:_engine/de9c1cb5-a11c-4f0d-b436-7e37378702c3/images/2d2145f5-9406-40c3-b85a-c17be924dd16/b64c8bac-f978-441e-b71c-2fa1d0fc078f) . Is it possible to persist that and get ha up and running ? I think it was not persisted in vdsm, am I right ? I found your post http://lists.ovirt.org/pipermail/users/2015-December/036305.html that looked like a similar Problem. (tar -cO * | dd of=/rhev/data-center/mnt/blockSD/$sdUUID/images/$conf_image_UUID/$conf_volume_UUID oflag=direct) ? Bye Am 18.05.2016 um 11:38 schrieb Simone Tiraboschi:
The issue is here: 2016-05-18 00:16:17 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:''hosted_engine_1'' You are quoting it twice and so it will try to use the string with the inner quote which is not a valid name.
Does it come from your custom '/root/engine_answers.conf'? if so please fix it there and try again.
On Wed, May 18, 2016 at 11:22 AM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Hello,
of course. See attached File.
Bye
Am 18.05.2016 um 11:13 schrieb Simone Tiraboschi:
It looks, correct.
Can you please share the whole log file?
On Wed, May 18, 2016 at 11:03 AM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Hello,
from the last setup Log:
2016-05-11 18:19:31 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:'hosted_engine_1'
Am 18.05.2016 um 10:59 schrieb Simone Tiraboschi:
On Wed, May 18, 2016 at 10:33 AM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Dear list,
i'm working for days on a hosted-engine setup on glusterfs and 'm stuck in the last phase of hosted-engine --deploy see Text output (for two days now :-( ). I've only ovirtmgmt Bridge up and running and all DNS-Names seen in the screenshot are resolvable. It shouldn't be related to your problem regarding multiple nic's in the engine-vm. As seen the hosted-engine setup is already succsessfull, engine-vm running, but the setup fails in registering the host (microcloud21...) in the engine.
[...]
|- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160517222119-ndcyxn.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160517223055-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=rxmgmt.databay.de <http://rxmgmt.databay.de>, CN=engine-mcIII.rxmgmt.databay.de.77792, Subject: C=US, O=rxmgmt.databay.de <http://rxmgmt.databay.de>, CN=engine-mcIII.rxmgmt.databay.de.77792, Fingerprint (SHA-1): D71A5EA7C5C2C2450F1473523C3A1E022945430 [ INFO ] Connecting to the Engine [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]: 1
Checking for oVirt-Engine status at engine-mcIII.rxmgmt.databay.de... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]:
Additionally I can see the error in the eninge-vm in /var/log/ovirt-engine/engine.log:
2016-05-17 14:08:15,296 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-13) [9277b0a] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server microcloud21.rxmgmt.databay.de <http://microcloud21.rxmgmt.databay.de>,VALIDATION_VDS_NAME_INVALID 2016-05-17 14:08:15,360 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Host name must be formed of alphanumeric characters, numbers or "-_."]
'microcloud21.rxmgmt.databay.de <http://microcloud21.rxmgmt.databay.de>' is the host address, in oVirt the host has also a name. Can you please look for OVEHOSTED_ENGINE/appHostName in your hosted-engine-setup logs?
The hostname that should be registered is the correct fqdn for the virtualization host that runs the engine-vm, points to the ip of the ovirtmgmt-Bridge and also the IP is reverse-resolvable to the hostname. I checked /etc/hostname of the host for non-printable characters and as you can see that is a valid hostname.
All is running on Centos 7 Versions: ovirt-engine-appliance.noarch 3.6-20160420.1.el7.centos @ovirt-3.6 ovirt-engine-sdk-python.noarch 3.6.5.0-1.el7.centos @ovirt-3.6 ovirt-host-deploy.noarch 1.4.1-1.el7.centos @ovirt-3.6 ovirt-hosted-engine-ha.noarch 1.3.5.3-1.1.el7 @centos-ovirt36 ovirt-hosted-engine-setup.noarch 1.3.5.0-1.1.el7 @centos-ovirt36 ovirt-setup-lib.noarch 1.0.1-1.el7.centos @ovirt-3.6 ovirt-vmconsole.noarch 1.0.0-1.el7.centos @ovirt-3.6 ovirt-vmconsole-host.noarch 1.0.0-1.el7.centos @ovirt-3.6
Since I already repeated the setup for many times now, I need your help. --
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de <mailto:rs@databay.de>*
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <mailto:rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <mailto:rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
-- *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <mailto:rs@databay.de> *Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------------------------------------------------ --------------9B2362D711CFA409D785EAEA Content-Type: multipart/related; boundary="------------27D3DA4256217B0B9C4B7B5B" --------------27D3DA4256217B0B9C4B7B5B Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>thanks a lot. Yesterday I stood at the edge off a cliff. Now I'm one step beyond...</p> <p>Setup finished sucsessfully, engine-vm was shutdown - and no HA Config, No running engine VM afterwards. :-(<br> </p> <p>Restart of ovirt-ha-agent/ovirt-ha-broker doesn't work. Reboot of Host neither. No (mounted) /var/run/ovirt-hosted-engine-ha/vm.conf.</p> <p>I can do "hosted-engine --connect-storage" which mounts my replica 3 gluster under /rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de:_engine.</p> <p>I found the tar file with configs in my volume and I am able to extract all the files (fhanswers.conf, hosted-engine.conf, broker.conf, vm.conf) from the conf volume which is a tar file (/rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de:_engine/de9c1cb5-a11c-4f0d-b436-7e37378702c3/images/2d2145f5-9406-40c3-b85a-c17be924dd16/b64c8bac-f978-441e-b71c-2fa1d0fc078f) . <br> </p> <p>Is it possible to persist that and get ha up and running ? I think it was not persisted in vdsm, am I right ?</p> <p>I found your post <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/pipermail/users/2015-December/036305.html">http://lists.ovirt.org/pipermail/users/2015-December/036305.html</a> that looked like a similar Problem.</p> <p>(tar -cO * | dd of=/rhev/data-center/mnt/blockSD/$sdUUID/images/$conf_image_UUID/$conf_volume_UUID oflag=direct) ?<br> </p> <p>Bye<br> </p> <p><br> </p> <div class="moz-cite-prefix">Am 18.05.2016 um 11:38 schrieb Simone Tiraboschi:<br> </div> <blockquote cite="mid:CAN8-ONqmN9Cc_PD6iYCsoHnH3XgOswFXc0dhoANcqEjiPO=QaQ@mail.gmail.com" type="cite"> <div dir="ltr"> <div>The issue is here:</div> <div>2016-05-18 00:16:17 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:''hosted_engine_1''<br> </div> <div>You are quoting it twice and so it will try to use the string with the inner quote which is not a valid name.<br> </div> <div><br> </div> <div>Does it come from your custom '/root/engine_answers.conf'? if so please fix it there and try again.</div> <div><br> </div> </div> <div class="gmail_extra"><br> <div class="gmail_quote">On Wed, May 18, 2016 at 11:22 AM, Ralf Schenk <span dir="ltr"><<a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>of course. See attached File.</p> <p>Bye<br> </p> <div> <div class="h5"> <br> <div>Am 18.05.2016 um 11:13 schrieb Simone Tiraboschi:<br> </div> <blockquote type="cite"> <div dir="ltr">It looks, correct. <div><br> </div> <div>Can you please share the whole log file?</div> </div> <div class="gmail_extra"><br> <div class="gmail_quote">On Wed, May 18, 2016 at 11:03 AM, Ralf Schenk <span dir="ltr"><<a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:rs@databay.de">rs@databay.de</a></a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>from the last setup Log:</p> <p><tt>2016-05-11 18:19:31 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:'hosted_engine_1'</tt><tt><br> </tt></p> <div> <div> <br> <div>Am 18.05.2016 um 10:59 schrieb Simone Tiraboschi:<br> </div> <blockquote type="cite"> <div dir="ltr"><br> <div class="gmail_extra"><br> <div class="gmail_quote">On Wed, May 18, 2016 at 10:33 AM, Ralf Schenk <span dir="ltr"><<a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:rs@databay.de">rs@databay.de</a></a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <p>Dear list,</p> <p>i'm working for days on a hosted-engine setup on glusterfs and 'm stuck in the last phase of hosted-engine --deploy see Text output (for two days now :-( ). I've only ovirtmgmt Bridge up and running and all DNS-Names seen in the screenshot are resolvable. It shouldn't be related to your problem regarding multiple nic's in the engine-vm. As seen the hosted-engine setup is already succsessfull, engine-vm running, but the setup fails in registering the host (microcloud21...) in the engine.</p> <p><tt><br> </tt></p> <p><tt>[...]</tt></p> <p><tt> |- [ INFO ] Starting engine service</tt><tt><br> </tt><tt> |- [ INFO ] Restarting httpd</tt><tt><br> </tt><tt> |- [ INFO ] Restarting ovirt-vmconsole proxy service</tt><tt><br> </tt><tt> |- [ INFO ] Stage: Clean up</tt><tt><br> </tt><tt> |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160517222119-ndcyxn.log</tt><tt><br> </tt><tt> |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160517223055-setup.conf'</tt><tt><br> </tt><tt> |- [ INFO ] Stage: Pre-termination</tt><tt><br> </tt><tt> |- [ INFO ] Stage: Termination</tt><tt><br> </tt><tt> |- [ INFO ] Execution of setup completed successfully</tt><tt><br> </tt><tt> |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS</tt><tt><br> </tt><tt>[ INFO ] Engine-setup successfully completed</tt><tt><br> </tt><tt>[ INFO ] Engine is still unreachable</tt><tt><br> </tt><tt>[ INFO ] Engine is still not reachable, waiting...</tt><tt><br> </tt><tt>[ INFO ] Engine replied: DB Up!Welcome to Health Status!</tt><tt><br> </tt><tt>[ INFO ] Acquiring internal CA cert from the engine</tt><tt><br> </tt><tt>[ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct:</tt><tt><br> </tt><tt>[ INFO ] Issuer: C=US, O=<a moz-do-not-send="true" href="http://rxmgmt.databay.de" target="_blank">rxmgmt.databay.de</a>, CN=engine-mcIII.rxmgmt.databay.de.77792, Subject: C=US, O=<a moz-do-not-send="true" href="http://rxmgmt.databay.de" target="_blank">rxmgmt.databay.de</a>, CN=engine-mcIII.rxmgmt.databay.de.77792, Fingerprint (SHA-1): D71A5EA7C5C2C2450F1473523C3A1E022945430</tt><tt><br> </tt><tt>[ INFO ] Connecting to the Engine</tt><tt><br> </tt><tt>[ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt> Please check Engine VM configuration.</tt><tt><br> </tt><tt><br> </tt><tt> Make a selection from the options below:</tt><tt><br> </tt><tt> (1) Continue setup - Engine VM configuration has been fixed</tt><tt><br> </tt><tt> (2) Abort setup</tt><tt><br> </tt><tt><br> </tt><tt> (1, 2)[1]: 1</tt><tt><br> </tt><tt><br> </tt><tt> Checking for oVirt-Engine status at engine-mcIII.rxmgmt.databay.de...</tt><tt><br> </tt><tt>[ INFO ] Engine replied: DB Up!Welcome to Health Status!</tt><tt><br> </tt><tt>[ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt> Please check Engine VM configuration.</tt><tt><br> </tt><tt><br> </tt><tt> Make a selection from the options below:</tt><tt><br> </tt><tt> (1) Continue setup - Engine VM configuration has been fixed</tt><tt><br> </tt><tt> (2) Abort setup</tt><tt><br> </tt><tt><br> </tt><tt> (1, 2)[1]:</tt><tt><br> </tt></p> <p>Additionally I can see the error in the eninge-vm in /var/log/ovirt-engine/engine.log:</p> <p><tt>2016-05-17 14:08:15,296 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-13) [9277b0a] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server <a moz-do-not-send="true" href="http://microcloud21.rxmgmt.databay.de" target="_blank">microcloud21.rxmgmt.databay.de</a>,VALIDATION_VDS_NAME_INVALID</tt><tt><br> </tt><tt>2016-05-17 14:08:15,360 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Host name must be formed of alphanumeric characters, numbers or "-_."]</tt></p> </div> </blockquote> <div class="gmail_quote"><br> </div> '<a moz-do-not-send="true" href="http://microcloud21.rxmgmt.databay.de" target="_blank">microcloud21.rxmgmt.databay.de</a>' is the host address, in oVirt the host has also a name.</div> <div class="gmail_quote">Can you please look for OVEHOSTED_ENGINE/appHostName in your hosted-engine-setup logs?<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <p><br> </p> The hostname that should be registered is the correct fqdn for the virtualization host that runs the engine-vm, points to the ip of the ovirtmgmt-Bridge and also the IP is reverse-resolvable to the hostname. <br> I checked /etc/hostname of the host for non-printable characters and as you can see that is a valid hostname.<br> <br> All is running on Centos 7 Versions: <br> <tt>ovirt-engine-appliance.noarch 3.6-20160420.1.el7.centos @ovirt-3.6</tt><tt><br> </tt><tt>ovirt-engine-sdk-python.noarch 3.6.5.0-1.el7.centos @ovirt-3.6</tt><tt><br> </tt><tt>ovirt-host-deploy.noarch 1.4.1-1.el7.centos @ovirt-3.6</tt><tt><br> </tt><tt>ovirt-hosted-engine-ha.noarch 1.3.5.3-1.1.el7 @centos-ovirt36</tt><tt><br> </tt><tt>ovirt-hosted-engine-setup.noarch 1.3.5.0-1.1.el7 @centos-ovirt36</tt><tt><br> </tt><tt>ovirt-setup-lib.noarch 1.0.1-1.el7.centos @ovirt-3.6</tt><tt><br> </tt><tt>ovirt-vmconsole.noarch 1.0.0-1.el7.centos @ovirt-3.6</tt><tt><br> </tt><tt>ovirt-vmconsole-host.noarch 1.0.0-1.el7.centos @ovirt-3.6</tt><tt><br> </tt><br> Since I already repeated the setup for many times now, I need your help.<br> <div>-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part8.8F347087.F1510FEF@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070" value="+492405408370" target="_blank">+49 (0) 24 05 / 40 83 70</a><br> fax <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759" value="+4924054083759" target="_blank">+49 (0) 24 05 / 40 83 759</a><br> mail <font color="#FF0000"><b><a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:rs@databay.de">rs@databay.de</a></a></b></font><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a moz-do-not-send="true" href="http://www.databay.de" target="_blank"><font color="#FF0000"><b><a class="moz-txt-link-abbreviated" href="http://www.databay.de">www.databay.de</a></b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </div> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br> <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br> <br> </blockquote> </div> <br> </div> </div> </blockquote> <br> <div>-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part15.CD9E20F6.1B5E1A95@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070" value="+492405408370" target="_blank">+49 (0) 24 05 / 40 83 70</a><br> fax <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759" value="+4924054083759" target="_blank">+49 (0) 24 05 / 40 83 759</a><br> mail <a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank"><font color="#FF0000"><b><a class="moz-txt-link-abbreviated" href="mailto:rs@databay.de">rs@databay.de</a></b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a moz-do-not-send="true" href="http://www.databay.de" target="_blank"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </div> </div> </div> </blockquote> </div> <br> </div> </blockquote> <br> <div>-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part20.4E9AE952.FB23BB2D@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070" value="+492405408370" target="_blank">+49 (0) 24 05 / 40 83 70</a><br> fax <a moz-do-not-send="true" href="tel:%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759" value="+4924054083759" target="_blank">+49 (0) 24 05 / 40 83 759</a><br> mail <a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank"><font color="#FF0000"><b>rs@databay.de</b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a moz-do-not-send="true" href="http://www.databay.de" target="_blank"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </div> </div> </div> </blockquote> </div> <br> </div> </blockquote> <br> <div class="moz-signature">-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part25.8656A912.044F1340@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon +49 (0) 24 05 / 40 83 70<br> fax +49 (0) 24 05 / 40 83 759<br> mail <a href="mailto:rs@databay.de"><font color="#FF0000"><b>rs@databay.de</b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a href="http://www.databay.de"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </body> </html> --------------27D3DA4256217B0B9C4B7B5B Content-Type: image/gif Content-Transfer-Encoding: base64 Content-ID: <part8.8F347087.F1510FEF@databay.de> R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------27D3DA4256217B0B9C4B7B5B Content-Type: image/gif Content-Transfer-Encoding: base64 Content-ID: <part15.CD9E20F6.1B5E1A95@databay.de> R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------27D3DA4256217B0B9C4B7B5B Content-Type: image/gif Content-Transfer-Encoding: base64 Content-ID: <part20.4E9AE952.FB23BB2D@databay.de> R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------27D3DA4256217B0B9C4B7B5B Content-Type: image/gif; name="logo_databay_email.gif" Content-Transfer-Encoding: base64 Content-ID: <part25.8656A912.044F1340@databay.de> Content-Disposition: inline; filename="logo_databay_email.gif" R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------27D3DA4256217B0B9C4B7B5B-- --------------9B2362D711CFA409D785EAEA--
On Wed, May 18, 2016 at 3:55 PM, Ralf Schenk <rs@databay.de> wrote:
Hello,
thanks a lot. Yesterday I stood at the edge off a cliff. Now I'm one step beyond...
Setup finished sucsessfully, engine-vm was shutdown - and no HA Config, No running engine VM afterwards. :-(
Do you have /etc/ovirt-hosted-engine/hosted-engine.conf ? that should be enough to mount the shared storage and start the VM extracting the VM configuration from there. Do you have any error from ovirt-ha-agent or from ovirt-ha-broker?
Restart of ovirt-ha-agent/ovirt-ha-broker doesn't work. Reboot of Host neither. No (mounted) /var/run/ovirt-hosted-engine-ha/vm.conf.
I can do "hosted-engine --connect-storage" which mounts my replica 3 gluster under /rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de: _engine.
I found the tar file with configs in my volume and I am able to extract all the files (fhanswers.conf, hosted-engine.conf, broker.conf, vm.conf) from the conf volume which is a tar file (/rhev/data-center/mnt/glusterSD/glusterfs.rxmgmt.databay.de:_engine/de9c1cb5-a11c-4f0d-b436-7e37378702c3/images/2d2145f5-9406-40c3-b85a-c17be924dd16/b64c8bac-f978-441e-b71c-2fa1d0fc078f) .
Is it possible to persist that and get ha up and running ? I think it was not persisted in vdsm, am I right ?
I found your post http://lists.ovirt.org/pipermail/users/2015-December/036305.html that looked like a similar Problem.
Here we had a bug but now the code is OK.
(tar -cO * | dd of=/rhev/data-center/mnt/blockSD/$sdUUID/images/$conf_image_UUID/$conf_volume_UUID oflag=direct) ?
Bye
Am 18.05.2016 um 11:38 schrieb Simone Tiraboschi:
The issue is here: 2016-05-18 00:16:17 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:''hosted_engine_1'' You are quoting it twice and so it will try to use the string with the inner quote which is not a valid name.
Does it come from your custom '/root/engine_answers.conf'? if so please fix it there and try again.
On Wed, May 18, 2016 at 11:22 AM, Ralf Schenk <rs@databay.de> wrote:
Hello,
of course. See attached File.
Bye
Am 18.05.2016 um 11:13 schrieb Simone Tiraboschi:
It looks, correct.
Can you please share the whole log file?
On Wed, May 18, 2016 at 11:03 AM, Ralf Schenk < <rs@databay.de> rs@databay.de> wrote:
Hello,
from the last setup Log:
2016-05-11 18:19:31 DEBUG otopi.context context.dumpEnvironment:510 ENV OVEHOSTED_ENGINE/appHostName=str:'hosted_engine_1'
Am 18.05.2016 um 10:59 schrieb Simone Tiraboschi:
On Wed, May 18, 2016 at 10:33 AM, Ralf Schenk < <rs@databay.de> rs@databay.de> wrote:
Dear list,
i'm working for days on a hosted-engine setup on glusterfs and 'm stuck in the last phase of hosted-engine --deploy see Text output (for two days now :-( ). I've only ovirtmgmt Bridge up and running and all DNS-Names seen in the screenshot are resolvable. It shouldn't be related to your problem regarding multiple nic's in the engine-vm. As seen the hosted-engine setup is already succsessfull, engine-vm running, but the setup fails in registering the host (microcloud21...) in the engine.
[...]
|- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160517222119-ndcyxn.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160517223055-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=rxmgmt.databay.de, CN=engine-mcIII.rxmgmt.databay.de.77792, Subject: C=US, O= rxmgmt.databay.de, CN=engine-mcIII.rxmgmt.databay.de.77792, Fingerprint (SHA-1): D71A5EA7C5C2C2450F1473523C3A1E022945430 [ INFO ] Connecting to the Engine [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]: 1
Checking for oVirt-Engine status at engine-mcIII.rxmgmt.databay.de... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ ERROR ] Cannot automatically add the host to cluster Default: Host name must be formed of alphanumeric characters, numbers or "-_."
Please check Engine VM configuration.
Make a selection from the options below: (1) Continue setup - Engine VM configuration has been fixed (2) Abort setup
(1, 2)[1]:
Additionally I can see the error in the eninge-vm in /var/log/ovirt-engine/engine.log:
2016-05-17 14:08:15,296 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-13) [9277b0a] CanDoAction of action 'AddVds' failed for user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server microcloud21.rxmgmt.databay.de,VALIDATION_VDS_NAME_INVALID 2016-05-17 14:08:15,360 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Host name must be formed of alphanumeric characters, numbers or "-_."]
'microcloud21.rxmgmt.databay.de' is the host address, in oVirt the host has also a name. Can you please look for OVEHOSTED_ENGINE/appHostName in your hosted-engine-setup logs?
The hostname that should be registered is the correct fqdn for the virtualization host that runs the engine-vm, points to the ip of the ovirtmgmt-Bridge and also the IP is reverse-resolvable to the hostname. I checked /etc/hostname of the host for non-printable characters and as you can see that is a valid hostname.
All is running on Centos 7 Versions: ovirt-engine-appliance.noarch 3.6-20160420.1.el7.centos @ovirt-3.6 ovirt-engine-sdk-python.noarch 3.6.5.0-1.el7.centos @ovirt-3.6 ovirt-host-deploy.noarch 1.4.1-1.el7.centos @ovirt-3.6 ovirt-hosted-engine-ha.noarch 1.3.5.3-1.1.el7 @centos-ovirt36 ovirt-hosted-engine-setup.noarch 1.3.5.0-1.1.el7 @centos-ovirt36 ovirt-setup-lib.noarch 1.0.1-1.el7.centos @ovirt-3.6 ovirt-vmconsole.noarch 1.0.0-1.el7.centos @ovirt-3.6 ovirt-vmconsole-host.noarch 1.0.0-1.el7.centos @ovirt-3.6
Since I already repeated the setup for many times now, I need your help. --
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail * <rs@databay.de>rs@databay.de <rs@databay.de>*
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen <http://www.databay.de>*www.databay.de <http://www.databay.de>*
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail <rs@databay.de>*rs@databay.de <rs@databay.de>*
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%2070> fax +49 (0) 24 05 / 40 83 759 <%2B49%20%280%29%2024%2005%20%2F%2040%2083%20759> mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
participants (2)
-
Ralf Schenk -
Simone Tiraboschi